Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1367

Support for retrieval from HPSS tape store using commands htar and hsi

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master, 4.9.1
    • Fix Version/s: master, 5.0.0, 4.9.2
    • Component/s: pegasus-transfer
    • Labels:
      None

      Description

      For work with ZTF pipelines, we need to put in support in pegasus transfer for retrieving from NERSC Tape Storage.
      https://docs.nersc.gov/filesystems/archive_access/

      For the initial version, we can assume that transfer jobs will run on NERSC nodes, so that tokens are not required.

      Also we would need to make sure that transfer jobs run in the xfer queue
      https://docs.nersc.gov/filesystems/archive/#order-large-retrievals

      Cannot transfer files using htar
      Htar requires the node you're on to accept incoming connections from its movers. This is not possible from a compute node at NERSC, so htar transfers will fail. Instead we recommend you use our special xfer queue for data transfers

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              vahi Karan Vahi
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: