Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1367

Support for retrieval from HPSS tape store using commands htar and hsi

    XMLWordPrintable

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • master, 4.9.1
    • master, 5.0.0, 4.9.2
    • CLI: pegasus-transfer
    • None

    Description

      For work with ZTF pipelines, we need to put in support in pegasus transfer for retrieving from NERSC Tape Storage.
      https://docs.nersc.gov/filesystems/archive_access/

      For the initial version, we can assume that transfer jobs will run on NERSC nodes, so that tokens are not required.

      Also we would need to make sure that transfer jobs run in the xfer queue
      https://docs.nersc.gov/filesystems/archive/#order-large-retrievals

      Cannot transfer files using htar
      Htar requires the node you're on to accept incoming connections from its movers. This is not possible from a compute node at NERSC, so htar transfers will fail. Instead we recommend you use our special xfer queue for data transfers

      Attachments

        Activity

          People

            vahi Karan Vahi
            vahi Karan Vahi
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: