-
Type: New Feature
-
Resolution: Fixed
-
Priority: Major
-
Affects Version/s: master, 4.9.1
-
Component/s: CLI: pegasus-transfer
-
None
For work with ZTF pipelines, we need to put in support in pegasus transfer for retrieving from NERSC Tape Storage.
https://docs.nersc.gov/filesystems/archive_access/
For the initial version, we can assume that transfer jobs will run on NERSC nodes, so that tokens are not required.
Also we would need to make sure that transfer jobs run in the xfer queue
https://docs.nersc.gov/filesystems/archive/#order-large-retrievals
Cannot transfer files using htar
Htar requires the node you're on to accept incoming connections from its movers. This is not possible from a compute node at NERSC, so htar transfers will fail. Instead we recommend you use our special xfer queue for data transfers