pegasus.transfer.bypass.input.staging breaks symlinking on the local site

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      In order to run on OSG, we need to set
      ```
      pegasus.transfer.bypass.input.staging=True
      ```
      however this breaks symlinking on the local site. Without this option set, local jobs can access their frame data with symlinks:
      ```
      { "type": "transfer",
      "id": 1,
      "src_urls": [

      { "site_label": "local", "url": "file:///home/dbrown/projects/osg/local-workflow-1/output/local-site-scratch/work/./main_ID0000001/./113494/H-H1_HOFT_C02-1134940160-4096.gwf", "checkpoint": "false" }

      ],
      "dest_urls": [

      { "site_label": "local", "url": "symlink://$PWD/113494/H-H1_HOFT_C02-1134940160-4096.gwf" }

      ] }
      ```

      However, when I set this, these become copy jobs:
      ```
      { "type": "transfer",
      "id": 1,
      "src_urls": [

      { "site_label": "local", "url": "file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134940160-4096.gwf", "priority": 400, "checkpoint": "false" }

      ,
      ],
      "dest_urls": [

      { "site_label": "local", "url": "file://$PWD/113494/H-H1_HOFT_C02-1134940160-4096.gwf" }

      ] }

      This is a blocker for us using OSG.

            Assignee:
            Karan Vahi
            Reporter:
            Duncan Brown
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: