Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1135

pegasus.transfer.bypass.input.staging breaks symlinking on the local site

XMLWordPrintable

      In order to run on OSG, we need to set
      ```
      pegasus.transfer.bypass.input.staging=True
      ```
      however this breaks symlinking on the local site. Without this option set, local jobs can access their frame data with symlinks:
      ```
      { "type": "transfer",
      "id": 1,
      "src_urls": [

      { "site_label": "local", "url": "file:///home/dbrown/projects/osg/local-workflow-1/output/local-site-scratch/work/./main_ID0000001/./113494/H-H1_HOFT_C02-1134940160-4096.gwf", "checkpoint": "false" }

      ],
      "dest_urls": [

      { "site_label": "local", "url": "symlink://$PWD/113494/H-H1_HOFT_C02-1134940160-4096.gwf" }

      ] }
      ```

      However, when I set this, these become copy jobs:
      ```
      { "type": "transfer",
      "id": 1,
      "src_urls": [

      { "site_label": "local", "url": "file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134940160-4096.gwf", "priority": 400, "checkpoint": "false" }

      ,
      ],
      "dest_urls": [

      { "site_label": "local", "url": "file://$PWD/113494/H-H1_HOFT_C02-1134940160-4096.gwf" }

      ] }

      This is a blocker for us using OSG.

            Assignee:
            vahi Karan Vahi
            Reporter:
            dbrown Duncan Brown
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: