Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1136

With bypass input staging some URLs are ending up in the wrong site

    XMLWordPrintable

Details

    Description

      I have LFNs in my DAX that have PFN entries for several sites, e.g.

      <file name="113449/H-H1_HOFT_C02-1134497792-4096.gwf">
      <pfn url="file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134497792-4096.gwf" site="osg"/>
      <pfn url="file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134497792-4096.gwf" site="local"/>
      <pfn url="root://xrootd-local.unl.edu/user/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134497792-4096.gwf" site="osg"/>
      </file>

      When I plan with bypass input staging turned on, the osg root URL turns up in my transfer job for executables on the local site, even though the root UWL is tagged with the osg pool attribute:

       { "type": "transfer",
         "id": 1,
         "src_urls": [
           { "site_label": "local", "url": "file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134940160-4096.gwf", "priority": 400, "checkpoint": "false" },
           { "site_label": "local", "url": "root://xrootd-local.unl.edu/user/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134940160-4096.gwf", "priority": 200, "checkpoint": "false" }
         ],
         "dest_urls": [
           { "site_label": "local", "url": "file://$PWD/113494/H-H1_HOFT_C02-1134940160-4096.gwf" }
         ] }

      This isn't a big deal for 4.7.1 as the priorities take care of making sure that I get the file URL that is the right one for the local site, but it would be good to fix this in 4.7.2.

      Attachments

        Activity

          People

            vahi Karan Vahi
            dbrown Duncan Brown
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: