Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1136

With bypass input staging some URLs are ending up in the wrong site

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 4.8.0
    • Fix Version/s: master, 4.8.0, 4.7.3
    • Component/s: pegasus-plan
    • Labels:
      None

      Description

      I have LFNs in my DAX that have PFN entries for several sites, e.g.

      <file name="113449/H-H1_HOFT_C02-1134497792-4096.gwf">
      <pfn url="file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134497792-4096.gwf" site="osg"/>
      <pfn url="file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134497792-4096.gwf" site="local"/>
      <pfn url="root://xrootd-local.unl.edu/user/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134497792-4096.gwf" site="osg"/>
      </file>

      When I plan with bypass input staging turned on, the osg root URL turns up in my transfer job for executables on the local site, even though the root UWL is tagged with the osg pool attribute:

       { "type": "transfer",
         "id": 1,
         "src_urls": [
           { "site_label": "local", "url": "file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134940160-4096.gwf", "priority": 400, "checkpoint": "false" },
           { "site_label": "local", "url": "root://xrootd-local.unl.edu/user/ligo/frames/O1/H1_HOFT_C02/H/1134/H-H1_HOFT_C02-1134940160-4096.gwf", "priority": 200, "checkpoint": "false" }
         ],
         "dest_urls": [
           { "site_label": "local", "url": "file://$PWD/113494/H-H1_HOFT_C02-1134940160-4096.gwf" }
         ] }

      This isn't a big deal for 4.7.1 as the priorities take care of making sure that I get the file URL that is the right one for the local site, but it would be good to fix this in 4.7.2.

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              dbrown Duncan Brown
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: