Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1014

Support for Fallback PFN while transferring raw input files

    XMLWordPrintable

    Details

      Description

      Original Email from Brian


      On Dec 1, 2015, at 12:03 PM, Brian Bockelman <bbockelm@cse.unl.edu> wrote:

      Hi,

      (I think I’ve got the right CC list; add as necessary)

      Right now, running LIGO PyCBC workflows have two different site names, “OSG” and “Stampede”. Both are accessed via GlideinWMS, so they appear as vanilla universe jobs in the same schedd.

      We’d like to merge this to a single “site” in the Pegasus site catalog in order to allow HTCondor to dynamically load-balance a workload across both resources. Since CVMFS is available at both locations, the only difference between the two sites is the input method.

      Would it be possible to specify a list of input methods for an entry in the site catalog? I.e., specify:

      1) Input from /cvmfs/oasis.opensciencegrid.org/ligo/frames via POSIX. If file not found or other error, then
      2) Input from gsiftp://red-gridftp.unl.edu//user/ligo/frames

      ? Would such a thing be a simple or complex change?

      This would allow us to use the local filesystem at Stampede (or OSG-LIGO sites like Nebraska, Syracuse, and GATech) but GridFTP otherwise.

      Thanks!

      Brian

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              dbrown Duncan Brown
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: