allow for direct use of singularity containers on CVMFS on compute site

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Major
    • master, 4.9.0
    • Affects Version/s: master
    • Component/s: Pegasus Planner
    • None

      Hi Karan,

      I've been experimenting with the current singularity implementation in 4.8.4 and hit the following problem. I want to use Brian's images from /cvmfs/singularity.opensciencegrid.org/pycbc/pycbc-el7:latest as they are pre-staged to all of our nodes, but I get the error

      2018.10.12 09:06:43.922 EDT: [FATAL ERROR]
      [1]: Unable to instantiate Transformation Catalog at edu.isi.pegasus.planner.catalog.transformation.TransformationFactory.loadInstance(TransformationFactory.java:172)
      [2]: edu.isi.pegasus.planner.catalog.transformation.impl.Text caught edu.isi.pegasus.planner.parser.ScannerException line 28: at edu.isi.pegasus.planner.parser.TransformationCatalogTextParser.parse(TransformationCatalogTextParser.java:236)
      [3]: Invalid suffix .opensciencegrid.org/pycbc/pycbc-el7:latest determined singularity image url url -> file:///cvmfs/singularity.opensciencegrid.org/pycbc/pycbc-el7:latest , protocol -> file , host -> , path -> /cvmfs/singularity.opensciencegrid.org/pycbc/pycbc-el7:latest , url-prefix -> file:// at edu.isi.pegasus.planner.catalog.transformation.classes.Container.computeLFN(Container.java:180)

      It looks like Pegasus doesn't like the file URL in my transformation catalog:

      tr pegasus::findrange:4.0 {

      site condorpool

      { pfn "gsiftp://sugwg-condor.phy.syr.edu/usr/bin/pegasus-keg" arch "x86_64" os "linux" # INSTALLED means pfn refers to path in the container. # STAGEABLE means the executable can be staged into the container type "STAGEABLE" #optional attribute to specify the container to use container "centos-pycbc" }

      }

      cont centos-pycbc

      { # can be either docker or singularity type "singularity" # URL to image in a docker|singularity hub OR # URL to an existing docker image exported as a tar file or singularity image image "file:///cvmfs/singularity.opensciencegrid.org/pycbc/pycbc-el7:latest" # optional site attribute to tell pegasus which site tar file # exists. useful for handling file URL's correctly image_site "pycbc" }

      Cheers,
      Duncan.

            Assignee:
            Karan Vahi
            Reporter:
            Duncan Brown
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: