Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-603

Enable workflows to reference local input files directly instead of symlinking them


    • Type: Icon: New Feature New Feature
    • Resolution: Fixed
    • Priority: Icon: Major Major
    • master, 5.0.0
    • Affects Version/s: master
    • Component/s: Pegasus Planner
    • None

      Add an option to Pegasus to allow users to specify that their pre-staged input files should be access directly if they are on a shared file system accessible to the compute node.

      Right now Pegasus offers an optimization where local input files are symlinked into the working directory instead of being copied. The goal of this feature is to extend the symlinking functionality so that Pegasus generates an argument string with the full path to the input file instead of generating a symlink.

      So, if a job had these arguments:

      <arguments>-i <file name="input.txt"></arguments>

      And input.txt was pre-staged to /mnt/scratch/user/input.txt, the resulting arguments would be:

      -i /mnt/scratch/user/input.txt

      Instead of:

      -i input.txt

            vahi Karan Vahi
            gideon Gideon Juve (Inactive)
            3 Start watching this issue