Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-603

Enable workflows to reference local input files directly instead of symlinking them

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master
    • Fix Version/s: master, 5.0.0
    • Component/s: Pegasus Planner
    • Labels:
      None

      Description

      Add an option to Pegasus to allow users to specify that their pre-staged input files should be access directly if they are on a shared file system accessible to the compute node.

      Right now Pegasus offers an optimization where local input files are symlinked into the working directory instead of being copied. The goal of this feature is to extend the symlinking functionality so that Pegasus generates an argument string with the full path to the input file instead of generating a symlink.

      So, if a job had these arguments:

      <arguments>-i <file name="input.txt"></arguments>

      And input.txt was pre-staged to /mnt/scratch/user/input.txt, the resulting arguments would be:

      -i /mnt/scratch/user/input.txt

      Instead of:

      -i input.txt

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              gideon Gideon Juve (Inactive)
            • Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: