Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-694

pegasus lite does not handle jobs where user specifies a stdin file

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master, 4.2
    • Fix Version/s: master, 4.2.1, 4.3
    • Component/s: Pegasus Planner
    • Labels:
      None

      Description

      For the imputation workflows in my dax i have a job

       <job id="ID0000001" name="update_map">
                      <argument>--vanilla --args mec-pilot-2010-11-19.map mec-pilot-2010-11-19.build37.map mec-pilot-2010-11-19.rsIDs.txt</argument>
                      <stdin name="MakeMaps.R" link="input"/>
                      <uses name="mec-pilot-2010-11-19.build37.map" link="output"/>
                      <uses name="MakeMaps.R" link="input"/>
                      <uses name="mec-pilot-2010-11-19.map" link="input"/>
                      <uses name="mec-pilot-2010-11-19.rsIDs.txt" link="output"/>
                      <uses name="metabochip_rsids_2011-12-21.txt" link="input"/>
              </job>

      The MakeMaps.R is a user provided input file that is the stdin for the job

      When we plan this job in Pegasus Lite mode we see

      input = /lfs1/work/page/dags/vahi/pegasus/imputation-mec-pilot/run0007/MakeMaps.R
      transfer_input_files = MakeMaps.R,mec-pilot-2010-11-19.map,metabochip_rsids_2011-12-21.txt,/lfs1/software/install/pegasus/pegasus-4.3.0cvs/share/pegasus/sh/pegasus-lite-common.sh

      In the PegasusLite mode, we ensure that the stdin file gets to the worker node directory. However, we don't unset the input condor key for the job. The job fails saying SUBMIT failed, since the file referred to by the input key is not present

       /lfs1/work/page/dags/vahi/pegasus/imputation-mec-pilot/run0007/MakeMaps.R does not exist.

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              vahi Karan Vahi
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: