Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-694

pegasus lite does not handle jobs where user specifies a stdin file


    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major Major
    • master, 4.2.1, 4.3
    • Affects Version/s: master, 4.2
    • Component/s: Pegasus Planner
    • None

      For the imputation workflows in my dax i have a job

      <job id="ID0000001" name="update_map">
      <argument>--vanilla --args mec-pilot-2010-11-19.map mec-pilot-2010-11-19.build37.map mec-pilot-2010-11-19.rsIDs.txt</argument>
      <stdin name="MakeMaps.R" link="input"/>
      <uses name="mec-pilot-2010-11-19.build37.map" link="output"/>
      <uses name="MakeMaps.R" link="input"/>
      <uses name="mec-pilot-2010-11-19.map" link="input"/>
      <uses name="mec-pilot-2010-11-19.rsIDs.txt" link="output"/>
      <uses name="metabochip_rsids_2011-12-21.txt" link="input"/>

      The MakeMaps.R is a user provided input file that is the stdin for the job

      When we plan this job in Pegasus Lite mode we see

      input = /lfs1/work/page/dags/vahi/pegasus/imputation-mec-pilot/run0007/MakeMaps.R
      transfer_input_files = MakeMaps.R,mec-pilot-2010-11-19.map,metabochip_rsids_2011-12-21.txt,/lfs1/software/install/pegasus/pegasus-4.3.0cvs/share/pegasus/sh/pegasus-lite-common.sh

      In the PegasusLite mode, we ensure that the stdin file gets to the worker node directory. However, we don't unset the input condor key for the job. The job fails saying SUBMIT failed, since the file referred to by the input key is not present

      /lfs1/work/page/dags/vahi/pegasus/imputation-mec-pilot/run0007/MakeMaps.R does not exist.

            vahi Karan Vahi
            vahi Karan Vahi
            1 Start watching this issue