pegasus lite does not handle jobs where user specifies a stdin file

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major
    • master, 4.2.1, 4.3
    • Affects Version/s: master, 4.2
    • Component/s: Pegasus Planner
    • None

      For the imputation workflows in my dax i have a job

      <job id="ID0000001" name="update_map">
      <argument>--vanilla --args mec-pilot-2010-11-19.map mec-pilot-2010-11-19.build37.map mec-pilot-2010-11-19.rsIDs.txt</argument>
      <stdin name="MakeMaps.R" link="input"/>
      <uses name="mec-pilot-2010-11-19.build37.map" link="output"/>
      <uses name="MakeMaps.R" link="input"/>
      <uses name="mec-pilot-2010-11-19.map" link="input"/>
      <uses name="mec-pilot-2010-11-19.rsIDs.txt" link="output"/>
      <uses name="metabochip_rsids_2011-12-21.txt" link="input"/>
      </job>

      The MakeMaps.R is a user provided input file that is the stdin for the job

      When we plan this job in Pegasus Lite mode we see

      input = /lfs1/work/page/dags/vahi/pegasus/imputation-mec-pilot/run0007/MakeMaps.R
      transfer_input_files = MakeMaps.R,mec-pilot-2010-11-19.map,metabochip_rsids_2011-12-21.txt,/lfs1/software/install/pegasus/pegasus-4.3.0cvs/share/pegasus/sh/pegasus-lite-common.sh

      In the PegasusLite mode, we ensure that the stdin file gets to the worker node directory. However, we don't unset the input condor key for the job. The job fails saying SUBMIT failed, since the file referred to by the input key is not present

      /lfs1/work/page/dags/vahi/pegasus/imputation-mec-pilot/run0007/MakeMaps.R does not exist.

            Assignee:
            Karan Vahi
            Reporter:
            Karan Vahi
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: