Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1226

hierarchal workflow with worker package staging fails

    XMLWordPrintable

Details

    Description

      If I have a workflow, with only DAX jobs and worker package staging enabled in the properties ( pegasus.transfer.worker.package=true) for the top level workflow, the top level workflow fails.

      The constructed stage_worker* job is incorrect as it refers in it's submit file as the executable, the location it is staging the pegasus worker package to!!!

      [bamboo@colo-vm63 045-hierarchy-sharedfs]$ pegasus-analyzer work/bamboo/pegasus/local-hierarchy/run0004

      ************************************Summary*************************************

       Submit Directory : work/bamboo/pegasus/local-hierarchy/run0004
       Total jobs : 6 (100.00%)
       # jobs succeeded : 1 (16.67%)
       # jobs failed : 1 (16.67%)
       # jobs held : 0 (0.00%)
       # jobs unsubmitted : 4 (66.67%)

      ******************************Failed jobs' details******************************

      ===================stage_worker_local_local-hierarchy_0_local===================

       last state: POST_SCRIPT_FAILED
             site: local
      submit file: stage_worker_local_local-hierarchy_0_local.sub
      output file: 00/00/stage_worker_local_local-hierarchy_0_local.out.001
       error file: 00/00/stage_worker_local_local-hierarchy_0_local.err.001

      -------------------------------Task #1 - Summary--------------------------------

      site : local
      hostname : -
      executable : /localhome/bamboo/045-hierarchy-sharedfs/work/local-site/scratch/bamboo/pegasus/local-hierarchy/run0004/pegasus-4.8.1dev/bin/pegasus-transfer
      arguments : --threads 2
      exitcode : 127
      working dir : /localhome/bamboo/045-hierarchy-sharedfs/work/bamboo/pegasus/local-hierarchy/run0004

      ---Job stderr file - 00/00/stage_worker_local_local-hierarchy_0_local.err.001---

      /usr/sbin/htcondor-user-job-wrapper: line 16: /localhome/bamboo/045-hierarchy-sharedfs/work/local-site/scratch/bamboo/pegasus/local-hierarchy/run0004/pegasus-4.8.1dev/bin/pegasus-transfer: No such file or directory

      Attachments

        Activity

          People

            vahi Karan Vahi
            vahi Karan Vahi
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: