Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1916

data management between parent compute job and a sub workflow job broken

XMLWordPrintable

      As part of PM-1766 the data dependency between a sub workflow and a parent compute job was supported.

      x-pegasus:

      {apiLang: python, createdBy: bamboo, createdOn: '05-31-23T15:26:25Z'}

      pegasus: 5.0.4
      name: local-hierarchy
      jobs:

      • type: job
        namespace: diamond
        version: '4.0'
        name: pre-preprocess
        id: ID0000001
        arguments: [-a, post-analyze, -T, '60', -i, f.input, -o, f.a]
        uses:
      • {lfn: f.a, type: output, stageOut: true, registerReplica: true}
      • {lfn: f.input, type: input}
      • type: pegasusWorkflow
        file: blackdiamond.yml
        id: ID0000002
        arguments: [--input-dir, input, --output-sites, local, -vvv, --force]
        uses:
      • {lfn: f.d, type: output, stageOut: true, registerReplica: true}
      • {lfn: blackdiamond.yml, forPlanning: true, type: input}
      • {lfn: f.a, type: input}
      • type: job
        namespace: diamond
        version: '4.0'
        name: post-analyze
        id: ID0000003
        arguments: [-a, post-analyze, -T, '60', -i, f.d, -o, f.e]
        uses:
      • {lfn: f.d, type: input}
      • {lfn: f.e, type: output, stageOut: true, registerReplica: true}

        jobDependencies:

      • id: ID0000001
        children: [ID0000002]
      • id: ID0000002
        children: [ID0000003]

      For the above workflow, the planning for sub workflow job with ID0000002 fails, saying it cannot find file f.input that is generated by the parent job with ID0000001

            Assignee:
            vahi Karan Vahi
            Reporter:
            vahi Karan Vahi
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: