-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major
-
Affects Version/s: master
-
Component/s: Workflow API Libraries
-
None
It could be useful to add a workflow into an existing workflow, without having to create a SubWorkflow job, and have a separate workflow yaml file.
For example, Ian Harry (LIGO/PyCBC), described the following scenario:
I have some root workflow A, and it has a SubWorkflow job B. Everything in B is known before running A. That is, there is nothing dynamic about the hierarchical workflow. The workflow is separated for organizational purposes (no 'generate_workflow' type job is run before the SubWorkflow job). Instead of first writing the workflow_b.yml file, and adding it to the SubWorkflow job, it would easier (and have less files) to just do something along the lines of this:
w1 = Workflow()
w.add_jobs(one)
w2 = Workflow()
w2.add_jobs(two, three)
w1.add_jobs( w2 )
w1.add_jobs( four )
- w1 should serialize w2 and internally manage it as w1.add_subwf( serialized(w2) ), and
- four should auto depend on w2 if four uses output of three or four
notes: https://docs.google.com/document/d/1e6rfkJDiXN3eKhSma7V_PmihfsgcaK9hisfOtPK6KrY/edit