Pegasus should automatically add data flow dependencies

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Type: New Feature
    • Resolution: Fixed
    • Priority: Major
    • master, 4.4.0
    • Affects Version/s: master
    • Component/s: Pegasus Planner
    • None

      AT the moment, the user needs to specify all the data flow dependencies in the DAX explicitly as edges via parent child relationships. there should be an option in the planner to automatically add dependencies on the basis of data dependencies implied by input and output files for jobs.

      For example if Job A creates an output file X and job B consumes it, then the planner should automatically add a dependency between A -> B if it does not exist already.

            Assignee:
            Karan Vahi
            Reporter:
            Karan Vahi
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: