Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-746

Pegasus should automatically add data flow dependencies

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Reopened
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master
    • Fix Version/s: master, 4.4.0
    • Component/s: pegasus-plan
    • Labels:
      None

      Description

      AT the moment, the user needs to specify all the data flow dependencies in the DAX explicitly as edges via parent child relationships. there should be an option in the planner to automatically add dependencies on the basis of data dependencies implied by input and output files for jobs.

      For example if Job A creates an output file X and job B consumes it, then the planner should automatically add a dependency between A -> B if it does not exist already.

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              vahi Karan Vahi
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: