Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1179 Integrity checking in Pegasus
  3. PM-1257

propagate checksums in hierarchal workflows

    XMLWordPrintable

Details

    Description

      In hierarchal workflows, we can have data dependencies between sub workflows i.e a file maybe created by a DAX job A that might be needed when the sub workflow corresponding to child DAX job B executes.

      Usually, we have handled this using cache files that the planner writes out in the submit directory, when a workflow is planned. However, this cache file does not contain any checksums of files that maybe created as the workflow is run.

      These checksums should also be propagated across workflows similar to cache files.

      Attachments

        Activity

          People

            vahi Karan Vahi
            vahi Karan Vahi
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: