pegasus-transfer to checksum files

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      For integrity checking in Pegasus, we have cases where a user may not have checksums available for all inputs. In that case, we want to be able to compute the checksums for raw inputs as part of the workflow.

      Two Options are possible
      1) One solution is to add jobs to the workflow to compute checksums for these input files.
      2) pegasus-transfer while pulling or pushing the data computes the checksums.

      2) has the added advantage of trying to compute checksums when a file is pulled for the workflow. In case of 1) the separate jobs are decoupled from the transfer jobs and may always need to pull in files to compute the checksums

            Assignee:
            Mats Rynge
            Reporter:
            Karan Vahi
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: