-
Type: Sub-task
-
Resolution: Fixed
-
Priority: Major
-
Affects Version/s: master
-
Component/s: CLI: pegasus-transfer, Integrity Checking, Planner: Transfer Module
-
None
For integrity checking in Pegasus, we have cases where a user may not have checksums available for all inputs. In that case, we want to be able to compute the checksums for raw inputs as part of the workflow.
Two Options are possible
1) One solution is to add jobs to the workflow to compute checksums for these input files.
2) pegasus-transfer while pulling or pushing the data computes the checksums.
2) has the added advantage of trying to compute checksums when a file is pulled for the workflow. In case of 1) the separate jobs are decoupled from the transfer jobs and may always need to pull in files to compute the checksums