-
Type: Bug
-
Resolution: Won't Fix
-
Priority: Minor
-
Affects Version/s: master, 5.0.1
-
Component/s: Pegasus Planner
-
None
Intermediate outputs within a clustered job, which are not used outside of that cluster as inputs to other jobs and when stage_out=False for those intermediate outputs, are still being sent back to the staging site. If the expectation is that they shouldn't be sent back, then this is a bug.
An example of this is illustrated in cluster-staging-pegasus.tar.gz. See workflow.py for the example workflow, and pegasus/test-cluster/run0001/00/00/merge_cluster.sh to see what transfers would be done.