Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1905

file dependencies between sub workflow and compute jobs broken

XMLWordPrintable

      The test is that we can use a file produced in a subworkflow as the input to a job. This worked previously, but seems to be leading to this planning error now.

      Planner invoked with following arguments --conf /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/pegasus.2602818085757992491.properties --dir /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l --relative-dir work/./sub1.dax_sub1 --relative-submit-dir work/././sub1.dax_sub1 --basename sub1 --sites condorpool_symlink,local --staging-site condorpool_symlink=local,local=local, --cache /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/./pegasus-plan_sub1.cache --inherited-rc-files /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/cont.dax-0.replica.store --cluster label,horizontal --output-map /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/./pegasus-plan_sub1.output.map --output-sites local --cleanup inplace --verbose --verbose --verbose --deferred /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/sub1.dax

      ..
      2023.02.20 19:17:20.206 UTC: [CONFIG] Transfer Refiner loaded is [Balanced Cluster Transfer Refiner( round robin distribution at file level)]
      2023.02.20 19:17:20.206 UTC: [CONFIG] ReplicaSelector loaded is [Regex]
      2023.02.20 19:17:20.207 UTC: [CONFIG] Submit Directory Mapper loaded is [Relative Submit Directory Mapper]
      2023.02.20 19:17:20.207 UTC: [CONFIG] Staging Mapper loaded is [Flat Directory Staging Mapper]
      2023.02.20 19:17:20.209 UTC: [DEBUG] SRM Server map is {}
      2023.02.20 19:17:20.209 UTC: [DEBUG] SRM Server map is {}
      2023.02.20 19:17:20.209 UTC: [DEBUG] Directory for job exe1_ID0_ID0000001 is exe1
      2023.02.20 19:17:20.209 UTC: [DEBUG]
      2023.02.20 19:17:20.209 UTC: [DEBUG] Job being traversed is exe1_ID0_ID0000001
      2023.02.20 19:17:20.209 UTC: [DEBUG] To be run at condorpool_symlink
      2023.02.20 19:17:20.209 UTC: [DEBUG] Parents of job:{}
      2023.02.20 19:17:20.211 UTC: [FATAL ERROR] edu.isi.pegasus.planner.mapper.MapperException: [Replica] Unable to retrive location from Mapper Replica Backend for lfn test_output.txt.2 for site local and operation put
      at edu.isi.pegasus.planner.mapper.output.Replica.mapAll(Replica.java:247)
      at edu.isi.pegasus.planner.transfer.generator.StageOut.constructFileTX(StageOut.java:431)
      at edu.isi.pegasus.planner.transfer.generator.StageOut.constructFileTX(StageOut.java:298)
      at edu.isi.pegasus.planner.refiner.TransferEngine.addTransferNodes(TransferEngine.java:284)
      at edu.isi.pegasus.planner.refiner.MainEngine.r

            Assignee:
            vahi Karan Vahi
            Reporter:
            iwharry Ian Harry
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: