file dependencies between sub workflow and compute jobs broken

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      The test is that we can use a file produced in a subworkflow as the input to a job. This worked previously, but seems to be leading to this planning error now.

      Planner invoked with following arguments --conf /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/pegasus.2602818085757992491.properties --dir /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l --relative-dir work/./sub1.dax_sub1 --relative-submit-dir work/././sub1.dax_sub1 --basename sub1 --sites condorpool_symlink,local --staging-site condorpool_symlink=local,local=local, --cache /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/./pegasus-plan_sub1.cache --inherited-rc-files /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/cont.dax-0.replica.store --cluster label,horizontal --output-map /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/pycbc-tmp_rvc_2r3l/work/./pegasus-plan_sub1.output.map --output-sites local --cleanup inplace --verbose --verbose --verbose --deferred /home/runner/work/pycbc/pycbc/examples/workflow/generic/simple_subworkflow_data/sub1.dax

      ..
      2023.02.20 19:17:20.206 UTC: [CONFIG] Transfer Refiner loaded is [Balanced Cluster Transfer Refiner( round robin distribution at file level)]
      2023.02.20 19:17:20.206 UTC: [CONFIG] ReplicaSelector loaded is [Regex]
      2023.02.20 19:17:20.207 UTC: [CONFIG] Submit Directory Mapper loaded is [Relative Submit Directory Mapper]
      2023.02.20 19:17:20.207 UTC: [CONFIG] Staging Mapper loaded is [Flat Directory Staging Mapper]
      2023.02.20 19:17:20.209 UTC: [DEBUG] SRM Server map is {}
      2023.02.20 19:17:20.209 UTC: [DEBUG] SRM Server map is {}
      2023.02.20 19:17:20.209 UTC: [DEBUG] Directory for job exe1_ID0_ID0000001 is exe1
      2023.02.20 19:17:20.209 UTC: [DEBUG]
      2023.02.20 19:17:20.209 UTC: [DEBUG] Job being traversed is exe1_ID0_ID0000001
      2023.02.20 19:17:20.209 UTC: [DEBUG] To be run at condorpool_symlink
      2023.02.20 19:17:20.209 UTC: [DEBUG] Parents of job:{}
      2023.02.20 19:17:20.211 UTC: [FATAL ERROR] edu.isi.pegasus.planner.mapper.MapperException: [Replica] Unable to retrive location from Mapper Replica Backend for lfn test_output.txt.2 for site local and operation put
      at edu.isi.pegasus.planner.mapper.output.Replica.mapAll(Replica.java:247)
      at edu.isi.pegasus.planner.transfer.generator.StageOut.constructFileTX(StageOut.java:431)
      at edu.isi.pegasus.planner.transfer.generator.StageOut.constructFileTX(StageOut.java:298)
      at edu.isi.pegasus.planner.refiner.TransferEngine.addTransferNodes(TransferEngine.java:284)
      at edu.isi.pegasus.planner.refiner.MainEngine.r

            Assignee:
            Karan Vahi
            Reporter:
            Ian Harry
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: