intersite transfers for large workflows broken

XMLWordPrintable

      for large workflows where the parent job of a compute job gets assigned a different relative directory (under the hierarchical naming scheme), the inter site transfer job created has an incorrect source and destination url. the source url ends up having the same relative dir prefix as the destination one

       

      /scitech/shared/projects/XENONnT/workflows-to-keep
wimp_ac_optimization-discovery_power-202409191126.analyzer.txt
wimp_ac_optimization-discovery_power-202409191126.tar.gz
 

       

       

      00/27/combine_ID0000613.sh: { "site_label": "staging-davs", "url": "gsidavs://xenon-gridftp.grid.uchicago.edu:2880/xenon/scratch/xudc/runs/00/23/wimp_ac_optimization-discovery_power-202409191126-6-combined_output.tar.gz" } Then later: 00/27/stage_inter_local_separate_ID0000614_0.out.000: 2024-09-19 14:37:09,089 INFO: gfal-copy -f -p -t 7200 -T 7200 'davs://xenon-gridftp.grid.uchicago.edu:2880/xenon/scratch/xudc/runs/00/24/wimp_ac_optimization-discovery_power-202409191126-6-combined_output.tar.gz' 'file:////scratch/xudc/workflows/wimp_ac_optimization-discovery_power-202409191126/scratch/runs/00/24/wimp_ac_optimization-discovery_power-202409191126-6-combined_output.tar.gz'

            Assignee:
            Karan Vahi
            Reporter:
            Mats Rynge
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: