intersite transfers for large workflows broken

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      for large workflows where the parent job of a compute job gets assigned a different relative directory (under the hierarchical naming scheme), the inter site transfer job created has an incorrect source and destination url. the source url ends up having the same relative dir prefix as the destination one

       

      /scitech/shared/projects/XENONnT/workflows-to-keep
      wimp_ac_optimization-discovery_power-202409191126.analyzer.txt
      wimp_ac_optimization-discovery_power-202409191126.tar.gz
       
      

       

       

      00/27/combine_ID0000613.sh: { "site_label": "staging-davs", "url": "gsidavs://xenon-gridftp.grid.uchicago.edu:2880/xenon/scratch/xudc/runs/00/23/wimp_ac_optimization-discovery_power-202409191126-6-combined_output.tar.gz" } Then later: 00/27/stage_inter_local_separate_ID0000614_0.out.000: 2024-09-19 14:37:09,089 INFO: gfal-copy -f -p -t 7200 -T 7200 'davs://xenon-gridftp.grid.uchicago.edu:2880/xenon/scratch/xudc/runs/00/24/wimp_ac_optimization-discovery_power-202409191126-6-combined_output.tar.gz' 'file:////scratch/xudc/workflows/wimp_ac_optimization-discovery_power-202409191126/scratch/runs/00/24/wimp_ac_optimization-discovery_power-202409191126-6-combined_output.tar.gz'

            Assignee:
            Karan Vahi
            Reporter:
            Mats Rynge
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: