Data Reuse with Bundle Refiner

XMLWordPrintable

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major
    • 2.4.3, 2.4.4, master, 3.0
    • Affects Version/s: 2.4.1, master, 3.0
    • Component/s: None
    • None

      In certain cases where data reuse happens with bundle transfer refiner, multiple stage out jobs are created with the same name. This results in the submit files being overwritten and also dagman complains saying the job already exists.

      Internally the bundle refiner relies on the level of the compute jobs to determine the stage out job to create.

      For the jobs that are deleted during data reuse level of 0 is assigned to them. The bug is triggered when a stage out job is created related to a top level undeleted compute job, and other stage out jobs are created for other levels > 0.

      Thus when the stage out jobs for the deleted jobs are added, they overwrite the stage out job for the higher level.

            Assignee:
            Karan Vahi
            Reporter:
            Karan Vahi
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: