planner support for recursive clustering

XMLWordPrintable

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major
    • master, 4.5.0, 4.4.2
    • Affects Version/s: master, 4.4.1
    • Component/s: Pegasus Planner
    • None

      Hi Karan,

      I have recently been investigating adding the pegasus clustering
      ability to our workflows. I took a look through the detailed and
      useful documentation page here:

      http://pegasus.isi.edu/wms/docs/latest/job_clustering.php

      I think that both "horizontal" and "label" clustering are things that
      we could make use of in our workflows.

      I have been able to make use of both individually (ie. creating a
      workflow that does only horizontal clustering or only label
      clustering). However, there is a specific section of the workflow
      where I would like to be able to first apply label clustering and then
      apply horizontal clustering.

      Trying to do this gives me strange error messages. Rather than put
      error messages in an email I created an example here:

      /home/spxiwh/testing/test_pegasus_cluster/1102089616-1102863616

      on sugar. You can run the failing planning command by doing:

      cd /home/spxiwh/testing/test_pegasus_cluster/1102089616-1102863616
      ./pycbc_basic_pegasus_plan main.dax /usr1/${USER}/log

      Note that the planning command works if I only use:

      --cluster horizontal

      or

      --cluster label

      but fails, with different errors, with either

      --cluster horizontal,label

      or

      --cluster label,horizontal

      What am I doing wrong here?

      Cheers
      Ian

            Assignee:
            Karan Vahi
            Reporter:
            Duncan Brown
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: