noop jobs created by Pegasus don't use DAGMan NOOP keyword

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major
    • master, 4.6.0, 4.5.3
    • Affects Version/s: master, 4.5.2
    • Component/s: Pegasus Planner
    • None

      Hi Karan,

      I’m running into a problem on LIGO clusters which have a wrapper around condor_submit that requires that accounting_group must be specified. If Pegasus creates noop jobs, then they are missing this keyword, even though it is specified for all jobs on the local site:

      <site handle="local" arch="x86_64" os="LINUX”>

      ...

      <profile namespace="condor" key="accounting_group">ligo.dev.o1.cbc.bbh.pycbcoffline</profile>
      </site>

      This is a show-stopper for data re-use for us so please can you make a patch? Longer term, is there any reason why you submit /bin/true rather than use the NOOP keyword in the dag?

      Cheers,
      Duncan.

      ######################################################################

      1. PEGASUS WMS GENERATED SUBMIT FILE
      2. DAG : main, Index = 0, Count = 1
      3. SUBMIT FILE NAME : noop_main_0.sub
        ######################################################################
        stream_error = false
        stream_output = false
        environment = "CONDOR_JOBID=$(cluster).$(process) PEGASUS_WF_UUID=e1dc331c-5093-462d-a9de-9fbd91a1dcfd PEGASUS_DAG_JOB_ID=noop_main_0 PEGASUS_SITE=local "
        concurrency_limits = pegasus_auxillary.createdir
        copy_to_spool = false
        error = /usr1/dbrown/pycbc-tmp.JaJXGnQ16J/work/./main_ID0000001.000/noop_main_0.err
        executable = /bin/true
        log = /usr1/dbrown/pycbc-tmp.JaJXGnQ16J/work/./main_ID0000001.000/main-0.log
        noop_job = true
        noop_job_exit_code = 0
        notification = NEVER
        output = /usr1/dbrown/pycbc-tmp.JaJXGnQ16J/work/./main_ID0000001.000/noop_main_0.out
        periodic_release = False
        periodic_remove = (JobStatus == 5) && ((CurrentTime - EnteredCurrentStatus) > 1800)
        priority = 800
        submit_event_user_notes = pool:local
        transfer_executable = false
        universe = local
        +pegasus_generator = "Pegasus"
        +pegasus_root_wf_uuid = "b2b984e7-12ef-4809-b7fd-6ee9b4b62697"
        +pegasus_wf_uuid = "e1dc331c-5093-462d-a9de-9fbd91a1dcfd"
        +pegasus_version = "4.5.2"
        +pegasus_wf_name = "main-0"
        +pegasus_wf_time = "20150924T090422-0400"
        +pegasus_wf_xformation = "pegasus::noop:1.0"
        +pegasus_wf_dax_job_id = "null"
        +pegasus_wf_dag_job_id = "noop_main_0"
        +pegasus_job_class = 6
        +pegasus_site = "local"
        +pegasus_job_runtime = 0
        +pegasus_cores = 1
        +pegasus_cluster_size = 1
        queue
        ######################################################################
      4. END OF SUBMIT FILE
        ######################################################################

            Assignee:
            Karan Vahi
            Reporter:
            Duncan Brown
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: