Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-987

noop jobs created by Pegasus don't use DAGMan NOOP keyword

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master, 4.5.2
    • Fix Version/s: master, 4.6.0, 4.5.3
    • Component/s: pegasus-plan
    • Labels:
      None

      Description

      Hi Karan,

      I’m running into a problem on LIGO clusters which have a wrapper around condor_submit that requires that accounting_group must be specified. If Pegasus creates noop jobs, then they are missing this keyword, even though it is specified for all jobs on the local site:

       <site handle="local" arch="x86_64" os="LINUX”>

      ...

         <profile namespace="condor" key="accounting_group">ligo.dev.o1.cbc.bbh.pycbcoffline</profile>
       </site>


      This is a show-stopper for data re-use for us so please can you make a patch? Longer term, is there any reason why you submit /bin/true rather than use the NOOP keyword in the dag?

      Cheers,
      Duncan.

      ######################################################################
      # PEGASUS WMS GENERATED SUBMIT FILE
      # DAG : main, Index = 0, Count = 1
      # SUBMIT FILE NAME : noop_main_0.sub
      ######################################################################
      stream_error = false
      stream_output = false
      environment = "CONDOR_JOBID=$(cluster).$(process) PEGASUS_WF_UUID=e1dc331c-5093-462d-a9de-9fbd91a1dcfd PEGASUS_DAG_JOB_ID=noop_main_0 PEGASUS_SITE=local "
      concurrency_limits = pegasus_auxillary.createdir
      copy_to_spool = false
      error = /usr1/dbrown/pycbc-tmp.JaJXGnQ16J/work/./main_ID0000001.000/noop_main_0.err
      executable = /bin/true
      log = /usr1/dbrown/pycbc-tmp.JaJXGnQ16J/work/./main_ID0000001.000/main-0.log
      noop_job = true
      noop_job_exit_code = 0
      notification = NEVER
      output = /usr1/dbrown/pycbc-tmp.JaJXGnQ16J/work/./main_ID0000001.000/noop_main_0.out
      periodic_release = False
      periodic_remove = (JobStatus == 5) && ((CurrentTime - EnteredCurrentStatus) > 1800)
      priority = 800
      submit_event_user_notes = pool:local
      transfer_executable = false
      universe = local
      +pegasus_generator = "Pegasus"
      +pegasus_root_wf_uuid = "b2b984e7-12ef-4809-b7fd-6ee9b4b62697"
      +pegasus_wf_uuid = "e1dc331c-5093-462d-a9de-9fbd91a1dcfd"
      +pegasus_version = "4.5.2"
      +pegasus_wf_name = "main-0"
      +pegasus_wf_time = "20150924T090422-0400"
      +pegasus_wf_xformation = "pegasus::noop:1.0"
      +pegasus_wf_dax_job_id = "null"
      +pegasus_wf_dag_job_id = "noop_main_0"
      +pegasus_job_class = 6
      +pegasus_site = "local"
      +pegasus_job_runtime = 0
      +pegasus_cores = 1
      +pegasus_cluster_size = 1
      queue
      ######################################################################
      # END OF SUBMIT FILE
      ######################################################################

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              dbrown Duncan Brown
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: