dump of braindump as json object when pegasus-plan is invoked with --json option is confusing

XMLWordPrintable

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major
    • master, 5.1.0
    • Affects Version/s: master, 5.0.6
    • Component/s: CLI: pegasus-plan
    • None

      as part of PM-1474 when pegasus-plan is invoked with --json option, all the logging of the planner goes to stderr ; while on stdout the contents of the braindump file get dumped as a json object

      this --json option is also passed when pegasus-plan is invoked from the python workflow api plan function.

      however, it seems in the default case the interleaving of stdout and stderr can be confusing to the users

      For example
      2024.01.31 15:53:10.344 CST: -----------------------------------------------------------------------
      2024.01.31 15:53:11.310 CST: Database version: '5.0.6dev' (sqlite:////home/rynge/.pegasus/workflow.db)
      2024.01.31 15:53:12.336 CST: Pegasus database was successfully created.
      2024.01.31 15:53:12.342 CST: Database version: '5.0.6dev' (sqlite:////home/rynge/workflows/runs/rynge/pegasus/wordfreq-workflow/run0002/wordfreq-workflow-0.replicas.db)
      2024.01.31 15:53:12.382 CST: Output replica catalog set to jdbc:sqlite:/home/rynge/workflows/runs/rynge/pegasus/wordfreq-workflow/run0002/wordfreq-workflow-0.replicas.db
      [WARNING] Submitting to condor wordfreq-workflow-0.dag.condor.sub

      {"user": "rynge", "grid_dn": null, "submit_hostname": "ap20.uc.osg-htc.org", "root_wf_uuid": "86ee470b-fa51-4d56-8339-3ccf8bd1e186", "wf_uuid": "86ee470b-fa51-4d56-8339-3ccf8bd1e186", "dax": "/home/rynge/git/tutorial-pegasus/wordfreq/workflow.yml", "dax_label": "wordfreq-workflow", "dax_index": "0", "dax_version": "5.0.4", "pegasus_wf_name": "wordfreq-workflow-0", "timestamp": "20240131T155309-0600", "basedir": "/home/rynge/workflows/runs", "submit_dir": "/home/rynge/workflows/runs/rynge/pegasus/wordfreq-workflow/run0002", "planner": "/usr/bin/pegasus-plan", "planner_version": "5.0.6dev", "planner_arguments": "\"--cluster horizontal --sites condorpool --output-sites local --staging-site condorpool=osdf --dir /home/rynge/workflows/runs --cleanup inplace --submit workflow.yml --json \"", "pegasus_build": "20230609121024Z", "jsd": "jobstate.log", "rundir": "run0002", "bindir": "/usr/bin", "vogroup": "pegasus", "uses_pmc": true, "properties": "pegasus.394061488026317849.properties", "condor_log": "wordfreq-workflow-0.log", "dag": "wordfreq-workflow-0.dag", "type": "dag", "notify": "wordfreq-workflow-0.notify", "script": null}

      2024.01.31 15:53:13.315 CST: Time taken to execute is 3.662 seconds

      Your workflow has been started and is running in the base directory:

      /home/rynge/workflows/runs/rynge/pegasus/wordfreq-workflow/run0002

          • To monitor the workflow you can run ***

      pegasus-status -l /home/rynge/workflows/runs/rynge/pegasus/wordfreq-workflow/run0002

            Assignee:
            Karan Vahi
            Reporter:
            Karan Vahi
            Watchers:
            1 Start watching this issue

              Created:
              Updated: