Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1692

Add any missing options from pegasus-plan cli to Workflow.plan() and Client.plan()

    XMLWordPrintable

    Details

      Description

      All options that should be accessible to the python api's Workflow.plan() from the pegasus-plan CLI should be added. Currently these are the following options:

       Options
       -b |--basename the basename prefix while constructing the per workflow files like .dag etc.
       -c |--cache comma separated list of replica cache files.
       --inherited-rc-files comma separated list of replica files. Locations mentioned in these have a lower priority
                             than the locations in the DAX file
       --cleanup the cleanup strategy to use. Can be none|inplace|leaf|constraint. Defaults to inplace.
       -C |--cluster comma separated list of clustering techniques to be applied to the workflow to
                             to cluster jobs in to larger jobs, to avoid scheduling overheads.
       --conf the path to the properties file to use for planning. Defaults to pegasus.properties file
                             in the current working directory
       --dir the directory where to generate the executable workflow.
       --relative-dir the relative directory to the base directory where to generate the concrete workflow.
       --relative-submit-dir the relative submit directory where to generate the concrete workflow. Overrides --relative-dir .
       -f |--force skip reduction of the workflow, resulting in build style dag.
       --force-replan force replanning for sub workflows in case of failure.
       -F |--forward any options that need to be passed ahead to pegasus-run in format option[=value]
                             where value can be optional. e.g -F nogrid will result in --nogrid . The option
                             can be repeated multiple times.
       -j |--job-prefix the prefix to be applied while construction job submit filenames
       -J |--json boolean option to log the final planner output in json format to the stdout.
                             All other log messages get logged to the stderr.
       -I |--input-dir comma separated list of optional input directories where the input files reside on submit host
       -O |--output-dir an optional output directory where the output files should be transferred to on submit host.
                             the directory specified is asscociated with the local-storage directory for the output site.
       -o |--output-sites comma separated list of output sites where the data products during workflow execution are
                             transferred to.
       -s |--sites comma separated list of executions sites on which to map the workflow.
       --staging-site comma separated list of key=value pairs , where the key is the execution site and value is the
                             staging site for that execution site.
       -r |--randomdir create random directories based on workflow label and the workflow uuid (listed in the
                             braindump file) on remote staging sites where data transfer jobs for the workflow are executed.
                             If the basename option is set, then instead of the workflow label, the basename is used for
                             generating the random directory name along with the workflow uuid.
       -R |--reuse comma separated list of submit directories of previous runs from which to pick up
                             output replica catalogs for data reuse.
       -S |--submit submit the executable workflow generated
       --staging-site comma separated list of key=value pairs, where key is the execution site and value is the
                             staging site
       -v |--verbose increases the verbosity of messages about what is going on
       -q |--quiet decreases the verbosity of messages about what is going on
       -V |--version displays the version of the Pegasus Workflow Management System
       -X[non standard java option] pass to jvm a non standard option . e.g. -Xmx1024m -Xms512m
       -h |--help generates this help.
       [abstract-workflow] the YAML input file that describes an abstract workflow. If not specified
                             the planner defaults to file *workflow.yml* in the current working directory.

      Additionally, these options need to be added to Client.plan().

        Attachments

          Activity

            People

            • Assignee:
              tanaka Ryan Tanaka
              Reporter:
              tanaka Ryan Tanaka
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: