The default job priorities for Pegasus workflows, in which auxiliary jobs such as staging and cleanup jobs gets higher priority, and priority based on level of the workflow, seems to work great in most cases. However, for large workflows using maxidle or maxjobs throttling, the jobs are not submitted in any particular order. Higher priority jobs might be available inside DAGMan while lower priority ones are submitted to the queue.

I suggest that we mirror the job priority we put in the .sub file to also be set in the DAG. This way the same priority is used for submitting as well as scheduling the jobs.

For more details on the DAG node priority:
https://research.cs.wisc.edu/htcondor/manual/v8.4/2_10DAGMan_Applications.html#SECTION003108300000000000000