Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-1347

pegasus will always try and transfer output when a code has checkpointed

XMLWordPrintable

      The job wrapper script created by pegasus-plan always tries to move the output files of the job (and will fail if they don't exist). This seems inconsistent with checkpointing a job, where the output file(s) might not exist until the job has actually finished.

      It should at least be documented that the output must be written in the case that checkpointing is used, but I would claim that it's better to allow the user to set a properties that says "only expect output if the job has not been killed by a checkpoint."

      echo -e "\n###################[Pegasus Lite] Staging out checkpoint files ###################" 1>&2

      1. stage out checkpoint files
        set +e
        pegasus-transfer --threads 1 1>&2 << 'EOF'
        [
        Unknown macro: { "type"}

        ]
        EOF
        ec=$?
        set -e
        if [ $ec -ne 0 ]; then
        echo " Ignoring failure while transferring chkpoint files. Exicode was $ec" 1>&2
        fi

      echo -e "\n#####################[Pegasus Lite] Staging out output files #####################" 1>&2

      1. stage out
        pegasus-transfer --threads 1 1>&2 << 'EOF'
        [
        Unknown macro: { "type"}

        ]
        EOF

            Assignee:
            vahi Karan Vahi
            Reporter:
            dbrown Duncan Brown
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: