pegasus will always try and transfer output when a code has checkpointed

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      The job wrapper script created by pegasus-plan always tries to move the output files of the job (and will fail if they don't exist). This seems inconsistent with checkpointing a job, where the output file(s) might not exist until the job has actually finished.

      It should at least be documented that the output must be written in the case that checkpointing is used, but I would claim that it's better to allow the user to set a properties that says "only expect output if the job has not been killed by a checkpoint."

      echo -e "\n###################[Pegasus Lite] Staging out checkpoint files ###################" 1>&2

      1. stage out checkpoint files
        set +e
        pegasus-transfer --threads 1 1>&2 << 'EOF'
        [
        Unknown macro: { "type"}

        ]
        EOF
        ec=$?
        set -e
        if [ $ec -ne 0 ]; then
        echo " Ignoring failure while transferring chkpoint files. Exicode was $ec" 1>&2
        fi

      echo -e "\n#####################[Pegasus Lite] Staging out output files #####################" 1>&2

      1. stage out
        pegasus-transfer --threads 1 1>&2 << 'EOF'
        [
        Unknown macro: { "type"}

        ]
        EOF

            Assignee:
            Karan Vahi
            Reporter:
            Duncan Brown
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: