improve checkpointing to handle asynchronous saving of checkpoint files back to staging site

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major
    • master, 5.1.0
    • Affects Version/s: master, 5.1.0
    • Component/s: None
    • None

      Improve checkpointing s.t. checkpoint files can be sent back to the staging site asynchronously while a job is running.

      Notes: https://docs.google.com/document/d/1Z44IKt8aiFRNhfX7RguS_zx5VM9KQ0Fz/edit

        1.
        		 implement background checkpointing process Open
        2.
        		 add changes to the Python API to support adding of checkpoint files by pattern Open
        3.
        		 refactor pegasus-transfer so that it can be invoked directly from pegasus-checkpoint Resolved

            Assignee:
            Karan Vahi
            Reporter:
            Ryan Tanaka [X] (Inactive)
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Archived: