Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-641 Improve Monitord performance (size and time) for large workflows
  3. PM-646

improve the expunge operation in monitord in replay mode

    XMLWordPrintable

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • master
    • master, 4.1
    • Monitord
    • None

    Description

      In case of large SCEC MySQL databases, the expunge process ( deleting already populated data in the mysql db for a workflow ) takes a long time, resulting in a spike in the memory usage.

      This is because of the way expunge has been implemented in the netlogger SQL Alchemy code.
      It triggers individual select statements , the results of which are then invidually deleted.

      The Expunge code already does things specific to a database backend
      1) for sqlite it just overwrites

      Hence, for mysql it is possible to rely on database triggers in the schema definition to do cascaded deletes

      Attachments

        Activity

          People

            vahi Karan Vahi
            vahi Karan Vahi
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: