Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-916

identify and generate the BP events for metadata

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master
    • Fix Version/s: master, 4.6.0
    • Component/s: Monitord, pegasus-plan
    • Labels:
      None

      Description

      planner generates the following type of events to capture the metadata in the dax into the bp file

      static.meta.start - indicates start of metadata events
      wf.meta - captures metadata events for the workflow
      task.meta - captures metadata associated with the task in the DAX
      rc.meta - captures metadata about a file
      wf.map.fille - associates the workflow, file and task that generates it
      static.meta.end - end of metadata events in the bp files

      sample events
      ts=2015-10-13T20:50:15.798371Z event=static.meta.start xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8

      ts=2015-10-13T20:50:15.798692Z event=xwf.meta xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 key="name" value="diamond"
      ts=2015-10-13T20:50:15.799129Z event=xwf.meta xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 key="createdby" value="Karan Vahi"

      ts=2015-10-13T20:50:15.799603Z event=task.meta xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 task.id="ID0000004" key="time" value="60"
      ts=2015-10-13T20:50:15.799907Z event=task.meta xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 task.id="ID0000004" key="transformation" value="analyze"

      ts=2015-10-13T20:50:15.805293Z event=rc.meta xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 lfn.id="f.a" key="raw_input" value="true"
      ts=2015-10-13T20:50:15.805593Z event=rc.meta xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 lfn.id="f.a" key="size" value="1024"


      ts=2015-10-13T20:50:15.805993Z event=wf.map.file xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 task.id="ID0000001" lfn.id="f.a"
      ts=2015-10-13T20:50:15.806300Z event=wf.map.file xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8 task.id="ID0000001" lfn.id="f.b2"

      ts=2015-10-13T20:50:15.807098Z event=static.meta.end xwf.id=9ab8e307-4f05-4e45-bb9c-01ffda6e59f8

        Attachments

          Activity

            People

            • Assignee:
              vahi Karan Vahi
              Reporter:
              vahi Karan Vahi
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: