Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-657

add support for a replica catalog implementation that takes in a directory where the files reside


      Users should be able to

      • set up a simple pipeline (probably without specifying files),
        • provide a directory with files in it
        • run the pipeline on the files in that dir and put it in another dir

      One way to do it is to do an implementation of the Replica Catalog interface in Pegasus that does the directory listing on the input directory, and creates a Replica Catalog in memory of the basis of the list of files in that directory.

      • The DAX remains abstract ( no physical paths specified for input files).
      • We will have to tinker with the Replica Factory also , as we may still want to register the output files in some other replica catalog

            vahi Karan Vahi
            vahi Karan Vahi
            1 Start watching this issue