Thanks for the followup! My main concern here is that the file is declared with two PFNs in the site catalog:

pfns:

• site: local
  pfn: file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O3/V1Online/V-V1Online-12399/V-V1Online-1239948000-2000.gwf
• site: osg
  pfn: file:///cvmfs/oasis.opensciencegrid.org/ligo/frames/O3/V1Online/V-V1Online-12399/V-V1Online-1239948000-2000.gwf

In the current case the site=local file is used, and then the file is correctly transferred with condorIO. This involves a transfer from the LIGO cluster to whatever remote OSG cluster we are running on. However, the site=osg file is visible on the remote cluster and should be used directly. So I contend that there should not be any condor file IO for this file and the planner should pick the site=osg URL for this file, not the site=local URL.

I think the stage-in question is less important. There are a few steps currently being done here:

1) The file is transferred from the location under /cvmfs to the local site scratch on site=local as a stage-in job.
2) The file is transferred to the OSG site via condor IO when the job starts.
3) The file is moved from the condor transfer location, to the directory pegasus expects the file to be in within the job's .sh script.

While stage (1) and (3) seem to be adding complexity, they aren't really a problem as long as they are symlinks (or moves) and not copies.

Does that make sense? The behaviour if transferring a file from site=local to site=osg seems correct, but I think the file should have been used directly from site=osg in this case.