When I use SCP for pegasus stage-out, the destination URL is a file and pegasus-transfer fails like so:

2011-05-03 13:57:25,132 INFO: Reading URL pairs from stdin
2011-05-03 13:57:25,133 INFO: PATH=/usr/local/globus/default/bin:/usr/local/pegasus/default/bin:/usr/bin:/bin
2011-05-03 13:57:25,133 INFO: LD_LIBRARY_PATH=/usr/local/globus/default/lib:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64
2011-05-03 13:57:25,133 INFO: Executing scp commands
Traceback (most recent call last):
File "/usr/local/pegasus/default/bin/pegasus-transfer", line 738, in <module>
scp()
File "/usr/local/pegasus/default/bin/pegasus-transfer", line 437, in scp
stats_add(url_pair.dst_path)
File "/usr/local/pegasus/default/bin/pegasus-transfer", line 595, in stats_add
s = os.stat(filename)
OSError: [Errno 2] No such file or directory: '/submit/diamond/outputs/f.b1'

This happens because the scp() function tries to stat the file before it has been transferred.

I fixed it like so:

— pegasus-transfer 2011-05-03 14:43:34.000000000 -0700
+++ pegasus-transfer-new 2011-05-03 14:43:51.000000000 -0700
@@ -434,12 +434,13 @@
prepare_local_dir(os.path.dirname(url_pair.dst_path))
cmd += " " + url_pair.src_host + ":" + url_pair.src_path
cmd += " " + url_pair.dst_path

• stats_add(url_pair.dst_path)
  else:
  cmd += " " + url_pair.src_path
  cmd += " " + url_pair.dst_host + ":" + url_pair.dst_path
  stats_add(url_pair.src_path)
  myexec(cmd, 1*60*60, False, delay_errors)
  + if url_pair.dst_proto == "file":
  + stats_add(url_pair.dst_path)

def webget():