pegasus-s3 cp s3:// to s3:// does not support files > 5GB

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Type: New Feature
    • Resolution: Done
    • Priority: Critical
    • master, 4.5.0
    • Affects Version/s: master, 4.4.0, 4.5.0
    • Component/s: Planner: Transfer Module
    • None

      pegasus-s3 fails when transferring file/key from s3 source to s3 dest when the src file is > 5GB

      Error Log:

      boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request
      <Error><Code>InvalidRequest</Code><Message>The specified copy source is larger than the maximum allowable size for a copy source: 5368709120</Message><RequestId>6A676A5A2E6B3668</RequestId><HostId>C6doHH6cT7K6k4SgFyv/oU/km+m0bcYyum6ruFKMAiDZ58qW7k0na5DdSoXNs7sZ</HostId></Error>

      Possible Solution:

      http://stackoverflow.com/questions/10355941/how-can-i-copy-files-bigger-than-5-gb-in-amazon-s3

            Assignee:
            Gideon Juve
            Reporter:
            Rajiv Mayani
            Archiver:
            Rajiv Mayani

              Created:
              Updated:
              Resolved:
              Archived: