Repository navigation
Client Side Encryption with S3EncryptionMultipartUploaderV2 fails on large files. #2512
Description
Describe the bug
When using S3EncryptionMultipartUploaderV2 with a file larger than 4GB a warning is thrown during encryption, and an exception is thrown during upload. This does not happen with smaller files.
Expected Behavior
Success on S3EncryptionMultipartUploaderV2::upload()
Current Behavior
When using S3EncryptionMultipartUploaderV2 with a file larger than 4GB this message appears:
PHP Warning: openssl_encrypt(): data is too long in libs\aws\aws-sdk-php\src\Crypto\AesGcmEncryptingStream.php on line 108
After a while, exception is thrown:
An exception occurred while completing a multipart upload: Error executing "CompleteMultipartUpload" on "encryption-test/lfmb_TESNICO.mp4?uploadId=345dd8b5-b470-4c92-8525-86298794e9e4"; AWS HTTP error: Client error: `POST encryption-test/lfmb_TESNICO.mp4?uploadId=345dd8b5-b470-4c92-8525-86298794e9e4` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed (truncated...)
MalformedXML (client): The XML you provided was not well-formed or did not validate against our published schema. - <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate against our published schema.</Message><Key>lfmb_TESNICO.mp4</Key><BucketName>encryption-test</BucketName><Resource>/encryption-test/lfmb_TESNICO.mp4</Resource><RequestId>170FDB5FFCA88848</RequestId><HostId>4bf9e65b-bbb0-488d-9da9-d54f60abb5c0</HostId></Error>
Reproduction Steps
$uploader = new S3EncryptionMultipartUploaderV2(
new Aws\S3\S3Client([
'version' => 'latest',
'region' => 'us-east-1',
'use_path_style_endpoint' => true,
'credentials' => [
'key' => $options['credentials']['key'],
'secret' => $options['credentials']['secret'],
]
]), ,fopen('path to a file larger than 4GB', 'r'),
[
'@MaterialsProvider' => $materialsProvider,
'@CipherOptions' => [ 'Cipher' => 'gcm', 'KeySize' => 128 ],
'@KmsEncryptionContext' => [],
'bucket' => $this->bucket,
'key' => 'thenamefile',
'concurrency' => 5,
'part_size' => null,
]
);
$uploader->upload();Possible Solution
openssl_encrypt is know have trouble with large files.
Maybe include a mechanism to encode by chunks.
Additional Information/Context
No response
SDK version used
"aws/aws-sdk-php": "^3.147"
Environment details (Version of PHP (php -v)? OS name and version, etc.)
PHP 7.4.30 (cli) (built: Jun 7 2022 15:36:03) ( NTS Visual C++ 2017 x64 )
Activity
yenfryherrerafeliz commentedon Sep 2, 2022
Hi @elt4n0, thanks for opening this issue. I have reproduced the reported behavior and indeed the issue is coming from openssl library. I am going to leave this issue under review so we can address this further, and will provide updates accordingly.
Thanks!
stobrien89 commentedon Dec 31, 2025
Hi @elt4n0,
Sorry for the delay in contact. We just recently released
S3EncryptionMultipartUploaderV3andS3EncryptionClientV3as a security fix for a vulnerabilities in the older versions. We recommend upgrading as soon as possible.About the 4GB limitation— I've discussed this with the AWS Cryptography team and they would prefer we don't encrypt/encode using chunking. Unfortunately this is a limitation present in all of the S3 Encryption implementations, and something we need to document.