Skip to content

Conversation

@bentsku
Copy link
Contributor

@bentsku bentsku commented Jun 13, 2025

Motivation

We've got a report in our Community Slack with a small sample (as represented by the new test) where Checksum handling did not work well with the UploadPartCopy operation: https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPartCopy.html

This was due because we simply did not handle checksums as this was not implemented in our previous implementation.

We now properly handle checksums when copying an object into a part, via our storage engine.

This was the error when calling ListParts:

An error occurred (InternalError) when calling the ListParts operation (reached max retries: 4): exception while calling s3.ListParts: 'NoneType' object has no attribute 'upper'

Changes

  • add new test covering the case of multipart uploads with checksum and UploadPartCopy
  • fix the behavior to retrieve the new checksum from the stored object, calculated when being stored
  • return the checksum field when the conditions are filled for UploadPartCopy
  • add a better check in ListParts to avoid unhandled exceptions

@bentsku bentsku added this to the 4.6 milestone Jun 13, 2025
@bentsku bentsku self-assigned this Jun 13, 2025
@bentsku bentsku added aws:s3 Amazon Simple Storage Service semver: patch Non-breaking changes which can be included in patch releases labels Jun 13, 2025
@github-actions
Copy link

S3 Image Test Results (AMD64 / ARM64)

  2 files    2 suites   8m 26s ⏱️
496 tests 446 ✅  50 💤 0 ❌
992 runs  892 ✅ 100 💤 0 ❌

Results for commit 420747a.

@github-actions
Copy link

Test Results - Preflight, Unit

21 613 tests  ±0   19 958 ✅ ±0   6m 16s ⏱️ -8s
     1 suites ±0    1 655 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit 420747a. ± Comparison against base commit 02ad74e.

@github-actions
Copy link

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 6s ⏱️ -1s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 420747a. ± Comparison against base commit 02ad74e.

@github-actions
Copy link

Test Results - Alternative Providers

987 tests   589 ✅  30m 17s ⏱️
  4 suites  398 💤
  4 files      0 ❌

Results for commit 420747a.

@github-actions
Copy link

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   1h 24m 40s ⏱️
2 237 tests 1 621 ✅ 616 💤 0 ❌
2 243 runs  1 621 ✅ 622 💤 0 ❌

Results for commit 420747a.

@github-actions
Copy link

LocalStack Community integration with Pro

    2 files      2 suites   1h 2m 8s ⏱️
2 213 tests 1 594 ✅ 619 💤 0 ❌
2 215 runs  1 594 ✅ 621 💤 0 ❌

Results for commit 420747a.

@bentsku bentsku requested a review from k-a-il June 13, 2025 14:26
Copy link
Contributor

@k-a-il k-a-il left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice quick fix 🌟 I took a quick look at the code and snapshots - everything looks good.

@bentsku bentsku merged commit df9ebe9 into master Jun 16, 2025
52 checks passed
@bentsku bentsku deleted the fix-s3-part-checksum branch June 16, 2025 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aws:s3 Amazon Simple Storage Service semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants