Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Adapter Checksum Algorithm set to NULL for wasabi #1850

Open
chrisgo opened this issue Feb 3, 2025 · 0 comments
Open

AWS Adapter Checksum Algorithm set to NULL for wasabi #1850

chrisgo opened this issue Feb 3, 2025 · 0 comments

Comments

@chrisgo
Copy link

chrisgo commented Feb 3, 2025

Bug Report

We were getting an error doing writes to wasabi for large files (16MB seems to be when it switches to multipart) that mentioned checksum Checksum Type mismatch occurred, expected checksum Type: null, actual checksum Type: crc32

There is a workaround but did bring our production systems down for a couple of days since this is also sporadic until we isolated this to "larger" files

This also seems to be working on the Amazon S3 service and ONLY for wasabi (this is the only other S3-compatible provider we have handy)

Q A
Flysystem Version ^3.10.0
Adapter Name flysystem-aws-s3-v3
Adapter version ^3.10.0
AWS SDK ^3.295.10

Summary

Tracing this down seems to lead to the ->upload() function being used instead of ->putObject(). The ->upload() function automatically switches to the multipart for large files. The ->putObject() seems to be unaffected

The problem seems to be due to this array https://github.com/thephpleague/flysystem/blob/3.x/src/AwsS3V3/AwsS3V3Adapter.php#L47 where it lists the 'ChecksumAlgorithm' as an AVAILABLE_OPTIONS

Then somewhere when it builds the config, it is setting this to NULL (not sure) https://github.com/thephpleague/flysystem/blob/3.x/src/AwsS3V3/AwsS3V3Adapter.php#L176 instead of taking the default (which seems to be CRC32) and this seems to be what is throwing the error above

How to reproduce

// Provider configuration 
$options = [
  ... // normal stuff per documentation
  'use_path_request_style' => true,  // not really needed for wasabi
];
$client = new \Aws\S3\S3Client($options);
$bucket = 'my-bucket'
$prefix = 'my-prefix';
// Usual way of instantiating the adapter
$adapter = new \League\Flysystem\AwsS3V3\AwsS3V3Adapter($client, $bucket, $prefix);
...

Workaround/Fix

// Provider configuration 
$options = [
  ... // normal stuff per documentation
  'use_path_request_style' => true,  // not really needed for wasabi
];
$client = new \Aws\S3\S3Client($options);
$bucket = 'my-bucket'
$prefix = 'my-prefix';
// 1/31/25 CG: Need to pass the $options (that has the 'ChecksumAlgorithm' = 'CRC32') all the way through to the adapter
$options['ChecksumAlgorithm'] = 'CRC32';
$adapter = new \League\Flysystem\AwsS3V3\AwsS3V3Adapter($client, $bucket, $prefix, null, null, $options);
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant