Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ObjectCopy and ObjectMove do not understand large objects #102

Open
majewsky opened this issue Nov 2, 2017 · 1 comment
Open

ObjectCopy and ObjectMove do not understand large objects #102

majewsky opened this issue Nov 2, 2017 · 1 comment

Comments

@majewsky
Copy link
Contributor

majewsky commented Nov 2, 2017

Connection.ObjectCopy (and, therefore, Connection.ObjectMove) issues only a single COPY request to copy an object. However, this leads to large objects being concatenated into a single object. The copy operation is therefore needlessly expensive, or may even fail if the object becomes too large because of the concatenation.

ObjectCopy and ObjectMove should (either automatically or upon explicit choice of the caller) detect large objects and copy only the manifest instead of the entire object.

This issue is especially severe because, at least for SLOs, the caller cannot work around this problem because of #101.

@ncw
Copy link
Owner

ncw commented Nov 9, 2017

I'd say that if you asked for a copy, then the segments should be copied too, otherwise you've just duplicated the manifest which doesn't preserve the data in the event of a delete, say.

Likewise the move should probably move the segments too.

Having ObjectCopy and ObjectMove do this automatically seems desirable. I'm a bit concerned about the copy or move only half completeing though...

I'm not sure of the right solution here.

@ncw ncw added the enhancement label Nov 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants