Skip to content

fix: stream object copies larger than cloud's CopyObject limit#7004

Merged
wjones127 merged 1 commit into
lance-format:mainfrom
vivek-bharathan:fix/large-object-copy-fallback
May 30, 2026
Merged

fix: stream object copies larger than cloud's CopyObject limit#7004
wjones127 merged 1 commit into
lance-format:mainfrom
vivek-bharathan:fix/large-object-copy-fallback

Conversation

@vivek-bharathan
Copy link
Copy Markdown
Contributor

copy now HEADs the source, and when it exceeds 5 GiB, falls back to a streaming multipart copy

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added the bug Something isn't working label May 29, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

❌ Patch coverage is 57.89474% with 32 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance-io/src/object_store.rs 57.89% 26 Missing and 6 partials ⚠️

📢 Thoughts on this report? Let us know!

Comment thread rust/lance-io/src/object_store.rs Outdated
Comment thread rust/lance-io/src/object_store.rs
@vivek-bharathan vivek-bharathan force-pushed the fix/large-object-copy-fallback branch from 7483987 to ee73b62 Compare May 29, 2026 22:15
Copy link
Copy Markdown
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me.

@cmccabe
Copy link
Copy Markdown
Contributor

cmccabe commented May 30, 2026

@vivek-bharathan : it looks like we need to remove this:

#[cfg_attr(coverage, coverage(off))]

because it's causing some Linux compiler errors:

    error[E0658]: the `#[coverage]` attribute is an experimental feature
    --> rust/lance-io/src/object_store.rs:1568:5
     |
1568 |     #[cfg_attr(coverage, coverage(off))]
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |

copy now HEADs the source, and when it exceeds 5 GiB, falls back to a streaming multipart copy
@vivek-bharathan vivek-bharathan force-pushed the fix/large-object-copy-fallback branch from ee73b62 to ce3b3f9 Compare May 30, 2026 04:03
@wjones127 wjones127 merged commit b2fcf62 into lance-format:main May 30, 2026
29 checks passed
@vivek-bharathan vivek-bharathan deleted the fix/large-object-copy-fallback branch May 30, 2026 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants