Skip to content

feat: support concurrent chunk uploads#60

Merged
TorstenDittmann merged 2 commits into
mainfrom
concurrent-chunk-uploads-1-9-x-minimal
May 21, 2026
Merged

feat: support concurrent chunk uploads#60
TorstenDittmann merged 2 commits into
mainfrom
concurrent-chunk-uploads-1-9-x-minimal

Conversation

@TorstenDittmann

Copy link
Copy Markdown
Contributor

This PR updates the SDK to support concurrent chunk uploads.

@greptile-apps

greptile-apps Bot commented May 21, 2026

Copy link
Copy Markdown

Greptile Summary

This PR refactors FileUpload to upload multi-chunk files with up to 8 concurrent workers and bumps the SDK version to v4.1.0. Chunk 0 is always uploaded synchronously to retrieve the server-assigned upload ID before the concurrent fan-out begins, and a done channel is used to signal workers to stop on the first error.

  • copyHeaders/copyParams helpers ensure each worker operates on its own per-request copies, preventing concurrent mutation.
  • A lastResult fallback is added so a non-nil final response is always returned even when no single response reports full completion.
  • Previously flagged concerns (no early cancellation, stale result, parseUploadId outer-err capture) are all addressed in this revision.

Confidence Score: 5/5

The concurrent upload implementation is safe to merge: goroutine lifecycle is correctly managed, the upload ID is fully resolved before any worker starts, and per-chunk copies of headers and params prevent concurrent mutation.

The core concurrency concerns are all handled correctly — done-channel cancellation fires at most once, workers can always drain into a fully-buffered results channel without blocking, os.File.ReadAt is safe for concurrent use via pread, and uploadId is written before goroutines start and only read afterward. The one new finding is a naming-collision style nit with no runtime impact.

No files require special attention; client/client.go is the only changed file and the concurrent logic is sound.

Important Files Changed

Filename Overview
client/client.go Refactors multi-chunk upload to use up to 8 concurrent workers with a jobs/results/done channel pattern; uploads chunk 0 synchronously to capture the upload ID, then fans out the remainder. Previously flagged issues have been addressed. One style issue: the added result = resp line sits immediately above a shadowing inner var result map[string]interface{} declaration.

Reviews (2): Last reviewed commit: "feat: support concurrent chunk uploads" | Re-trigger Greptile

Comment thread client/client.go Outdated
Comment thread client/client.go
Comment thread client/client.go
@TorstenDittmann TorstenDittmann merged commit 4611062 into main May 21, 2026
1 check passed
@TorstenDittmann TorstenDittmann deleted the concurrent-chunk-uploads-1-9-x-minimal branch May 21, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants