Skip to content

feat: write to temp dir first#164

Merged
LandonPatmore merged 2 commits into
v2from
content-integrity
Jun 3, 2026
Merged

feat: write to temp dir first#164
LandonPatmore merged 2 commits into
v2from
content-integrity

Conversation

@LandonPatmore

@LandonPatmore LandonPatmore commented May 26, 2026

Copy link
Copy Markdown
Contributor

Prior to this PR, we were writing all blobs to their final destination. If a pull stopped mid way through, push code would see the blob existing even though it was corrupted.

We now added an extra step to our pull code: temp -> final folder move. This will allow us to make sure any blobs put in the final folder by koci have been SHA + size checked. Any mid progress blobs will be separate, thus operations like push will not see them unless properly verified.

This speeds up our existence check in the final folder by only doing a size check instead of a full hash check like we were doing prior.

Added tests.

Benchmarks below show that this change did not hinder performance.

These tests were ran on a M4 Pro MBP on a wired 2.5gbps connection on a layer 3 switch.

Date: 2026-05-26 13:30
Machine: macOS-26.4.1-arm64-arm-64bit-Mach-O
JVM: openjdk version "21.0.11" 2026-04-21 LTS
Go: go version go1.26.3 darwin/arm64
Iterations: 31

Pull

Size oras-go mean v1 mean current mean v1 vs oras current vs oras current vs v1
4MB 32.9ms 35.3ms 33.9ms 1.1x slower ~same ~same
52MB 258.6ms 260.7ms 258.5ms ~same ~same ~same
474MB 2.28s 2.25s 2.25s ~same ~same ~same
1.1GB 5.46s 5.44s 5.44s ~same ~same ~same
181MB 863.6ms 868.0ms 865.5ms ~same ~same ~same

@LandonPatmore

LandonPatmore commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

Comment thread koci/src/main/kotlin/com/defenseunicorns/koci/api/Layout.kt
Comment thread koci/src/test/kotlin/com/defenseunicorns/koci/LayoutTest.kt
@LandonPatmore LandonPatmore merged commit f341b32 into v2 Jun 3, 2026
9 checks passed
@LandonPatmore LandonPatmore deleted the content-integrity branch June 3, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants