refactor(protoc): migrate to protocompile experimental compiler with embedded proto deps#404
Open
Kybxd wants to merge 5 commits into
Open
refactor(protoc): migrate to protocompile experimental compiler with embedded proto deps#404Kybxd wants to merge 5 commits into
Kybxd wants to merge 5 commits into
Conversation
The experimental compiler only accepts source files via source.Opener, so
vendor required .proto deps under internal/x/xproto/protoc/embedded/ and
expose them through go:embed:
- tableau/protobuf/*.proto: copied from proto/tableau/protobuf/
- buf/validate/validate.proto: exported from buf.build/bufbuild/protovalidate
at the commit pinned in buf.lock
gen_embedded.sh (invoked via go:generate) refreshes the embedded tree and
explicitly forwards buf.lock's protovalidate commit to `buf export`, since
ad-hoc `buf export` does not consult buf.lock. The embedded/ tree is
committed so `go install ...@latest` works without buf or go generate.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #404 +/- ##
==========================================
+ Coverage 73.38% 74.44% +1.05%
==========================================
Files 88 89 +1
Lines 11300 9154 -2146
==========================================
- Hits 8293 6815 -1478
+ Misses 2440 1762 -678
- Partials 567 577 +10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Protobuf Edition 2024 is not understood by
bufbuild/protocompile'sstable
Compiler. To unblock Edition 2024 schemas in this codebase, wefollow the same path Buf itself took in
buf v1.69.0 and
switch the internal
protocwrapper to protocompile's experimentalcompiler, which has Edition 2024 support.
What changes vs. the stable compiler
The experimental compiler exclusively consumes source files via
source.Opener. It no longer accepts pre-compiledFileDescriptorsthrough
protoregistry.GlobalFiles— which is how the stable compilerused to pick up
tableau/protobuf/*.protoandbuf/validate/validate.proto. We need a new mechanism to make thoseimports resolvable at compile time.
How
Vendor the required
.protofiles underinternal/x/xproto/protoc/embedded/and expose them via
go:embed:tableau/protobuf/tableau.protoproto/tableau/protobuf/tableau/protobuf/wellknown.protoproto/tableau/protobuf/buf/validate/validate.protobuf exportofbuf.build/bufbuild/protovalidateat the commit pinned inbuf.lockStandard imports (
google/protobuf/*) continue to be served bysource.WKTsand need no embedding.Refresh workflow
A
go:generatedirective inembed.goinvokesgen_embedded.sh, which:embedded/and recreatesembedded/tableau/protobuf/.proto/tableau/protobuf/*.proto(top-level only — matchesbuf.yamlexcludes forinternal/andunittest/).buf.lockand forwards it tobuf export buf.build/bufbuild/protovalidate:<commit>.Why is
embedded/checked into version control?So that downstream consumers of
go install github.com/tableauio/tableau/cmd/tableauc@latestcan buildwithout
bufinstalled and without runninggo generate—proxy.golang.orgonly packages git-tracked files.Upgrading protovalidate