Add tools wkp descriptors command and update WKP generator with source_code_info support#148
Merged
Merged
Conversation
Contributor
🔍 Vulnerabilities of
|
| digest | sha256:0d7fd5d6c2baee88d240986abef6308f7265150bc71b1121e3ba5a9669b99dfc |
| vulnerabilities |     |
| platform | linux/amd64 |
| size | 161 MB |
| packages | 505 |
📦 Base Image ubuntu:24.04
| also known as |
|
| digest | sha256:023f8a753c22258c9fe2d0005a7d28258038da7d620e9f93e9ad78aa266f9f11 |
| vulnerabilities |    |
maoueh
force-pushed
the
feature/tools-wkp-descriptors
branch
from
4e90e77 to
d500805
Compare
New command `firecore tools wkp descriptors [output-file]` exports all well-known blockchain protobuf descriptors as a self-contained, binary google.protobuf.FileDescriptorSet. The set includes every transitive import so consumers can build a descriptor registry with no external resolution. Output is deterministic (stable topological + alphabetical ordering) for CI "is it up to date?" regenerate-and-diff workflows. The WKP generator is updated to use the BSR Connect API (buf.registry.module.v1.FileDescriptorSetService/GetFileDescriptorSet) instead of the Reflect v1beta1 API. A single call now returns both the resolved commit ID (used in the documentation URL comment in each generated file) and the FileDescriptorSet with source_code_info included via includeSourceCodeInfo:true, restoring proto field/message comments that were previously stripped by the reflection protocol. BUFBUILD_AUTH_TOKEN is now optional (public modules are freely accessible; a warning is emitted when absent). Deduplication across modules uses a name-keyed map and warns when the same file path appears with different content (BSR module version skew). Logging switched from the standard library to the StreamingFast logging library (zlog) and all error handling uses cli.NoError/cli.Ensure throughout.
Upgrades testcontainers-go from v0.40.0 to v0.42.0, which dropped github.com/docker/docker in favour of the split github.com/moby/moby/api and github.com/moby/moby/client modules. Updates the single direct import in relayer/relayer_e2e_test.go from docker/docker to moby/moby/api. Removes GHSA-x744-4wpc-v9h2, GHSA-x86f-5xw2-fm2r, and GHSA-rg2x-37c3-w2rh flagged by the dependency-review CI check.
maoueh
force-pushed
the
feature/tools-wkp-descriptors
branch
from
d500805 to
e70e677
Compare
| req.Header.Set("Authorization", "Bearer "+authToken) | ||
| } | ||
|
|
||
| resp, err := http.DefaultClient.Do(req) |
Contributor
There was a problem hiding this comment.
Should this have a timeout in case the BSR hangs ?
Contributor
Author
There was a problem hiding this comment.
A context now sets a 60s timeout for the whole operation.
- Remove unexported blockTypeNames() helper and its test — not needed by the command itself, was test-only utility with no clear owner - Add 60s context timeout to BSR HTTP request in fetchFileDescriptorSet to prevent generator hanging indefinitely on slow/unresponsive BSR
UlysseCorbeil
approved these changes
Jun 18, 2026
testcontainers-go v0.42 uses moby/moby/api types which redesigned Port (now a struct) and PortBinding (HostIP is netip.Addr). Update HostConfig.PortBindings to use network.PortMap/PortBinding and cast nat.Port to string for wait.ForListeningPort.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
firecore tools wkp descriptors [output-file]— exports all well-known blockchain protobuf descriptors as a self-contained binarygoogle.protobuf.FileDescriptorSet. Output is deterministic (stable topological + alphabetical ordering) for CI regenerate-and-diff workflows. Supports-for stdout; default filename iswell-known-descriptors.binpb. 22 files, ~109 KB, covers 12 chain block types (Ethereum, Solana, Near, Bitcoin, Cosmos, Antelope, Beacon, Starknet, Stellar, Tron, Arweave, ACME) plus Google well-known types.proto/generatorto use the BSR Connect API (buf.registry.module.v1.FileDescriptorSetService/GetFileDescriptorSet) instead of the Reflect v1beta1 API. A single call now returns the resolved commit ID (restored in documentation URL comments in generated files) and theFileDescriptorSetwithsource_code_infoincluded (includeSourceCodeInfo: true), restoring proto field/message comments that were previously stripped by the reflection protocol.BUFBUILD_AUTH_TOKENis now optional for public modules. Generator logging switched to the StreamingFastzloglibrary; all error handling usescli.NoError/cli.Ensure. Deduplication across modules uses a name-keyed map and warns on content mismatch (version skew).Test plan
go test ./cmd/tools/wkp/...— 9 tests covering export round-trip, determinism, self-contained registry validation, Ethereum block type resolution, no-duplicate-files, topological order, block type name list, write-to-file, and source_code_info statusgo build ./...— clean buildfirecore tools wkp descriptors /tmp/out.binpb— runs end-to-end and logs file count + byte size.gofiles now contain exact commit hashes in documentation URL comments and the embedded descriptors carrysource_code_info