Design: Managing forked WASM dependencies#850
Conversation
pgrayy
force-pushed
the
designs/fork-management
branch
from
f0b53a8 to
6da25e7
Compare
|
|
||
| ## Solution | ||
|
|
||
| Use **git subtrees** to import each fork's source directly into sdk-typescript. |
There was a problem hiding this comment.
The biggest con to me is that this source code ends up in the history forever; e.g. there's no way to strip it later without rewriting history. Whereas a submodule being a pointer [effectively] means the overhead is trivial once we remove them
There was a problem hiding this comment.
That seems like a feature to me though no? At one point we vended these dependencies and getting old builds working means simply checking out the old history. Hermetic builds.
There was a problem hiding this comment.
That is a good call out. It would permanently alter the size of sdk-typescript. Based on the metrics, it would end up being roughly 30MB (calculated from the WASM repo .git sizes) of dead weight once removed.
To mitigate, we could try slimming down wasmtime and jco even more than what was proposed.
There was a problem hiding this comment.
Yeah, what Patrick said; it's a permanent size bump for what is/should be a temporary solution
There was a problem hiding this comment.
So no subtree? Commit your fork into the repo without history, and we can override from your fork if we need to downstream anything?
There was a problem hiding this comment.
In my experience submodules are just incredibly clunky and make it difficult to use git tools to split history or something. Memory is fuzzy here. My understanding is that they'd poison our repo and be hard to remove but don't take my word for it.
There was a problem hiding this comment.
Commit your fork into the repo without history, and we can override from your fork if we need to downstream anything?
Yeah effectively
In my experience submodules are just incredibly clunky
If you're doing active development and changing things often, this is true - it's why I don't suggest it as an alternative to a monorepository. But for infrequent changes or "pinning" at a specific commit, it works pretty well.
AFAIK there's no poisoning as it's more like a file/pointer more than anything. Deleting them is a couple of configs, whereas subtrees you have the entire history as part of your repo forever.
There was a problem hiding this comment.
Ignore me, conflating LFS and submodules for some reason. Need a nap.
| "scripts": { | ||
| "dev:link-forks": "npm link ./forks/componentize-js ./forks/jco" | ||
| } | ||
| } |
There was a problem hiding this comment.
Could continue to add to these commands over time to offer more conveniences if need be. But I feel something like this is a reasonable start.
There was a problem hiding this comment.
It balances the development flow (with friction) with the consumer flow quite well IMHO
|
|
||
| Publish fork artifacts to **npm** and **PyPI** under the `@strands-agents` scope. Use a dedicated fork repo with CI that tests against sdk-typescript before publishing. Use **git submodules** in sdk-typescript as an optional local development convenience. | ||
|
|
||
| ### Distribution |
There was a problem hiding this comment.
The biggest pro is that this streamlines development for those who don't need changes to these packages. The biggest con is that it is a much higher bar for those that are making the changes to the packages.
How confident are we that changes will be infrequent?
3 participants
Proposes using git subtrees with slim branches to house forked WASM dependencies (wasmtime, wasmtime-py, componentize-js, jco) within sdk-typescript. Includes size analysis and comparison with alternatives (submodules, LFS).