Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 19 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Responsibilities:
- Track upstream `google-ai-edge/LiteRT-LM` releases.
- Fetch or build native LiteRT-LM libraries per platform.
- Preserve upstream LiteRT-LM's C runtime ABI as the FFI boundary.
- Embed the small StreamProxy callback-copy helper into runtime libraries used
by asynchronous FFI clients.
- Embed the small LiteRtLmBridge callback helper into runtime libraries used by
asynchronous FFI clients.
- Package web assets around official LiteRT-LM/LiteRT.js distribution paths.
- Publish Apple Swift Package Manager XCFramework zip assets built from the
same bridge runtimes as the native release payload.
Expand Down Expand Up @@ -53,11 +53,13 @@ GPU/NPU validation; web should use JavaScript interop instead of FFI.
- `web/`: web package scaffold for JS/Wasm/WebGPU/WebNN integration.
- `tools/fetch_upstream.py`: resolves and downloads upstream release assets.
- `tools/build_upstream_runtime.py`: builds upstream LiteRT-LM C runtime
libraries from tagged source with Bazel/Bazelisk, embeds StreamProxy symbols
into source-built runtime libraries, and stages them for release.
libraries from tagged source with Bazel/Bazelisk through the repo-owned
`native/bridge` Bazel package, embeds LiteRtLmBridge symbols into
source-built runtime libraries without patching upstream source files, and
stages them for release.
- `tools/package_ios_runtime.py`: extracts official upstream
`CLiteRTLM.xcframework` slices and stages `LiteRtLm.framework` plus
`CLiteRTLM.framework`, with the wrapper embedding StreamProxy symbols and
`CLiteRTLM.framework`, with the wrapper embedding LiteRtLmBridge symbols and
re-exporting `CLiteRTLM`.
- `tools/package_apple_xcframeworks.py`: packages iOS framework wrappers, a
macOS `LiteRtLm.framework` wrapper around the source-built runtime, and macOS
Expand Down Expand Up @@ -93,11 +95,11 @@ python3 tools/validate_artifacts.py
- `Validate`: validates package metadata and checks Python/web tooling on
pushes and pull requests.
- `Native Build & Release`: manually packages a selected upstream LiteRT-LM tag.
It builds upstream C runtime libraries with embedded StreamProxy symbols for
It builds upstream C runtime libraries with embedded LiteRtLmBridge symbols for
Android arm64/x64, macOS arm64/x64, Linux x64/arm64, and Windows x64, copies
upstream `prebuilt/` companion libraries for Android, Apple, Linux, and
Windows, converts official upstream `CLiteRTLM.xcframework` slices into iOS
framework runtime archives with an embedded-StreamProxy wrapper, packages
framework runtime archives with an embedded LiteRtLmBridge wrapper, packages
Apple SPM XCFramework zips from the same runtime payloads, includes the
official upstream release assets, then publishes a GitHub release with
`manifest.json` and `SHA256SUMS`.
Expand All @@ -115,14 +117,18 @@ package hooks and Swift Package manifests. When moving to a new LiteRT-LM tag:
2. Verify the release contains runtime archives, official upstream assets,
Apple SPM XCFramework zips, `manifest.json`, and `SHA256SUMS`.
3. Update downstream `llamadart` hook pins, SPM URLs, and SPM checksums
together so native-assets and SPM consumers use the same StreamProxy-enabled
together so native-assets and SPM consumers use the same bridge-enabled
runtime build.

The release workflow uses upstream's public C API (`c/engine.h`) as the
production FFI boundary. Downstream loaders should bind directly to the runtime
library for the selected platform. StreamProxy symbols are embedded into the
same runtime library surface; no standalone StreamProxy runtime artifact is part
of the release contract.
library for the selected platform. Source-built native runtimes are assembled
from a repo-owned Bazel package selected ahead of the upstream source tree with
`--package_path`, so the workflow does not edit upstream LiteRT-LM files.
LiteRtLmBridge symbols are embedded into the same runtime library surface; no
standalone bridge runtime artifact is part of the release contract. The bridge
currently exports the `stream_proxy_*` compatibility symbols used by
asynchronous callback loaders.

Apple SPM consumers should depend on the release's direct
`litert-lm-native-apple-*-xcframework-<tag>.zip` assets. The `LiteRtLm`
Expand All @@ -142,4 +148,5 @@ architecture, runtime kind (`native` or `web`), and accelerator metadata, then
verify checksums before bundling or loading the files.

Upstream LiteRT-LM's native C ABI is the compatibility boundary. This repository
does not add a second wrapper ABI unless a future upstream change requires it.
does not add a second model wrapper ABI unless a future upstream change requires
it; bridge helpers remain narrow FFI utilities around that runtime surface.
22 changes: 13 additions & 9 deletions docs/current_upstream.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,23 @@
# Current Upstream Snapshot

As of the first scaffold, the latest upstream release resolved by
As of `2026-06-08`, the latest upstream release resolved by
`tools/fetch_upstream.py --latest --metadata-only` is:

- Repository: `google-ai-edge/LiteRT-LM`
- Tag: `v0.12.0`
- Published: `2026-05-18T20:53:57Z`
- Release URL: `https://github.com/google-ai-edge/LiteRT-LM/releases/tag/v0.12.0`
- Asset: `CLiteRTLM.xcframework.zip`
- Tag: `v0.13.1`
- Published: `2026-06-03T20:52:11Z`
- Release URL: `https://github.com/google-ai-edge/LiteRT-LM/releases/tag/v0.13.1`
- Assets:
- `CLiteRTLM.xcframework.zip`
- `CLiteRTLM_mac.xcframework.zip`

Older release `v0.11.0` exposed standalone CLI assets for Android, iOS
simulator, Linux x64, macOS arm64, and Windows x64. The native repo should not
assume that upstream releases always publish the same asset matrix.
Older releases have exposed different asset matrices. For example, `v0.11.0`
included standalone CLI assets for Android, iOS simulator, Linux x64, macOS
arm64, and Windows x64, while `v0.13.1` publishes Apple XCFramework archives.
The native repo should not assume that upstream releases always publish the same
asset matrix.

The first production implementation should support both modes:
Production packaging should support both modes:

- consume official upstream artifacts when they exist and expose the required
ABI/API surface
Expand Down
15 changes: 9 additions & 6 deletions docs/platform_strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,24 @@ avoids publishing a second model wrapper ABI that does not add behavior.
The release automation publishes these runtime artifact groups:

- upstream LiteRT-LM C runtime libraries built from the tagged source archive,
with StreamProxy symbols embedded for downstream FFI streaming on source-built
platforms
with LiteRtLmBridge symbols embedded by the repo-owned `native/bridge` Bazel
package for downstream FFI streaming on source-built platforms
- upstream `prebuilt/` companion libraries copied from the tagged source archive
- official upstream release assets, including the iOS `CLiteRTLM.xcframework`
archive when Google publishes it
- iOS framework-style runtime wrappers derived from `CLiteRTLM.xcframework`,
with StreamProxy symbols embedded in `LiteRtLm.framework/LiteRtLm` and
with LiteRtLmBridge symbols embedded in `LiteRtLm.framework/LiteRtLm` and
upstream symbols re-exported from `CLiteRTLM.framework/CLiteRTLM`
- Apple Swift Package Manager XCFramework zips produced from the same iOS
wrappers, macOS source-built runtime, and macOS companion dylibs used by the
native-assets payloads

The upstream C runtime is the production FFI target for downstream packages. If
we later need a repo-owned compatibility layer, it should be introduced as a real
wrapper over upstream LiteRT-LM rather than a scaffold library.
The upstream C runtime is the production FFI target for downstream packages.
LiteRtLmBridge is limited to narrow FFI helpers around that runtime surface. It
currently exports the `stream_proxy_*` compatibility symbols used by downstream
streaming callbacks. Source-built native runtimes use Bazel `--package_path` to
build the repo-owned bridge package alongside upstream LiteRT-LM, without
patching upstream source files.

SPM artifacts are intentionally split by binary target. `LiteRtLm` carries the
primary iOS runtime/wrapper and a macOS framework wrapper around the
Expand Down
42 changes: 42 additions & 0 deletions native/bridge/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package(default_visibility = ["//visibility:public"])

cc_library(
name = "litert_lm_bridge",
srcs = ["litert_lm_bridge.c"],
alwayslink = True,
linkopts = select({
"@platforms//os:android": ["-ldl"],
"@platforms//os:linux": ["-ldl"],
"//conditions:default": [],
}),
)

cc_binary(
name = "libLiteRtLm.so",
linkshared = True,
deps = [
":litert_lm_bridge",
"//c:engine",
"//schema/capabilities:capabilities_c",
],
)

cc_binary(
name = "libLiteRtLm.dylib",
linkshared = True,
deps = [
":litert_lm_bridge",
"//c:engine",
"//schema/capabilities:capabilities_c",
],
)

cc_binary(
name = "LiteRtLm.dll",
linkshared = True,
deps = [
":litert_lm_bridge",
"//c:engine",
"//schema/capabilities:capabilities_c",
],
)
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
#include <stdlib.h>
#include <string.h>

// LiteRtLmBridge hosts downstream FFI bridge helpers. The stream_proxy_* exports
// are the current streaming callback ABI and stay stable for compatibility.
#if defined(_WIN32)
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#define STREAM_PROXY_EXPORT __declspec(dllexport)
#define LITERT_LM_BRIDGE_EXPORT __declspec(dllexport)
#else
#include <dlfcn.h>
#define STREAM_PROXY_EXPORT __attribute__((visibility("default")))
#define LITERT_LM_BRIDGE_EXPORT __attribute__((visibility("default")))
#endif

typedef void (*stream_proxy_callback_t)(
Expand Down Expand Up @@ -57,7 +59,7 @@ static void stream_proxy_forward(
error_copy);
}

STREAM_PROXY_EXPORT void *stream_proxy_load_global(const char *path) {
LITERT_LM_BRIDGE_EXPORT void *stream_proxy_load_global(const char *path) {
if (path == NULL) {
return NULL;
}
Expand All @@ -69,7 +71,7 @@ STREAM_PROXY_EXPORT void *stream_proxy_load_global(const char *path) {
#endif
}

STREAM_PROXY_EXPORT void *stream_proxy_create(
LITERT_LM_BRIDGE_EXPORT void *stream_proxy_create(
stream_proxy_callback_t dart_callback,
void *dart_data,
stream_proxy_callback_t *out_proxy_callback) {
Expand All @@ -89,10 +91,10 @@ STREAM_PROXY_EXPORT void *stream_proxy_create(
return context;
}

STREAM_PROXY_EXPORT void stream_proxy_delete(void *callback_data) {
LITERT_LM_BRIDGE_EXPORT void stream_proxy_delete(void *callback_data) {
free(callback_data);
}

STREAM_PROXY_EXPORT void stream_proxy_free_string(char *value) {
LITERT_LM_BRIDGE_EXPORT void stream_proxy_free_string(char *value) {
free(value);
}
Loading