Skip to content

Add LiteRT-LM LoRA C API support#7

Open
leehack wants to merge 2 commits into
mainfrom
litert-lora-support
Open

Add LiteRT-LM LoRA C API support#7
leehack wants to merge 2 commits into
mainfrom
litert-lora-support

Conversation

@leehack

@leehack leehack commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a build-time upstream patch that exports litert_lm_session_config_set_lora_file and litert_lm_session_config_set_audio_lora_file
  • wire scoped text LoRA files through resource management into the LiteRT compiled-model executor
  • preserve active LoRA IDs across context clone/restore and inject active LoRA buffers into decode inputs

Why

LiteRT-LM v0.13.1 has internal LoRA manager/session-config support, but the public C wrapper and text executor path do not expose enough behavior for Dart FFI clients. This enables the native runtime bundle needed by llamadart to support one LiteRT-LM LoRA adapter at scale 1.0.

Companion llamadart PR: leehack/llamadart#206

Validation

  • python3 -m py_compile tools/build_upstream_runtime.py
  • python3 tools/build_upstream_runtime.py --upstream-tag v0.13.1 --platform macos --arch arm64 --jobs 8
  • nm -gU bin/macos/arm64/libLiteRtLm.dylib | rg 'litert_lm_session_config_set_(audio_)?lora_file|litert_lm_conversation_create'
  • git diff --check

Runtime smoke used by companion PR

With this locally built macOS runtime, llamadart generated successfully through both ModelParams.loras and engine.setLora(...) using:

  • model: /private/tmp/litert-lm-v0.13.1-src/runtime/testdata/test_lm.litertlm
  • adapter: /private/tmp/litert-lm-v0.13.1-src/runtime/testdata/test_lora_rank32_f16_all_ones.tflite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant