QUAYNOR

A lightweight, blazing fast AI inference engine written in Rust.

Embed local LLMs in your app: load GGUF checkpoints, chat on-device or on the GPU, and keep data off the cloud. Bindings available for Python, Flutter, React Native, and Swift.

Documentation: www.quaynor.site

At a glance

• Runtime: Rust core that loads GGUF models and using Vulkan or Metal where the platform enables GPU backends.
• Features: Chat with streaming completions, Minijinja chat templates, tokenizer helpers, optional embeddings and cross-encoder reranking, and GBNF grammar-based tool calling—surfaced consistently across bindings where supported.
• This repo: Shared engine and most bindings live under quaynor/ as a Cargo workspace; the Swift layer is a separate SwiftPM package under quaynor/swift/. For module layout and contribution workflow, see dev_guide.md.

---

Why use it

• Offline inference - No inference API keys; models stay on disk or load from URLs / Hugging Face paths you choose.
• One chat-style API across bindings - Chat/ask patterns align so you can move ideas between Python, Flutter, React Native, and Swift without relearning primitives.
• Production-oriented features - Streaming replies, bounded context sizing, embeddings and cross-encoder reranking where supported, grammar-based tool calling wired from native functions (Python) or equivalents in mobile bindings.

Rough capability map:

Area	Notes
Chat & streaming	Token streaming and full-string completion helpers (e.g. .completed()).
Tool calling	Grammar-constrained tool use; decorate Python functions or declare tools in RN/Flutter per docs.
Hardware	Vulkan (desktop/Android where applicable), Metal (Apple).
Modalities	Vision and audio pipelines where the model supports them; see docs for model quirks.

---

Flutter

quaynor on pub.dev

flutter pub add quaynor

import 'package:quaynor/quaynor.dart' as quaynor;

void main() async {
  await quaynor.Quaynor.init();
  final chat = await quaynor.Chat.fromPath(
    modelPath: 'huggingface:bartowski/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf',
  );
  final msg = await chat.ask('Is a zebra black or white?').completed();
  print(msg);
}

---

React Native

react-native-quaynor on npm

npm install react-native-quaynor

import { Chat } from "react-native-quaynor";

const chat = await Chat.fromPath({
  modelPath: "hf://bartowski/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf",
});

const msg = await chat.ask("Is a zebra black or white?").completed();
console.log(msg);

---

Python

quaynor on PyPI

pip install quaynor

from quaynor import Chat

chat = Chat("huggingface:bartowski/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf")
for token in chat.ask("Is a zebra black or white?"):
    print(token, end="", flush=True)

Wait for the full string with .completed():

full = chat.ask("Why is the sky blue?").completed()

Tool calling:

import math
from quaynor import tool, Chat

@tool(description="Area of a circle from radius")
def circle_area(radius: float) -> str:
    return f"{math.pi * radius ** 2:.2f}"

chat = Chat("./path/to/model.gguf", tools=[circle_area])

---

Platforms

Desktop (Windows, Linux, macOS): Python, Flutter, and React Native. Swift package distribution is available for Apple platforms through Swift Package Manager. Android and iOS: Flutter and React Native bindings (Metal on Apple, Vulkan where enabled on Android).

---

Swift

Swift Package docs

SwiftPM is Apple's package manager for Swift and Xcode projects.

dependencies: [
    .package(url: "https://github.com/iBz-04/quaynor.git", from: "0.1.2")
]

import Quaynor

let model = try await Model.fromPath(
    path: "huggingface:bartowski/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf"
)
let chat = Chat(model: model)
let text = try await chat.ask("Is a zebra black or white?").completed()
print(text)

The Swift package is distributed through Swift Package Manager for iOS and macOS using the published QuaynorFFI.xcframework release artifact. The public Swift API includes Model, Chat, TokenStream, Encoder, CrossEncoder, Prompt, SamplerPresets, Tool, CachedModel, and ChatStats. Model also supports Hugging Face downloads, cache inspection with getCachedModels(), and cache deletion with deleteCachedModel(modelPath:).

---

Contributing

Issues, discussions, and PRs are welcome. For codebase layout and conventions, see dev_guide.md. Setup and binding-specific guides live on www.quaynor.site and in repo labels.

---

License

GNU GENERAL PUBLIC LICENSE