Skip to content

docs: README header covers all four model families (main) #710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JustVugg wants to merge 1 commit into from
Closed

docs: README header covers all four model families (main) #710

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 23 additions & 16 deletions README.it.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,32 @@
<a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.zh-TW.md">繁體中文</a> · Italiano
</p>

**Motore piccolo, modello immenso.** Esplora **GLM-5.2 (MoE da 744 miliardi di
parametri)** su hardware consumer ed eterogeneo — in C puro, senza dipendenze
del motore, trattando storage, RAM e VRAM come un'unica gerarchia di inferenza.

> **Colibrì è un motore di inferenza sperimentale e una piattaforma di ricerca.**
> Il suo obiettivo principale è migliorare le prestazioni di inferenza lungo
> l'intero confine software/hardware — formati dei modelli, gerarchia di memoria,
> I/O dello storage, piazzamento, scheduling, kernel, speculazione e sovrapposizione
> CPU/GPU — affinché i grandi modelli dipendano meno da hardware raro e costino meno.

Colibrì è intenzionalmente un luogo dove verificare idee di sistema aggressive,
non un runtime di produzione con SLA. Gli esperimenti devono dimostrare il proprio
valore con misure end-to-end riproducibili; la policy predefinita **non cambia mai
silenziosamente la precisione del modello né la semantica del router**. Una memoria
veloce insufficiente può ridurre la velocità, ma non ridefinire il modello di nascosto.
**Motore piccolo, modello immenso.** Esegui **modelli MoE di frontiera — da 744
miliardi a 2,8 mila miliardi di parametri** — su hardware consumer ed eterogeneo,
in C puro e senza dipendenze del motore, trattando storage, RAM e VRAM come
un'unica gerarchia di inferenza.

Oggi girano quattro famiglie: **GLM-5.2** (744B), **Inkling** (975B), **Kimi K3**
(2,8T) e **OLMoE** (7B) — un file C ciascuna, la stessa interfaccia `coli chat` /
`coli serve` / `coli web`. [Elenco completo](README.md#other-supported-models)

> **Colibrì è un motore di inferenza che puoi usare oggi, e una piattaforma di
> ricerca aperta.** Il suo obiettivo principale è migliorare le prestazioni di
> inferenza lungo l'intero confine software/hardware — formati dei modelli,
> gerarchia di memoria, I/O dello storage, piazzamento, scheduling, kernel,
> speculazione e sovrapposizione CPU/GPU — affinché i grandi modelli dipendano
> meno da hardware raro e costino meno.

Colibrì è intenzionalmente un luogo dove verificare idee di sistema aggressive —
quindi **nessuno SLA sulla velocità, e una garanzia dura sulla semantica**: gli
esperimenti devono dimostrare il proprio valore con misure end-to-end riproducibili;
la policy predefinita **non cambia mai silenziosamente la precisione del modello né
la semantica del router**. Una memoria veloce insufficiente può ridurre la velocità,
ma non ridefinire il modello di nascosto.

```
$ ./coli chat
🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
🐦 colibri v1.3.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
Expand Down
39 changes: 22 additions & 17 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,31 @@
English · <a href="README.zh-CN.md">简体中文</a> · <a href="README.zh-TW.md">繁體中文</a> · <a href="README.it.md">Italiano</a>
</p>

**Tiny engine, immense model.** Explore **GLM-5.2 (744B-parameter MoE)** across
consumer and heterogeneous hardware — in pure C, with zero engine dependencies,
by treating storage, RAM, and VRAM as one inference hierarchy.

> **Colibrì is an experimental inference engine and research platform.** Its
> primary goal is to pursue inference-side performance across the entire
> software/hardware boundary — model formats, memory hierarchy, storage I/O,
> placement, scheduling, kernels, speculation, and CPU/GPU overlap — so large
> models depend less on scarce hardware and cost less to run.

Colibrì treats VRAM, RAM, and storage as one managed memory hierarchy. It is
deliberately a place to test aggressive systems ideas, not a production runtime
with an SLA. Experiments must earn their place through reproducible end-to-end
measurements, and the default policy **never silently changes model precision
or router semantics**. Insufficient fast memory may reduce speed; it must not
quietly redefine the model.
**Tiny engine, immense model.** Run **frontier MoE models — 744B to 2.8T
parameters** — on consumer and heterogeneous hardware, in pure C with zero
engine dependencies, by treating storage, RAM, and VRAM as one inference
hierarchy.

Four families run today: **GLM-5.2** (744B), **Inkling** (975B), **Kimi K3**
(2.8T) and **OLMoE** (7B) — one C file each, the same `coli chat` /
`coli serve` / `coli web` front end. [Full roster ↓](#other-supported-models)

> **Colibrì is an inference engine you can run today, and an open research
> platform.** Its primary goal is to pursue inference-side performance across
> the entire software/hardware boundary — model formats, memory hierarchy,
> storage I/O, placement, scheduling, kernels, speculation, and CPU/GPU
> overlap — so large models depend less on scarce hardware and cost less to run.

Colibrì treats VRAM, RAM, and storage as one managed memory hierarchy, and it is
deliberately a place to test aggressive systems ideas — so there is **no SLA on
speed, and a hard guarantee on semantics**: experiments must earn their place
through reproducible end-to-end measurements, and the default policy **never
silently changes model precision or router semantics**. Insufficient fast memory
may reduce speed; it must not quietly redefine the model.

```
$ ./coli chat
🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
🐦 colibri v1.3.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
Expand Down
22 changes: 13 additions & 9 deletions README.zh-CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,24 @@
<a href="README.md">English</a> · 简体中文 · <a href="README.zh-TW.md">繁體中文</a> · <a href="README.it.md">Italiano</a>
</p>

**小巧引擎，庞大模型。**在消费级与异构硬件上探索 **GLM-5.2（744B 参数的
MoE）**——以引擎零依赖的纯 C 实现，将存储、RAM 与 VRAM 视为统一的推理层级。
**小巧引擎，庞大模型。**在消费级与异构硬件上运行**前沿 MoE 模型——从 744B 到
2.8T 参数**——以引擎零依赖的纯 C 实现，将存储、RAM 与 VRAM 视为统一的推理层级。

> **Colibrì 是一个实验性推理引擎与研究平台。**它的首要目标是在完整的软硬件边界上
> 追求推理侧性能——模型格式、内存层级、存储 I/O、放置、调度、内核、推测解码以及
> CPU/GPU 重叠执行——让大模型减少对稀缺硬件的依赖，并降低运行成本。
目前可运行四个模型家族：**GLM-5.2**（744B）、**Inkling**（975B）、**Kimi K3**
（2.8T）与 **OLMoE**（7B）——各自一个 C 文件，共用同一套 `coli chat` /
`coli serve` / `coli web` 前端。[完整列表](README.md#other-supported-models)

Colibrì 刻意用于验证激进的系统思路，而不是提供 SLA 的生产运行时。实验必须通过
可复现的端到端测量证明价值；默认策略**绝不会在未告知的情况下改变模型精度或路由语义**。
高速内存不足可以降低速度，但不能悄悄重新定义模型。
> **Colibrì 既是今天就能运行的推理引擎，也是一个开放的研究平台。**它的首要目标是在
> 完整的软硬件边界上追求推理侧性能——模型格式、内存层级、存储 I/O、放置、调度、内核、
> 推测解码以及 CPU/GPU 重叠执行——让大模型减少对稀缺硬件的依赖，并降低运行成本。

Colibrì 刻意用于验证激进的系统思路——因此**对速度不作 SLA 承诺，对语义则给出硬性保证**：
实验必须通过可复现的端到端测量证明价值；默认策略**绝不会在未告知的情况下改变模型精度或
路由语义**。高速内存不足可以降低速度，但不能悄悄重新定义模型。

```
$ ./coli chat
🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
🐦 colibri v1.3.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
Expand Down
22 changes: 13 additions & 9 deletions README.zh-TW.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,24 @@
<a href="README.md">English</a> · <a href="README.zh-CN.md">简体中文</a> · 繁體中文 · <a href="README.it.md">Italiano</a>
</p>

**小巧引擎，龐大模型。**在消費級與異質硬體上探索 **GLM-5.2（744B 參數的
MoE）**——以引擎零相依套件的純 C 實作，將儲存、RAM 與 VRAM 視為統一的推論階層。
**小巧引擎，龐大模型。**在消費級與異質硬體上執行**前沿 MoE 模型——從 744B 到
2.8T 參數**——以引擎零相依套件的純 C 實作，將儲存、RAM 與 VRAM 視為統一的推論階層。

> **Colibrì 是一個實驗性推論引擎與研究平台。**它的首要目標是在完整的軟硬體邊界上
> 追求推論側效能——模型格式、記憶體階層、儲存 I/O、配置、排程、核心、推測解碼以及
> CPU/GPU 重疊執行——讓大型模型減少對稀缺硬體的依賴，並降低執行成本。
目前可執行四個模型家族：**GLM-5.2**（744B）、**Inkling**（975B）、**Kimi K3**
（2.8T）與 **OLMoE**（7B）——各自一個 C 檔案，共用同一套 `coli chat` /
`coli serve` / `coli web` 前端。[完整清單](README.md#other-supported-models)

Colibrì 刻意用於驗證激進的系統構想，而不是提供 SLA 的生產執行環境。實驗必須透過
可重現的端到端測量證明價值；預設策略**絕不會在未告知的情況下改變模型精度或路由語意**。
高速記憶體不足可以降低速度，但不能悄悄重新定義模型。
> **Colibrì 既是今天就能執行的推論引擎，也是一個開放的研究平台。**它的首要目標是在
> 完整的軟硬體邊界上追求推論側效能——模型格式、記憶體階層、儲存 I/O、配置、排程、核心、
> 推測解碼以及 CPU/GPU 重疊執行——讓大型模型減少對稀缺硬體的依賴，並降低執行成本。

Colibrì 刻意用於驗證激進的系統構想——因此**對速度不作 SLA 承諾，對語意則給出硬性保證**：
實驗必須透過可重現的端到端測量證明價值；預設策略**絕不會在未告知的情況下改變模型精度或
路由語意**。高速記憶體不足可以降低速度，但不能悄悄重新定義模型。

```
$ ./coli chat
🐦 colibri v1.1.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
🐦 colibri v1.3.0 — GLM-5.2 · 744B MoE · int4 · streaming CPU
✓ ready in 32s · resident 9.9 GB
› ciao!
◆ Ciao! 😊 Come posso aiutarti oggi?
Expand Down
Loading