Skip to content

README.md

Latest commit

 

History

History
66 lines (50 loc) · 3.61 KB

README.md

File metadata and controls

66 lines (50 loc) · 3.61 KB
Raw

🧠 Tiny Transformer on Arduino UNO 🚀

Arduino UNO Transformer RAM 2KB Flash 32KB
A microscopic Generative AI model running entirely on an 8-bit microcontroller.

🌟 Overview

Have you ever wondered if an Arduino UNO with its humble 16 MHz clock speed and 2 KB of RAM could run a modern AI architecture? The answer is... YES!

This project implements a highly optimized, fully functional Transformer Neural Network directly in C/C++ for the Arduino platform. It performs text completion character-by-character, right on the edge.

✨ Features

  • 🧩 Self-Attention Mechanism: A simplified version of the core building block behind ChatGPT and other large language models.
  • 📉 Int8 Quantization: Neural network weights are quantized to 8-bit integers (int8_t) to fit into the microscopic Flash memory limits.
  • 💾 PROGMEM Storage: Model weights are stored in the Arduino's Flash memory mapping, preserving valuable SRAM for intermediate calculations.
  • 🎲 Temperature Sampling: Includes a stochastic text generation loop to produce creative and randomized outputs instead of a deterministic loop.
  • ⚡ Zero External Dependencies: Pure math, pure C/C++. Doesn't require Wi-Fi, external RAM, or an SD card. Completely standalone offline inference!

🛠️ Hardware Requirements

  • Arduino UNO R3 (or any ATmega328P based board)
  • No extra shields or sensors are required! (We use analog noise from an unconnected A0 pin to seed randomness).

🧮 How it Works Under the Hood

  1. Embedding: The input text is tokenized into IDs and passed alongside internal positional embeddings.
  2. Attention Phase: Matrix-vector multiplications compute Query (Q), Key (K), and Value (V) arrays. It learns the "context" behind the letters.
  3. Feed Forward Network: A small Multi-Layer Perceptron (MLP) mapping adds non-linear activations.
  4. De-quantization: Real-time conversion interpolates 8-bit integers back to floating points (float) using specific scaling factors.
  5. Softmax Output: Computes the final probability distribution of characters in the minuscule 32-character vocabulary.

🚀 Getting Started

  1. Clone the repository.
  2. Open tiny_transformer_uno.ino in the standard Arduino IDE.
  3. Ensure that tiny_model.h is present in the same sketch folder.
  4. Upload the sketch to your Arduino UNO.
  5. Open the Serial Monitor (Set BAUD rate to 115200 and ending to Newline or Both NL & CR).

💬 Usage

Just type lowercase English text (or Translit) and hit Enter. The model will respond based on its pre-trained corpus distributions.

Tiny Transformer LLM on Arduino UNO
Type lowercase english/translit text and press Enter.
Example: hello | privet kak dela | arduino uno

you> hello
tiny-llm>  world how are you

Note: This model is incredibly small. Expect it to act more like a clever Markov Chain / predictive keyboard than a true conversational agent.

🤝 Roadmap / Future Ideas

  • Explore larger models on the ESP32 (more RAM/Flash).
  • Enhance training corpus and optimize tokenization (BPE?).
  • Add int4 quantization support to fit a slightly bigger model into the UNO.

⚖️ License

This project is open-source. Feel free to fork, experiment, and push the limits of what microcontrollers can do!