HandGen — Handwriting Generator

Type text and render it onto a lined notebook page — either in a handwriting font of your choice, or as AI-synthesized cursive that draws your actual words stroke by stroke using an Alex Graves recurrent neural network — then view the page as a 3D crumpled sheet of paper lit with Three.js.

Live demo: https://handgen.onrender.com

Hosted on Render's free tier, so the first request after idle takes ~30 s to wake the server. Once it's up, generation is immediate.

---

Interface

The UI was redesigned from the original prototype into a futuristic, mobile-responsive "handwriting studio" — a dark glassmorphism theme with neon cyan/violet accents — generated with Google Stitch (via its MCP server) and then hand-integrated so all the original functionality kept working. The Django app now serves this single, unified UI.

Before → After (desktop)

Original prototype	Redesigned (Stitch)

Mobile (responsive)

Original on mobile	Redesigned on mobile

The Stitch design concept the integration was based on is saved at screenshots/stitch-mockup.png.

Features

• AI handwriting synthesis ("Use AI"). Renders your typed text as genuine, flowing cursive using a pretrained Alex Graves handwriting-synthesis network — run entirely in your browser via onnxruntime-web, so the demo needs no GPU and no per-request server compute. Text is split into short chunks, each laid out along the ruled lines and wrapped to fill the page. A Bias slider controls neatness (higher = steadier, more legible strokes). Progress shows in an off-canvas status pill so the paper stays clean.
• Handwriting fonts. Eight Google Fonts pre-loaded (Pacifico, Caveat, Architects Daughter, Cookie, Covered By Your Grace, Gochi Hand, Great Vibes, Homemade Apple).
• Upload your own font. Drop in a .ttf or .otf and the text re-renders in it. The font stays local to your browser — nothing is uploaded.
• Adjustable font size and skew. Sliders for both.
• Notebook-paper canvas. Red header and margin rules, evenly-spaced blue horizontal rules, gray paper background, and a "Date://___" stamp.
• 3D crumpled paper view. Click Render 3D to map the 2D notebook canvas onto a 99×99 plane in Three.js, with vertex displacement from a mix of Perlin, Simplex, and Worley noise to simulate folds and wrinkles. Orbit by dragging inside the 3D view; press Close or Esc to exit.
• Responsive — works on desktop and mobile.

How "Use AI" works — Alex Graves handwriting synthesis

The AI mode implements the handwriting model from Alex Graves' Generating Sequences With Recurrent Neural Networks (2013, arXiv:1308.0850). The paper describes two related networks, and this project contains both:

The two networks

• Prediction network — learns the dynamics of a pen. Given the strokes so far, it predicts a probability distribution over the next pen move. It produces convincing handwriting-like motion but does not spell out specific text.
• Synthesis network — adds a learned attention window that slides along the input characters, so the strokes actually transcribe the requested words. This is what powers the "Use AI" button.

The core idea (mixture density output)

Handwriting is a sequence of tiny pen moves (Δx, Δy, pen_up). Predicting the single next point with squared error fails — after any stroke the next move is genuinely ambiguous, and the average of the options is a dead blur. So instead the network, at every timestep, outputs the parameters of a mixture of 2-D Gaussians plus a pen-lift probability:

• e — probability the pen lifts (sigmoid),
• π₁…π_M — mixture weights (softmax, sum to 1),
• μ, σ, ρ per component — each Gaussian's centre, spread, and correlation.

It's trained to maximise the likelihood of the true next move (negative-log-likelihood loss). To draw, you sample a move from this distribution, feed it back in, and repeat — the pen draws itself. Graves' bias b ≥ 0 (the UI's Bias slider) shrinks every σ → σ·e^(−b) and sharpens the mixture choice, trading variety for neatness.

What runs in the browser

docs-free and serverless on the client side: onnx_synthesis.js drives a pretrained synthesis model (synthesis_network_52.onnx) step-by-step through onnxruntime-web:

• 3 stacked LSTM layers (400 units each),
• a soft attention window of 10 Gaussian components whose centre κ only ever moves forward along the one-hot character string (φ(t,u) = Σ αᵏ exp(−βᵏ(κᵏ−u)²)), telling the net which character it is currently drawing,
• a 20-component bivariate mixture-density output + pen-lift, sampled each step (Box–Muller + Cholesky for the correlated Gaussian).

The model and inference recipe are adapted from the MIT-licensed pytorch-handwriting-synthesis-toolkit by Evgenii Dolotov (the only change is a built-in bivariate sampler replacing the multivariate-normal dependency). Layout and two legibility tricks live in CanvasScript.js: a trailing drift-trim removes the stray slant the model leaves after a word, and a "sacrificial successor" trick — append a throwaway glyph, then cut the strokes at the exact step the attention window hands off to it — keeps the final letter of every chunk complete.

From scratch, by hand (the part I'm proudest of)

HandGen/HandWriter/handwriting_rnn/ is a pure-NumPy implementation of the prediction network — no PyTorch, no TensorFlow. The LSTM forward/backward pass, the mixture-density layer, the NLL loss, the hand-derived backpropagation, an Adam optimiser, and gradient clipping are all written out explicitly, and the analytic gradients are confirmed against finite differences to ~1e-10 by gradcheck.py. It's a 1-LSTM-layer (100 hidden units), 20-mixture model trained on procedurally-generated strokes. The goal was to understand every equation in the paper by building it, not importing it. Full write-up: handwriting_rnn/README.md.

The two engines, and how they're wired

Engine	Where	What it does
Pretrained synthesis (ONNX)	in the browser	Primary. Writes your actual typed text.
Pretrained synthesis (PyTorch)	server /writer/api/generate	Fallback if the browser path is unavailable and PyTorch is installed.
From-scratch NumPy prediction	server /writer/api/generate	Last-resort fallback (handwriting-like motion, doesn't transcribe).

The server endpoint (views.py → generate) is only used as a fallback; on the live demo the in-browser ONNX path is what runs, so the model executes on your machine and the server just serves static files.

Tech stack

• Frontend: vanilla JS, jQuery, Tailwind CSS (CDN), Three.js r84, noisejs.
• AI inference: onnxruntime-web (WASM) in the browser, running a pretrained Graves synthesis model. Uses multi-threaded WASM when the page is cross-origin isolated, single-threaded otherwise (always functional).
• Design: Google Stitch generated the glassmorphism design system ("Digital Ink Synthesis").
• Backend: Django 3.1 (Python 3.7+), serving the UI and an /api/generate fallback endpoint. Deployed on Render via render.yaml with gunicorn + WhiteNoise; SECRET_KEY/DEBUG read from environment variables.

How it works (rendering pipeline)

1. Text → 2D canvas (fonts). On Write, CanvasScript.js paints the gray paper, red margin/header rules and blue lines, then renders each line in the chosen font with ctx.fillText.
2. Text → 2D canvas (AI). With Use AI on, the text is chunked and each chunk is generated stroke-by-stroke by the synthesis network, then scaled to the ruled-line height, placed on the next line, and wrapped — drawing real connected cursive.
3. 2D canvas → 3D texture. On Render 3D, the canvas becomes a THREE.CanvasTexture on a MeshPhongMaterial over a PlaneGeometry(5, 5, 99, 99).
4. Procedural crumpling. Each of the plane's 10 000 vertices gets a Z offset from a Worley fold-distance term + low-frequency Perlin + mid-frequency Simplex + five octaves of Simplex (Brownian) detail.
5. Camera + lighting. A PerspectiveCamera, an AmbientLight, a white PointLight, and OrbitControls scoped to the renderer canvas.

Running locally

pip install "Django>=3.1,<3.2"
cd HandGen
python manage.py migrate          # creates the local SQLite db
python manage.py runserver
# then visit http://127.0.0.1:8000/

The "Use AI" button runs in the browser, so it works locally with no extra setup. (Multi-threaded WASM only engages when the page is cross-origin isolated; locally it falls back to single-threaded — slower but fully functional.)

Project structure

.
├── HandGen/                              # Django project
│   ├── manage.py
│   ├── HandGen/                          # project config (settings, urls, wsgi)
│   └── HandWriter/                       # the single Django app
│       ├── views.py                      # UI view + /api/generate fallback endpoint
│       ├── templates/HandWriter/base.html# the unified (redesigned) UI
│       ├── static/assets/                # JS engine + AI model:
│       │   ├── CanvasScript.js           #   2D canvas + AI layout engine
│       │   ├── onnx_synthesis.js         #   in-browser Graves synthesis (onnxruntime-web)
│       │   ├── synthesis_network_52.onnx #   pretrained synthesis weights
│       │   ├── three.js / OrbitControls.js / logic.js / main.css
│       ├── handwriting_rnn/              # from-scratch NumPy Graves prediction net (gradient-checked)
│       └── synthesis/                    # vendored pretrained PyTorch model (server fallback)
│
├── screenshots/                          # before/after UI images used by this README
├── render.yaml                           # Render.com Blueprint (gunicorn + WhiteNoise)
├── .mcp.json                             # Stitch MCP server config (remote HTTP endpoint)
├── refresh-stitch-token.ps1              # refreshes the hourly Stitch access token
└── README.md

Known issues / limitations

• AI synthesis is best-effort. The pretrained model occasionally renders an uncommon word as a scribble or a slightly malformed glyph; the layout retries obvious collapses and trims drift, but it isn't perfect on every word.
• Style slider is not wired — only the Bias slider currently affects AI output.
• Skew slider value is read into a JS variable but never applied to the canvas transform.
• SECRET_KEY has a public dev fallback in HandGen/HandGen/settings.py. Production reads DJANGO_SECRET_KEY from the environment (Render injects a generated one); the committed fallback must never be used for a real deployment.

Roadmap

• Wire the Style slider to a learned style/primer for the synthesis network, and the Skew slider to ctx.transform.
• Voronoi crease network for sharper, more paper-like folds.
• PNG export of the 3D rendered view.
• More paper presets: graph, dotted, parchment, legal-pad yellow.
• Migrate Three.js off r84.

Stitch MCP setup (for redesign work)

The futuristic UI was generated through Google Stitch's MCP server. To drive Stitch yourself:

1. Have a Google Cloud project with the Stitch API enabled, and gcloud installed and authenticated (gcloud auth login).
2. .mcp.json points your MCP client at the official remote endpoint https://stitch.googleapis.com/mcp using a bearer token from the STITCH_TOKEN env var.
3. The token expires hourly — run ./refresh-stitch-token.ps1 to refresh it, then restart your MCP client.

No credentials are committed; the service-account key and token are gitignored / env-only.

License

Not yet specified. Until a license is added, default copyright applies — please do not redistribute without permission.