Codeframe

Cluster, code, and tabulate open-ended survey responses, locally.

Codeframe takes a column of free text answers, groups them into themes, drafts an editable code frame (labels, descriptions, keywords, exemplars), applies it to every response, and writes a coded dataset, a frequency table, and exemplar quotes. It turns the chore that makes teams ignore open-ended questions into a few seconds of work.

It runs fully offline. The default pipeline uses TF-IDF and scikit-learn, so there is no model to download, no API key, and no data leaving your machine. Optional backends (sentence embeddings for semantics, or a local LLM via Ollama for assignment) can be switched on, but are never required.

The gap it fills

The tools that code open-ended survey responses well (Displayr, Ascribe, and the like) are proprietary, priced for enterprises, and cloud based, which rules them out for sensitive or consented survey data. General CAQDAS tools target deep coding of long transcripts, not high volume categorization of short answers, and generic clustering libraries do not produce a researcher style code frame with frequencies and exemplars. On GitHub there is no maintained open source product for this. Codeframe fills that gap. See ../OPPORTUNITIES.md for the evidence.

Full scope (what this project ships)

Core:

• Read responses from CSV, TSV, or XLSX, choosing the text column.
• Cluster responses into candidate themes (KMeans, with automatic selection of the number of themes by silhouette score, or a count you set).
• Draft a code frame: for each theme, a label and keywords from the most distinctive terms, a description, and exemplar responses.
• Save and load the code frame as human editable YAML (this is the review step).
• Apply a code frame to responses with multi label assignment and an uncertain bucket for low confidence rows.
• Output a coded dataset (original columns plus code ids, labels, an uncertain flag, and a binary column per code), a frequency table, exemplar quotes per code, and a JSON summary.

Two ways to run:

• codeframe code does everything in one step and assigns each response to the theme it was clustered into (complete, coherent coverage of your data).
• codeframe draft then codeframe apply lets you edit the code frame in between, and applies it by similarity so responses that do not fit any code go to the uncertain bucket.

Optional and opt in:

• A sentence embedding backend for semantic clustering and assignment.
• A local LLM assigner via Ollama.
• A Gradio dashboard to browse frequencies, exemplars, and the coded data.

Install

Requires Python 3.10 or newer (developed and tested on 3.14).

python -m venv .venv
# Windows PowerShell:  .venv\Scripts\Activate.ps1
# Git Bash / Linux / macOS:  source .venv/bin/activate
pip install -e .

Exact pinned environment:

pip install -r requirements-lock.txt
pip install -e . --no-deps

Optional extras:

pip install -e ".[embedding]"  # semantic backend (sentence-transformers)
pip install -e ".[dashboard]"  # Gradio review dashboard
pip install -e ".[dev]"        # pytest and hypothesis for the test suite

Usage

One step

codeframe code survey.csv --column feedback --clusters 3 --out out/

Writes to out/: codeframe.yaml, coded.csv, frequencies.csv, exemplars.csv, and summary.json. Omit --clusters to let it choose the number of themes automatically. Example frequency table:

code_id,label,count,percent
C3,Expensive,7,58.3
C1,Helpful,3,25.0
C2,Wait,2,16.7
UNCERTAIN,(uncertain),0,0.0

Draft, edit, then apply

# 1. Draft a code frame and open codeframe.yaml to edit labels and merge codes
codeframe draft survey.csv --column feedback --clusters 3 --out codeframe.yaml

# 2. Apply your edited code frame, writing a coded dataset and frequency table
codeframe apply survey.csv --column feedback \
  --codeframe codeframe.yaml --out coded.csv --freq frequencies.csv

Run python -m codeframe ... if you prefer not to install the console script.

Optional dashboard

pip install -e ".[dashboard]"
codeframe dashboard survey.csv --column feedback --clusters 3

Configuration

Settings come from defaults, an optional TOML file (--config), and environment variables. A TOML file:

[codeframe]
backend = "tfidf"          # or "embedding" for clustering
assigner = "keyword"       # or "embedding" or "llm"
n_clusters = 0             # 0 means auto select
assign_threshold = 0.18    # min similarity to assign a code
uncertain_threshold = 0.10 # below this, a response is uncertain
multi_label = true

Environment variables: CODEFRAME_BACKEND and CODEFRAME_OLLAMA_HOST (used only by the optional LLM assigner). No secrets are read or stored.

How the code frame is built and applied

The drafter names each cluster from its most distinctive TF-IDF terms and picks the responses closest to the cluster centre as exemplars. Each code is represented for matching by its label, keywords, and exemplar text, so the keyword assigner recognizes real wording (including synonyms in the exemplars), not just the label. Clustering quality improves with more responses; for small datasets, set --clusters or enable the embedding backend, and always review the drafted code frame before trusting the numbers.

How to run the tests

pip install -e ".[dev]"
pytest

The suite (35 tests) builds its own survey data (CSV and XLSX) in temporary directories and runs fully offline and deterministically. It covers data I/O and formula injection safety, vectorization, clustering, drafting, both assignment paths, YAML load safety, the full pipelines, the report outputs, the CLI, and the security properties below.

Architecture notes

• models.py the code, code frame, assignment, and coded dataset types.
• config.py backends, thresholds, and limits.
• io_data.py reading responses and writing sanitized output.
• vectorize.py TF-IDF and optional embedding vectorizers.
• cluster.py KMeans clustering with automatic K.
• draft.py building a code frame from clusters.
• codeframe_io.py YAML read and write of the code frame.
• assign.py similarity based and cluster based assignment.
• llm.py the optional Ollama assigner.
• pipeline.py the draft, apply, and end to end pipelines.
• report.py coded dataset, frequencies, exemplars, and summary.
• cli.py the Typer command line interface.

Security notes

• Spreadsheet formula injection is prevented. Survey text is user controlled, so a response beginning with =, +, -, @, or a control character could execute as a formula when the exported CSV or XLSX is opened in a spreadsheet. Every cell written to output is sanitized (dangerous cells are prefixed with a single quote). There are tests for this, including an end to end one.
• Safe YAML only. The code frame is read with yaml.safe_load, which cannot construct arbitrary Python objects, so a malicious code frame file cannot run code. There is a test that a !!python/object payload is rejected.
• Bounded input. Files larger than a configurable limit (default 50 MB) and row counts over a configurable cap are refused, bounding memory and time.
• Deterministic and offline by default. Clustering uses a fixed random seed, so runs are reproducible. No network calls happen unless you explicitly enable the LLM assigner, and then only to the local Ollama host you configure, with a timeout. The host is read from the environment and never logged.
• No unsafe execution. There is no eval, no exec, no pickle, and no yaml.load. Configuration is read with the standard library tomllib. XLSX is read with openpyxl, which does not execute macros.

License

Apache-2.0. See LICENSE.

Automatic coding is an aid, not a substitute for judgment. Review the drafted code frame and spot check the assignments, especially the uncertain bucket, before reporting results.