InsightCoder

InsightCoder is a lightweight research product for coding open-ended survey and interview responses. It turns a CSV dataset into suggested qualitative codes, evidence snippets, a theme matrix, and an exportable analysis draft.

The project is designed around a common research workflow: collect enterprise survey text, build an initial code frame, let AI suggest codes with evidence, review the suggestions manually, and export structured results for analysis.

Why This Project Exists

Open-ended enterprise surveys often contain valuable but messy evidence. Researchers need more than a generic chatbot summary:

• every code should be traceable to a sentence-level evidence snippet;
• the code frame should be editable by the researcher;
• AI output should stay in a human review workflow;
• results should export to CSV and Markdown for follow-up analysis.

InsightCoder demonstrates this workflow with the theme of how generative AI affects enterprise labor demand, skills, workflows, compliance, and hiring.

Features

• CSV import for open-ended responses.
• Editable code frame with keyword, semantic, and hybrid coding modes.
• Codebook boundaries with positive examples, negative examples, and exclusion rules.
• Evidence-first suggestions with confidence scores.
• Human review status, final codes, and reviewer notes.
• Theme matrix grouped by industry.
• CSV export for coded responses.
• Topic-variable export for Excel, Python, or Stata analysis.
• Markdown report export for analysis drafts.
• Manual-label evaluation with exact match, precision, recall, and F1.
• Dependency-free Python CLI for reproducible batch processing.
• Optional LM Studio / OpenAI-compatible local embedding backend.
• Optional sentence-transformers backend for offline embedding inference.
• Static browser demo that can run directly from app/index.html.

Project Structure

InsightCoder/
├── app/                         # Static web demo
├── data/                        # Sample enterprise survey and gold-label datasets
├── docs/                        # Method notes and product brief
├── src/                         # Python CLI and core coding logic
└── tests/                       # Unit tests

Quick Start

Run the Python CLI:

python3 src/insightcoder.py data/sample_enterprise_ai_survey.csv \
  -o exports/coded_responses.csv \
  --variables exports/topic_variables.csv \
  --report exports/report.md \
  --gold-labels data/manual_labeled_sample.csv \
  --evaluation exports/evaluation.md \
  --json

The default CLI engine is hybrid: a transparent keyword baseline plus a local lightweight semantic embedding baseline. This default does not call OpenAI or any cloud model.

Use keyword-only mode:

python3 src/insightcoder.py data/sample_enterprise_ai_survey.csv --engine keyword

Use a local LM Studio / OpenAI-compatible embedding server:

python3 src/insightcoder.py data/sample_enterprise_ai_survey.csv \
  --engine hybrid \
  --embedding-backend lm-studio \
  --embedding-url http://localhost:1234/v1 \
  --model-name text-embedding-nomic-embed-text-v1.5 \
  --top-k 1

Use optional sentence-transformers embeddings:

pip install -r requirements-optional.txt
python3 src/insightcoder.py data/sample_enterprise_ai_survey.csv \
  --engine hybrid \
  --embedding-backend sentence-transformers \
  --model-name paraphrase-multilingual-MiniLM-L12-v2

Run tests:

python3 -m unittest discover -s tests

Open the static demo:

app/index.html

Review Loop

InsightCoder keeps AI suggestions separate from researcher's final labels:

• ai_code_ids: model-generated candidate codes.
• final_code_ids: manually reviewed final codes.
• review_status: pending, accepted, revised, or rejected.
• review_note: reason for accepting or changing the suggestion.

The web demo lets the reviewer remove or add final codes for each response. The CLI can also consume gold_code_ids or a separate gold-label CSV for evaluation.

See docs/method-workflow.md for the full method pipeline.

Research Exports

The CLI can export two analysis-ready files:

• coded_responses.csv: original text plus AI codes, evidence, final codes, and review metadata.
• topic_variables.csv: one-hot topic variables such as efficiency_gain, data_security, and role_restructuring.

This makes the output usable in Excel, Python, or Stata.

Input Format

The default CSV expects these fields:

• enterprise_id
• industry
• company_size
• role_group
• response

You can pass another text column to the CLI with --text-field.

Product Positioning

InsightCoder is not positioned as a full academic CAQDAS replacement or a fine-tuned BERT classifier. It is a focused product prototype for one high-frequency job-to-be-done: helping researchers and analysts convert open-ended survey evidence into reviewable structured themes.

The current version uses a hybrid architecture: a keyword baseline for interpretability, a local lightweight embedding baseline for semantic matching without model downloads, and an optional sentence-transformers backend for real embedding inference when the user has the model environment ready.

When a local OpenAI-compatible server is available, InsightCoder can use its /v1/embeddings endpoint. In the current local setup this was tested with LM Studio exposing text-embedding-nomic-embed-text-v1.5. This is not an OpenAI model; it is a local embedding backend behind an OpenAI-compatible API shape. For this model, --top-k 1 is recommended in the sample dataset because broader semantic recall can introduce too many false positives.

The current version also includes a small manually labeled validation sample. This is intentionally small, but it demonstrates the expected research workflow: AI proposes codes, the researcher reviews them, and the system reports exact match, precision, recall, and F1 before those labels become variables.

The long-term version would add LLM-assisted codebook generation, supervised classification after enough reviewed labels accumulate, inter-coder agreement, project-level audit logs, and source document management.