Multimodal AI Study Assistant

Handwriting → Notes → Flashcards + Quiz + Mind Maps

This project is an AI-powered study assistant that converts handwritten notes into structured learning material. It uses Qwen2.5-VL for handwriting OCR and Gemini for intelligent post-processing and study-tool generation.

---

Features

✍️ Handwriting OCR

• Upload notebook images.
• Uses Qwen2.5-VL to transcribe handwritten text.
• Preserves line breaks and document structure.

🤖 AI Correction

• Optional Gemini-powered OCR cleanup.

• Fixes:

  • OCR mistakes
  • Character substitutions
  • Misspelled words
  • Formatting issues

📝 Notes Management

• Save extracted text as study notes.
• Persistent local storage using JSON.
• Delete individual notes.
• Clear all notes.

🃏 Flashcard Generation

Generate AI-powered flashcards from saved notes.

Each flashcard contains:

• Front (question)
• Back (answer)

Useful for active recall and spaced repetition.

❓ Quiz Generation

Generate multiple-choice quizzes from notes.

Each question contains:

• Question
• Four answer choices
• Correct answer
• Explanation

🔍 Misconception Detection

Gemini analyzes notes and identifies:

• Misconceptions
• Missing concepts
• Ambiguous explanations
• Areas needing clarification

🗺️ Mind Map Generation

Automatically creates a visual SVG mind map showing:

• Central topic
• Major concepts
• Subtopics
• Concept relationships

🌐 Local Web Interface

Built with Flask and a fully custom HTML/CSS/JavaScript frontend.

---

Architecture

Notebook Image
      │
      ▼
Qwen2.5-VL OCR
      │
      ▼
Raw Text
      │
      ▼
Gemini Correction
      │
      ▼
Saved Notes
      │
      ├── Flashcards
      ├── Quiz
      ├── Misconception Analysis
      └── Mind Map

---

Project Structure

.
├── app.py
├── backend.py
├── inkwell_notes.json
├── README.md
└── requirements.txt

---

Requirements

Python

Recommended:

Python 3.10+

---

Install Dependencies

pip install flask
pip install torch torchvision accelerate transformers
pip install bitsandbytes
pip install google-genai
pip install qwen-vl-utils

---

Environment Variables

Gemini API Key

Windows:

set GEMINI_API_KEY=your_key_here

Linux / macOS:

export GEMINI_API_KEY=your_key_here

---

Optional Qwen Model

Default:

Qwen/Qwen2.5-VL-7B-Instruct

Override:

export QWEN_MODEL_PATH=Qwen/Qwen2.5-VL-(2/3)B-Instruct

or

set QWEN_MODEL_PATH=Qwen/Qwen2.5-VL-(2/3)B-Instruct

---

Running

Start the application:

python app.py

Open the URL in your browser.

---

API Endpoints

OCR

POST /api/ocr

Upload:

image=<file>

Response:

{
  "text": "...",
  "stats": "..."
}

---

Get Notes

GET /api/notes

---

Save Note

POST /api/notes

Body:

{
  "text": "my note"
}

---

Delete Note

DELETE /api/notes/<index>

---

Clear Notes

POST /api/notes/clear

---

Generate Flashcards

POST /api/flashcards

---

Generate Quiz

POST /api/quiz

---

Analyze Misconceptions

POST /api/misconceptions

---

Generate Mind Map

POST /api/mindmap

---

Notes Storage

All notes are stored locally in:

inkwell_notes.json

Example:

[
  {
    "id": 0,
    "title": "Introduction to Biology",
    "text": "...",
    "date": "Jun 13, 2026"
  }
]

---

Models Used

OCR

Qwen2.5-VL

• Vision-language model
• Handwriting transcription
• Local inference

Reasoning & Generation

Gemini 2.5 Flash

Used for:

• OCR correction
• Flashcards
• Quizzes
• Misconception analysis
• Mind maps

---

Future Improvements

• PDF upload support
• Spaced repetition scheduling
• User authentication
• Multi-user support
• Note search
• Study analytics dashboard
• Dark mode
• Mobile-responsive UI improvements

---