Refactor backend structure, add user providers, notes/chat/auth routes and improved OCR/model pipeline

[HaiweiZuo]

Motivation

• Restructure the backend into clearer packages (core, agents, routes) and move runtime/env handling into backend/.env.example so configuration and provider credentials are managed properly.
• Add user-level provider configuration and persistence in the database so each user can manage their own LLM/OCR providers instead of relying on a global .env key.
• Provide new features: notebook (notes) agent + API, richer chat/teaching flows, authentication with email verification, and erase/ensexam inference for pre-processing images to improve OCR quality.
• Improve OCR→LLM workflow robustness (batching, overlap handling, deduplication, image path normalization and export correctness) to reduce errors caused by cross-page content and LLM path mangling.

Description

• Reorganized code layout: moved config.py, llm.py, state.py, and mail helpers into backend/core/; moved agent modules under backend/agents/; added backend/routes/ with Blueprints (auth, upload, questions, chat, stats, settings, notes) and a route registration helper.
• Database and CRUD refactor: reworked db/ into modular crud submodules (users, questions, providers, notes, chat, split_records, stats, email_verification, tags) and extended ORM models with ProviderConfig, Note, NoteTagMapping, EmailVerification, ChatSession changes, and session_version on User.
• New APIs and features: authentication endpoints with email verification (/api/auth), user provider settings (/api/settings/config), notes pipeline and endpoints (/api/notes + agents.note), chat sessions/messages/streaming with SSE and teach agent integration, upload/erase/ocr/split/export flow implemented in routes/upload.py.
• OCR / Agent / Workflow improvements: caching OCR results, building overlapping batches with is_primary flags, new deduplication using text similarity and section-aware rules, propagation and normalization of section_title, fixing leading images ownership, and normalizing image paths (e.g. imgs/ → /images/).
• Model / inference updates: added EnsExam generator/refine network rework, an InferenceEngine for erase model with lazy loading and sliding-window inference, included model weight managed via Git LFS (backend/models/weight/best.pth), and adjusted test scripts and configs accordingly.
• Misc: removed top-level .env.example in favor of backend/.env.example, added .gitattributes for LFS, updated .gitignore, refreshed docs (README.md, CLAUDE.md) to reflect architecture and operational changes, and added backend/package-lock.json.

Testing

• Ran backend unit tests with cd backend && pytest tests/ -q, which completed successfully.
• Executed the models inference smoke test script backend/models/test_erase.py to validate the EnsExam inference changes, which ran without errors on the test environment.
• Performed basic API smoke checks for upload/ocr/split and notes creation (local manual runs) to verify end-to-end flow; no automated failures encountered.

---

Codex Task