VCM-B: Vietnamese Question Answering System

An intelligent multi-pipeline QA system built for the VNPT AI Hackathon 2025

Features • Architecture • Installation • Usage • Configuration

---

Overview

VCM-B is a high-performance Vietnamese Question Answering system designed to handle multiple question types efficiently. It leverages dual-thread parallel processing, Retrieval-Augmented Generation (RAG), and specialized pipelines for different question categories.

Features

• Smart Question Routing - Automatically classifies questions into 3 specialized pipelines
• Math Pipeline - Handles mathematical calculations with code execution
• Reading Comprehension Pipeline - Context extraction and understanding
• Knowledge Pipeline - RAG-enhanced knowledge retrieval
• Parallel Processing - Dual-thread architecture for optimal performance
• RAG Integration - ChromaDB-powered vector search with Vietnamese text support
• Batch Processing - Configurable batch sizes for API efficiency

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Input Questions                          │
└────────────────────────────┬────────────────────────────────────┘
                             │
                    ┌────────▼────────┐
                    │  Question Router │
                    └────────┬────────┘
                             │
         ┌───────────────────┼───────────────────┐
         │                   │                   │
   ┌─────▼─────┐      ┌──────▼──────┐     ┌─────▼─────┐
   │ Math      │      │ Knowledge   │     │ Reading   │
   │ Pipeline  │      │ Pipeline    │     │ Comprehension│
   └─────┬─────┘      └──────┬──────┘     └─────┬─────┘
         │                   │                  │
         │            ┌──────▼──────┐    ┌──────▼──────┐
         │            │ RAG Thread  │    │  Context    │
         │            │ (ChromaDB)  │    │  Extractor  │
         │            └──────┬──────┘    └──────┬──────┘
         │                   │                  │
         └───────────────────┴──────────────────┘
                             │
                    ┌────────▼────────┐
                    │   API Thread    │
                    │ (LLM Inference) │
                    └────────┬────────┘
                             │
                    ┌────────▼────────┐
                    │    Answers      │
                    └─────────────────┘

Pipeline Classification

Pipeline	Description	Model	Features
Toán học	Math calculations	Large LLM	Code generation & execution
Kiến thức	Knowledge questions	Large LLM	RAG context retrieval
Đọc hiểu	Reading comprehension	Small LLM	Context extraction

Installation

Prerequisites

• Python 3.8 or higher
• pip (Python package manager)

Setup

1. Clone the repository

   git clone https://github.com/your-username/VCM-B.git
   cd VCM-B

2. Install dependencies

   pip install -r requirements.txt

3. Configure API credentials

   Create an api_keys.json file with your VNPT AI API credentials:

   [
     {"authorization": "...", "tokenKey": "...", "llmApiName": "LLM small", "tokenId": "..."},
     {"authorization": "...", "tokenKey": "...", "llmApiName": "LLM embedings", "tokenId": "..."},
     {"authorization": "...", "tokenKey": "...", "llmApiName": "LLM large", "tokenId": "..."}
   ]

4. Prepare your dataset

   Place your test data in data/test.json in the following format:

   [
     {
       "qid": "question_001",
       "question": "Your question text here",
       "choices": ["Option A", "Option B", "Option C", "Option D"]
     }
   ]

Usage

Run the QA Pipeline

python main.py

Output

The system generates two output files in output_log/run_<timestamp>/:

• predictions.json - Detailed predictions with metadata
• submission.csv - Submission format (qid, answer)

Example Output

=== QA System V2 (Parallel) | 2024-12-26 20:00:00 ===
[Load] 500 câu từ data/test.json
[Phân loại] Toán: 50 | Kiến thức: 350 | Đọc hiểu: 100

[Xử lý]
  Đã thêm 50 câu toán vào hàng đợi
  Đang trích xuất ngữ cảnh cho 100 câu đọc hiểu...
[RAG Thread] Bắt đầu, 350 câu kiến thức
[API Thread] Bắt đầu
...
[Hoàn tất] Tổng cộng: 500 đáp án
[Output] output_log/run_20241226_200000/

Configuration

Edit config.py to customize system behavior:

# Batch settings
EXTRACT_BATCH_SIZE = 10  # Context extraction batch size
SMALL_BATCH_SIZE = 15    # Small model batch size
LARGE_BATCH_SIZE = 10    # Large model batch size
MATH_BATCH_SIZE = 5      # Math pipeline batch size

# Paths
DATASET_FILE = "data/test.json"
API_KEYS_FILE = "api_keys.json"

RAG Configuration

Configure RAG settings in rag/config.py:

DEFAULT_COLLECTION = "your_collection_name"
RERANK_TOP_K = 5  # Number of documents to retrieve

Project Structure

VCM-B/
├── main.py                 # Entry point
├── config.py               # Global configuration
├── requirements.txt        # Dependencies
├── api_keys.json          # API credentials (not in repo)
├── data/
│   └── test.json          # Input dataset
├── models/
│   └── llm_client.py      # LLM API client
├── pipeline/
│   ├── qa_pipeline_v2.py  # Main pipeline (V2 with threading)
│   ├── router.py          # Question classification
│   ├── context_extractor.py # Context extraction
│   ├── math_pipeline.py   # Math problem solver
│   └── code_executor.py   # Safe code execution
├── rag/
│   ├── retriever.py       # Document retrieval
│   ├── vectorstore.py     # Vector database operations
│   ├── chunker.py         # Text chunking
│   ├── reranker.py        # Result reranking
│   └── hf_embedder.py     # HuggingFace embeddings
└── output_log/            # Output directory

Dependencies

Package	Purpose
requests	HTTP client for API calls
chromadb	Vector database for RAG
underthesea	Vietnamese NLP toolkit
sentence-transformers	Embedding models
transformers	HuggingFace models
torch	Deep learning framework

Performance Tips

1. Adjust batch sizes based on your API rate limits
2. Pre-populate RAG database with relevant knowledge documents
3. Monitor the output log for debugging failed predictions
4. Use validation set to tune routing heuristics

License

This project is developed for the VNPT AI Hackathon 2025.

---