diff --git a/TRIAL_SYSTEM_IMPLEMENTATION.md b/TRIAL_SYSTEM_IMPLEMENTATION.md deleted file mode 100644 index 662ebab6..00000000 --- a/TRIAL_SYSTEM_IMPLEMENTATION.md +++ /dev/null @@ -1,171 +0,0 @@ -# 7-Day Trial System Implementation Summary - -## ๐ŸŽฏ **Objective Completed** -Replaced the Free plan with a 7-day trial using Stripe's manual capture system. Users must now provide payment upfront but are only charged after 7 days if they don't cancel. - ---- - -## โœ… **What We've Implemented (Phase 1)** - -### 1. **Core System Changes** -- **Removed FREE plan entirely** from credit configuration -- **Added TRIAL plan type** with 500 credits -- **Updated all plan fallbacks** to default to Standard instead of Free -- **Implemented manual capture** using Stripe's [authorization and capture](https://docs.stripe.com/payments/place-a-hold-on-a-payment-method) system - -### 2. **Credit Management Overhaul** -- **Zero credits for non-subscribers**: Users without subscription get 0 credits -- **Immediate Standard access**: Trial users get 500 credits instantly -- **Better error handling**: Clear "upgrade required" messages when credits = 0 -- **Trial credit tracking**: Redis stores trial-specific metadata - -### 3. **New API Endpoints** -``` -POST /api/trial/start - Starts trial after payment authorization -POST /api/trial/cancel - Cancels trial and removes access immediately -POST /api/checkout-sessions - Updated to use manual capture instead of subscriptions -``` - -### 4. **Stripe Integration** -- **Manual capture setup**: 7-day authorization hold on customer's card -- **Payment intent tracking**: Store payment intent ID for trial management -- **Automatic cancellation**: Cancel payment intent if user cancels trial - -### 5. **User Experience Updates** -- **Pricing page**: Removed Free tier, added "Start 7-Day Trial" button -- **Immediate feedback**: Clear error messages when credits are insufficient -- **Trial status**: Users see Standard plan benefits immediately - -### 6. **Data Migration** -- **Downgrade script**: Sets all existing free users to 0 credits -- **Redis schema update**: Added trial-specific fields (paymentIntentId, trialEndDate, etc.) - ---- - -## ๐Ÿ“‹ **Implementation Details** - -### **Trial Flow:** -1. User clicks "Start 7-Day Trial" โ†’ Redirects to checkout -2. Stripe authorizes card (no charge) โ†’ Creates payment intent with `capture_method: 'manual'` -3. User gets immediate access to 500 Standard plan credits -4. After 7 days: Stripe automatically captures payment OR user cancels and card is not charged - -### **Credit System:** -- **Trial Users**: 500 credits, planType = 'STANDARD', status = 'trial' -- **Downgraded Users**: 0 credits, can browse but can't use features -- **Paid Users**: Normal credit allocation based on their plan - -### **Cancellation Logic:** -- **During trial**: Immediate access removal + Stripe payment intent cancellation -- **After trial converts**: Access until month end (standard cancellation flow) - ---- - -## ๐Ÿšง **Still Needed (Phase 2)** - -### 1. **Automated Payment Capture** (High Priority) -- Set up Stripe scheduled webhooks or cron job to capture payments on day 7 -- Handle failed captures (expired cards, insufficient funds) -- Convert trial status to 'active' after successful capture - -### 2. **Webhook Updates** (High Priority) -- Update webhook handlers for new payment intent events: - - `payment_intent.succeeded` (trial converted to paid) - - `payment_intent.payment_failed` (capture failed) - - `payment_intent.canceled` (trial canceled) - -### 3. **UI/UX Enhancements** (Medium Priority) -- Trial countdown display in dashboard -- "Cancel Trial" button in account settings -- Better upgrade prompts when credits = 0 -- Trial status indicators throughout the app - -### 4. **Frontend Integration** (Medium Priority) -- Update checkout success page to call `/api/trial/start` -- Add trial cancellation UI components -- Update credit balance displays for trial users - -### 5. **Remaining Free Plan References** (Low Priority) -- Clean up any remaining Free plan references in: - - Feature access control - - Email templates - - UI components - - Error messages - ---- - -## ๐Ÿ”ง **How to Deploy** - -### **Before Deployment:** -```bash -# 1. Run the downgrade script (ONCE) -cd Auto-Analyst-CS/auto-analyst-frontend -node scripts/run-downgrade.js - -# 2. Verify all free users have been downgraded -# Check Redis: HGETALL user:*:credits should show total: "0" -``` - -### **After Deployment:** -1. Test the trial flow with Stripe test cards -2. Verify trial users get immediate Standard access -3. Test trial cancellation flow -4. Monitor for any remaining Free plan fallbacks - ---- - -## ๐Ÿ’ฐ **Business Impact** - -### **Positive Changes:** -- โœ… **Immediate revenue impact**: All users must provide payment info -- โœ… **Higher conversion**: Trial users experience full Standard features -- โœ… **Reduced free-rider problem**: No more permanent free users -- โœ… **Better user qualification**: Payment info acts as user quality filter - -### **Considerations:** -- โš ๏ธ **Conversion tracking needed**: Monitor trial โ†’ paid conversion rates -- โš ๏ธ **Support volume**: May increase due to payment authorization questions -- โš ๏ธ **Competitive positioning**: Ensure trial period is competitive - ---- - -## ๐Ÿ“Š **Technical Architecture Changes** - -```mermaid -graph TD - A[User] --> B[Pricing Page] - B --> C[Start 7-Day Trial] - C --> D[Stripe Checkout - Manual Capture] - D --> E[Payment Authorized] - E --> F[Trial Started - 500 Credits] - - F --> G{User Action} - G -->|Cancels| H[Cancel Payment Intent] - G -->|No Action| I[Auto-capture on Day 7] - - H --> J[0 Credits - Upgrade Required] - I --> K[Active Standard Plan] -``` - ---- - -## โฐ **Estimated Timeline for Phase 2** - -- **Automated capture system**: 2-3 days -- **Webhook updates**: 1-2 days -- **UI/UX enhancements**: 2-3 days -- **Testing & deployment**: 1-2 days - -**Total Phase 2**: ~6-10 days - ---- - -## ๐ŸŽ‰ **Ready to Go Live** - -The core system is functional and ready for initial deployment. Users can: -- โœ… Start trials with payment authorization -- โœ… Get immediate Standard plan access -- โœ… Cancel trials to avoid charges -- โœ… Be blocked from features when credits = 0 - -Phase 2 will add automation and polish, but the fundamental business requirement is met! \ No newline at end of file diff --git a/auto-analyst-backend/docs/README.md b/auto-analyst-backend/docs/README.md new file mode 100644 index 00000000..f50cf5b6 --- /dev/null +++ b/auto-analyst-backend/docs/README.md @@ -0,0 +1,254 @@ +# Auto-Analyst Backend Documentation + +This directory contains comprehensive documentation for the Auto-Analyst backend - a sophisticated multi-agent AI platform for data analysis built with FastAPI, DSPy, and modern Python technologies. + +## ๐Ÿ“ Documentation Structure + +### **๐Ÿ—๏ธ Architecture** (`/architecture/`) +- **[System Architecture](./architecture/architecture.md)** - Comprehensive overview of backend system design, components, and data flow patterns + +### **๐Ÿš€ Development** (`/development/`) +- **[Development Workflow](./development/development_workflow.md)** - Complete development guide with patterns, best practices, and code organization principles + +### **๐Ÿ”ง System** (`/system/`) +- **[Database Schema](./system/database-schema.md)** - Complete database schema with all tables, relationships, and performance optimization +- **[Shared DataFrame System](./system/shared_dataframe.md)** - Inter-agent data sharing and session management + +### **๐ŸŒ API** (`/api/`) +- **[API Endpoints Overview](./api/endpoints.md)** - Main API reference hub +- **[Route Documentation](./api/routes/)** - Detailed endpoint documentation: + - **[Core Routes](./api/routes/core.md)** - File uploads, sessions, authentication + - **[Chat Routes](./api/routes/chats.md)** - Chat and messaging endpoints + - **[Code Routes](./api/routes/code.md)** - Code execution and processing + - **[Analytics Routes](./api/routes/analytics.md)** - Usage analytics and monitoring + - **[Deep Analysis Routes](./api/routes/deep_analysis.md)** - Multi-agent analysis system + - **[Template Routes](./api/routes/templates.md)** - Agent template management + - **[Feedback Routes](./api/routes/feedback.md)** - User feedback and rating system + +### **๐Ÿ› Troubleshooting** (`/troubleshooting/`) +- **[Troubleshooting Guide](./troubleshooting/troubleshooting.md)** - Common issues, debugging tools, and solutions + +## ๐ŸŽฏ Backend Overview + +### **Tech Stack** +- **FastAPI** - Modern async Python web framework +- **DSPy** - AI agent orchestration and LLM integration +- **SQLAlchemy** - Database ORM with PostgreSQL/SQLite support +- **Plotly** - Interactive data visualizations +- **Pandas/NumPy** - Data manipulation and analysis +- **Scikit-learn** - Machine learning models +- **Statsmodels** - Statistical analysis + +### **Core Features** +- **Multi-Agent System** - 4+ specialized AI agents for different analysis tasks +- **Template System** - User-customizable agent configurations +- **Deep Analysis** - Multi-step analytical workflows with streaming progress +- **Session Management** - Stateful user sessions with shared data context +- **Code Execution** - Safe Python code execution environment +- **Real-time Streaming** - WebSocket support for live analysis updates + +### **Agent Types** +1. **Data Preprocessing Agent** - Data cleaning and preparation +2. **Statistical Analytics Agent** - Statistical analysis using statsmodels +3. **Machine Learning Agent** - ML modeling with scikit-learn +4. **Data Visualization Agent** - Interactive charts with Plotly +5. **Feature Engineering Agent** (Premium) - Advanced feature creation +6. **Polars Agent** (Premium) - High-performance data processing + +## ๐Ÿš€ Quick Start Guide + +### **1. Environment Setup** + +```bash +# Navigate to backend directory +cd Auto-Analyst-CS/auto-analyst-backend + +# Create virtual environment +python -m venv venv +source venv/bin/activate # Linux/Mac +venv\Scripts\activate # Windows + +# Install dependencies +pip install -r requirements.txt +``` + +### **2. Environment Configuration** + +Create `.env` file with required variables: + +```env +# Database Configuration +DATABASE_URL=sqlite:///./chat_database.db + +# AI Model Configuration +OPENAI_API_KEY=your-openai-api-key +MODEL_PROVIDER=openai # openai, anthropic, groq, gemini +MODEL_NAME=gpt-4o-mini +TEMPERATURE=0.7 +MAX_TOKENS=6000 + +# Optional: Additional AI Providers +ANTHROPIC_API_KEY=your-anthropic-key +GROQ_API_KEY=your-groq-key +GEMINI_API_KEY=your-gemini-key + +# Security +ADMIN_API_KEY=your-admin-key + +# Application Settings +ENVIRONMENT=development +FRONTEND_URL=http://localhost:3000/ +``` + +### **3. Database Initialization** + +```bash +# Initialize database and default agents +python -c " +from src.db.init_db import init_database +from src.db.init_default_agents import initialize_default_agents +init_database() +initialize_default_agents() +print('โœ… Database and agents initialized successfully') +" +``` + +### **4. Start Development Server** + +```bash +# Start the FastAPI server +python app.py + +# Or with uvicorn for more control +uvicorn app:app --reload --host 0.0.0.0 --port 8000 +``` + +### **5. Verify Installation** + +- **API Documentation**: `http://localhost:8000/docs` +- **Health Check**: `http://localhost:8000/health` +- **Interactive API**: `http://localhost:8000/redoc` + +## ๐Ÿ”ง Development Workflow + +### **Adding New Agents** + +1. **Define Agent Signature** in `src/agents/agents.py` +2. **Add Configuration** to `agents_config.json` +3. **Register Agent** in loading system +4. **Test Integration** with multi-agent pipeline + +### **Adding New API Endpoints** + +1. **Create Route File** in `src/routes/` +2. **Define Pydantic Models** for request/response +3. **Implement Endpoints** with proper error handling +4. **Register Router** in `app.py` +5. **Update Documentation** + +### **Database Changes** + +1. **Modify Models** in `src/db/schemas/models.py` +2. **Create Migration**: `alembic revision --autogenerate -m "description"` +3. **Apply Migration**: `alembic upgrade head` +4. **Update Documentation** + +## ๐Ÿ“Š System Architecture + +### **Request Processing Flow** +``` +HTTP Request โ†’ FastAPI Router โ†’ Route Handler โ†’ Business Logic โ†’ +Database/Agent System โ†’ AI Model โ†’ Response Processing โ†’ JSON Response +``` + +### **Agent Execution Flow** +``` +User Query โ†’ Session Manager โ†’ Agent Selection โ†’ Context Preparation โ†’ +DSPy Chain โ†’ AI Model โ†’ Code Generation โ†’ Execution โ†’ Response Formatting +``` + +### **Deep Analysis Workflow** +``` +Goal Input โ†’ Question Generation โ†’ Planning โ†’ Multi-Agent Execution โ†’ +Code Synthesis โ†’ Result Compilation โ†’ HTML Report Generation +``` + +## ๐Ÿงช Testing & Validation + +### **API Testing** +```bash +# Interactive documentation +open http://localhost:8000/docs + +# cURL examples +curl -X GET "http://localhost:8000/health" +curl -X POST "http://localhost:8000/chat/preprocessing_agent" \ + -H "Content-Type: application/json" \ + -d '{"query": "Clean this dataset", "session_id": "test"}' +``` + +### **Agent Testing** +```python +# Test individual agents +from src.agents.agents import preprocessing_agent +import dspy + +# Configure DSPy +lm = dspy.LM('openai/gpt-4o-mini', api_key='your-key') +dspy.configure(lm=lm) + +# Test agent +agent = dspy.ChainOfThought(preprocessing_agent) +result = agent(goal='clean data', dataset='test dataset') +print(result) +``` + +## ๐Ÿ”’ Security & Production + +### **Security Features** +- **Session-based authentication** with secure session management +- **API key protection** for admin endpoints +- **Input validation** using Pydantic models +- **Error handling** with proper HTTP status codes +- **CORS configuration** for frontend integration + +### **Production Considerations** +- **PostgreSQL database** for production deployment +- **Environment variable management** for secrets +- **Logging configuration** for monitoring +- **Rate limiting** for API protection +- **Performance optimization** for large datasets + +## ๐Ÿ“ˆ Monitoring & Analytics + +The backend includes comprehensive analytics for: +- **Usage tracking** - API endpoint usage and performance +- **Model usage** - AI model consumption and costs +- **User analytics** - User behavior and engagement +- **Error monitoring** - System health and error tracking +- **Performance metrics** - Response times and throughput + +## ๐Ÿค Contributing + +1. **Follow coding standards** defined in development workflow +2. **Add comprehensive tests** for new features +3. **Update documentation** for all changes +4. **Use proper error handling** patterns +5. **Submit detailed pull requests** with clear descriptions + +--- + +## ๐Ÿ“– Detailed Documentation + +For specific implementation details, refer to the organized documentation in each subdirectory: + +- **[Getting Started Guide](./getting_started.md)** - Complete setup walkthrough +- **[Architecture Documentation](./architecture/)** - System design and components +- **[Development Guides](./development/)** - Workflow and best practices +- **[API Reference](./api/)** - Complete endpoint documentation +- **[System Documentation](./system/)** - Database and core systems +- **[Troubleshooting](./troubleshooting/)** - Debugging and solutions + +--- + +**Need help?** Check the troubleshooting guide or refer to the comprehensive documentation in each section. \ No newline at end of file diff --git a/auto-analyst-backend/docs/endpoints.md b/auto-analyst-backend/docs/api/endpoints.md similarity index 100% rename from auto-analyst-backend/docs/endpoints.md rename to auto-analyst-backend/docs/api/endpoints.md diff --git a/auto-analyst-backend/docs/routes/analytics.md b/auto-analyst-backend/docs/api/routes/analytics.md similarity index 100% rename from auto-analyst-backend/docs/routes/analytics.md rename to auto-analyst-backend/docs/api/routes/analytics.md diff --git a/auto-analyst-backend/docs/routes/chats.md b/auto-analyst-backend/docs/api/routes/chats.md similarity index 100% rename from auto-analyst-backend/docs/routes/chats.md rename to auto-analyst-backend/docs/api/routes/chats.md diff --git a/auto-analyst-backend/docs/routes/code.md b/auto-analyst-backend/docs/api/routes/code.md similarity index 100% rename from auto-analyst-backend/docs/routes/code.md rename to auto-analyst-backend/docs/api/routes/code.md diff --git a/auto-analyst-backend/docs/routes/core.md b/auto-analyst-backend/docs/api/routes/core.md similarity index 100% rename from auto-analyst-backend/docs/routes/core.md rename to auto-analyst-backend/docs/api/routes/core.md diff --git a/auto-analyst-backend/docs/routes/deep_analysis.md b/auto-analyst-backend/docs/api/routes/deep_analysis.md similarity index 100% rename from auto-analyst-backend/docs/routes/deep_analysis.md rename to auto-analyst-backend/docs/api/routes/deep_analysis.md diff --git a/auto-analyst-backend/docs/routes/feedback.md b/auto-analyst-backend/docs/api/routes/feedback.md similarity index 100% rename from auto-analyst-backend/docs/routes/feedback.md rename to auto-analyst-backend/docs/api/routes/feedback.md diff --git a/auto-analyst-backend/docs/routes/templates.md b/auto-analyst-backend/docs/api/routes/templates.md similarity index 100% rename from auto-analyst-backend/docs/routes/templates.md rename to auto-analyst-backend/docs/api/routes/templates.md diff --git a/auto-analyst-backend/docs/architecture.md b/auto-analyst-backend/docs/architecture/architecture.md similarity index 100% rename from auto-analyst-backend/docs/architecture.md rename to auto-analyst-backend/docs/architecture/architecture.md diff --git a/auto-analyst-backend/docs/development_workflow.md b/auto-analyst-backend/docs/development/development_workflow.md similarity index 100% rename from auto-analyst-backend/docs/development_workflow.md rename to auto-analyst-backend/docs/development/development_workflow.md diff --git a/auto-analyst-backend/docs/system/database-schema.md b/auto-analyst-backend/docs/system/database-schema.md new file mode 100644 index 00000000..8a328d75 --- /dev/null +++ b/auto-analyst-backend/docs/system/database-schema.md @@ -0,0 +1,289 @@ +# Auto-Analyst Database Schema Documentation + +## ๐Ÿ“‹ Overview + +The Auto-Analyst backend uses a relational database schema designed for scalability and data integrity. The schema supports both **SQLite** (development) and **PostgreSQL** (production) databases through SQLAlchemy ORM. + +### **Database Features** +- **User Management** - Authentication and user data +- **Chat System** - Conversation sessions and message history +- **AI Model Tracking** - Usage analytics and cost monitoring +- **Code Execution** - Code generation and execution tracking +- **Agent Templates** - Customizable AI agent configurations +- **Deep Analysis** - Multi-step analysis reports and results +- **User Feedback** - Rating and feedback system + +--- + +## ๐Ÿ—„๏ธ Database Tables + +### **1. Users Table (`users`)** + +**Purpose**: Core user authentication and profile management + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `user_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique user identifier | +| `username` | `STRING` | UNIQUE, NOT NULL | User's display name | +| `email` | `STRING` | UNIQUE, NOT NULL | User's email address | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Account creation timestamp | + +**Relationships:** +- **One-to-Many**: `chats` (User โ†’ Chat sessions) +- **One-to-Many**: `usage_records` (User โ†’ Model usage tracking) +- **One-to-Many**: `deep_analysis_reports` (User โ†’ Analysis reports) +- **One-to-Many**: `template_preferences` (User โ†’ Agent preferences) + +--- + +### **2. Chats Table (`chats`)** + +**Purpose**: Conversation sessions and chat organization + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `chat_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique chat session identifier | +| `user_id` | `INTEGER` | FOREIGN KEY โ†’ `users.user_id`, CASCADE DELETE | Chat owner (nullable for anonymous) | +| `title` | `STRING` | DEFAULT: 'New Chat' | Human-readable chat title | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Chat creation timestamp | + +**Relationships:** +- **Many-to-One**: `user` (Chat โ†’ User) +- **One-to-Many**: `messages` (Chat โ†’ Messages) +- **One-to-Many**: `usage_records` (Chat โ†’ Model usage) + +--- + +### **3. Messages Table (`messages`)** + +**Purpose**: Individual messages within chat conversations + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `message_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique message identifier | +| `chat_id` | `INTEGER` | FOREIGN KEY โ†’ `chats.chat_id`, CASCADE DELETE | Parent chat session | +| `sender` | `STRING` | NOT NULL | Message sender: 'user' or 'ai' | +| `content` | `TEXT` | NOT NULL | Message content (text/markdown) | +| `timestamp` | `DATETIME` | DEFAULT: UTC NOW | Message creation time | + +**Relationships:** +- **Many-to-One**: `chat` (Message โ†’ Chat) +- **One-to-One**: `feedback` (Message โ†’ Feedback) + +--- + +### **4. Model Usage Table (`model_usage`)** + +**Purpose**: AI model usage tracking for analytics and billing + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `usage_id` | `INTEGER` | PRIMARY KEY | Unique usage record identifier | +| `user_id` | `INTEGER` | FOREIGN KEY โ†’ `users.user_id`, SET NULL | User who triggered the usage | +| `chat_id` | `INTEGER` | FOREIGN KEY โ†’ `chats.chat_id`, SET NULL | Associated chat session | +| `model_name` | `STRING(100)` | NOT NULL | AI model used (e.g., 'gpt-4o-mini') | +| `provider` | `STRING(50)` | NOT NULL | Model provider ('openai', 'anthropic', etc.) | +| `prompt_tokens` | `INTEGER` | DEFAULT: 0 | Input tokens consumed | +| `completion_tokens` | `INTEGER` | DEFAULT: 0 | Output tokens generated | +| `total_tokens` | `INTEGER` | DEFAULT: 0 | Total tokens (input + output) | +| `query_size` | `INTEGER` | DEFAULT: 0 | Query size in characters | +| `response_size` | `INTEGER` | DEFAULT: 0 | Response size in characters | +| `cost` | `FLOAT` | DEFAULT: 0.0 | Cost in USD for this usage | +| `timestamp` | `DATETIME` | DEFAULT: UTC NOW | Usage timestamp | +| `is_streaming` | `BOOLEAN` | DEFAULT: FALSE | Whether response was streamed | +| `request_time_ms` | `INTEGER` | DEFAULT: 0 | Request processing time (milliseconds) | + +**Relationships:** +- **Many-to-One**: `user` (Usage โ†’ User) +- **Many-to-One**: `chat` (Usage โ†’ Chat) + +--- + +### **5. Code Executions Table (`code_executions`)** + +**Purpose**: Track code generation and execution attempts + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `execution_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique execution identifier | +| `message_id` | `INTEGER` | FOREIGN KEY โ†’ `messages.message_id`, CASCADE DELETE | Associated message | +| `chat_id` | `INTEGER` | FOREIGN KEY โ†’ `chats.chat_id`, CASCADE DELETE | Parent chat session | +| `user_id` | `INTEGER` | FOREIGN KEY โ†’ `users.user_id`, SET NULL | User who triggered execution | +| `initial_code` | `TEXT` | NULLABLE | First version of generated code | +| `latest_code` | `TEXT` | NULLABLE | Most recent code version | +| `is_successful` | `BOOLEAN` | DEFAULT: FALSE | Whether execution succeeded | +| `output` | `TEXT` | NULLABLE | Execution output (including errors) | +| `model_provider` | `STRING(50)` | NULLABLE | AI model provider used | +| `model_name` | `STRING(100)` | NULLABLE | AI model name used | +| `failed_agents` | `TEXT` | NULLABLE | JSON list of failed agent names | +| `error_messages` | `TEXT` | NULLABLE | JSON map of error messages by agent | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Execution creation time | +| `updated_at` | `DATETIME` | DEFAULT: UTC NOW, ON UPDATE | Last update timestamp | + +--- + +### **6. Message Feedback Table (`message_feedback`)** + +**Purpose**: User feedback and model settings for messages + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `feedback_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique feedback identifier | +| `message_id` | `INTEGER` | FOREIGN KEY โ†’ `messages.message_id`, CASCADE DELETE | Associated message | +| `rating` | `INTEGER` | NULLABLE | Star rating (1-5 scale) | +| `model_name` | `STRING(100)` | NULLABLE | Model used for this message | +| `model_provider` | `STRING(50)` | NULLABLE | Model provider used | +| `temperature` | `FLOAT` | NULLABLE | Temperature setting used | +| `max_tokens` | `INTEGER` | NULLABLE | Max tokens setting used | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Feedback creation time | +| `updated_at` | `DATETIME` | DEFAULT: UTC NOW, ON UPDATE | Last update timestamp | + +**Relationships:** +- **One-to-One**: `message` (Feedback โ†” Message) + +--- + +### **7. Deep Analysis Reports Table (`deep_analysis_reports`)** + +**Purpose**: Store comprehensive multi-agent analysis reports + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `report_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique report identifier | +| `report_uuid` | `STRING(100)` | UNIQUE, NOT NULL | Frontend-generated UUID | +| `user_id` | `INTEGER` | FOREIGN KEY โ†’ `users.user_id`, CASCADE DELETE | Report owner | +| `goal` | `TEXT` | NOT NULL | Analysis objective/question | +| `status` | `STRING(20)` | NOT NULL, DEFAULT: 'pending' | Status: 'pending', 'running', 'completed', 'failed' | +| `start_time` | `DATETIME` | DEFAULT: UTC NOW | Analysis start time | +| `end_time` | `DATETIME` | NULLABLE | Analysis completion time | +| `duration_seconds` | `INTEGER` | NULLABLE | Total analysis duration | +| `deep_questions` | `TEXT` | NULLABLE | Generated analytical questions | +| `deep_plan` | `TEXT` | NULLABLE | Analysis execution plan | +| `summaries` | `JSON` | NULLABLE | Array of analysis summaries | +| `analysis_code` | `TEXT` | NULLABLE | Generated Python code | +| `plotly_figures` | `JSON` | NULLABLE | Array of Plotly figure data | +| `synthesis` | `JSON` | NULLABLE | Array of synthesis insights | +| `final_conclusion` | `TEXT` | NULLABLE | Final analysis conclusion | +| `html_report` | `TEXT` | NULLABLE | Complete HTML report | +| `progress_percentage` | `INTEGER` | DEFAULT: 0 | Progress percentage (0-100) | +| `total_tokens_used` | `INTEGER` | DEFAULT: 0 | Total tokens consumed | +| `estimated_cost` | `FLOAT` | DEFAULT: 0.0 | Estimated cost in USD | +| `credits_consumed` | `INTEGER` | DEFAULT: 0 | Credits deducted for analysis | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Report creation time | +| `updated_at` | `DATETIME` | DEFAULT: UTC NOW, ON UPDATE | Last update timestamp | + +**Relationships:** +- **Many-to-One**: `user` (Report โ†’ User) + +--- + +### **8. Agent Templates Table (`agent_templates`)** + +**Purpose**: Store predefined AI agent configurations + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `template_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique template identifier | +| `template_name` | `STRING(100)` | UNIQUE, NOT NULL | Internal template name | +| `display_name` | `STRING(200)` | NULLABLE | User-friendly display name | +| `description` | `TEXT` | NOT NULL | Template description | +| `prompt_template` | `TEXT` | NOT NULL | Agent behavior instructions | +| `icon_url` | `STRING(500)` | NULLABLE | Template icon URL | +| `category` | `STRING(50)` | NULLABLE | Template category | +| `is_premium_only` | `BOOLEAN` | DEFAULT: FALSE | Requires premium subscription | +| `variant_type` | `STRING(20)` | DEFAULT: 'individual' | 'planner', 'individual', or 'both' | +| `is_active` | `BOOLEAN` | DEFAULT: TRUE | Template is active/available | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Template creation time | + +**Relationships:** +- **One-to-Many**: `user_preferences` (Template โ†’ User preferences) + +--- + +### **9. User Template Preferences Table (`user_template_preferences`)** + +**Purpose**: Track user preferences and usage for agent templates + +| Column | Type | Constraints | Description | +|--------|------|-------------|-------------| +| `preference_id` | `INTEGER` | PRIMARY KEY, AUTO INCREMENT | Unique preference identifier | +| `user_id` | `INTEGER` | FOREIGN KEY โ†’ `users.user_id`, CASCADE DELETE | User who owns preference | +| `template_id` | `INTEGER` | FOREIGN KEY โ†’ `agent_templates.template_id`, CASCADE DELETE | Associated template | +| `is_enabled` | `BOOLEAN` | DEFAULT: TRUE | Whether user has template enabled | +| `usage_count` | `INTEGER` | DEFAULT: 0 | Number of times user used template | +| `last_used_at` | `DATETIME` | NULLABLE | Last time user used template | +| `created_at` | `DATETIME` | DEFAULT: UTC NOW | Preference creation time | + +**Relationships:** +- **Many-to-One**: `user` (Preference โ†’ User) +- **Many-to-One**: `template` (Preference โ†’ Template) + +**Constraints:** +- **Unique**: `(user_id, template_id)` - One preference per user per template + +--- + +## ๐Ÿ”— Entity Relationship Diagram + +``` +Users (1) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ (Many) Chats + โ”‚ โ”‚ + โ”‚ โ”œโ”€โ”€ (Many) Messages + โ”‚ โ”‚ โ”‚ + โ”‚ โ”‚ โ””โ”€โ”€ (1) MessageFeedback + โ”‚ โ”‚ + โ”‚ โ””โ”€โ”€ (Many) CodeExecutions + โ”‚ + โ”œโ”€โ”€ (Many) ModelUsage + โ”‚ + โ”œโ”€โ”€ (Many) DeepAnalysisReports + โ”‚ + โ””โ”€โ”€ (Many) UserTemplatePreferences + โ”‚ + โ””โ”€โ”€ (Many) AgentTemplates +``` + +--- + +## ๐Ÿ“Š Database Performance + +### **Optimized Indexes** + +```sql +-- High-performance queries +CREATE INDEX idx_messages_chat_timestamp ON messages(chat_id, timestamp DESC); +CREATE INDEX idx_model_usage_user_time ON model_usage(user_id, timestamp DESC); +CREATE INDEX idx_model_usage_model_time ON model_usage(model_name, timestamp DESC); +CREATE INDEX idx_reports_user_time ON deep_analysis_reports(user_id, created_at DESC); +``` + +### **Cascade Deletion Rules** + +| Parent โ†’ Child | Rule | Description | +|----------------|------|-------------| +| `users` โ†’ `chats` | CASCADE | Delete all user chats when user deleted | +| `chats` โ†’ `messages` | CASCADE | Delete all chat messages when chat deleted | +| `messages` โ†’ `feedback` | CASCADE | Delete feedback when message deleted | +| `users` โ†’ `model_usage` | SET NULL | Keep usage records for analytics | + +--- + +## ๐Ÿ›ก๏ธ Security & Maintenance + +### **Data Protection** +- User data isolated by `user_id` +- Sensitive fields require encryption in production +- Automatic cleanup of anonymous data after 90 days + +### **Regular Maintenance** +```sql +-- Clean old anonymous chats +DELETE FROM chats WHERE user_id IS NULL AND created_at < DATE_SUB(NOW(), INTERVAL 90 DAY); + +-- Update statistics for query optimization +ANALYZE users, chats, messages, model_usage; +``` + +--- + +This schema supports the full Auto-Analyst application with optimized performance, data integrity, and scalability for both development and production environments. \ No newline at end of file diff --git a/auto-analyst-backend/docs/shared_dataframe.md b/auto-analyst-backend/docs/system/shared_dataframe.md similarity index 100% rename from auto-analyst-backend/docs/shared_dataframe.md rename to auto-analyst-backend/docs/system/shared_dataframe.md diff --git a/auto-analyst-backend/docs/troubleshooting.md b/auto-analyst-backend/docs/troubleshooting/troubleshooting.md similarity index 100% rename from auto-analyst-backend/docs/troubleshooting.md rename to auto-analyst-backend/docs/troubleshooting/troubleshooting.md diff --git a/auto-analyst-frontend/app/account/page.tsx b/auto-analyst-frontend/app/account/page.tsx index 9747bfe8..33c290e3 100644 --- a/auto-analyst-frontend/app/account/page.tsx +++ b/auto-analyst-frontend/app/account/page.tsx @@ -145,9 +145,6 @@ export default function AccountPage() { }) syncResult = await syncRes.json() - if (syncResult.success) { - console.log(`Subscription synced: ${syncResult.before.redisStatus} โ†’ ${syncResult.after.stripeStatus}`) - } } catch (syncError) { console.log('Subscription sync not available or failed:', syncError) // Continue with regular refresh even if sync fails diff --git a/auto-analyst-frontend/app/api/analytics/feedback/detailed/route.ts b/auto-analyst-frontend/app/api/analytics/feedback/detailed/route.ts index 9723cc54..f15e866b 100644 --- a/auto-analyst-frontend/app/api/analytics/feedback/detailed/route.ts +++ b/auto-analyst-frontend/app/api/analytics/feedback/detailed/route.ts @@ -11,7 +11,6 @@ export async function GET(request: NextRequest) { // Create the backend URL with query parameters const backendEndpoint = `${backendUrl}/analytics/feedback/detailed?${searchParams.toString()}` - console.log(`Proxying request to: ${backendEndpoint}`) // Make the request to the backend const response = await fetch(backendEndpoint, { diff --git a/auto-analyst-frontend/app/api/debug/force-zero-credits/route.ts b/auto-analyst-frontend/app/api/debug/force-zero-credits/route.ts index f5e75bed..8184e4a8 100644 --- a/auto-analyst-frontend/app/api/debug/force-zero-credits/route.ts +++ b/auto-analyst-frontend/app/api/debug/force-zero-credits/route.ts @@ -14,15 +14,11 @@ export async function POST(request: NextRequest) { const userId = token.sub - console.log(`[DEBUG] Force zeroing credits for user ${userId}`) // Get current state before const beforeCredits = await redis.hgetall(KEYS.USER_CREDITS(userId)) const beforeSubscription = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - - console.log('Before - Credits:', beforeCredits) - console.log('Before - Subscription:', beforeSubscription) - + // Force set zero credits multiple ways await creditUtils.setZeroCredits(userId) @@ -41,16 +37,11 @@ export async function POST(request: NextRequest) { const afterCredits = await redis.hgetall(KEYS.USER_CREDITS(userId)) const afterSubscription = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - console.log('After - Credits:', afterCredits) - console.log('After - Subscription:', afterSubscription) - // Test the refresh logic const refreshResult = await subscriptionUtils.refreshCreditsIfNeeded(userId) - console.log('Refresh result:', refreshResult) // Get final state const finalCredits = await redis.hgetall(KEYS.USER_CREDITS(userId)) - console.log('Final - Credits:', finalCredits) return NextResponse.json({ success: true, diff --git a/auto-analyst-frontend/app/api/debug/sync-subscription/route.ts b/auto-analyst-frontend/app/api/debug/sync-subscription/route.ts index e06bd06f..aeb7d51b 100644 --- a/auto-analyst-frontend/app/api/debug/sync-subscription/route.ts +++ b/auto-analyst-frontend/app/api/debug/sync-subscription/route.ts @@ -35,13 +35,10 @@ export async function POST(request: NextRequest) { const stripeSubscriptionId = currentSubscriptionData.stripeSubscriptionId as string - console.log(`Syncing subscription ${stripeSubscriptionId} for user ${userId}`) // Get current status from Stripe const subscription = await stripe.subscriptions.retrieve(stripeSubscriptionId) - console.log(`Stripe subscription status: ${subscription.status}`) - console.log(`Redis subscription status: ${currentSubscriptionData.status}`) // Check if subscription is scheduled for cancellation const isCancelingAtPeriodEnd = subscription.cancel_at_period_end && subscription.status === 'active' @@ -74,14 +71,13 @@ export async function POST(request: NextRequest) { // Handle specific status transitions if (currentSubscriptionData.status === 'trialing' && subscription.status === 'active') { - console.log(`Trial to active transition detected during sync for user ${userId}`) updateData.trialEndedAt = new Date().toISOString() updateData.trialToActiveDate = new Date().toISOString() } await redis.hset(KEYS.USER_SUBSCRIPTION(userId), updateData) - console.log(`Successfully synced subscription status for user ${userId}`) + // console.log(`Successfully synced subscription status for user ${userId}`) return NextResponse.json({ success: true, diff --git a/auto-analyst-frontend/app/api/trial/cancel/route.ts b/auto-analyst-frontend/app/api/trial/cancel/route.ts index 7a28564f..23150540 100644 --- a/auto-analyst-frontend/app/api/trial/cancel/route.ts +++ b/auto-analyst-frontend/app/api/trial/cancel/route.ts @@ -63,13 +63,13 @@ export async function POST(request: NextRequest) { await stripe.subscriptions.cancel(stripeSubscriptionId, { prorate: false // Don't prorate since it's a trial cancellation }) - console.log(`Canceled trial subscription ${stripeSubscriptionId} for user ${userId}`) + // console.log(`Canceled trial subscription ${stripeSubscriptionId} for user ${userId}`) } else if (stripeSubscription.status === 'active') { // For active subscriptions, cancel at period end await stripe.subscriptions.update(stripeSubscriptionId, { cancel_at_period_end: true }) - console.log(`Scheduled cancellation for subscription ${stripeSubscriptionId} for user ${userId}`) + // console.log(`Scheduled cancellation for subscription ${stripeSubscriptionId} for user ${userId}`) } else { console.log(`Subscription ${stripeSubscriptionId} already in status: ${stripeSubscription.status}`) } @@ -114,7 +114,7 @@ export async function POST(request: NextRequest) { trialStartDate: isTrial ? '' : subscriptionData.trialStartDate || '' }) - console.log(`${isTrial ? 'Trial' : 'Legacy subscription'} canceled for user ${userId}, access removed immediately`) + // console.log(`${isTrial ? 'Trial' : 'Legacy subscription'} canceled for user ${userId}, access removed immediately`) } else { // For post-trial cancellations: Maintain access until period end // Don't change credits - let them keep access until billing cycle ends @@ -131,7 +131,7 @@ export async function POST(request: NextRequest) { : now.toISOString() }) - console.log(`Subscription scheduled for cancellation at period end for user ${userId}`) + // console.log(`Subscription scheduled for cancellation at period end for user ${userId}`) } diff --git a/auto-analyst-frontend/app/api/trial/start/route.ts b/auto-analyst-frontend/app/api/trial/start/route.ts index 54b31e4f..9d7801c8 100644 --- a/auto-analyst-frontend/app/api/trial/start/route.ts +++ b/auto-analyst-frontend/app/api/trial/start/route.ts @@ -162,7 +162,6 @@ export async function POST(request: NextRequest) { // Store subscription data in Redis await redis.hset(KEYS.USER_SUBSCRIPTION(userId), subscriptionData) - console.log(`Started trial for user ${userId} with subscription ${subscription.id}`) return NextResponse.json({ success: true, diff --git a/auto-analyst-frontend/app/api/user/cancel-subscription/route.ts b/auto-analyst-frontend/app/api/user/cancel-subscription/route.ts index c6616b18..66ae3737 100644 --- a/auto-analyst-frontend/app/api/user/cancel-subscription/route.ts +++ b/auto-analyst-frontend/app/api/user/cancel-subscription/route.ts @@ -38,11 +38,6 @@ export async function POST(request: NextRequest) { const stripeSubscriptionId = subscriptionData.stripeSubscriptionId as string const isLegacyUser = !stripeSubscriptionId || !stripeSubscriptionId.startsWith('sub_') - // For legacy users, we'll skip Stripe API calls and just update Redis - if (isLegacyUser) { - console.log(`Legacy user ${userId} canceling - using Redis-only flow`) - } - try { let canceledSubscription = null @@ -53,7 +48,6 @@ export async function POST(request: NextRequest) { canceledSubscription = await stripe.subscriptions.update(stripeSubscriptionId, { cancel_at_period_end: true, }) - console.log(`Scheduled Stripe cancellation for subscription ${stripeSubscriptionId} for user ${userId}`) } else { console.log(`Legacy user ${userId} - skipping Stripe API calls, updating Redis only`) } diff --git a/auto-analyst-frontend/app/api/webhooks/route.ts b/auto-analyst-frontend/app/api/webhooks/route.ts index 96a3170e..c1ab6a88 100644 --- a/auto-analyst-frontend/app/api/webhooks/route.ts +++ b/auto-analyst-frontend/app/api/webhooks/route.ts @@ -159,7 +159,6 @@ export async function POST(request: NextRequest) { let rawBody: Buffer try { rawBody = await getRawBody(request.body as unknown as Readable) - console.log(`๐Ÿ“จ Webhook received: body length=${rawBody.length}, signature present=${!!signature}`) } catch (bodyError) { console.error('โŒ Failed to read webhook body:', bodyError) return NextResponse.json({ error: 'Failed to read request body' }, { status: 400 }) @@ -169,7 +168,6 @@ export async function POST(request: NextRequest) { let event try { event = stripe.webhooks.constructEvent(rawBody, signature, webhookSecret) - console.log(`โœ… Webhook signature verified: event=${event.type}, id=${event.id}`) } catch (err: any) { console.error(`โŒ Webhook signature verification failed:`) console.error(` Error: ${err.message}`) @@ -203,13 +201,11 @@ export async function POST(request: NextRequest) { // All checkouts now use trial subscriptions, handled by trial/start endpoint // This webhook is kept for logging purposes only const session = event.data.object as Stripe.Checkout.Session - console.log(`Checkout session completed: ${session.id} - handled by trial flow`) return NextResponse.json({ received: true }) } case 'customer.subscription.updated': { const subscription = event.data.object as Stripe.Subscription - console.log(`Subscription updated: ${subscription.id}, status: ${subscription.status}`) const customerId = subscription.customer as string if (!customerId) { @@ -229,7 +225,6 @@ export async function POST(request: NextRequest) { const currentSubscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) const currentStatus = currentSubscriptionData?.status - console.log(`Subscription status change for user ${userId}: ${currentStatus} -> ${subscription.status}`) // Always sync the status with Stripe, but handle special cases const updateData: any = { @@ -240,7 +235,6 @@ export async function POST(request: NextRequest) { // Handle specific status transitions if (currentStatus === 'trialing' && subscription.status === 'active') { - console.log(`Trial to active transition detected for user ${userId}`) updateData.trialEndedAt = new Date().toISOString() updateData.trialToActiveDate = new Date().toISOString() } @@ -256,7 +250,6 @@ export async function POST(request: NextRequest) { // Update subscription data await redis.hset(KEYS.USER_SUBSCRIPTION(userId), updateData) - console.log(`Updated subscription status to ${subscription.status} for user ${userId}`) return NextResponse.json({ received: true }) } @@ -352,7 +345,6 @@ export async function POST(request: NextRequest) { case 'invoice.payment_succeeded': { const invoice = event.data.object as Stripe.Invoice - console.log(`Invoice payment succeeded: ${invoice.id}, billing_reason: ${invoice.billing_reason}`) // Check if this is for a subscription payment if (invoice.subscription) { @@ -382,10 +374,8 @@ export async function POST(request: NextRequest) { stripeSubscriptionStatus: subscription.status // Keep Stripe status in sync }) - console.log(`User ${userId} trial ended successfully, subscription is now active`) } else if (invoice.billing_reason === 'subscription_create') { // This is the initial subscription creation payment (if any) - console.log(`Initial subscription payment for user ${userId}`) await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { status: subscription.status, // Use Stripe's status @@ -444,15 +434,12 @@ export async function POST(request: NextRequest) { case 'payment_intent.payment_failed': { const paymentIntent = event.data.object as Stripe.PaymentIntent - console.log(`Payment intent failed: ${paymentIntent.id}`) // Check if this is a trial payment intent by looking at metadata if (paymentIntent.metadata?.isTrial === 'true') { const userId = paymentIntent.metadata?.userId if (userId) { - console.log(`Trial payment authorization failed for user ${userId}`) - // Prevent trial access by ensuring credits remain at 0 await creditUtils.setZeroCredits(userId) @@ -463,8 +450,6 @@ export async function POST(request: NextRequest) { paymentFailedAt: new Date().toISOString(), failureReason: 'Payment authorization failed during trial signup' }) - - console.log(`Trial access prevented for user ${userId} due to payment authorization failure`) } } @@ -473,14 +458,12 @@ export async function POST(request: NextRequest) { case 'payment_intent.canceled': { const paymentIntent = event.data.object as Stripe.PaymentIntent - console.log(`Payment intent canceled: ${paymentIntent.id}`) // Check if this is a trial payment intent if (paymentIntent.metadata?.isTrial === 'true') { const userId = paymentIntent.metadata?.userId if (userId) { - console.log(`Trial payment intent canceled for user ${userId}`) // Ensure user doesn't get trial access await creditUtils.setZeroCredits(userId) @@ -493,7 +476,7 @@ export async function POST(request: NextRequest) { cancelReason: 'Payment intent canceled during trial signup' }) - console.log(`Trial access prevented for user ${userId} due to payment intent cancellation`) + // console.log(`Trial access prevented for user ${userId} due to payment intent cancellation`) } } @@ -517,7 +500,7 @@ export async function POST(request: NextRequest) { if (userKey) { const userId = userKey.toString() - console.log(`Trial setup failed for user ${userId}`) + // console.log(`Trial setup failed for user ${userId}`) // Cancel the trial subscription since setup failed await stripe.subscriptions.cancel(subscriptionId) @@ -533,7 +516,7 @@ export async function POST(request: NextRequest) { failureReason: 'Payment method setup failed during trial signup' }) - console.log(`Trial access prevented for user ${userId} due to setup failure`) + // console.log(`Trial access prevented for user ${userId} due to setup failure`) } } catch (error) { console.error('Error handling setup intent failure:', error) @@ -546,12 +529,11 @@ export async function POST(request: NextRequest) { case 'payment_intent.requires_action': { const paymentIntent = event.data.object as Stripe.PaymentIntent - console.log(`Payment intent requires action (3D Secure): ${paymentIntent.id}`) + // console.log(`Payment intent requires action (3D Secure): ${paymentIntent.id}`) // For trial payment intents, log the authentication requirement if (paymentIntent.metadata?.isTrial === 'true') { const userId = paymentIntent.metadata?.userId - console.log(`Trial payment requires 3D Secure authentication for user ${userId}`) // Don't grant trial access until authentication is complete // The payment will either succeed (triggering payment_intent.succeeded) diff --git a/auto-analyst-frontend/app/checkout/page.tsx b/auto-analyst-frontend/app/checkout/page.tsx index bd024063..eda128e5 100644 --- a/auto-analyst-frontend/app/checkout/page.tsx +++ b/auto-analyst-frontend/app/checkout/page.tsx @@ -99,7 +99,6 @@ export default function CheckoutPage() { const handleBillingCycleChange = (newCycle: 'monthly' | 'yearly') => { if (newCycle === billingCycle || paymentLoading) return - console.log(`๐Ÿ”„ Billing cycle change: ${billingCycle} โ†’ ${newCycle}`) setPaymentLoading(true) setBillingCycle(newCycle) @@ -115,7 +114,6 @@ export default function CheckoutPage() { setPaymentError('') setPromoError('') - console.log(`๐Ÿงน Cleared old setup intent, creating new one for ${newCycle} plan`) // Add a small delay to show loading state setTimeout(() => { @@ -180,7 +178,7 @@ export default function CheckoutPage() { setDiscountInfo(null) } - console.log(`Created new setup intent: ${data.setupIntentId} for ${planData.name} ${planData.cycle} plan ($${planData.amount})`) + } } catch (err) { console.error('Error creating payment intent:', err) diff --git a/auto-analyst-frontend/components/analytics/FeedbackAnalytics.tsx b/auto-analyst-frontend/components/analytics/FeedbackAnalytics.tsx index 16202bc1..0e2765ec 100644 --- a/auto-analyst-frontend/components/analytics/FeedbackAnalytics.tsx +++ b/auto-analyst-frontend/components/analytics/FeedbackAnalytics.tsx @@ -66,7 +66,6 @@ const FeedbackAnalytics = () => { setLoading(true) try { const apiKey = getAPIKey() - console.log('Fetching feedback summary with API key:', apiKey ? 'Key provided' : 'No key') // Check if API key is missing if (!apiKey) { @@ -88,7 +87,6 @@ const FeedbackAnalytics = () => { } const data = await response.json() - console.log('Received feedback summary data:', data) setSummary(data) } catch (error) { console.error('Error fetching feedback summary:', error) @@ -107,7 +105,6 @@ const FeedbackAnalytics = () => { setLoadingDetailed(true) try { const apiKey = getAPIKey() - console.log('Fetching detailed feedback with API key:', apiKey ? 'Key provided' : 'No key') // Check if API key is missing if (!apiKey) { @@ -134,7 +131,6 @@ const FeedbackAnalytics = () => { url += `&model_name=${encodeURIComponent(selectedModel)}` } - console.log('Fetching detailed feedback from URL:', url) const response = await fetch(url) if (!response.ok) { @@ -144,7 +140,6 @@ const FeedbackAnalytics = () => { } const data = await response.json() - console.log('Received detailed feedback data:', data) if (!data.feedback || !Array.isArray(data.feedback)) { console.warn('Feedback data is missing or not an array', data) diff --git a/auto-analyst-frontend/components/chat/ChatInput.tsx b/auto-analyst-frontend/components/chat/ChatInput.tsx index 86dd838f..0f433df4 100644 --- a/auto-analyst-frontend/components/chat/ChatInput.tsx +++ b/auto-analyst-frontend/components/chat/ChatInput.tsx @@ -266,7 +266,6 @@ const ChatInput = forwardRef< const isFromAccountsPage = referrer.includes('/account') || referrer.includes('/pricing'); if ((navigationFlag || isFromAccountsPage) && session) { - console.log('Refreshing credits due to navigation from account page (focus event)') // Refresh credits when coming back from accounts/pricing page setTimeout(() => { checkCredits(); diff --git a/auto-analyst-frontend/components/chat/ChatWindow.tsx b/auto-analyst-frontend/components/chat/ChatWindow.tsx index 915d1202..786b571f 100644 --- a/auto-analyst-frontend/components/chat/ChatWindow.tsx +++ b/auto-analyst-frontend/components/chat/ChatWindow.tsx @@ -110,12 +110,10 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Add the fetchLatestCode function before extractCodeFromMessages const fetchLatestCode = useCallback(async (messageId: number) => { if (!messageId) { - console.log("No message_id provided, skipping latest code fetch"); return null; } try { - console.log(`Fetching latest code for message_id: ${messageId}`); const response = await axios.post(`${API_URL}/code/get-latest-code`, { message_id: messageId @@ -125,7 +123,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess }, }); - console.log("Latest code response:", response.data); if (response.data.found) { return response.data; @@ -140,7 +137,7 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Extract code blocks from messages with support for latest code const extractCodeFromMessages = useCallback(async (messagesToExtract: ChatMessage[], messageIndex: number) => { - logger.log("messagesToExtract", messagesToExtract); + // logger.log("messagesToExtract", messagesToExtract); // First, try to fetch the latest code if we have a message_id const message = messagesToExtract[0]; @@ -151,7 +148,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // If we have latest code from a previous execution, use it if (latestCodeData && latestCodeData.latest_code) { - console.log(`Using latest code from database for message_id: ${actualMessageId}`); // Get the language from the code (assuming Python for now) const language = 'python'; // Default to Python @@ -295,7 +291,7 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // If we have a current message with a message_id, make sure it's set in the canvas if (currentMessageIndex !== null && messages[currentMessageIndex] && messages[currentMessageIndex].message_id) { // Log to debug - logger.log(`Setting message_id in canvas: ${messages[currentMessageIndex].message_id} for message index ${currentMessageIndex}`); + // logger.log(`Setting message_id in canvas: ${messages[currentMessageIndex].message_id} for message index ${currentMessageIndex}`); // Fetch the latest code for this message_id const messageId = messages[currentMessageIndex].message_id; @@ -326,14 +322,13 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Log the message ID for debugging const messageId = currentMessage.message_id; - logger.log(`Processing AI message at index ${lastAiMessageIndex} with message_id: ${messageId}`); + // logger.log(`Processing AI message at index ${lastAiMessageIndex} with message_id: ${messageId}`); // Skip if this is an empty message if (!currentMessage.text || (typeof currentMessage.text === 'string' && currentMessage.text.trim() === '') || (typeof currentMessage.text === 'object' && currentMessage.text.type === 'plotly')) { // Skip empty text messages or plotly messages (which don't contain code) - logger.log("Skipping empty message or plotly chart"); return; } @@ -555,8 +550,8 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess return; } - console.log("Code canvas executed with result:", result); - console.log("For code entry:", codeEntry); + // console.log("Code canvas executed with result:", result); + // console.log("For code entry:", codeEntry); // Get the unique message identifier const messageId = codeEntry.messageIndex; @@ -616,7 +611,7 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Add output if available if (result.error) { // Add error output - console.log("Adding error output:", result.error); + // console.log("Adding error output:", result.error); newOutputs[messageId] = [ { type: 'error', @@ -627,7 +622,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess ]; } else if (result.output) { // Add text output - console.log("Adding text output:", result.output); newOutputs[messageId] = [ { type: 'output', @@ -640,7 +634,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Add plotly outputs if any if (result.plotly_outputs && result.plotly_outputs.length > 0) { - console.log("Adding plotly outputs:", result.plotly_outputs); // Process all plotly outputs const plotlyOutputItems: CodeOutput[] = []; @@ -648,7 +641,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess result.plotly_outputs.forEach((plotlyOutput: string) => { try { const plotlyContent = plotlyOutput.replace(/```plotly\n|\n```/g, ""); - console.log("Parsed plotly content:", plotlyContent); const plotlyData = JSON.parse(plotlyContent); plotlyOutputItems.push({ @@ -673,7 +665,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Add matplotlib outputs if any if (result.matplotlib_outputs && result.matplotlib_outputs.length > 0) { - console.log("Adding matplotlib outputs:", result.matplotlib_outputs); // Process all matplotlib outputs const matplotlibOutputItems: CodeOutput[] = []; @@ -681,7 +672,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess result.matplotlib_outputs.forEach((matplotlibOutput: string) => { try { const matplotlibContent = matplotlibOutput.replace(/```matplotlib\n|\n```/g, ""); - console.log("Parsed matplotlib content length:", matplotlibContent.length); matplotlibOutputItems.push({ type: 'matplotlib', @@ -756,7 +746,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Set the message ID in the session first so the backend knows which message this code belongs to if (messageId) { try { - console.log(`Setting message_id in backend: ${messageId}`); await axios.post(`${API_URL}/set-message-info`, { message_id: messageId }, { @@ -780,7 +769,6 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess }, }) - console.log("Code execution response:", response.data); // Pass execution result to handleCodeCanvasExecute handleCodeCanvasExecute(entryId, response.data); @@ -800,7 +788,7 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess setCodeFixes(prev => { const updatedFixes = { ...prev }; updatedFixes[codeId] = (prev[codeId] || 0) + 1; - console.log("ChatWindow: Updated fix counts:", updatedFixes); + // console.log("ChatWindow: Updated fix counts:", updatedFixes); return updatedFixes; }); @@ -844,7 +832,7 @@ const ChatWindow: React.FC = ({ messages, isLoading, onSendMess // Auto-run the fixed code after a short delay to ensure state is updated setTimeout(() => { if (codeEntry.language === "python") { - console.log("Auto-running fixed code for entry:", codeId); + // console.log("Auto-running fixed code for entry:", codeId); executeCodeFromChatWindow(codeId, fixedCode, codeEntry.language); toast({ diff --git a/auto-analyst-frontend/components/chat/CodeCanvas.tsx b/auto-analyst-frontend/components/chat/CodeCanvas.tsx index 35aa4443..ba710888 100644 --- a/auto-analyst-frontend/components/chat/CodeCanvas.tsx +++ b/auto-analyst-frontend/components/chat/CodeCanvas.tsx @@ -163,7 +163,6 @@ const CodeCanvas: React.FC = ({ if (sessionResponse.data && sessionResponse.data.current_message_id) { sessionMessageId = sessionResponse.data.current_message_id; - console.log(`Using current message ID from session: ${sessionMessageId}`); // Set this message ID for the current execution await axios.post(`${API_URL}/set-message-info`, { @@ -190,7 +189,7 @@ const CodeCanvas: React.FC = ({ }, }) - console.log("Code execution response:", response.data); + // console.log("Code execution response:", response.data); // Mark execution as complete const updatedEntries = [...codeEntries]; @@ -236,7 +235,7 @@ const CodeCanvas: React.FC = ({ // Pass execution result to parent component if (onCodeExecute) { - console.log("Passing execution results to parent:", response.data); + // console.log("Passing execution results to parent:", response.data); onCodeExecute(entryId, response.data); } @@ -384,7 +383,6 @@ const CodeCanvas: React.FC = ({ // Update the backend with message ID if available if (messageId) { - console.log(`Setting message_id in backend for edit: ${messageId}`); axios.post(`${API_URL}/set-message-info`, { message_id: messageId }, { @@ -421,7 +419,6 @@ const CodeCanvas: React.FC = ({ .then(sessionResponse => { const sessionMessageId = sessionResponse.data?.current_message_id; if (sessionMessageId) { - console.log(`Using current message ID from session for edit: ${sessionMessageId}`); // Update the session with this message ID axios.post(`${API_URL}/set-message-info`, { diff --git a/auto-analyst-frontend/components/chat/CreditExhaustedModal.tsx b/auto-analyst-frontend/components/chat/CreditExhaustedModal.tsx index 2988c12a..028600a6 100644 --- a/auto-analyst-frontend/components/chat/CreditExhaustedModal.tsx +++ b/auto-analyst-frontend/components/chat/CreditExhaustedModal.tsx @@ -40,7 +40,7 @@ export default function CreditExhaustedModal({
Available options:
    -
  • โ€ข Start a {TrialUtils.getTrialDisplayText()} free trial with {TrialUtils.getTrialCredits()} credits
    • +
  • โ€ข Start a {TrialUtils.getTrialDisplayText()} with {TrialUtils.getTrialCredits()} credits
  • โ€ข Upgrade to Standard plan for 500 credits/month
  • โ€ข Contact us for enterprise plans
diff --git a/auto-analyst-frontend/components/chat/MessageFeedback.tsx b/auto-analyst-frontend/components/chat/MessageFeedback.tsx index bf83a0d6..b596a38f 100644 --- a/auto-analyst-frontend/components/chat/MessageFeedback.tsx +++ b/auto-analyst-frontend/components/chat/MessageFeedback.tsx @@ -41,10 +41,10 @@ const MessageFeedback = ({ messageId, chatId }: MessageFeedbackProps) => { } try { - console.log(`Fetching feedback from ${API_URL}/feedback/message/${messageId}`) + // console.log(`Fetching feedback from ${API_URL}/feedback/message/${messageId}`) const response = await axios.get(`${API_URL}/feedback/message/${messageId}`) if (response.data) { - console.log("Received existing feedback:", response.data) + // console.log("Received existing feedback:", response.data) setExistingFeedback(response.data) setRating(response.data.rating) setIsLocked(true) // Lock the feedback if it already exists @@ -54,7 +54,7 @@ const MessageFeedback = ({ messageId, chatId }: MessageFeedbackProps) => { // Handle 404 (no feedback yet) differently than other errors if (axios.isAxiosError(error)) { if (error.response?.status === 404) { - console.log("No existing feedback found (404 is expected)") + // console.log("No existing feedback found (404 is expected)") setHasError(false) } else { console.error("Error fetching feedback:", error) @@ -114,7 +114,7 @@ const MessageFeedback = ({ messageId, chatId }: MessageFeedbackProps) => { setIsSubmitting(true) setRating(selectedRating) setHasError(false) - console.log(`Submitting rating ${selectedRating} for message ${messageId}`) + // console.log(`Submitting rating ${selectedRating} for message ${messageId}`) try { console.log(`POST to ${API_URL}/feedback/message/${messageId}`, { @@ -135,7 +135,7 @@ const MessageFeedback = ({ messageId, chatId }: MessageFeedbackProps) => { max_tokens: modelSettings?.max_tokens }) - console.log("Rating submitted successfully:", response.data) + // console.log("Rating submitted successfully:", response.data) setExistingFeedback(response.data) setShowThankYou(true) setIsLocked(true) // Lock feedback after submission diff --git a/auto-analyst-frontend/components/deep-analysis/DeepAnalysisSidebar.tsx b/auto-analyst-frontend/components/deep-analysis/DeepAnalysisSidebar.tsx index 54ff64f4..e8cdb90a 100644 --- a/auto-analyst-frontend/components/deep-analysis/DeepAnalysisSidebar.tsx +++ b/auto-analyst-frontend/components/deep-analysis/DeepAnalysisSidebar.tsx @@ -254,7 +254,6 @@ export default function DeepAnalysisSidebar({ // Check if user has access to Deep Analysis feature if (!deepAnalysisAccess.hasAccess) { - console.log('[Deep Analysis] Feature access denied:', deepAnalysisAccess.reason) setShowPremiumUpgradeModal(true) return } @@ -268,7 +267,6 @@ export default function DeepAnalysisSidebar({ const hasEnough = await hasEnoughCredits(deepAnalysisCost) if (!hasEnough) { - console.log(`[Deep Analysis] Insufficient credits. Required: ${deepAnalysisCost}, Available: ${remainingCredits}`) // Store the required credits amount for the modal setRequiredCredits(deepAnalysisCost) @@ -359,7 +357,6 @@ export default function DeepAnalysisSidebar({ } else if (trimmedLine.startsWith('{')) { data = JSON.parse(trimmedLine) } else { - console.log('Skipping non-JSON line:', trimmedLine) continue } @@ -471,7 +468,6 @@ export default function DeepAnalysisSidebar({ } if (userIdForCredits) { - console.log(`[Deep Analysis] Deducting ${deepAnalysisCost} credits for user ${userIdForCredits}`) // Deduct credits through API call const response = await fetch('/api/user/deduct-credits', { @@ -487,7 +483,6 @@ export default function DeepAnalysisSidebar({ }) if (response.ok) { - console.log('[Deep Analysis] Credits deducted successfully') // Refresh the credits display in the UI if (checkCredits) { @@ -648,7 +643,7 @@ export default function DeepAnalysisSidebar({ } if (userIdForCredits) { - console.log(`[Deep Analysis] Deducting ${deepAnalysisCost} credits for user ${userIdForCredits}`) + // console.log(`[Deep Analysis] Deducting ${deepAnalysisCost} credits for user ${userIdForCredits}`) // Deduct credits through API call const response = await fetch('/api/user/deduct-credits', { @@ -664,7 +659,7 @@ export default function DeepAnalysisSidebar({ }) if (response.ok) { - console.log('[Deep Analysis] Credits deducted successfully') + // console.log('[Deep Analysis] Credits deducted successfully') // Refresh the credits display in the UI if (checkCredits) { @@ -866,7 +861,7 @@ export default function DeepAnalysisSidebar({ if (buffer.trim()) { try { const data = JSON.parse(buffer.trim()) - console.log('Processing final buffer data:', data) + // console.log('Processing final buffer data:', data) } catch (error) { console.warn('Failed to parse final buffer:', error) } @@ -1090,7 +1085,7 @@ export default function DeepAnalysisSidebar({ requestData.report_uuid = currentReport.id; } - console.log('Sending analysis data to backend:', requestData); + // console.log('Sending analysis data to backend:', requestData); const response = await fetch(`${API_URL}/deep_analysis/download_report`, { method: 'POST', diff --git a/auto-analyst-frontend/components/landing/StatsTicker.tsx b/auto-analyst-frontend/components/landing/StatsTicker.tsx index 2551d39d..a325d349 100644 --- a/auto-analyst-frontend/components/landing/StatsTicker.tsx +++ b/auto-analyst-frontend/components/landing/StatsTicker.tsx @@ -73,7 +73,7 @@ export default function StatsTicker() { throw new Error('Failed to fetch ticker data') } const data = await response.json() - console.log('Ticker data fetched:', data) // Debug log + // console.log('Ticker data fetched:', data) // Debug log setTickerData(data) setError(null) } catch (err) { diff --git a/auto-analyst-frontend/docs/README.md b/auto-analyst-frontend/docs/README.md index 4a8c5b52..422a4273 100644 --- a/auto-analyst-frontend/docs/README.md +++ b/auto-analyst-frontend/docs/README.md @@ -1,46 +1,235 @@ -# Auto-Analyst Documentation +# Auto-Analyst Frontend Documentation -This documentation covers the payment processing, subscription management, and data storage systems used in Auto-Analyst. +This directory contains comprehensive guides for understanding, developing, and contributing to the Auto-Analyst frontend. -## Table of Contents +## ๐Ÿ“ Frontend Documentation Structure -### Core Systems -- [๐Ÿ”„ Redis Data Schema](./redis-schema.md) - Complete Redis data structure and key patterns -- [๐Ÿ’ณ Stripe Integration](./stripe-integration.md) - Payment processing and subscription management -- [๐ŸŽฏ Webhooks](./webhooks.md) - Event handling and data synchronization -- [๐Ÿ’ฐ Credit System](./credit-system.md) - Credit management and billing logic +### **๐Ÿ—๏ธ Architecture** (`/architecture/`) +- **Component Structure** - Detailed component hierarchy and patterns +- **State Management** - Context providers and store implementations +- **Tech Stack** - Next.js, TypeScript, and styling framework details -### API Reference -- [๐Ÿ“ก API Endpoints](./api-endpoints.md) - Complete list of working endpoints -- [๐Ÿ” Authentication](./authentication.md) - User authentication and session management +### **๐ŸŽจ Features** (`/features/`) +- **Chat System** - AI conversation interface implementation +- **Default Agents** - [Automated agent setup and configuration](./features/default-agents.md) +- **User Interface** - Component library and design patterns +- **Real-time Features** - WebSocket integration and live updates -### Workflows -- [๐Ÿ†“ Trial System](./trial-system.md) - 7-day trial implementation with Stripe manual capture -- [โŒ Cancellation Flows](./cancellation-flows.md) - Different cancellation scenarios and handling +### **๐Ÿ”— Communication** (`/communication/`) +- **API Integration** - Backend communication patterns and endpoints +- **Authentication** - NextAuth.js setup and session management +- **Data Fetching** - API client libraries and data flow -## Quick Start +### **โš™๏ธ Development** (`/development/`) +- **Environment Setup** - [Complete environment variables guide](./development/environment-setup.md) +- **Development Workflow** - Local development and testing procedures -1. **Environment Setup**: Ensure you have the required environment variables set up (see [Stripe Integration](./stripe-integration.md)) -2. **Redis Configuration**: Review the [Redis Schema](./redis-schema.md) for data structure -3. **Webhook Setup**: Configure Stripe webhooks as described in [Webhooks](./webhooks.md) +### **๐Ÿ”ง System** (`/system/`) +- **Authentication** - NextAuth.js setup and user management +- **Redis Schema** - Data storage and caching patterns +- **Webhooks** - Event processing and subscription management +- **Model Registry** - [AI model management and configuration](./system/model-registry.md) +- **Middleware** - [Route protection and request handling](./system/middleware.md) -## System Overview +### **๐Ÿ’ณ Billing** (`/billing/`) +- **Credit System** - Usage tracking and credit management +- **Trial System** - [Complete 2-day trial architecture with Stripe](./billing/trial-system.md) +- **Credit Configuration** - [Centralized credit and trial management](./billing/credit-configuration.md) +- **Stripe Integration** - Payment processing and subscriptions -Auto-Analyst uses a hybrid approach combining: -- **Stripe** for payment processing and subscription management -- **Redis** for fast user data access and caching -- **Webhooks** for real-time synchronization between Stripe and our system -- **Credit-based billing** for usage tracking and limits +### **๐Ÿ”„ User Flows** (`/user-flows/`) +- **Cancellation Flows** - Subscription cancellation processes +- **Onboarding** - User registration and trial activation +- **Payment Flows** - Checkout and billing workflows -## Key Features +### **๐Ÿ“Š State Management** (`/state-management/`) +- **Stores** - Zustand store implementations and patterns +- **Context Providers** - React context for global state +- **Local State** - Component-level state management -- โœ… 7-day free trial with payment authorization -- โœ… Automatic trial-to-paid conversion -- โœ… Real-time credit tracking -- โœ… Subscription management (upgrade, downgrade, cancel) -- โœ… Webhook-based data synchronization -- โœ… Comprehensive error handling +## ๐Ÿš€ Frontend Quick Start -## Support +### **Development Setup** -For questions about this documentation or the system implementation, please refer to the specific documentation files or contact the development team. \ No newline at end of file +```bash +# Navigate to frontend directory +cd auto-analyst-frontend + +# Install dependencies +npm install + +# Set up environment variables +cp .env.example .env.local + +# Start development server +npm run dev +``` + +### **Required Environment Variables** + +Essential variables for frontend development: +```bash +NEXT_PUBLIC_API_URL=http://localhost:8000 # Backend API URL +NEXTAUTH_URL=http://localhost:3000 # Frontend URL +NEXTAUTH_SECRET=your-secret-key # Session encryption +GOOGLE_CLIENT_ID=your-google-client-id # OAuth authentication +UPSTASH_REDIS_REST_URL=your-redis-url # Redis connection +``` + +**๐Ÿ“– See [Environment Setup Guide](./development/environment-setup.md) for complete configuration.** + +## ๐ŸŽฏ Frontend-Specific Features + +### **๐Ÿค– Chat Interface** +- **Real-time Messaging** - WebSocket-powered chat with AI agents +- **Code Execution** - Live Python code execution with syntax highlighting +- **File Uploads** - CSV and Excel file processing for data analysis +- **Message History** - Persistent chat sessions with Redis storage + +### **๐Ÿ” Authentication System** +- **Google OAuth** - Primary authentication via NextAuth.js +- **Session Management** - Secure session handling with Redis +- **Admin Access** - Temporary admin login for analytics dashboard +- **Guest Mode** - Limited trial access for non-authenticated users + +### **๐Ÿ’ณ Credit Management** +- **Usage Tracking** - Real-time credit consumption monitoring +- **Tier System** - Different AI models with varying credit costs (1-20 credits) +- **Trial Management** - 2-day trial with 500 credits +- **Subscription Integration** - Stripe-powered billing interface + +### **๐Ÿ“Š Admin Dashboard** +- **Usage Analytics** - User activity and platform statistics +- **Cost Analysis** - Model usage and cost tracking +- **User Management** - User activity monitoring and management +- **Real-time Metrics** - Live platform performance data + +## ๐Ÿ”ง Development Workflow + +### **Available Scripts** + +```bash +npm run dev # Start development server on localhost:3000 +npm run build # Build optimized production bundle +npm run start # Start production server +npm run lint # Run ESLint for code quality +npm run type-check # TypeScript type checking +``` + +### **Component Development** + +```bash +# Component structure +components/ +โ”œโ”€โ”€ ui/ # Reusable UI components (shadcn/ui) +โ”œโ”€โ”€ chat/ # Chat interface components +โ”œโ”€โ”€ admin/ # Admin dashboard components +โ”œโ”€โ”€ analytics/ # Analytics visualization components +โ””โ”€โ”€ [feature]/ # Feature-specific components +``` + +### **Styling Guidelines** + +- **Tailwind CSS** - Utility-first CSS framework +- **shadcn/ui** - Pre-built component library +- **Responsive Design** - Mobile-first approach +- **Dark Mode** - Theme switching support + +## ๐Ÿ—๏ธ Frontend Architecture + +### **Tech Stack** +- **Next.js 14** - React framework with App Router +- **TypeScript** - Full type safety +- **Tailwind CSS** - Styling framework +- **NextAuth.js** - Authentication +- **Zustand** - State management +- **React Query** - Server state management + +### **Key Patterns** +- **Component Composition** - Reusable, composable components +- **Custom Hooks** - Logic abstraction and reusability +- **Context Providers** - Global state management +- **API Client Libraries** - Centralized API communication + +### **Performance Optimizations** +- **Code Splitting** - Automatic route-based splitting +- **Image Optimization** - Next.js Image component +- **Bundle Analysis** - Webpack bundle analyzer +- **Caching Strategies** - Redis and browser caching + +## ๐Ÿงช Testing + +### **Testing Strategy** +- **Unit Tests** - Component and utility function testing +- **Integration Tests** - API integration and user flow testing +- **E2E Tests** - Full application workflow testing + +### **Testing Tools** +```bash +# Testing commands (to be implemented) +npm run test # Run unit tests +npm run test:e2e # Run end-to-end tests +npm run test:watch # Watch mode for development +``` + +## ๐Ÿ“ฑ Responsive Design + +### **Breakpoints** +- **Mobile**: `< 640px` +- **Tablet**: `640px - 1024px` +- **Desktop**: `> 1024px` + +### **Mobile-First Approach** +- Progressive enhancement from mobile to desktop +- Touch-friendly interfaces +- Optimized performance for mobile devices + +## ๐Ÿ”’ Security Considerations + +### **Frontend Security** +- **Environment Variables** - Proper secret management +- **API Security** - Request authentication and validation +- **XSS Protection** - Input sanitization and CSP headers +- **CSRF Protection** - Built-in NextAuth.js protection + +### **Authentication Security** +- **Secure Sessions** - HttpOnly cookies with secure flags +- **Token Validation** - JWT token verification +- **Route Protection** - Middleware-based route guarding + +## ๐Ÿ“ˆ Performance Monitoring + +### **Core Web Vitals** +- **LCP** - Largest Contentful Paint optimization +- **FID** - First Input Delay minimization +- **CLS** - Cumulative Layout Shift prevention + +### **Monitoring Tools** +- **Vercel Analytics** - Performance metrics +- **Error Tracking** - Error monitoring and reporting +- **User Analytics** - User behavior tracking + +## ๐Ÿค Contributing to Frontend + +### **Code Standards** +- Follow TypeScript best practices +- Use ESLint and Prettier for code formatting +- Write comprehensive component documentation +- Implement proper error handling + +### **Component Guidelines** +- Create reusable, composable components +- Use TypeScript interfaces for props +- Implement proper accessibility (a11y) +- Follow naming conventions + +### **Pull Request Process** +1. Create feature branch from `main` +2. Implement changes with tests +3. Update relevant documentation +4. Submit PR with clear description +5. Address review feedback + +--- + +For project-wide documentation and backend information, see the [main project documentation](../../docs/README.md). \ No newline at end of file diff --git a/auto-analyst-frontend/docs/architecture/components.md b/auto-analyst-frontend/docs/architecture/components.md new file mode 100644 index 00000000..2ad3544f --- /dev/null +++ b/auto-analyst-frontend/docs/architecture/components.md @@ -0,0 +1,298 @@ +# Component Architecture + +The Auto-Analyst frontend uses a hierarchical component structure with clear separation of concerns and reusable patterns. + +## ๐Ÿ›๏ธ Component Hierarchy + +``` +App +โ”œโ”€โ”€ Layout (app/layout.tsx) +โ”‚ โ””โ”€โ”€ ClientLayout +โ”‚ โ”œโ”€โ”€ SessionProvider +โ”‚ โ”œโ”€โ”€ ThemeProvider +โ”‚ โ”œโ”€โ”€ CreditProvider +โ”‚ โ””โ”€โ”€ Page Components +โ””โ”€โ”€ API Routes (app/api/*) +``` + +## ๐Ÿ“ Component Organization + +### **Feature-Based Structure** +``` +components/ +โ”œโ”€โ”€ chat/ # Chat-specific components +โ”œโ”€โ”€ analytics/ # Analytics dashboard components +โ”œโ”€โ”€ admin/ # Admin panel components +โ”œโ”€โ”€ ui/ # Reusable UI components +โ”œโ”€โ”€ landing/ # Landing page components +โ””โ”€โ”€ features/ # Feature-specific components (Conditionally renders content based on feature access) +``` + +### **Component Categories** + +#### **1. Layout Components** +- `ClientLayout.tsx` - Main app wrapper with providers +- `ResponsiveLayout.tsx` - Responsive container +- `AuthProvider.tsx` - Authentication wrapper + +#### **2. Feature Components** +- `ChatInterface.tsx` - Main chat container (1,682 lines) +- `ChatInput.tsx` - Message input handling (2,457 lines) +- `ChatWindow.tsx` - Message display area +- `AnalyticsLayout.tsx` - Analytics dashboard layout + +#### **3. UI Components (shadcn/ui)** +- `Button`, `Input`, `Dialog` - Basic form elements +- `Card`, `Table`, `Tabs` - Layout components +- `Alert`, `Toast` - Notification components + +## ๐ŸŽฏ Component Patterns + +### **1. Container/Presentation Pattern** + +```typescript +// Container Component (Smart) +const ChatInterface: React.FC = () => { + const [messages, setMessages] = useState([]) + const { remainingCredits } = useCredits() + + return ( + + + + + ) +} + +// Presentation Component (Dumb) +interface ChatWindowProps { + messages: Message[] +} + +const ChatWindow: React.FC = ({ messages }) => ( +
+ {messages.map(message => ( + + ))} +
+) +``` + +### **2. Context Provider Pattern** + +```typescript +// Provider Component +const CreditProvider: React.FC<{ children: ReactNode }> = ({ children }) => { + const [remainingCredits, setRemainingCredits] = useState(0) + + const checkCredits = async () => { + // Credit checking logic + } + + return ( + + {children} + + ) +} + +// Consumer Hook +const useCredits = () => { + const context = useContext(CreditContext) + if (!context) throw new Error('useCredits must be used within CreditProvider') + return context +} +``` + +### **3. Compound Component Pattern** + +```typescript +// Main Component +const ChatInterface = { + Container: ChatContainer, + Window: ChatWindow, + Input: ChatInput, + Sidebar: ChatSidebar +} + +// Usage + + + + + +``` + +## ๐Ÿ”„ Component Lifecycle + +### **1. Mount Sequence** +1. Layout providers initialize +2. Authentication state loads +3. User data (credits, subscription) fetches +4. Feature components render +5. Real-time connections establish + +### **2. Update Patterns** +- **Optimistic Updates**: UI updates immediately, syncs with server +- **Progressive Enhancement**: Base functionality works, enhanced features load +- **Error Boundaries**: Graceful error handling per feature + +## ๐Ÿงฉ Key Component Deep Dive + +### **ChatInterface.tsx** (Main Chat Container) + +**Responsibilities:** +- Chat message orchestration +- Agent selection and routing +- Credit validation before actions +- File upload handling +- Real-time message streaming + +**Key Features:** +```typescript +interface ChatInterfaceState { + messages: ChatMessage[] + isLoading: boolean + activeChatId: number | null + selectedAgent: string + remainingCredits: number +} +``` + +### **ChatInput.tsx** (User Input Handler) + +**Responsibilities:** +- Message composition and validation +- File attachment handling +- Command suggestions +- Credit cost preview +- Code execution triggers + +**Key Features:** +- Auto-resize textarea +- Drag & drop file upload +- Agent command shortcuts +- Real-time cost calculation + +### **MessageContent.tsx** (Message Rendering) + +**Responsibilities:** +- Markdown rendering +- Code syntax highlighting +- Plotly chart display +- Interactive code blocks +- Message actions (copy, edit, delete) + +## ๐ŸŽจ Styling Patterns + +### **1. Tailwind CSS Classes** +```typescript +const buttonVariants = { + default: "bg-primary text-primary-foreground hover:bg-primary/90", + destructive: "bg-destructive text-destructive-foreground hover:bg-destructive/90", + outline: "border border-input bg-background hover:bg-accent", +} +``` + +### **2. CSS Modules for Complex Components** +```typescript +// For components with complex animations or layouts +import styles from './ChatInterface.module.css' + +const ChatInterface = () => ( +
+
+ {/* Content */} +
+
+) +``` + +### **3. Dynamic Class Names** +```typescript +import { cn } from '@/lib/utils' + +const Button = ({ variant, className, ...props }) => ( +
} + onError={(error) => console.error('Feature error:', error)} + > + {children} + + ) +} +``` + +### **3. Loading States** +```typescript +const DataComponent = () => { + const { data, isLoading, error } = useSWR('/api/data') + + if (isLoading) return + if (error) return + if (!data) return + + return +} +``` + +### **4. Accessibility** +```typescript +const AccessibleButton = ({ children, ...props }) => ( + +) +``` + +## ๐Ÿ“Š Component Metrics + +### **Large Components** (Consider refactoring) +- `ChatInterface.tsx`: 1,682 lines +- `ChatInput.tsx`: 2,457 lines +- `ChatWindow.tsx`: 1,698 lines + +### **Reusable UI Components** +- 27 components in `/components/ui/` +- Used across 50+ other components +- Consistent design system + +### **Feature Components** +- 24 chat-related components +- 8 analytics components +- 5 admin components \ No newline at end of file diff --git a/auto-analyst-frontend/docs/architecture/overview.md b/auto-analyst-frontend/docs/architecture/overview.md new file mode 100644 index 00000000..b1bbfd33 --- /dev/null +++ b/auto-analyst-frontend/docs/architecture/overview.md @@ -0,0 +1,259 @@ +# Frontend Architecture Overview + +The Auto-Analyst frontend is built with **Next.js 14** using the App Router pattern, providing a modern, scalable architecture for an AI-powered analytics platform. + +## ๐Ÿ—๏ธ Tech Stack + +### **Core Framework** +- **Next.js 13** - React framework with App Router +- **React 18** - UI library with concurrent features +- **TypeScript** - Type safety and developer experience + +### **Styling & UI** +- **Tailwind CSS** - Utility-first CSS framework +- **shadcn/ui** - Pre-built accessible components +- **Framer Motion** - Animation library +- **next-themes** - Theme management + +### **State Management** +- **React Context** - Global state for auth, credits +- **Zustand** - Client-side state store +- **SWR** - Data fetching and caching + +### **Authentication** +- **NextAuth.js** - Authentication framework +- **Google OAuth** - Social login integration +- **JWT Tokens** - Session management + +### **Backend Communication** +- **Axios** - HTTP client for API calls +- **Redis** - Session and cache storage +- **WebSockets/SSE** - Real-time communication + +## ๐ŸŽฏ Key Features + +### **1. AI Chat Interface** +- Multi-agent conversation system +- Real-time code execution +- File upload and dataset processing +- Interactive data visualization + +### **2. Credit System** +- Usage-based billing +- Model tier management +- Real-time credit tracking +- Subscription integration + +### **3. Analytics Dashboard** +- Usage analytics +- Performance monitoring +- User management +- Revenue tracking + +### **4. Admin Panel** +- User management +- System monitoring +- Configuration management +- Analytics reporting + +## ๐Ÿ›๏ธ Architecture Patterns + +### **1. Provider Pattern** +```typescript + + + + + {children} + + + + +``` + +### **2. API Route Middleware** +``` +Frontend โ†’ Next.js API Routes โ†’ Backend API + โ†“ + Redis Cache +``` + +### **3. Component Composition** +- Feature-based folder structure +- Reusable UI components +- Container/Presentation pattern +- Compound components + +## ๐Ÿ”„ Data Flow + +### **Authentication Flow** +``` +User Login โ†’ NextAuth โ†’ Session โ†’ Redis โ†’ Context โ†’ Components +``` + +### **Chat Message Flow** +``` +User Input โ†’ Credit Check โ†’ Backend API โ†’ Response Stream โ†’ UI Update +``` + +### **State Management Flow** +``` +Component โ†’ Hook โ†’ Context/Store โ†’ API โ†’ Backend โ†’ Redis +``` + +## ๐Ÿ“ Project Structure + +``` +auto-analyst-frontend/ +โ”œโ”€โ”€ app/ # Next.js App Router +โ”‚ โ”œโ”€โ”€ api/ # API routes (middleware) +โ”‚ โ”œโ”€โ”€ chat/ # Chat pages +โ”‚ โ”œโ”€โ”€ analytics/ # Analytics dashboard +โ”‚ โ””โ”€โ”€ admin/ # Admin panel +โ”œโ”€โ”€ components/ # React components +โ”‚ โ”œโ”€โ”€ ui/ # shadcn/ui components +โ”‚ โ”œโ”€โ”€ chat/ # Chat components +โ”‚ โ”œโ”€โ”€ analytics/ # Analytics components +โ”‚ โ””โ”€โ”€ admin/ # Admin components +โ”œโ”€โ”€ lib/ # Utilities and configurations +โ”‚ โ”œโ”€โ”€ api/ # API client functions +โ”‚ โ”œโ”€โ”€ contexts/ # React contexts +โ”‚ โ”œโ”€โ”€ hooks/ # Custom hooks +โ”‚ โ”œโ”€โ”€ store/ # Zustand stores +โ”‚ โ””โ”€โ”€ utils/ # Helper functions +โ”œโ”€โ”€ config/ # Configuration files +โ””โ”€โ”€ types/ # TypeScript definitions +``` + +## ๐Ÿ”Œ Backend Integration + +### **API Configuration** +```typescript +// config/api.ts +const API_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000'; +``` + +### **Communication Patterns** + +#### **1. Direct API Calls** +```typescript +// Direct calls to Python FastAPI backend +const response = await axios.post(`${API_URL}/chat`, { + message: userMessage, + agent: selectedAgent +}); +``` + +#### **2. Next.js API Middleware** +```typescript +// app/api/user/credits/route.ts +export async function GET(request: NextRequest) { + const token = await getToken({ req: request }); + const creditsData = await redis.hgetall(KEYS.USER_CREDITS(token.sub)); + return NextResponse.json(creditsData); +} +``` + +#### **3. Real-time Communication** +```typescript +// WebSocket/SSE for chat streaming +const eventSource = new EventSource(`${API_URL}/chat/stream`); +eventSource.onmessage = (event) => { + const data = JSON.parse(event.data); + updateMessageContent(data); +}; +``` + +## ๐ŸŽจ Design System + +### **Color Scheme** +- **Primary**: Blue (#3B82F6) +- **Secondary**: Purple (#8B5CF6) +- **Success**: Green (#10B981) +- **Warning**: Orange (#F59E0B) +- **Danger**: Red (#EF4444) + +### **Typography** +- **Font**: Inter (Google Fonts) +- **Headings**: Font weights 600-700 +- **Body**: Font weight 400 +- **Code**: JetBrains Mono + +### **Spacing** +- **Base**: 4px (0.25rem) +- **Scale**: 4, 8, 12, 16, 20, 24, 32, 40, 48, 64px +- **Consistent**: Using Tailwind spacing scale + +## ๐Ÿ”’ Security Considerations + +### **Authentication** +- JWT token validation +- Secure session storage +- CSRF protection +- Rate limiting + +### **Data Protection** +- Input sanitization +- XSS prevention +- HTTPS enforcement +- Secure headers + +### **API Security** +- Authentication middleware +- Request validation +- Error handling +- Audit logging + +## ๐Ÿ“Š Performance Optimizations + +### **Code Splitting** +- Route-based code splitting +- Dynamic imports for heavy components +- Tree shaking for unused code + +### **Caching** +- Redis for session data +- Browser caching for static assets +- SWR for API response caching + +### **Bundle Optimization** +- Next.js built-in optimizations +- Image optimization +- CSS purging +- Compression + +## ๐Ÿงช Testing Strategy + +### **Unit Testing** +- Jest for utility functions +- React Testing Library for components +- Mock API responses + +### **Integration Testing** +- End-to-end flows +- API integration tests +- Authentication flows + +### **Performance Testing** +- Lighthouse audits +- Bundle analysis +- Core Web Vitals monitoring + +## ๐Ÿš€ Deployment + +### **Build Process** +```bash +npm run build # Next.js production build +npm run start # Production server +``` + +### **Environment Variables** +- `NEXT_PUBLIC_API_URL` - Backend API URL +- `NEXTAUTH_SECRET` - NextAuth secret +- `GOOGLE_CLIENT_ID` - OAuth client ID +- `UPSTASH_REDIS_REST_URL` - Redis connection + +### **Hosting** +- **Vercel** - Recommended for Next.js +- **AWS/Azure** - Alternative cloud providers +- **Docker** - Containerized deployment \ No newline at end of file diff --git a/auto-analyst-frontend/docs/authentication.md b/auto-analyst-frontend/docs/authentication.md deleted file mode 100644 index e512ec76..00000000 --- a/auto-analyst-frontend/docs/authentication.md +++ /dev/null @@ -1,663 +0,0 @@ -# Authentication - -This document covers the authentication system in Auto-Analyst, including NextAuth.js configuration, session management, and API protection. - -## Overview - -Auto-Analyst uses **NextAuth.js** for authentication with support for multiple providers and JWT-based sessions. Authentication is required for all core features including trials, subscriptions, and chat functionality. - -## Authentication Providers - -### Google OAuth -Primary authentication method for user sign-in. - -```typescript -// lib/auth.ts -import GoogleProvider from "next-auth/providers/google" - -const providers = [ - GoogleProvider({ - clientId: process.env.GOOGLE_CLIENT_ID!, - clientSecret: process.env.GOOGLE_CLIENT_SECRET!, - authorization: { - params: { - prompt: "consent", - access_type: "offline", - response_type: "code" - } - } - }) -] -``` - -### Required Environment Variables -```bash -GOOGLE_CLIENT_ID=your_google_client_id -GOOGLE_CLIENT_SECRET=your_google_client_secret -NEXTAUTH_SECRET=your_nextauth_secret -NEXTAUTH_URL=http://localhost:3000 # or your production URL -``` - -## NextAuth Configuration - -### Main Configuration -**File**: `app/api/auth/[...nextauth]/route.ts` - -```typescript -import NextAuth from "next-auth" -import GoogleProvider from "next-auth/providers/google" -import { NextAuthOptions } from "next-auth" - -const authOptions: NextAuthOptions = { - providers: [ - GoogleProvider({ - clientId: process.env.GOOGLE_CLIENT_ID!, - clientSecret: process.env.GOOGLE_CLIENT_SECRET! - }) - ], - - session: { - strategy: "jwt", - maxAge: 30 * 24 * 60 * 60, // 30 days - updateAge: 24 * 60 * 60 // 24 hours - }, - - jwt: { - maxAge: 30 * 24 * 60 * 60 // 30 days - }, - - callbacks: { - async jwt({ token, user, account }) { - // Store user info in JWT token - if (user) { - token.sub = user.id - token.email = user.email - token.name = user.name - token.picture = user.image - } - return token - }, - - async session({ session, token }) { - // Send properties to the client - if (token) { - session.user.id = token.sub - session.user.email = token.email - session.user.name = token.name - session.user.image = token.picture - } - return session - }, - - async signIn({ user, account, profile }) { - // Always allow sign in for valid Google accounts - return true - }, - - async redirect({ url, baseUrl }) { - // Redirect to chat page after successful login - if (url.startsWith("/")) return `${baseUrl}${url}` - else if (new URL(url).origin === baseUrl) return url - return `${baseUrl}/chat` - } - }, - - pages: { - signIn: "/login", - signOut: "/signout", - error: "/login" - } -} - -const handler = NextAuth(authOptions) -export { handler as GET, handler as POST } -``` - -## Session Management - -### Client-Side Session Access - -```typescript -'use client' -import { useSession, signIn, signOut } from "next-auth/react" - -function SessionComponent() { - const { data: session, status } = useSession() - - if (status === "loading") return

Loading...

- - if (status === "unauthenticated") { - return ( - - ) - } - - return ( -
-

Signed in as {session.user?.email}

- -
- ) -} -``` - -### Server-Side Session Access - -```typescript -// API Routes -import { getToken } from "next-auth/jwt" - -export async function GET(request: NextRequest) { - const token = await getToken({ req: request }) - - if (!token) { - return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) - } - - const userId = token.sub - const userEmail = token.email - - // Use user data... -} -``` - -```typescript -// Server Components -import { getServerSession } from "next-auth/next" -import { authOptions } from "@/app/api/auth/[...nextauth]/route" - -export default async function ServerComponent() { - const session = await getServerSession(authOptions) - - if (!session) { - redirect('/login') - } - - return
Welcome {session.user?.name}
-} -``` - -## Session Provider Setup - -### Root Layout Configuration -**File**: `app/layout.tsx` - -```typescript -import { SessionProvider } from "next-auth/react" -import { AuthProvider } from "@/components/AuthProvider" - -export default function RootLayout({ - children, -}: { - children: React.ReactNode -}) { - return ( - - - - {children} - - - - ) -} -``` - -### Auth Provider Component -**File**: `components/AuthProvider.tsx` - -```typescript -'use client' -import { SessionProvider } from "next-auth/react" - -export function AuthProvider({ children }: { children: React.ReactNode }) { - return {children} -} -``` - -## API Route Protection - -### Standard Protection Pattern - -```typescript -// app/api/example/route.ts -import { NextRequest, NextResponse } from 'next/server' -import { getToken } from 'next-auth/jwt' - -export async function GET(request: NextRequest) { - // Authenticate user - const token = await getToken({ req: request }) - - if (!token?.sub) { - return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) - } - - const userId = token.sub - const userEmail = token.email - - try { - // Your API logic here - return NextResponse.json({ success: true }) - } catch (error) { - return NextResponse.json( - { error: 'Internal server error' }, - { status: 500 } - ) - } -} -``` - -### Reusable Auth Middleware - -```typescript -// lib/auth-middleware.ts -import { NextRequest } from 'next/server' -import { getToken } from 'next-auth/jwt' - -export async function requireAuth(request: NextRequest) { - const token = await getToken({ req: request }) - - if (!token?.sub) { - throw new Error('Unauthorized') - } - - return { - userId: token.sub, - userEmail: token.email, - userName: token.name, - userImage: token.picture - } -} - -// Usage in API routes -export async function GET(request: NextRequest) { - try { - const { userId, userEmail } = await requireAuth(request) - // Your protected logic here - } catch (error) { - return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) - } -} -``` - -## Frontend Authentication Components - -### Login Page -**File**: `app/login/page.tsx` - -```typescript -'use client' -import { signIn, getSession } from "next-auth/react" -import { useEffect } from "react" -import { useRouter } from "next/navigation" -import { Button } from "@/components/ui/button" - -export default function LoginPage() { - const router = useRouter() - - useEffect(() => { - // Redirect if already authenticated - getSession().then((session) => { - if (session) { - router.push('/chat') - } - }) - }, [router]) - - const handleSignIn = () => { - signIn('google', { callbackUrl: '/chat' }) - } - - return ( -
-
-
-

Sign in to Auto-Analyst

-

- Access your AI-powered analytics platform -

-
- - -
-
- ) -} -``` - -### Sign Out Page -**File**: `app/signout/page.tsx` - -```typescript -'use client' -import { signOut, useSession } from "next-auth/react" -import { useEffect } from "react" -import { useRouter } from "next/navigation" - -export default function SignOutPage() { - const { data: session } = useSession() - const router = useRouter() - - useEffect(() => { - if (session) { - signOut({ callbackUrl: '/' }) - } else { - router.push('/') - } - }, [session, router]) - - return ( -
-
-

Signing you out...

-

Thank you for using Auto-Analyst

-
-
- ) -} -``` - -### Navigation Auth State -**File**: `components/layout.tsx` - -```typescript -'use client' -import { useSession, signIn, signOut } from "next-auth/react" -import { Button } from "@/components/ui/button" -import { Avatar, AvatarFallback, AvatarImage } from "@/components/ui/avatar" - -function AuthButton() { - const { data: session, status } = useSession() - - if (status === "loading") { - return
- } - - if (status === "unauthenticated") { - return ( - - ) - } - - return ( -
- - - - {session.user?.name?.charAt(0) || 'U'} - - - - -
- ) -} -``` - -## Route Protection - -### Page-Level Protection - -```typescript -// app/chat/page.tsx -import { getServerSession } from "next-auth/next" -import { authOptions } from "@/app/api/auth/[...nextauth]/route" -import { redirect } from "next/navigation" - -export default async function ChatPage() { - const session = await getServerSession(authOptions) - - if (!session) { - redirect('/login') - } - - return ( -
-

Welcome to Chat, {session.user?.name}

- {/* Chat interface */} -
- ) -} -``` - -### Middleware Protection - -```typescript -// middleware.ts -import { withAuth } from "next-auth/middleware" - -export default withAuth( - function middleware(req) { - // Additional middleware logic if needed - }, - { - callbacks: { - authorized: ({ token }) => !!token - }, - } -) - -export const config = { - matcher: [ - '/chat/:path*', - '/account/:path*', - '/analytics/:path*', - '/api/user/:path*', - '/api/trial/:path*', - '/api/checkout-sessions/:path*' - ] -} -``` - -## User Profile Management - -### Storing User Data - -```typescript -// When user signs in, store profile in Redis -export async function storeUserProfile(userId: string, userData: any) { - await redis.hset(KEYS.USER_PROFILE(userId), { - email: userData.email, - name: userData.name || 'User', - image: userData.image || '', - joinedDate: new Date().toISOString().split('T')[0], - lastLogin: new Date().toISOString() - }) -} -``` - -### Accessing User Profile - -```typescript -// lib/redis.ts - profileUtils -export const profileUtils = { - async getUserProfile(userId: string) { - const profile = await redis.hgetall(KEYS.USER_PROFILE(userId)) - return profile || {} - }, - - async updateUserProfile(userId: string, updates: any) { - await redis.hset(KEYS.USER_PROFILE(userId), { - ...updates, - lastUpdated: new Date().toISOString() - }) - } -} -``` - -## Session Security - -### JWT Configuration - -```typescript -// Secure JWT settings -const jwtOptions = { - secret: process.env.NEXTAUTH_SECRET, - encryption: true, - maxAge: 30 * 24 * 60 * 60, // 30 days - algorithm: 'HS256' -} -``` - -### Session Validation - -```typescript -// Enhanced token validation -export async function validateSession(request: NextRequest) { - const token = await getToken({ req: request }) - - if (!token) { - throw new Error('No session token') - } - - // Check token expiration - if (Date.now() >= token.exp * 1000) { - throw new Error('Token expired') - } - - // Validate user still exists - const userExists = await redis.exists(KEYS.USER_PROFILE(token.sub)) - if (!userExists) { - throw new Error('User not found') - } - - return token -} -``` - -## Error Handling - -### Authentication Errors - -```typescript -// Common authentication error patterns -export function handleAuthError(error: any) { - switch (error.type) { - case 'OAuthAccountNotLinked': - return 'Account already exists with different provider' - case 'EmailCreateAccount': - return 'Account creation failed' - case 'Callback': - return 'Authentication callback failed' - case 'OAuthCallback': - return 'OAuth provider callback failed' - case 'OAuthCreateAccount': - return 'Failed to create OAuth account' - case 'SessionRequired': - return 'Please sign in to continue' - default: - return 'Authentication failed' - } -} -``` - -### Error Pages - -```typescript -// app/login/error/page.tsx -'use client' -import { useSearchParams } from 'next/navigation' - -export default function AuthError() { - const searchParams = useSearchParams() - const error = searchParams.get('error') - - const errorMessage = handleAuthError({ type: error }) - - return ( -
-
-

- Authentication Error -

-

{errorMessage}

- -
-
- ) -} -``` - -## Testing Authentication - -### Local Development - -```bash -# Set up Google OAuth for localhost -# In Google Cloud Console: -# - Create OAuth 2.0 credentials -# - Add http://localhost:3000 to authorized origins -# - Add http://localhost:3000/api/auth/callback/google to redirect URIs -``` - -### Test User Flows - -1. **Sign In Flow** - ```typescript - // Test successful sign in - await signIn('google') - // Should redirect to /chat - ``` - -2. **Protected Route Access** - ```typescript - // Access protected route without auth - // Should redirect to /login - ``` - -3. **Session Persistence** - ```typescript - // Refresh page after sign in - // Should maintain session - ``` - -4. **Sign Out Flow** - ```typescript - // Test sign out - await signOut() - // Should clear session and redirect - ``` - -## Best Practices - -### Security -1. **Always validate sessions** server-side -2. **Use HTTPS** in production -3. **Secure environment variables** properly -4. **Implement CSRF protection** (built into NextAuth) -5. **Monitor authentication events** - -### Performance -1. **Cache session data** appropriately -2. **Minimize token size** -3. **Use efficient session storage** -4. **Implement session refresh** logic - -### User Experience -1. **Clear error messages** for auth failures -2. **Smooth redirect flows** after sign in -3. **Loading states** during authentication -4. **Remember user preferences** across sessions - -### Development -1. **Consistent auth patterns** across the app -2. **Centralized auth logic** in utilities -3. **Comprehensive error handling** -4. **Authentication testing** in CI/CD \ No newline at end of file diff --git a/auto-analyst-frontend/docs/billing/credit-configuration.md b/auto-analyst-frontend/docs/billing/credit-configuration.md new file mode 100644 index 00000000..01e183d5 --- /dev/null +++ b/auto-analyst-frontend/docs/billing/credit-configuration.md @@ -0,0 +1,284 @@ +# Credit Configuration Guide + +This guide explains how to configure credits, trials, and billing settings using the centralized `credits-config.ts` file. + +## Overview + +All credit-related configurations are managed in `lib/credits-config.ts` to ensure consistency across the application. This includes: + +- Trial periods and credits +- Plan credit allocations +- Feature costs +- Credit thresholds and warnings + +## Trial Configuration + +### Changing Trial Duration + +To modify the trial period, update the `TRIAL_CONFIG` object: + +```typescript +// lib/credits-config.ts +export const TRIAL_CONFIG: TrialConfig = { + duration: 2, // Change this number + unit: 'days', // 'days' | 'hours' | 'minutes' | 'weeks' + displayText: '2-Day Free Trial', // Update display text + credits: 500 // Trial credits +} +``` + +**Examples:** +```typescript +// 7-day trial +{ + duration: 7, + unit: 'days', + displayText: '7-Day Free Trial', + credits: 1000 +} + +// 48-hour trial +{ + duration: 48, + unit: 'hours', + displayText: '48-Hour Trial', + credits: 200 +} +``` + +### Trial Utilities + +The `TrialUtils` class provides helper methods: + +```typescript +// Get trial duration in milliseconds +TrialUtils.getTrialDurationMs() + +// Get trial end timestamp for Stripe +TrialUtils.getTrialEndTimestamp() + +// Check if trial has expired +TrialUtils.isTrialExpired(trialEndDate) + +// Get display text +TrialUtils.getTrialDisplayText() // "2-Day Free Trial" +``` + +## Plan Credit Configuration + +### Modifying Plan Credits + +Update credit allocations in the `PLAN_CREDITS` object: + +```typescript +export const PLAN_CREDITS: Record = { + 'Trial': { + total: 500, // Trial credits + displayName: 'Trial Plan', + type: 'TRIAL', + isUnlimited: false + }, + 'Standard': { + total: 1000, // Monthly Standard credits + displayName: 'Standard Plan', + type: 'STANDARD', + isUnlimited: false + }, + 'Pro': { + total: 999999, // "Unlimited" threshold + displayName: 'Pro Plan', + type: 'PRO', + isUnlimited: true + } +} +``` + +### Adding New Plans + +To add a new plan type: + +1. **Add to TypeScript types:** +```typescript +export type PlanType = 'TRIAL' | 'STANDARD' | 'PRO' | 'ENTERPRISE' +export type PlanName = 'Trial' | 'Standard' | 'Pro' | 'Enterprise' +``` + +2. **Add plan configuration:** +```typescript +'Enterprise': { + total: 5000, + displayName: 'Enterprise Plan', + type: 'ENTERPRISE', + isUnlimited: false, + minimum: 1000 +} +``` + +3. **Update normalization logic:** +```typescript +private static normalizePlanName(planName: string): PlanName { + const normalized = planName.toLowerCase().trim() + + if (normalized.includes('enterprise')) return 'Enterprise' + // ... existing logic +} +``` + +## Feature Costs + +### Configuring Feature Pricing + +Update feature costs in the `FEATURE_COSTS` object: + +```typescript +export const FEATURE_COSTS = { + DEEP_ANALYSIS: 50, // Current: 50 credits + CODE_GENERATION: 10, // Example: New feature + DATASET_UPLOAD: 25, // Example: New feature +} +``` + +### Using Feature Costs + +```typescript +// Check if user can afford a feature +const canAfford = CreditConfig.canAffordDeepAnalysis(userCredits) + +// Get feature cost +const cost = CreditConfig.getDeepAnalysisCost() // 50 + +// Custom feature check +const hasCredits = userCredits >= FEATURE_COSTS.CODE_GENERATION +``` + +## Credit Thresholds + +### Warning and Limit Configuration + +```typescript +export const CREDIT_THRESHOLDS: CreditThresholds = { + unlimitedThreshold: 99999, // When to show "Unlimited" + defaultTrial: 500, // Default trial credits + warningThreshold: 80 // Warn at 80% usage +} +``` + +### Usage Examples + +```typescript +// Check if plan appears unlimited +const isUnlimited = CreditConfig.isUnlimitedTotal(999999) // true + +// Check if user should be warned +const shouldWarn = CreditConfig.shouldWarnLowCredits(400, 500) // true (80% used) + +// Format credits for display +const display = CreditConfig.formatCreditTotal(999999) // "Unlimited" +``` + +## Environment-Specific Configuration + +### Development vs Production + +For different environments, you can use environment variables: + +```typescript +// Example: Environment-specific trial credits +export const TRIAL_CONFIG: TrialConfig = { + duration: parseInt(process.env.TRIAL_DURATION || '2'), + unit: 'days', + displayText: `${process.env.TRIAL_DURATION || '2'}-Day Free Trial`, + credits: parseInt(process.env.TRIAL_CREDITS || '500') +} + +// Non-authenticated user limits +const MAX_FREE_QUERIES = process.env.NODE_ENV === 'development' ? 20000 : 0 +``` + +## Utility Functions + +### CreditConfig Class Methods + +```typescript +// Plan operations +CreditConfig.getCreditsForPlan('Standard') // Get plan credits +CreditConfig.isUnlimitedPlan('Pro') // Check if unlimited +CreditConfig.getAllPlans() // Get all plan configs + +// Credit calculations +CreditConfig.calculateUsagePercentage(250, 500) // 50% +CreditConfig.hasCreditsRemaining(450, 500) // true +CreditConfig.formatRemainingCredits(450, 500) // "50" + +// Date utilities +CreditConfig.getNextResetDate() // Next month reset +CreditConfig.getTrialEndDate() // Trial end date +``` + +## Impact of Changes + +### What Gets Updated Automatically + +When you modify `credits-config.ts`, these components update automatically: + +- **Trial displays** (duration, credits shown in UI) +- **Plan comparisons** (pricing page, upgrade prompts) +- **Credit warnings** (low credit notifications) +- **Usage calculations** (progress bars, percentages) +- **Feature access** (deep analysis, premium features) + +### Manual Updates Required + +Some areas may need manual updates: + +- **Stripe configuration** (trial periods in payment setup) +- **Database migrations** (if adding new plan types) +- **API documentation** (if changing credit costs) +- **Marketing copy** (landing page, emails) + +## Best Practices + +### 1. Test Changes Thoroughly +```bash +# Test different scenarios +npm run test:credits +npm run test:trial-flows +``` + +### 2. Coordinate with Backend +Ensure backend credit costs match frontend configuration: + +```python +# auto-analyst-backend/src/utils/model_registry.py +MODEL_TIERS = { + "tier1": {"credits": 1}, # Must match frontend MODEL_TIERS + "tier2": {"credits": 3}, + "tier3": {"credits": 5}, + "tier4": {"credits": 20} +} +``` + +> **Note**: Model-specific configurations are managed through the Model Registry. See [Model Registry Guide](../system/model-registry.md) for adding new models. + +### 3. Document Changes +Update this documentation when making significant changes to credit configuration. + +### 4. Consider Migration Path +When changing existing plans, provide migration logic for existing users. + +## Troubleshooting + +### Common Issues + +1. **Trial not reflecting changes**: Clear Redis cache and restart application +2. **Credit display incorrect**: Check `CreditProvider` context updates +3. **Stripe trial mismatch**: Update Stripe configuration to match `TRIAL_CONFIG` + +### Debug Utilities + +```typescript +// Debug current configuration +console.log('Trial Config:', TrialUtils.getTrialConfig()) +console.log('All Plans:', CreditConfig.getAllPlans()) +console.log('Feature Costs:', FEATURE_COSTS) +``` \ No newline at end of file diff --git a/auto-analyst-frontend/docs/billing/credit-system.md b/auto-analyst-frontend/docs/billing/credit-system.md new file mode 100644 index 00000000..0437f35d --- /dev/null +++ b/auto-analyst-frontend/docs/billing/credit-system.md @@ -0,0 +1,126 @@ +# Credit System Documentation + +The Auto-Analyst credit system manages user usage and billing based on AI model consumption. + +## Credit Overview + +### Credit Types +- **Standard Credits**: Used for basic models (1-5 credits per query) +- **Premium Credits**: Used for advanced models (10-20 credits per query) +- **Unlimited**: Pro plan users have unlimited usage + +### Model Tiers +```typescript +// lib/model-registry.ts +export const MODEL_TIERS = { + "tier1": { "credits": 1, "models": ["claude-3-5-haiku", "llama3-8b"] }, + "tier2": { "credits": 3, "models": ["gpt-4o-mini", "o1-mini"] }, + "tier3": { "credits": 5, "models": ["gpt-4o", "claude-3-5-sonnet"] }, + "tier4": { "credits": 20, "models": ["o1-pro", "claude-3-opus"] } +} +``` + +## Credit Management + +### Credit Provider (React Context) +```typescript +// lib/contexts/credit-context.tsx +const CreditProvider = ({ children }) => { + const [remainingCredits, setRemainingCredits] = useState(0) + const [isChatBlocked, setIsChatBlocked] = useState(false) + + const checkCredits = async () => { + // Fetch from API or Redis + const credits = await fetch('/api/user/credits') + setRemainingCredits(credits.remaining) + } + + const deductCredits = async (amount) => { + // Optimistic update + server sync + setRemainingCredits(prev => prev - amount) + return await creditUtils.deductCredits(userId, amount) + } +} +``` + +### Credit Storage +- **Redis**: Server-side credit tracking +- **Local Storage**: Client-side caching +- **Context**: Real-time UI updates + +## Credit Flow + +### 1. Pre-flight Check +```typescript +const handleMessageSend = async (message) => { + const cost = getModelCreditCost(selectedModel) + + if (!await hasEnoughCredits(cost)) { + setInsufficientCreditsModalOpen(true) + return + } + + // Proceed with message +} +``` + +### 2. Credit Deduction +```typescript +// Deduct after successful API response +if (response.ok) { + await deductCredits(modelCost) +} +``` + +### 3. Credit Reset +```typescript +// Monthly/yearly reset based on subscription +const resetUserCredits = async (userId) => { + const subscription = await getSubscription(userId) + const newCredits = getPlanCredits(subscription.plan) + + await redis.hset(KEYS.USER_CREDITS(userId), { + total: newCredits, + used: '0', + resetDate: getNextResetDate(subscription.interval) + }) +} +``` + +## UI Components + +### Credit Balance Display +```typescript +const CreditBalance = () => { + const { remainingCredits, isLoading } = useCredits() + + return ( +
+ {isLoading ? ( + + ) : ( + {remainingCredits} credits + )} +
+ ) +} +``` + +### Insufficient Credits Modal +- Triggered when user lacks credits +- Options to upgrade subscription +- Shows current usage and limits + +## Credit Analytics + +### Usage Tracking +- Model usage by user +- Cost analysis per query +- Credit consumption patterns +- Popular model preferences + +### Admin Dashboard +- Total credit usage +- Revenue per credit +- User consumption analytics +- Credit allocation efficiency \ No newline at end of file diff --git a/auto-analyst-frontend/docs/billing/stripe-integration.md b/auto-analyst-frontend/docs/billing/stripe-integration.md new file mode 100644 index 00000000..a14c0677 --- /dev/null +++ b/auto-analyst-frontend/docs/billing/stripe-integration.md @@ -0,0 +1,79 @@ +# Stripe Integration Documentation + +This document covers the Stripe payment processing integration for Auto-Analyst subscriptions and billing. + +## Overview + +Auto-Analyst uses Stripe for handling: +- Subscription management +- Payment processing +- Webhook handling +- Promo codes and discounts +- Invoice management + +## Configuration + +### Stripe Keys +```bash +STRIPE_SECRET_KEY=sk_test_... +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_... +STRIPE_WEBHOOK_SECRET=whsec_... +``` + +### Product Configuration +Located in pricing configuration files with Stripe product IDs for each plan. + +## Payment Flow + +### 1. Checkout Session Creation +```typescript +// app/api/create-checkout-session/route.ts +const session = await stripe.checkout.sessions.create({ + payment_method_types: ['card'], + line_items: [{ + price: priceId, + quantity: 1, + }], + mode: 'subscription', + success_url: `${origin}/checkout/success?session_id={CHECKOUT_SESSION_ID}`, + cancel_url: `${origin}/checkout/error`, + metadata: { + userId: userId + } +}) +``` + +### 2. Webhook Processing +```typescript +// app/api/webhooks/route.ts +export async function POST(request: Request) { + const sig = request.headers.get('stripe-signature') + const event = stripe.webhooks.constructEvent(body, sig, webhookSecret) + + switch (event.type) { + case 'invoice.payment_succeeded': + await handlePaymentSucceeded(event.data.object) + break + case 'customer.subscription.deleted': + await handleSubscriptionDeleted(event.data.object) + break + } +} +``` + +## Subscription Management + +### Plan Types +- **Trial**: 500 credits, temporary access +- **Standard**: 500 credits monthly +- **Pro**: Unlimited credits + +### Credit Allocation +Credits are automatically allocated based on subscription tier via webhook processing. + +## Security + +- Webhook signature verification +- Secure API key management +- PCI compliance through Stripe +- No sensitive payment data stored locally \ No newline at end of file diff --git a/TRIAL_SYSTEM_FINAL.md b/auto-analyst-frontend/docs/billing/trial-system.md similarity index 94% rename from TRIAL_SYSTEM_FINAL.md rename to auto-analyst-frontend/docs/billing/trial-system.md index b08b4991..e46d67ad 100644 --- a/TRIAL_SYSTEM_FINAL.md +++ b/auto-analyst-frontend/docs/billing/trial-system.md @@ -2,10 +2,10 @@ ## ๐Ÿ“‹ **System Overview** -The Auto-Analyst app now uses a **7-day trial system** where: +The Auto-Analyst app now uses a **2-day trial system** where: - โŒ **No Free Plan** - Users get 0 credits without subscription - โœ… **Trial Required** - All new users must authorize payment to access features -- ๐Ÿ’ณ **Payment After Trial** - Stripe charges at day 7 unless canceled +- ๐Ÿ’ณ **Payment After Trial** - Stripe charges at day 2 unless canceled - ๐Ÿ›ก๏ธ **Webhook Protected** - All logic handled via Stripe webhooks --- @@ -18,7 +18,7 @@ User clicks "Start Trial" โ†“ Checkout page (/checkout) โ†“ -Stripe subscription with 7-day trial +Stripe subscription with 2-day trial โ†“ Payment method authorization (no charge) โ†“ @@ -33,16 +33,16 @@ Redirect to /account ### **2. Trial Cancellation Flow** ``` -During Trial (0-7 days): +During Trial (0-2 days): User cancels โ†’ Credits = 0 immediately โ†’ No charge ever -After Trial (7+ days): +After Trial (2+ days): User cancels โ†’ Keep access until month end โ†’ Final cleanup via webhook ``` ### **3. Payment Capture Flow** ``` -Day 7: Stripe auto-captures payment +Day 2: Stripe auto-captures payment โ†“ invoice.payment_succeeded webhook โ†“ @@ -152,7 +152,7 @@ User keeps 500 credits for full month | ๐Ÿ’ณ Card declined during signup | `payment_intent.payment_failed` | No trial access | | โŒ User cancels payment | `payment_intent.canceled` | No trial access | | ๐Ÿ” 3D Secure fails | `setup_intent.setup_failed` | No trial access | -| โฐ Day 7 payment fails | `invoice.payment_failed` | Credits โ†’ 0 | +| โฐ Day 2 payment fails | `invoice.payment_failed` | Credits โ†’ 0 | | ๐Ÿšซ User cancels trial | `/api/trial/cancel` | Immediate access removal | | ๐Ÿ“… User cancels after trial | `/api/trial/cancel` | Access until period end | diff --git a/auto-analyst-frontend/docs/cancellation-flows.md b/auto-analyst-frontend/docs/cancellation-flows.md deleted file mode 100644 index e44cdbf2..00000000 --- a/auto-analyst-frontend/docs/cancellation-flows.md +++ /dev/null @@ -1,552 +0,0 @@ -# Cancellation Flows - -This document covers all cancellation scenarios in Auto-Analyst, including trial cancellations, subscription cancellations, and the different handling logic for each case. - -## Overview - -Auto-Analyst handles two distinct cancellation scenarios with different behaviors: - -1. **Trial Cancellation** (0-7 days): Immediate access removal, no charges -2. **Post-Payment Cancellation** (7+ days): Access maintained until period end - -## Cancellation Types - -### 1. Trial Cancellation (Immediate) - -**When**: User cancels during 7-day trial period -**Effect**: Immediate access removal, no charges ever made -**Implementation**: Cancel Stripe subscription immediately - -```typescript -// Subscription status: 'trialing' -await stripe.subscriptions.cancel(subscriptionId, { - prorate: false -}) - -// Immediate credit removal -await creditUtils.setZeroCredits(userId) -``` - -### 2. Active Subscription Cancellation (End of Period) - -**When**: User cancels after trial conversion (paid subscription) -**Effect**: Access maintained until billing period ends -**Implementation**: Set `cancel_at_period_end: true` - -```typescript -// Subscription status: 'active' -await stripe.subscriptions.update(subscriptionId, { - cancel_at_period_end: true -}) - -// Access maintained until period_end -// Credits preserved until cancellation date -``` - -## Implementation - -### Unified Cancellation Endpoint - -**File**: `app/api/trial/cancel/route.ts` - -```typescript -export async function POST(request: NextRequest) { - const token = await getToken({ req: request }) - const userId = token.sub - - // Get user's subscription data - const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - const subscriptionId = subscriptionData.stripeSubscriptionId - - if (!subscriptionId) { - return NextResponse.json( - { error: 'No subscription found' }, - { status: 404 } - ) - } - - // Retrieve current subscription from Stripe - const subscription = await stripe.subscriptions.retrieve(subscriptionId) - - if (subscription.status === 'trialing') { - // TRIAL CANCELLATION: Immediate - return await handleTrialCancellation(userId, subscriptionId) - } else if (subscription.status === 'active') { - // ACTIVE CANCELLATION: End of period - return await handleActiveCancellation(userId, subscription) - } else { - return NextResponse.json( - { error: `Cannot cancel subscription with status: ${subscription.status}` }, - { status: 400 } - ) - } -} -``` - -### Trial Cancellation Handler - -```typescript -async function handleTrialCancellation( - userId: string, - subscriptionId: string -) { - try { - // Cancel subscription immediately - await stripe.subscriptions.cancel(subscriptionId, { - prorate: false - }) - - // Mark as trial cancellation in Redis - await redis.hset(KEYS.USER_CREDITS(userId), { - trialCanceled: 'true', - canceledAt: new Date().toISOString() - }) - - // Update subscription status - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'canceled', - displayStatus: 'canceled', - canceledAt: new Date().toISOString(), - subscriptionCanceled: 'true' - }) - - console.log(`Trial canceled for user ${userId}`) - - return NextResponse.json({ - success: true, - canceled: true, - creditsRemoved: true, - message: 'Trial canceled successfully. You will not be charged.' - }) - } catch (error) { - console.error('Trial cancellation error:', error) - return NextResponse.json( - { error: 'Failed to cancel trial' }, - { status: 500 } - ) - } -} -``` - -### Active Subscription Cancellation Handler - -```typescript -async function handleActiveCancellation( - userId: string, - subscription: Stripe.Subscription -) { - try { - // Set cancel at period end - const updatedSubscription = await stripe.subscriptions.update( - subscription.id, - { cancel_at_period_end: true } - ) - - const periodEnd = new Date(subscription.current_period_end * 1000) - - // Update Redis with cancellation info - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - cancel_at_period_end: 'true', - willCancelAt: periodEnd.toISOString(), - displayStatus: 'canceling', - lastUpdated: new Date().toISOString() - }) - - console.log(`Active subscription marked for cancellation: ${userId}`) - - return NextResponse.json({ - success: true, - canceledAtPeriodEnd: true, - accessUntil: periodEnd.toISOString(), - message: `Your subscription will cancel on ${periodEnd.toLocaleDateString()}. You'll maintain access until then.` - }) - } catch (error) { - console.error('Active cancellation error:', error) - return NextResponse.json( - { error: 'Failed to cancel subscription' }, - { status: 500 } - ) - } -} -``` - -## Webhook Handling - -### Subscription Deletion Webhook - -**Event**: `customer.subscription.deleted` - -```typescript -case 'customer.subscription.deleted': { - const subscription = event.data.object as Stripe.Subscription - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - if (!userId) { - console.error('No userId found for customer:', subscription.customer) - break - } - - // Update subscription status - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'canceled', - stripeSubscriptionStatus: 'canceled', - displayStatus: 'canceled', - canceledAt: new Date().toISOString(), - subscriptionDeleted: 'true', - lastUpdated: new Date().toISOString(), - syncedAt: new Date().toISOString() - }) - - // Set credits to 0 for canceled subscriptions - await creditUtils.setZeroCredits(userId) - - console.log(`Subscription deleted for user ${userId}, credits set to 0`) - break -} -``` - -### Subscription Update Webhook - -**Event**: `customer.subscription.updated` - -```typescript -case 'customer.subscription.updated': { - const subscription = event.data.object as Stripe.Subscription - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - // Handle cancel_at_period_end status - if (subscription.status === 'active' && subscription.cancel_at_period_end) { - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'active', - displayStatus: 'canceling', - cancel_at_period_end: 'true', - willCancelAt: new Date(subscription.current_period_end * 1000).toISOString(), - lastUpdated: new Date().toISOString(), - syncedAt: new Date().toISOString() - }) - } else { - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: subscription.status, - displayStatus: subscription.status, - cancel_at_period_end: subscription.cancel_at_period_end.toString(), - lastUpdated: new Date().toISOString(), - syncedAt: new Date().toISOString() - }) - } - - break -} -``` - -## Credit Management During Cancellation - -### Trial Cancellation Credit Logic - -```typescript -// Immediate credit removal for trial cancellations -async function handleTrialCreditRemoval(userId: string) { - await redis.hset(KEYS.USER_CREDITS(userId), { - total: '0', - used: '0', - trialCanceled: 'true', - canceledAt: new Date().toISOString(), - lastUpdate: new Date().toISOString() - }) - - console.log(`Credits removed for trial cancellation: ${userId}`) -} -``` - -### Active Subscription Credit Logic - -```typescript -// Credits preserved until period end -async function handleActiveCreditPreservation( - userId: string, - periodEndDate: Date -) { - // Credits remain active until cancellation date - // No immediate changes to credit allocation - - await redis.hset(KEYS.USER_CREDITS(userId), { - willResetAt: periodEndDate.toISOString(), - pendingCancellation: 'true', - lastUpdate: new Date().toISOString() - }) - - console.log(`Credits preserved until ${periodEndDate.toISOString()} for user ${userId}`) -} -``` - -## Status Display Logic - -### Frontend Status Mapping - -```typescript -function getDisplayStatus(subscription: SubscriptionData) { - // Use displayStatus if available (for UI-specific states) - if (subscription.displayStatus) { - return { - status: subscription.displayStatus, - color: getStatusColor(subscription.displayStatus), - message: getStatusMessage(subscription.displayStatus) - } - } - - // Fallback to subscription status - return { - status: subscription.status, - color: getStatusColor(subscription.status), - message: getStatusMessage(subscription.status) - } -} - -function getStatusColor(status: string) { - switch (status) { - case 'trialing': return 'blue' - case 'active': return 'green' - case 'canceling': return 'amber' // Special UI state - case 'canceled': return 'red' - case 'past_due': return 'orange' - default: return 'gray' - } -} - -function getStatusMessage(status: string) { - switch (status) { - case 'trialing': return 'Trial period active' - case 'active': return 'Subscription active' - case 'canceling': return 'Will cancel at period end' - case 'canceled': return 'Subscription canceled' - case 'past_due': return 'Payment past due' - default: return 'Unknown status' - } -} -``` - -### Account Page Display - -```typescript -// components/account/SubscriptionStatus.tsx -function SubscriptionStatus({ subscription }: { subscription: SubscriptionData }) { - const displayStatus = getDisplayStatus(subscription) - - return ( -
- - {displayStatus.status} - - - {subscription.status === 'canceling' && ( -
- Cancels on {new Date(subscription.willCancelAt).toLocaleDateString()} -
- )} - - {subscription.status === 'trialing' && ( -
- Trial ends {new Date(subscription.trialEndDate).toLocaleDateString()} -
- )} -
- ) -} -``` - -## Cancellation Prevention - -### Retry Logic for Failed Payments - -```typescript -// Webhook: invoice.payment_failed -case 'invoice.payment_failed': { - const invoice = event.data.object as Stripe.Invoice - - // Don't immediately cancel - Stripe will retry - console.log(`Payment failed for invoice ${invoice.id}`, { - attemptCount: invoice.attempt_count, - nextPaymentAttempt: invoice.next_payment_attempt - }) - - // Update status to past_due but maintain access - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'past_due', - displayStatus: 'past_due', - lastPaymentFailure: new Date().toISOString() - }) - - break -} -``` - -### Win-Back Campaigns - -```typescript -// Trigger win-back email for canceling subscriptions -async function triggerWinBackCampaign(userId: string, cancellationReason?: string) { - const userData = await redis.hgetall(KEYS.USER_PROFILE(userId)) - - // Send targeted email based on cancellation timing - if (subscription.status === 'trialing') { - await sendEmail(userData.email, 'trial-cancellation-winback') - } else { - await sendEmail(userData.email, 'subscription-cancellation-winback') - } -} -``` - -## Reactivation - -### Resubscribe Flow - -```typescript -// Allow users to reactivate canceled subscriptions -async function reactivateSubscription(userId: string, subscriptionId: string) { - try { - // Remove cancel_at_period_end flag - const subscription = await stripe.subscriptions.update(subscriptionId, { - cancel_at_period_end: false - }) - - // Update Redis status - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'active', - displayStatus: 'active', - cancel_at_period_end: 'false', - reactivatedAt: new Date().toISOString(), - lastUpdated: new Date().toISOString() - }) - - // Restore credits if needed - await subscriptionUtils.refreshCreditsIfNeeded(userId) - - return { success: true, reactivated: true } - } catch (error) { - console.error('Reactivation failed:', error) - throw error - } -} -``` - -## Error Handling - -### Cancellation Errors - -```typescript -// Handle common cancellation errors -async function handleCancellationError(error: any, userId: string) { - switch (error.code) { - case 'resource_missing': - // Subscription not found - await cleanupOrphanedSubscription(userId) - break - - case 'subscription_canceled': - // Already canceled - await syncCancellationStatus(userId) - break - - default: - console.error('Unexpected cancellation error:', error) - throw error - } -} -``` - -### Data Consistency - -```typescript -// Ensure Redis and Stripe stay in sync -async function validateCancellationState(userId: string) { - const redisData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - const stripeSubscription = await stripe.subscriptions.retrieve( - redisData.stripeSubscriptionId - ) - - // Check for discrepancies - if (redisData.status !== stripeSubscription.status) { - console.warn('Status mismatch detected:', { - userId, - redis: redisData.status, - stripe: stripeSubscription.status - }) - - // Sync with Stripe as source of truth - await syncSubscriptionStatus(userId, stripeSubscription) - } -} -``` - -## Testing Cancellation Flows - -### Test Scenarios - -1. **Trial Cancellation** - ```bash - # Start trial, then cancel within 7 days - POST /api/trial/cancel - # Expected: Immediate access removal, no charge - ``` - -2. **Active Cancellation** - ```bash - # Convert trial to paid, then cancel - POST /api/trial/cancel - # Expected: Access until period end - ``` - -3. **Webhook Cancellation** - ```bash - stripe trigger customer.subscription.deleted - # Expected: Cleanup in Redis, credits zeroed - ``` - -4. **Reactivation** - ```bash - # Cancel subscription, then reactivate - # Expected: Restored access and credits - ``` - -## Monitoring - -### Cancellation Metrics - -- **Trial Cancellation Rate**: % of trials canceled before conversion -- **Churn Rate**: % of paid subscriptions canceled per month -- **Reactivation Rate**: % of canceled users who resubscribe -- **Time to Cancel**: Average days from signup to cancellation - -### Alerting - -```typescript -// Monitor for unusual cancellation patterns -const cancellationMetrics = { - trialCancellations: await getTrialCancellations('last_24h'), - activeCancellations: await getActiveCancellations('last_24h'), - reactivations: await getReactivations('last_24h') -} - -if (cancellationMetrics.trialCancellations > threshold.trial) { - await sendAlert('High trial cancellation rate detected') -} -``` - -## Best Practices - -### Cancellation UX -1. **Clear consequences**: Explain what happens when canceling -2. **Retention offers**: Present alternatives before confirming -3. **Easy reactivation**: Allow users to easily resubscribe -4. **Feedback collection**: Understand why users are canceling - -### Technical Implementation -1. **Immediate feedback**: Confirm cancellation success instantly -2. **Data consistency**: Keep Redis and Stripe synchronized -3. **Graceful degradation**: Handle edge cases smoothly -4. **Comprehensive logging**: Track all cancellation events - -### Business Logic -1. **Preserve trial conversion**: Don't make trials too restrictive -2. **Honor commitments**: Maintain access for paid periods -3. **Win-back opportunities**: Engage canceled users appropriately -4. **Analytics driven**: Use data to improve retention \ No newline at end of file diff --git a/auto-analyst-frontend/docs/communication/api-integration.md b/auto-analyst-frontend/docs/communication/api-integration.md new file mode 100644 index 00000000..39638aae --- /dev/null +++ b/auto-analyst-frontend/docs/communication/api-integration.md @@ -0,0 +1,430 @@ +# API Integration & Backend Communication + +This document explains how the Auto-Analyst frontend communicates with the backend services. + +## ๐ŸŒ Communication Architecture + +``` +Frontend (Next.js) โ†โ†’ Next.js API Routes โ†โ†’ Python FastAPI Backend + โ†“ + Redis Cache +``` + +## ๐Ÿ”ง Configuration + +### **API Base URL** +```typescript +// config/api.ts +const API_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:8000'; +export default API_URL; +``` + +### **Environment Variables** +```bash +NEXT_PUBLIC_API_URL=http://localhost:8000 # Backend API URL +UPSTASH_REDIS_REST_URL=redis://... # Redis connection +UPSTASH_REDIS_REST_TOKEN=... # Redis auth token +``` + +## ๐Ÿ›ฃ๏ธ Communication Patterns + +### **1. Direct Backend API Calls** + +Used for real-time chat interactions and immediate responses. + +```typescript +// lib/api/auth.ts +export async function loginUser(username: string, email: string, sessionId?: string) { + const response = await fetch(`${API_BASE_URL}/auth/login`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ + username, + email, + session_id: sessionId || localStorage.getItem('sessionId') || undefined + }), + }); + + if (!response.ok) { + throw new Error(`Login failed: ${response.statusText}`); + } + + const data = await response.json(); + localStorage.setItem('sessionId', data.session_id); + localStorage.setItem('userId', String(data.user_id)); + return data; +} +``` + +### **2. Next.js API Route Middleware** + +Used for authentication, data processing, and Redis operations. + +```typescript +// app/api/user/credits/route.ts +export async function GET(request: NextRequest) { + try { + // Authentication + const token = await getToken({ req: request }); + if (!token?.sub) { + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }); + } + + // Redis data access + const creditsHash = await redis.hgetall(KEYS.USER_CREDITS(token.sub)); + + // Data processing + const creditsTotal = parseInt(creditsHash.total as string); + const creditsUsed = parseInt(creditsHash.used as string || '0'); + + return NextResponse.json({ + used: creditsUsed, + total: creditsTotal, + resetDate: creditsHash.resetDate, + }); + } catch (error) { + return NextResponse.json({ error: 'Failed to fetch credits' }, { status: 500 }); + } +} +``` + +### **3. Real-time Communication** + +Used for chat streaming and live updates. + +```typescript +// Chat streaming example +const processMessage = async (message: string) => { + const response = await fetch(`${API_URL}/chat/stream`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ + message, + session_id: sessionId, + agent: selectedAgent, + }), + }); + + const reader = response.body?.getReader(); + const decoder = new TextDecoder(); + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + const chunk = decoder.decode(value); + const lines = chunk.split('\n'); + + for (const line of lines) { + if (line.startsWith('data: ')) { + const data = JSON.parse(line.slice(6)); + updateMessageContent(data); + } + } + } +}; +``` + +## ๐Ÿ“Š Analytics API Integration + +### **Admin Analytics** +```typescript +// lib/api/analytics.ts +export async function fetchUsageSummary( + adminKey: string, + startDate?: string, + endDate?: string +) { + let url = `${API_BASE_URL}/analytics/usage/summary`; + + const params = new URLSearchParams(); + if (startDate) params.append('start_date', startDate); + if (endDate) params.append('end_date', endDate); + + if (params.toString()) { + url += `?${params.toString()}`; + } + + const response = await fetch(url, { + method: 'GET', + headers: { + 'Content-Type': 'application/json', + 'X-Admin-API-Key': adminKey, + }, + }); + + if (!response.ok) { + throw new Error(`Error fetching usage summary: ${response.statusText}`); + } + + return response.json(); +} +``` + +### **User Analytics** +```typescript +// User-specific data access +export async function fetchUserData(userId: string) { + const response = await fetch('/api/user/data', { + method: 'GET', + headers: { + 'Authorization': `Bearer ${token}`, + }, + }); + + return response.json(); +} +``` + +## ๐Ÿ” Authentication Integration + +### **NextAuth Configuration** +```typescript +// app/api/auth/[...nextauth]/route.ts +const handler = NextAuth({ + providers: [ + GoogleProvider({ + clientId: process.env.GOOGLE_CLIENT_ID!, + clientSecret: process.env.GOOGLE_CLIENT_SECRET!, + }), + CredentialsProvider({ + name: "Temporary Login", + credentials: { + password: { label: "Temporary Code", type: "password" }, + }, + async authorize(credentials) { + if (credentials?.password === process.env.NEXT_PUBLIC_ANALYTICS_ADMIN_PASSWORD) { + return { + id: "admin", + name: "Administrator", + email: "admin@example.com", + }; + } + return null; + } + }) + ], + callbacks: { + async signIn({ user, account, profile }) { + // Save user profile to Redis + if (user && user.email) { + await profileUtils.saveUserProfile(user.id || user.email, { + email: user.email, + name: user.name || '', + image: user.image || '', + joinedDate: new Date().toISOString().split('T')[0], + role: 'Free' + }); + } + return true; + }, + }, +}); +``` + +### **Session Management** +```typescript +// Session access in components +const { data: session, status } = useSession(); + +// Session access in API routes +const token = await getToken({ req: request }); +const userId = token?.sub; +``` + +## ๐Ÿ“ก Data Fetching Patterns + +### **SWR for Client-side Fetching** +```typescript +import useSWR from 'swr'; + +const fetcher = (url: string) => fetch(url).then(res => res.json()); + +function UserCredits() { + const { data, error, isLoading } = useSWR('/api/user/credits', fetcher); + + if (error) return
Failed to load credits
; + if (isLoading) return
Loading...
; + + return
Credits: {data.remaining}
; +} +``` + +### **React Query Alternative** +```typescript +import { useQuery } from '@tanstack/react-query'; + +function useUserCredits() { + return useQuery({ + queryKey: ['userCredits'], + queryFn: () => fetch('/api/user/credits').then(res => res.json()), + refetchInterval: 30000, // Refetch every 30 seconds + }); +} +``` + +## ๐Ÿš€ WebSocket Integration + +### **Real-time Chat Updates** +```typescript +class ChatWebSocket { + private ws: WebSocket | null = null; + + connect(sessionId: string) { + this.ws = new WebSocket(`ws://localhost:8000/ws/${sessionId}`); + + this.ws.onmessage = (event) => { + const data = JSON.parse(event.data); + this.handleMessage(data); + }; + + this.ws.onclose = () => { + // Reconnection logic + setTimeout(() => this.connect(sessionId), 5000); + }; + } + + sendMessage(message: string) { + if (this.ws?.readyState === WebSocket.OPEN) { + this.ws.send(JSON.stringify({ message })); + } + } + + private handleMessage(data: any) { + // Update UI with new message + updateChatInterface(data); + } +} +``` + +## ๐Ÿ”„ Error Handling + +### **Global Error Handler** +```typescript +// lib/api/error-handler.ts +export class APIError extends Error { + constructor( + message: string, + public status: number, + public code?: string + ) { + super(message); + } +} + +export async function handleAPIResponse(response: Response) { + if (!response.ok) { + const errorData = await response.json().catch(() => ({})); + throw new APIError( + errorData.message || 'Request failed', + response.status, + errorData.code + ); + } + return response.json(); +} +``` + +### **Component Error Boundaries** +```typescript +function APIErrorBoundary({ children }: { children: ReactNode }) { + return ( + ( +
+

API Error

+

{error.message}

+ +
+ )} + > + {children} + + ); +} +``` + +## ๐Ÿ“ˆ Performance Optimization + +### **Request Caching** +```typescript +// Cache API responses +const cache = new Map(); + +async function cachedFetch(url: string, options?: RequestInit) { + const cacheKey = `${url}-${JSON.stringify(options)}`; + + if (cache.has(cacheKey)) { + return cache.get(cacheKey); + } + + const response = await fetch(url, options); + const data = await response.json(); + + cache.set(cacheKey, data); + return data; +} +``` + +### **Request Debouncing** +```typescript +import { debounce } from 'lodash'; + +const debouncedSearch = debounce(async (query: string) => { + const results = await fetch(`/api/search?q=${query}`); + return results.json(); +}, 300); +``` + +## ๐Ÿ” Monitoring & Logging + +### **API Request Logging** +```typescript +// lib/utils/logger.ts +export const logger = { + log: (message: string, data?: any) => { + console.log(`[${new Date().toISOString()}] ${message}`, data); + }, + + error: (message: string, error?: any) => { + console.error(`[${new Date().toISOString()}] ERROR: ${message}`, error); + }, + + api: (method: string, url: string, status: number, duration: number) => { + console.log(`[API] ${method} ${url} - ${status} (${duration}ms)`); + } +}; +``` + +### **Request Interceptors** +```typescript +// Add request/response interceptors +axios.interceptors.request.use((config) => { + const startTime = Date.now(); + config.metadata = { startTime }; + return config; +}); + +axios.interceptors.response.use( + (response) => { + const duration = Date.now() - response.config.metadata.startTime; + logger.api( + response.config.method?.toUpperCase() || 'GET', + response.config.url || '', + response.status, + duration + ); + return response; + }, + (error) => { + logger.error('API Request Failed', error); + return Promise.reject(error); + } +); +``` \ No newline at end of file diff --git a/auto-analyst-frontend/docs/credit-system.md b/auto-analyst-frontend/docs/credit-system.md deleted file mode 100644 index 123b6dbd..00000000 --- a/auto-analyst-frontend/docs/credit-system.md +++ /dev/null @@ -1,468 +0,0 @@ -# Credit System - -This document covers the complete credit management system in Auto-Analyst, including allocation, usage tracking, and billing logic. - -## Overview - -Auto-Analyst uses a credit-based billing system where users consume credits for various features like chat analysis, code execution, and deep analysis. Credits are allocated based on subscription plans and reset monthly. - -## Credit Configuration - -### Plan Types and Credits - -Located in `lib/credits-config.ts`: - -```typescript -export const PLAN_CREDITS: Record = { - 'Trial': { - total: 500, - displayName: 'Trial Plan', - type: 'TRIAL', - isUnlimited: false, - minimum: 0 - }, - 'Standard': { - total: 500, - displayName: 'Standard Plan', - type: 'STANDARD', - isUnlimited: false, - minimum: 0 - }, - 'Pro': { - total: 999999, - displayName: 'Pro Plan', - type: 'PRO', - isUnlimited: true, - minimum: 0 - } -} -``` - -### Credit Utilities - -```typescript -export const CreditConfig = { - // Get credits by plan type - getCreditsByType: (type: PlanType) => PLAN_CREDITS[type], - - // Get credits by plan name - getCreditsForPlan: (planName: string) => { /* implementation */ }, - - // Check if unlimited credits - isUnlimitedTotal: (total: number) => total >= 999999, - - // Get next reset date - getNextResetDate: () => { - const now = new Date() - const nextMonth = new Date(now.getFullYear(), now.getMonth() + 1, 1) - return nextMonth.toISOString().split('T')[0] - } -} -``` - -## Credit Lifecycle - -### 1. Credit Initialization - -**For Trial Users**: -```typescript -// app/api/trial/start/route.ts -await creditUtils.initializeTrialCredits(userId, { - total: 500, - resetDate: CreditConfig.getNextResetDate(), - paymentIntentId: paymentIntent.id -}) -``` - -**For Subscription Users**: -```typescript -// Handled by webhook: invoice.payment_succeeded -await subscriptionUtils.refreshCreditsIfNeeded(userId) -``` - -### 2. Credit Reset Schedule - -Credits reset monthly on the 1st of each month: - -```typescript -// lib/redis.ts - refreshCreditsIfNeeded() -const now = new Date() -const resetDate = new Date(creditsData.resetDate) - -if (now >= resetDate) { - // Reset credits to plan amount - const planCredits = CreditConfig.getCreditsByType(planType) - - await redis.hset(KEYS.USER_CREDITS(userId), { - total: planCredits.credits.toString(), - used: '0', - resetDate: CreditConfig.getNextResetDate(), - lastUpdate: new Date().toISOString() - }) -} -``` - -### 3. Credit Usage Tracking - -#### Deduction Process -```typescript -// app/api/user/deduct-credits/route.ts -export async function POST(request: NextRequest) { - const { amount, description } = await request.json() - const token = await getToken({ req: request }) - const userId = token.sub - - // Attempt to deduct credits - const success = await creditUtils.deductCredits(userId, amount) - - if (!success) { - return NextResponse.json( - { - error: "Insufficient credits", - required: amount, - available: await creditUtils.getRemainingCredits(userId) - }, - { status: 402 } - ) - } - - // Return updated credit info - const remaining = await creditUtils.getRemainingCredits(userId) - return NextResponse.json({ - success: true, - remainingCredits: remaining, - deducted: amount, - description - }) -} -``` - -#### Atomic Deduction Logic -```typescript -// lib/redis.ts - creditUtils.deductCredits() -async deductCredits(userId: string, amount: number): Promise { - const creditsHash = await redis.hgetall(KEYS.USER_CREDITS(userId)) - - if (!creditsHash || !creditsHash.total) { - return false // No credits available - } - - const total = parseInt(creditsHash.total) - const used = parseInt(creditsHash.used || '0') - const available = total - used - - if (available < amount) { - return false // Insufficient credits - } - - // Atomic update - await redis.hset(KEYS.USER_CREDITS(userId), { - used: (used + amount).toString(), - lastUpdate: new Date().toISOString() - }) - - return true -} -``` - -## Feature Credit Costs - -### Current Credit Pricing - -```typescript -export const FEATURE_COSTS = { - /** Cost for deep analysis - premium feature for paid users */ - DEEP_ANALYSIS: 50, -} - -// Model credit costs are based on MODEL_TIERS configuration: -// - Basic models: 1-2 credits -// - Standard models: 3-5 credits -// - Premium models: 6-10 credits -// - Premium+ models: 11-20 credits -``` - -### Usage Implementation - -```typescript -// For deep analysis feature -const creditsNeeded = FEATURE_COSTS.DEEP_ANALYSIS // 50 credits - -const deductResponse = await fetch('/api/user/deduct-credits', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - amount: creditsNeeded, - description: 'Deep analysis report generation' - }) -}) - -if (deductResponse.status === 402) { - // Show upgrade prompt - insufficient credits - showInsufficientCreditsModal() - return -} - -// For AI model usage (chat messages) -const modelCost = getModelCreditCost(selectedModel) // 1-20 credits based on model tier -// Credit deduction happens automatically in backend based on model used -``` - -## Credit Storage Schema - -### Redis Structure - -```typescript -// Key: user:${userId}:credits -{ - total: string // "500" - used: string // "150" - resetDate: string // "2024-02-01" - lastUpdate: string // ISO timestamp - resetInterval: string // "month" - - // Trial-specific fields - isTrialCredits?: string // "true" - paymentIntentId?: string // Stripe payment intent - - // Cancellation tracking - trialCanceled?: string // "true" - subscriptionDeleted?: string // "true" - - // Admin fields - nextTotalCredits?: string // For plan changes - pendingDowngrade?: string // "true" -} -``` - -### Data Validation - -```typescript -function validateCreditData(creditsData: any) { - const total = parseInt(creditsData.total || '0') - const used = parseInt(creditsData.used || '0') - - // Validation rules - if (total < 0) throw new Error('Total credits cannot be negative') - if (used < 0) throw new Error('Used credits cannot be negative') - if (used > total && total < 999999) { - throw new Error('Used credits cannot exceed total') - } - - return { total, used, remaining: total - used } -} -``` - -## Credit Refresh Logic - -### Automatic Refresh Triggers - -1. **User Data API Call** (`/api/user/data`) -2. **Credit Check API Call** (`/api/user/credits`) -3. **Webhook Events** (payment succeeded, subscription updated) -4. **Navigation Events** (chat page load with refresh flag) - -### Refresh Implementation - -```typescript -// lib/redis.ts - subscriptionUtils.refreshCreditsIfNeeded() -async refreshCreditsIfNeeded(userId: string): Promise { - const creditsData = await redis.hgetall(KEYS.USER_CREDITS(userId)) - const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - - // Check if credits need reset (monthly) - const now = new Date() - const resetDate = new Date(creditsData.resetDate || now) - - if (now >= resetDate) { - await this.resetMonthlyCredits(userId) - return true - } - - // Check for plan changes - if (this.hasPlanChanged(creditsData, subscriptionData)) { - await this.updateCreditsForPlanChange(userId) - return true - } - - // Check for cancellations - if (this.shouldZeroCredits(creditsData, subscriptionData)) { - await creditUtils.setZeroCredits(userId) - return true - } - - return false -} -``` - -### Reset Logic - -```typescript -async resetMonthlyCredits(userId: string) { - const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - const planType = subscriptionData.planType || 'STANDARD' - const planCredits = CreditConfig.getCreditsByType(planType) - - await redis.hset(KEYS.USER_CREDITS(userId), { - total: planCredits.credits.toString(), - used: '0', - resetDate: CreditConfig.getNextResetDate(), - lastUpdate: new Date().toISOString(), - resetInterval: 'month' - }) - - console.log(`Credits reset for user ${userId}: ${planCredits.credits} credits`) -} -``` - -## Credit Preservation - -### Successful Trial Conversion -```typescript -// Credits are preserved when trial converts to paid -// Only genuine cancellations zero out credits - -function shouldZeroCredits(creditsData: any, subscriptionData: any): boolean { - // Genuine trial cancellation - if (creditsData.trialCanceled === 'true') return true - - // Subscription deleted (not converted) - if (creditsData.subscriptionDeleted === 'true') return true - - // Status is canceled and marked for cleanup - if (subscriptionData.status === 'canceled' && - subscriptionData.subscriptionCanceled === 'true') return true - - return false -} -``` - -### Plan Downgrade Protection -```typescript -// Credits are preserved during plan changes within billing period -// Only reset to new plan amount at next billing cycle - -async updateCreditsForPlanChange(userId: string) { - const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - const newPlanType = subscriptionData.planType - const newPlanCredits = CreditConfig.getCreditsByType(newPlanType) - - // Set next month's credit allocation - await redis.hset(KEYS.USER_CREDITS(userId), { - nextTotalCredits: newPlanCredits.credits.toString(), - pendingDowngrade: newPlanCredits.credits < currentTotal ? 'true' : 'false' - }) -} -``` - -## Credit Display - -### Frontend Components - -#### Credit Balance Display -```typescript -// components/chat/CreditBalance.tsx -function CreditBalance() { - const [credits, setCredits] = useState({ used: 0, total: 0 }) - const [isRefreshing, setIsRefreshing] = useState(false) - - const refreshCredits = async () => { - setIsRefreshing(true) - const response = await fetch('/api/user/credits') - const data = await response.json() - setCredits(data) - setIsRefreshing(false) - } - - const percentage = credits.total === Infinity ? 100 : - Math.round(((credits.total - credits.used) / credits.total) * 100) - - return ( -
- {isRefreshing ? ( - - ) : ( - {credits.total - credits.used} / {credits.total} - )} - -
- ) -} -``` - -#### Usage Warning -```typescript -function InsufficientCreditsModal({ requiredCredits, availableCredits }) { - return ( - - - - Insufficient Credits - - You need {requiredCredits} credits but only have {availableCredits} remaining. - Upgrade your plan to continue using this feature. - - - - - - - - ) -} -``` - -## Monitoring and Analytics - -### Credit Usage Tracking -```typescript -// Log credit usage for analytics -console.log('Credit deduction:', { - userId, - amount, - feature: description, - remainingCredits, - timestamp: new Date().toISOString() -}) -``` - -### Key Metrics -- Average credits per user per month -- Feature usage patterns -- Credit exhaustion rates -- Conversion rates from credit exhaustion - -### Alerts -- Users approaching credit limits -- Unusual usage patterns -- Failed credit deductions -- System errors in credit processing - -## Best Practices - -### Credit Management -1. **Always check credits** before feature usage -2. **Handle insufficient credits gracefully** with upgrade prompts -3. **Use atomic operations** for credit deduction -4. **Preserve credits** during legitimate state transitions -5. **Reset credits monthly** regardless of billing cycle - -### Error Handling -1. **Graceful degradation** when credit system fails -2. **Clear error messages** for users -3. **Retry logic** for temporary failures -4. **Logging** for debugging and monitoring - -### Performance -1. **Cache credit data** in component state -2. **Batch credit operations** when possible -3. **Use Redis efficiently** with hash operations -4. **Minimize API calls** with smart refresh logic - -### Security -1. **Validate all credit operations** server-side -2. **Prevent credit manipulation** via client -3. **Audit credit changes** with timestamps -4. **Rate limit** credit-related endpoints \ No newline at end of file diff --git a/auto-analyst-frontend/docs/development/environment-setup.md b/auto-analyst-frontend/docs/development/environment-setup.md new file mode 100644 index 00000000..5bede454 --- /dev/null +++ b/auto-analyst-frontend/docs/development/environment-setup.md @@ -0,0 +1,320 @@ +# Environment Setup Guide + +This guide covers all environment variables required for the Auto-Analyst frontend application. + +## Required Environment Files + +Create these files in the frontend root directory: + +- `.env.local` - Local development (not committed to git) +- `.env.example` - Template file (committed to git) + +## Environment Variables Reference + +### **๐Ÿ”ง Core Configuration** + +```bash +# Backend API Configuration +NEXT_PUBLIC_API_URL=http://localhost:8000 # Backend FastAPI URL + +# NextAuth Configuration +NEXTAUTH_URL=http://localhost:3000 # Frontend URL +NEXTAUTH_SECRET=your-nextauth-secret-key # Session encryption key (generate with openssl rand -base64 32) +``` + +### **๐Ÿ” Authentication & OAuth** + +```bash +# Google OAuth (Primary authentication) +GOOGLE_CLIENT_ID=your-google-client-id +GOOGLE_CLIENT_SECRET=your-google-client-secret + +# Admin Authentication +NEXT_PUBLIC_ANALYTICS_ADMIN_PASSWORD=your-admin-password # Temporary admin access +``` + +### **๐Ÿ’ณ Payment & Billing** + +```bash +# Stripe Configuration +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_... # Stripe public key +STRIPE_SECRET_KEY=sk_test_... # Stripe secret key +STRIPE_WEBHOOK_SECRET=whsec_... # Webhook verification secret +``` + +### **๐Ÿ“ง Email Configuration** + +```bash +# SMTP Settings +SMTP_HOST=smtp.gmail.com # Email server host +SMTP_PORT=587 # Email server port +SMTP_USER=your-email@domain.com # SMTP username +SMTP_PASS=your-app-password # SMTP password (use app password for Gmail) +SMTP_FROM=noreply@your-domain.com # From email address +SALES_EMAIL=sales@your-domain.com # Sales contact email +``` + +### **๐Ÿ—„๏ธ Database & Caching** + +```bash +# Redis Configuration (Upstash or self-hosted) +UPSTASH_REDIS_REST_URL=https://your-redis-url +UPSTASH_REDIS_REST_TOKEN=your-redis-token +``` + +### **๐Ÿค– AI Model Defaults** + +```bash +# Model Configuration Defaults +DEFAULT_MODEL_PROVIDER=openai # Default AI provider +DEFAULT_PUBLIC_MODEL=gpt-4o-mini # Default model +DEFAULT_TEMPERATURE=0.7 # Default temperature +PUBLIC_DEFAULT_MAX_TOKENS=6000 # Default token limit +NEXT_PUBLIC_OPENAI_API_KEY= # Optional: Public OpenAI key (not recommended) +``` + +### **๐Ÿ“Š Trial & Credits** + +```bash +# Trial Configuration +NEXT_PUBLIC_FREE_TRIAL_LIMIT=0 # Free queries for non-authenticated users (0 in production) +TRIAL_DURATION=2 # Trial duration (days) +TRIAL_CREDITS=500 # Credits given during trial +``` + +### **๐ŸŒ Deployment & Environment** + +```bash +# Environment Settings +NODE_ENV=development # development | production +FRONTEND_URL=https://your-domain.com # Production frontend URL +``` + +## Environment-Specific Configuration + +### **Development (.env.local)** + +```bash +# Development configuration +NEXT_PUBLIC_API_URL=http://localhost:8000 +NEXTAUTH_URL=http://localhost:3000 +NEXTAUTH_SECRET=development-secret-key-change-in-production +GOOGLE_CLIENT_ID=your-dev-google-client-id +GOOGLE_CLIENT_SECRET=your-dev-google-client-secret +NEXT_PUBLIC_ANALYTICS_ADMIN_PASSWORD=admin123 +NEXT_PUBLIC_FREE_TRIAL_LIMIT=20000 +NODE_ENV=development + +# Email (optional in development) +SMTP_HOST=smtp.gmail.com +SMTP_PORT=587 +SMTP_USER=your-dev-email@gmail.com +SMTP_PASS=your-app-password +SMTP_FROM=dev@your-domain.com +SALES_EMAIL=dev@your-domain.com + +# Redis (required) +UPSTASH_REDIS_REST_URL=your-redis-url +UPSTASH_REDIS_REST_TOKEN=your-redis-token +``` + +### **Production (.env.production)** + +```bash +# Production configuration +NEXT_PUBLIC_API_URL=https://api.your-domain.com +NEXTAUTH_URL=https://your-domain.com +NEXTAUTH_SECRET=secure-production-secret-key +GOOGLE_CLIENT_ID=your-prod-google-client-id +GOOGLE_CLIENT_SECRET=your-prod-google-client-secret +NEXT_PUBLIC_ANALYTICS_ADMIN_PASSWORD=secure-admin-password +NEXT_PUBLIC_FREE_TRIAL_LIMIT=0 +NODE_ENV=production + +# Stripe (production) +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_live_... +STRIPE_SECRET_KEY=sk_live_... +STRIPE_WEBHOOK_SECRET=whsec_... + +# Email (production) +SMTP_HOST=smtp.your-email-provider.com +SMTP_PORT=587 +SMTP_USER=noreply@your-domain.com +SMTP_PASS=secure-email-password +SMTP_FROM=noreply@your-domain.com +SALES_EMAIL=sales@your-domain.com + +# Redis (production) +UPSTASH_REDIS_REST_URL=https://prod-redis-url +UPSTASH_REDIS_REST_TOKEN=prod-redis-token +``` + +## Security Best Practices + +### **Secret Generation** + +```bash +# Generate secure secrets +openssl rand -base64 32 # For NEXTAUTH_SECRET +openssl rand -hex 32 # Alternative format +``` + +### **Environment File Security** + +1. **Never commit `.env.local`** - Add to `.gitignore` +2. **Use different keys** for development and production +3. **Rotate secrets** regularly in production +4. **Use environment variables** in deployment platforms + +### **Key Security Guidelines** + +- **NEXTAUTH_SECRET**: Must be unique and secure (32+ characters) +- **API Keys**: Never expose secret keys in client-side code +- **SMTP Passwords**: Use app passwords, not account passwords +- **Admin Passwords**: Use strong, unique passwords + +## Setup Instructions + +### **1. Copy Template** + +```bash +cp .env.example .env.local +``` + +### **2. Configure Google OAuth** + +1. Go to [Google Cloud Console](https://console.cloud.google.com/) +2. Create/select project +3. Enable Google+ API +4. Create OAuth 2.0 credentials +5. Add authorized redirect URI: `http://localhost:3000/api/auth/callback/google` +6. Copy Client ID and Secret to `.env.local` + +### **3. Configure Stripe (if using billing)** + +1. Create [Stripe Account](https://stripe.com/) +2. Get publishable and secret keys from dashboard +3. Set up webhook endpoint (see [Webhook Setup Guide](../system/webhooks.md)) +4. Add keys to `.env.local` + +### **4. Configure Redis** + +**Option A: Upstash (Recommended)** +1. Create [Upstash Account](https://upstash.com/) +2. Create Redis database +3. Copy REST URL and token + +**Option B: Self-hosted Redis** +```bash +# Install Redis locally +brew install redis # macOS +# or use Docker +docker run -d -p 6379:6379 redis:alpine +``` + +### **5. Configure Email (Optional)** + +**Gmail Setup:** +1. Enable 2-factor authentication +2. Generate app password +3. Use app password in `SMTP_PASS` + +**Other Providers:** +- AWS SES +- SendGrid +- Mailgun +- Custom SMTP server + +## Validation + +### **Check Configuration** + +```bash +# Verify environment variables are loaded +npm run dev + +# Check in browser console +console.log(process.env.NEXT_PUBLIC_API_URL) +``` + +### **Test Services** + +1. **Authentication**: Try Google login +2. **Email**: Test contact form submission +3. **Redis**: Check session persistence +4. **API**: Verify backend communication + +## Troubleshooting + +### **Common Issues** + +1. **NextAuth Error**: Check `NEXTAUTH_SECRET` and `NEXTAUTH_URL` +2. **Google OAuth Failed**: Verify redirect URI configuration +3. **API Connection Failed**: Check `NEXT_PUBLIC_API_URL` +4. **Email Not Sending**: Verify SMTP credentials and app password +5. **Redis Connection**: Check URL format and token + +### **Debug Commands** + +```bash +# Check environment variables +echo $NEXT_PUBLIC_API_URL + +# Verify NextAuth configuration +curl http://localhost:3000/api/auth/providers + +# Test Redis connection +# (Check browser network tab for Redis calls) +``` + +### **Environment Validation Script** + +```typescript +// scripts/validate-env.ts +const requiredVars = [ + 'NEXT_PUBLIC_API_URL', + 'NEXTAUTH_SECRET', + 'GOOGLE_CLIENT_ID', + 'GOOGLE_CLIENT_SECRET', + 'UPSTASH_REDIS_REST_URL', + 'UPSTASH_REDIS_REST_TOKEN' +] + +requiredVars.forEach(varName => { + if (!process.env[varName]) { + console.error(`โŒ Missing required environment variable: ${varName}`) + } else { + console.log(`โœ… ${varName} is set`) + } +}) +``` + +## Deployment Considerations + +### **Vercel Deployment** + +Add environment variables in Vercel dashboard: +1. Go to Project Settings โ†’ Environment Variables +2. Add all production variables +3. Set different variables for preview/development branches + +### **Docker Deployment** + +```dockerfile +# Pass environment variables to container +ENV NEXT_PUBLIC_API_URL=${NEXT_PUBLIC_API_URL} +ENV NEXTAUTH_SECRET=${NEXTAUTH_SECRET} +# ... other variables +``` + +### **CI/CD Pipeline** + +```yaml +# Example GitHub Actions +env: + NEXT_PUBLIC_API_URL: ${{ secrets.NEXT_PUBLIC_API_URL }} + NEXTAUTH_SECRET: ${{ secrets.NEXTAUTH_SECRET }} + # ... other secrets +``` + +This comprehensive environment setup ensures your Auto-Analyst frontend works correctly across all environments and features. \ No newline at end of file diff --git a/auto-analyst-frontend/docs/features/chat-system.md b/auto-analyst-frontend/docs/features/chat-system.md new file mode 100644 index 00000000..4280f311 --- /dev/null +++ b/auto-analyst-frontend/docs/features/chat-system.md @@ -0,0 +1,535 @@ +# Chat System Documentation + +The Auto-Analyst chat system is the core feature of the application, enabling AI-powered data analysis conversations. + +## ๐ŸŽฏ Overview + +The chat system consists of several interconnected components that handle: +- Multi-agent AI conversations +- Real-time message streaming +- Code execution and visualization +- File upload and dataset processing +- Credit management integration + +## ๐Ÿ—๏ธ Component Architecture + +### **Core Components** + +``` +ChatInterface (Main Container) +โ”œโ”€โ”€ Sidebar (Chat History & Agent Selection) +โ”œโ”€โ”€ ChatWindow (Message Display) +โ”‚ โ”œโ”€โ”€ MessageContent (Individual Messages) +โ”‚ โ”œโ”€โ”€ CodeCanvas (Code Execution) +โ”‚ โ””โ”€โ”€ PlotlyChart (Data Visualization) +โ”œโ”€โ”€ ChatInput (User Input) +โ”‚ โ”œโ”€โ”€ FileUpload +โ”‚ โ”œโ”€โ”€ AgentSuggestions +โ”‚ โ””โ”€โ”€ CommandSuggestions +โ””โ”€โ”€ Various Modals & Overlays +``` + +## ๐Ÿงฉ Key Components + +### **1. ChatInterface.tsx** (1,682 lines) + +The main orchestrator component that manages the entire chat experience. + +**Key Responsibilities:** +- Message flow coordination +- Agent selection and routing +- Credit validation +- File upload handling +- Real-time streaming management + +**State Management:** +```typescript +interface ChatInterfaceState { + messages: ChatMessage[] + isLoading: boolean + isSidebarOpen: boolean + agents: AgentInfo[] + activeChatId: number | null + abortController: AbortController | null + userId: number | null + isAdmin: boolean +} +``` + +**Credit Integration:** +```typescript +const { remainingCredits, checkCredits, hasEnoughCredits } = useCredits() + +const handleMessageSend = async (message: string) => { + const creditCost = getModelCreditCost(modelSettings.model) + + if (!await hasEnoughCredits(creditCost)) { + setInsufficientCreditsModalOpen(true) + return + } + + // Process message... +} +``` + +### **2. ChatInput.tsx** (2,457 lines) + +Handles all user input including text, files, and commands. + +**Key Features:** +- **Auto-resize textarea** - Grows with content +- **File drag & drop** - Multiple file upload support +- **Agent shortcuts** - Quick agent selection via @mentions +- **Command suggestions** - Auto-complete for common operations +- **Credit cost preview** - Shows cost before sending + +**File Upload Integration:** +```typescript +const handleFileUpload = async (files: FileList) => { + for (const file of files) { + const formData = new FormData() + formData.append('file', file) + formData.append('session_id', sessionId) + + const response = await fetch(`${API_URL}/upload`, { + method: 'POST', + body: formData, + }) + + const result = await response.json() + addMessage({ + text: `File uploaded: ${file.name}`, + sender: 'ai', + timestamp: new Date().toISOString() + }) + } +} +``` + +### **3. ChatWindow.tsx** (1,698 lines) + +Displays the conversation history and manages message rendering. + +**Features:** +- **Message streaming** - Real-time content updates +- **Syntax highlighting** - Code block rendering +- **Copy functionality** - Easy message copying +- **Message actions** - Edit, delete, regenerate +- **Auto-scroll** - Follows conversation flow + +**Message Types:** +```typescript +interface ChatMessage { + text: string | PlotlyMessage + sender: "user" | "ai" + agent?: string + message_id?: number + chat_id?: number | null + timestamp?: string +} + +interface PlotlyMessage { + type: "plotly" + data: any + layout: any +} +``` + +### **4. MessageContent.tsx** (720 lines) + +Renders individual messages with rich content support. + +**Supported Content:** +- **Markdown** - Full markdown support +- **Code blocks** - Syntax highlighted code +- **Interactive plots** - Plotly.js integration +- **Tables** - Data table rendering +- **LaTeX** - Mathematical expressions + +**Code Execution:** +```typescript +const CodeBlock = ({ code, language }: { code: string; language: string }) => { + const [output, setOutput] = useState('') + const [isExecuting, setIsExecuting] = useState(false) + + const executeCode = async () => { + setIsExecuting(true) + + try { + const response = await fetch(`${API_URL}/execute`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + code, + language, + session_id: sessionId + }) + }) + + const result = await response.json() + setOutput(result.output) + } catch (error) { + setOutput(`Error: ${error.message}`) + } finally { + setIsExecuting(false) + } + } + + return ( +
+ + {code} + + + {output && 
{output}
} +
+ ) +} +``` + +## ๐Ÿค– Agent System + +### **Agent Types** +The system supports multiple specialized agents: + +```typescript +interface AgentInfo { + name: string + description: string +} + +// Available agents fetched from backend +const agents = [ + { name: "data_analyst", description: "Data analysis specialist" }, + { name: "python_developer", description: "Python programming expert" }, + { name: "visualization_expert", description: "Data visualization specialist" }, + { name: "ml_engineer", description: "Machine learning expert" } +] +``` + +### **Agent Selection** +```typescript +const AgentSuggestions = ({ onAgentSelect }: { onAgentSelect: (agent: string) => void }) => { + return ( +
+ {agents.map(agent => ( + + ))} +
+ ) +} +``` + +## ๐Ÿ“ก Real-time Communication + +### **Message Streaming** +```typescript +const processRegularMessage = async (message: string, controller: AbortController) => { + const response = await fetch(`${API_URL}/chat`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + message, + session_id: sessionId, + user_id: userId, + agent: selectedAgent, + model: modelSettings.model + }), + signal: controller.signal + }) + + const reader = response.body?.getReader() + const decoder = new TextDecoder() + let aiMessageContent = '' + + while (true) { + const { done, value } = await reader.read() + if (done) break + + const chunk = decoder.decode(value, { stream: true }) + const lines = chunk.split('\n') + + for (const line of lines) { + if (line.startsWith('data: ')) { + try { + const data = JSON.parse(line.slice(6)) + + if (data.content) { + aiMessageContent += data.content + updateMessage(messageId, { text: aiMessageContent }) + } + + if (data.finished) { + setIsLoading(false) + await saveMessage(aiMessageContent) + } + } catch (error) { + console.error('Error parsing streaming data:', error) + } + } + } + } +} +``` + +### **WebSocket Alternative** +```typescript +class ChatWebSocketManager { + private ws: WebSocket | null = null + private reconnectAttempts = 0 + private maxReconnectAttempts = 5 + + connect(sessionId: string) { + this.ws = new WebSocket(`ws://localhost:8000/ws/chat/${sessionId}`) + + this.ws.onopen = () => { + console.log('WebSocket connected') + this.reconnectAttempts = 0 + } + + this.ws.onmessage = (event) => { + const data = JSON.parse(event.data) + this.handleMessage(data) + } + + this.ws.onclose = () => { + this.handleReconnect(sessionId) + } + + this.ws.onerror = (error) => { + console.error('WebSocket error:', error) + } + } + + sendMessage(message: ChatMessage) { + if (this.ws?.readyState === WebSocket.OPEN) { + this.ws.send(JSON.stringify(message)) + } + } + + private handleReconnect(sessionId: string) { + if (this.reconnectAttempts < this.maxReconnectAttempts) { + setTimeout(() => { + this.reconnectAttempts++ + this.connect(sessionId) + }, Math.pow(2, this.reconnectAttempts) * 1000) + } + } +} +``` + +## ๐Ÿ“Š Data Visualization + +### **Plotly Integration** +```typescript +const PlotlyChart = ({ data, layout }: { data: any; layout: any }) => { + const plotRef = useRef(null) + + useEffect(() => { + if (plotRef.current && data && layout) { + Plotly.newPlot(plotRef.current, data, layout, { + responsive: true, + displayModeBar: true, + displaylogo: false, + modeBarButtonsToRemove: ['pan2d', 'lasso2d'] + }) + } + }, [data, layout]) + + return
+} +``` + +### **Chart Types Supported** +- **Line charts** - Time series data +- **Bar charts** - Categorical data +- **Scatter plots** - Correlation analysis +- **Heatmaps** - Matrix visualization +- **Box plots** - Statistical distributions +- **Histograms** - Data distribution +- **3D plots** - Multi-dimensional data + +## ๐Ÿ’พ State Management + +### **Chat History Store (Zustand)** +```typescript +interface ChatHistoryStore { + messages: ChatMessage[] + addMessage: (message: ChatMessage) => string + updateMessage: (id: string, updatedMessage: Partial) => void + clearMessages: () => void +} + +const useChatHistoryStore = create()( + persist( + (set) => ({ + messages: [], + addMessage: (message) => { + const id = Math.random().toString(36).substring(7) + set((state) => ({ + messages: [...state.messages, { ...message, id }], + })) + return id + }, + updateMessage: (id, updatedMessage) => { + set((state) => ({ + messages: state.messages.map((message) => + message.id === id ? { ...message, ...updatedMessage } : message + ), + })) + }, + clearMessages: () => set({ messages: [] }), + }), + { name: 'chat-history' } + ) +) +``` + +### **Session Management** +```typescript +const useSessionStore = create((set) => ({ + sessionId: '', + setSessionId: (id: string) => set({ sessionId: id }), + generateNewSession: () => { + const newId = uuidv4() + set({ sessionId: newId }) + return newId + }, +})) +``` + +## ๐ŸŽจ UI Features + +### **Message Actions** +```typescript +const MessageActions = ({ message }: { message: ChatMessage }) => { + const [showActions, setShowActions] = useState(false) + + return ( +
setShowActions(true)} + onMouseLeave={() => setShowActions(false)} + > +
+ {message.text} +
+ + {showActions && ( +
+ + + +
+ )} +
+ ) +} +``` + +### **Typing Indicators** +```typescript +const TypingIndicator = () => ( +
+
+ + + +
+ AI is thinking... +
+) +``` + +### **Auto-scroll Behavior** +```typescript +const useChatScroll = (messages: ChatMessage[]) => { + const messagesEndRef = useRef(null) + const [shouldAutoScroll, setShouldAutoScroll] = useState(true) + + const scrollToBottom = useCallback(() => { + if (shouldAutoScroll && messagesEndRef.current) { + messagesEndRef.current.scrollIntoView({ behavior: 'smooth' }) + } + }, [shouldAutoScroll]) + + useEffect(() => { + scrollToBottom() + }, [messages, scrollToBottom]) + + const handleScroll = useCallback((e: React.UIEvent) => { + const { scrollTop, scrollHeight, clientHeight } = e.currentTarget + const isAtBottom = scrollHeight - scrollTop <= clientHeight + 100 + setShouldAutoScroll(isAtBottom) + }, []) + + return { messagesEndRef, handleScroll } +} +``` + +## ๐Ÿ”ง Performance Optimizations + +### **Message Virtualization** +For large chat histories, the system implements virtual scrolling: + +```typescript +const VirtualizedChatWindow = ({ messages }: { messages: ChatMessage[] }) => { + const { FixedSizeList: List } = require('react-window') + + const MessageItem = ({ index, style }: { index: number; style: any }) => ( +
+ +
+ ) + + return ( + + {MessageItem} + + ) +} +``` + +### **Debounced Input** +```typescript +const useDebouncedInput = (callback: (value: string) => void, delay: number) => { + const [inputValue, setInputValue] = useState('') + const debouncedValue = useDebounce(inputValue, delay) + + useEffect(() => { + if (debouncedValue) { + callback(debouncedValue) + } + }, [debouncedValue, callback]) + + return [inputValue, setInputValue] as const +} +``` + +### **Memoized Components** +```typescript +const MemoizedMessageContent = React.memo(MessageContent, (prevProps, nextProps) => { + return prevProps.message.id === nextProps.message.id && + prevProps.message.text === nextProps.message.text +}) +``` \ No newline at end of file diff --git a/DEFAULT_AGENTS_SETUP.md b/auto-analyst-frontend/docs/features/default-agents.md similarity index 100% rename from DEFAULT_AGENTS_SETUP.md rename to auto-analyst-frontend/docs/features/default-agents.md diff --git a/auto-analyst-frontend/docs/redis-schema.md b/auto-analyst-frontend/docs/redis-schema.md deleted file mode 100644 index 6626f81b..00000000 --- a/auto-analyst-frontend/docs/redis-schema.md +++ /dev/null @@ -1,292 +0,0 @@ -# Redis Data Schema - -This document describes the complete Redis data structure used in Auto-Analyst for user management, subscriptions, and credit tracking. - -## Overview - -Auto-Analyst uses **Upstash Redis** with a hash-based storage pattern for efficient data access and atomic operations. All user data is organized using consistent key patterns. - -## Environment Variables - -```bash -UPSTASH_REDIS_REST_URL=your_redis_url -UPSTASH_REDIS_REST_TOKEN=your_redis_token -``` - -## Key Patterns - -### Core Keys -```typescript -export const KEYS = { - USER_PROFILE: (userId: string) => `user:${userId}:profile`, - USER_SUBSCRIPTION: (userId: string) => `user:${userId}:subscription`, - USER_CREDITS: (userId: string) => `user:${userId}:credits`, -}; - -// Additional keys -stripe:customer:${customerId} -> userId (string mapping) -``` - -## Data Structures - -### 1. User Profile (`user:${userId}:profile`) - -**Hash Structure:** -```typescript -{ - email: string // User's email address - name: string // User's display name - image: string // Profile image URL - joinedDate: string // ISO date string (YYYY-MM-DD) - role: string // Plan display name (e.g., "Standard Plan") -} -``` - -**Example:** -```json -{ - "email": "user@example.com", - "name": "John Doe", - "image": "https://example.com/avatar.jpg", - "joinedDate": "2024-01-15", - "role": "Standard Plan" -} -``` - -### 2. User Subscription (`user:${userId}:subscription`) - -**Hash Structure:** -```typescript -{ - // Core subscription data - plan: string // Plan display name - planType: string // Plan type (TRIAL, STANDARD, PRO, etc.) - status: string // Subscription status - displayStatus?: string // UI display status (for canceling subscriptions) - amount: string // Amount as string (e.g., "15.00") - interval: string // Billing interval (month, year) - - // Dates and timestamps - purchaseDate: string // ISO timestamp - renewalDate: string // Next billing date (YYYY-MM-DD) - lastUpdated: string // ISO timestamp - - // Stripe integration - stripeCustomerId: string // Stripe customer ID - stripeSubscriptionId: string // Stripe subscription ID - stripeSubscriptionStatus: string // Direct Stripe status - - // Trial management - trialStartDate?: string // ISO timestamp - trialEndDate?: string // ISO timestamp - trialEndedAt?: string // ISO timestamp - trialToActiveDate?: string // ISO timestamp - - // Cancellation tracking - canceledAt?: string // ISO timestamp - subscriptionCanceled?: string // "true" if canceled - cancel_at_period_end?: string // "true"/"false" - willCancelAt?: string // ISO timestamp - - // Yearly subscription support - isYearly?: boolean // True for yearly plans - nextMonthlyReset?: string // Next credit reset for yearly plans - - // Synchronization - syncedAt?: string // Last sync timestamp -} -``` - -**Example:** -```json -{ - "plan": "Standard Plan", - "planType": "STANDARD", - "status": "active", - "amount": "15.00", - "interval": "month", - "purchaseDate": "2024-01-15T10:30:00.000Z", - "renewalDate": "2024-02-15", - "lastUpdated": "2024-01-20T15:45:00.000Z", - "stripeCustomerId": "cus_ABC123", - "stripeSubscriptionId": "sub_DEF456", - "stripeSubscriptionStatus": "active" -} -``` - -### 3. User Credits (`user:${userId}:credits`) - -**Hash Structure:** -```typescript -{ - // Core credit data - total: string // Total credits as string - used: string // Used credits as string - resetDate: string // Next reset date (YYYY-MM-DD) - lastUpdate: string // ISO timestamp - resetInterval: string // "month" (always monthly) - - // Trial credits - isTrialCredits?: string // "true" if trial credits - paymentIntentId?: string // Associated payment intent - - // Cancellation tracking - trialCanceled?: string // "true" if trial was canceled - subscriptionDeleted?: string // "true" if subscription deleted - downgradedAt?: string // ISO timestamp - canceledAt?: string // ISO timestamp - - // Admin operations - nextTotalCredits?: string // Scheduled credit amount - pendingDowngrade?: string // "true" if downgrade pending -} -``` - -**Example:** -```json -{ - "total": "500", - "used": "150", - "resetDate": "2024-02-01", - "lastUpdate": "2024-01-20T15:45:00.000Z", - "resetInterval": "month" -} -``` - -### 4. Stripe Customer Mapping (`stripe:customer:${customerId}`) - -**Simple String Value:** -```typescript -// Key: stripe:customer:cus_ABC123 -// Value: "user123" -``` - -This mapping allows webhooks to find the correct user ID from Stripe customer events. - -## Data Operations - -### Reading Data -```typescript -// Get all subscription data -const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - -// Get specific field -const planType = await redis.hget(KEYS.USER_SUBSCRIPTION(userId), 'planType') - -// Get multiple fields -const creditInfo = await redis.hmget(KEYS.USER_CREDITS(userId), ['total', 'used', 'resetDate']) -``` - -### Writing Data -```typescript -// Update multiple fields atomically -await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'active', - lastUpdated: new Date().toISOString(), - stripeSubscriptionStatus: 'active' -}) - -// Set single field -await redis.hset(KEYS.USER_CREDITS(userId), 'used', '160') -``` - -### Atomic Operations -```typescript -// Check and update credits atomically -const creditsHash = await redis.hgetall(KEYS.USER_CREDITS(userId)) -if (creditsHash && creditsHash.total) { - const total = parseInt(creditsHash.total) - const used = parseInt(creditsHash.used || '0') - - if (total - used >= requiredCredits) { - await redis.hset(KEYS.USER_CREDITS(userId), { - used: (used + requiredCredits).toString(), - lastUpdate: new Date().toISOString() - }) - return true - } -} -return false -``` - -## Data Consistency - -### Webhook Synchronization -All Stripe events are synchronized to Redis via webhooks to ensure data consistency: - -1. **Subscription Updates**: `customer.subscription.updated` โ†’ Update subscription status -2. **Payment Success**: `invoice.payment_succeeded` โ†’ Activate subscription -3. **Payment Failure**: `invoice.payment_failed` โ†’ Handle failed payments -4. **Subscription Deletion**: `customer.subscription.deleted` โ†’ Clean up data - -### Error Handling -```typescript -try { - const result = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - return result || {} -} catch (error) { - console.error('Redis error:', error) - return {} // Graceful fallback -} -``` - -## Performance Considerations - -### Batch Operations -```typescript -// Use pipeline for multiple operations -const pipeline = redis.pipeline() -pipeline.hset(KEYS.USER_SUBSCRIPTION(userId), subscriptionData) -pipeline.hset(KEYS.USER_CREDITS(userId), creditData) -await pipeline.exec() -``` - -### Caching Strategy -- **TTL**: No TTL set on user data (persistent) -- **Invalidation**: Data updated via webhooks and API calls -- **Fallback**: API endpoints provide fallback for missing data - -## Monitoring - -### Key Metrics to Monitor -- Redis memory usage -- Connection count -- Operation latency -- Error rates - -### Debug Endpoints -- `/api/debug/redis-check` - Check Redis data for specific user -- `/api/debug/sync-subscription` - Manual sync with Stripe - -## Migration Patterns - -### Adding New Fields -```typescript -// Safe field addition (backward compatible) -const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) -const newField = subscriptionData.newField || 'defaultValue' -``` - -### Data Migration Script Example -```typescript -async function migrateSubscriptions() { - const keys = await redis.keys('user:*:subscription') - - for (const key of keys) { - const data = await redis.hgetall(key) - if (!data.planType) { - await redis.hset(key, 'planType', 'STANDARD') - } - } -} -``` - -## Best Practices - -1. **Always use hash operations** for user data -2. **Include timestamps** for debugging and auditing -3. **Handle missing data gracefully** with fallbacks -4. **Use atomic operations** for credit deduction -5. **Validate data types** when reading from Redis -6. **Log errors** but don't fail the user experience -7. **Use consistent key patterns** across the application \ No newline at end of file diff --git a/auto-analyst-frontend/docs/state-management/stores.md b/auto-analyst-frontend/docs/state-management/stores.md new file mode 100644 index 00000000..fa86cc2f --- /dev/null +++ b/auto-analyst-frontend/docs/state-management/stores.md @@ -0,0 +1,690 @@ +# State Management Documentation + +The Auto-Analyst frontend uses a hybrid state management approach combining React Context, Zustand stores, and local component state. + +## ๐Ÿ—๏ธ Architecture Overview + +``` +Global State Layer +โ”œโ”€โ”€ React Context (Authentication, Credits, Analysis) +โ”œโ”€โ”€ Zustand Stores (Client-side persistence) +โ”œโ”€โ”€ Redis Cache (Server-side persistence) +โ””โ”€โ”€ Component State (Local UI state) +``` + +## ๐ŸŒ React Context Providers + +### **1. Credit Provider** +Manages user credit balance and usage tracking. + +```typescript +// lib/contexts/credit-context.tsx +interface CreditContextType { + remainingCredits: number + isLoading: boolean + checkCredits: () => Promise + hasEnoughCredits: (amount: number) => Promise + deductCredits: (amount: number) => Promise + isChatBlocked: boolean + creditResetDate: string | null +} + +const CreditContext = createContext(undefined) + +export function CreditProvider({ children }: { children: ReactNode }) { + const { data: session } = useSession() + const [remainingCredits, setRemainingCredits] = useState(0) + const [isLoading, setIsLoading] = useState(true) + const [isChatBlocked, setIsChatBlocked] = useState(false) + + // Get user identifier + const getUserId = (): string => { + if (session?.user?.email) { + return session.user.email + } else if (typeof window !== 'undefined' && localStorage.getItem('isAdmin') === 'true') { + return 'admin-user' + } else { + // Guest user handling + const guestId = typeof window !== 'undefined' ? + (localStorage.getItem('guestUserId') || `guest-${Date.now()}`) : + `guest-${Date.now()}`; + + if (typeof window !== 'undefined' && !localStorage.getItem('guestUserId')) { + localStorage.setItem('guestUserId', guestId) + } + return guestId + } + } + + // Fetch current credit balance + const checkCredits = async () => { + try { + setIsLoading(true) + const userId = session?.user ? ((session.user as any).sub || session.user.id) : getUserId() + + let currentCredits = 100; // Default + let resetDate = null; + + try { + // Try API endpoint first + const response = await fetch('/api/user/credits'); + if (response.ok) { + const data = await response.json(); + currentCredits = data.total === 999999 ? Infinity : data.total - data.used; + resetDate = data.resetDate; + } else { + // Fall back to Redis + currentCredits = await creditUtils.getRemainingCredits(userId); + } + } catch (error) { + currentCredits = 10; // Fallback + } + + setRemainingCredits(currentCredits); + setIsChatBlocked(currentCredits <= 0); + + // Cache for offline usage + if (typeof window !== 'undefined') { + localStorage.setItem(`user_credits_${userId}`, currentCredits.toString()); + localStorage.setItem(`user_credits_updated_${userId}`, Date.now().toString()); + } + } catch (error) { + console.error('Error checking credits:', error); + setRemainingCredits(100); + setIsChatBlocked(false); + } finally { + setIsLoading(false); + } + }; + + // Credit validation + const hasEnoughCredits = async (amount: number): Promise => { + const hasEnough = remainingCredits >= amount; + if (!hasEnough) { + setIsChatBlocked(true); + } + return hasEnough; + } + + // Credit deduction + const deductCredits = async (amount: number): Promise => { + try { + const userId = getUserId(); + + if (remainingCredits < amount) { + setIsChatBlocked(true); + return false; + } + + let success = false; + + try { + success = await creditUtils.deductCredits(userId, amount); + } catch (redisError) { + success = true; // Fallback to local state + } + + if (success) { + const newBalance = remainingCredits - amount; + setRemainingCredits(newBalance); + setIsChatBlocked(newBalance <= 0); + + // Keep local storage in sync + if (typeof window !== 'undefined') { + localStorage.setItem(`user_credits_${userId}`, newBalance.toString()); + } + } + + return success; + } catch (error) { + console.error('[CREDIT-CONTEXT] Error in deductCredits:', error); + return false; + } + }; + + return ( + + {children} + + ) +} + +export function useCredits() { + const context = useContext(CreditContext) + if (!context) { + throw new Error('useCredits must be used within CreditProvider') + } + return context +} +``` + +### **2. Deep Analysis Provider** +Manages deep analysis feature state. + +```typescript +// lib/contexts/deep-analysis-context.tsx +interface DeepAnalysisContextType { + isAnalysisRunning: boolean + currentStep: string + progress: number + results: AnalysisResult[] + startAnalysis: (data: any) => Promise + stopAnalysis: () => void + clearResults: () => void +} + +export function DeepAnalysisProvider({ children }: { children: ReactNode }) { + const [isAnalysisRunning, setIsAnalysisRunning] = useState(false) + const [currentStep, setCurrentStep] = useState('') + const [progress, setProgress] = useState(0) + const [results, setResults] = useState([]) + + const startAnalysis = async (data: any) => { + setIsAnalysisRunning(true) + setProgress(0) + setCurrentStep('Initializing...') + + try { + const response = await fetch('/api/deep-analysis/start', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(data) + }) + + // Handle streaming response for progress updates + const reader = response.body?.getReader() + const decoder = new TextDecoder() + + while (true) { + const { done, value } = await reader.read() + if (done) break + + const chunk = decoder.decode(value) + const lines = chunk.split('\n') + + for (const line of lines) { + if (line.startsWith('data: ')) { + const data = JSON.parse(line.slice(6)) + + if (data.step) setCurrentStep(data.step) + if (data.progress) setProgress(data.progress) + if (data.result) setResults(prev => [...prev, data.result]) + } + } + } + } catch (error) { + console.error('Analysis error:', error) + } finally { + setIsAnalysisRunning(false) + setCurrentStep('') + setProgress(100) + } + } + + const stopAnalysis = () => { + setIsAnalysisRunning(false) + setCurrentStep('') + } + + const clearResults = () => { + setResults([]) + setProgress(0) + } + + return ( + + {children} + + ) +} +``` + +## ๐Ÿ—„๏ธ Zustand Stores + +### **1. Chat History Store** +Persists chat messages across sessions. + +```typescript +// lib/store/chatHistoryStore.ts +export interface ChatMessage { + text: string | { + type: "plotly" + data: any + layout: any + } + sender: "user" | "ai" + agent?: string + id?: string + message_id?: number + chat_id?: number + timestamp?: string +} + +interface ChatHistoryStore { + messages: ChatMessage[] + addMessage: (message: ChatMessage) => string + updateMessage: (id: string, updatedMessage: Partial) => void + clearMessages: () => void +} + +export const useChatHistoryStore = create()( + persist( + (set) => ({ + messages: [], + addMessage: (message) => { + const id = Math.random().toString(36).substring(7) + set((state: ChatHistoryStore) => ({ + messages: [...state.messages, { ...message, id }], + })) + return id + }, + updateMessage: (id, updatedMessage) => { + set((state: ChatHistoryStore) => ({ + messages: state.messages.map((message) => + message.id === id ? { ...message, ...updatedMessage } : message + ), + })) + }, + clearMessages: () => set({ messages: [] }), + }), + { + name: 'chat-history', + // Custom storage with compression + storage: { + getItem: (name) => { + const str = localStorage.getItem(name) + if (!str) return null + try { + return JSON.parse(str) + } catch { + return null + } + }, + setItem: (name, value) => { + localStorage.setItem(name, JSON.stringify(value)) + }, + removeItem: (name) => localStorage.removeItem(name), + }, + } + ) +) +``` + +### **2. User Subscription Store** +Manages subscription and billing information. + +```typescript +// lib/store/userSubscriptionStore.ts +interface Subscription { + plan: string + status: string + amount: number + interval: string + planType: string + nextBilling?: string + cancelAtPeriodEnd?: boolean +} + +interface UserSubscriptionStore { + subscription: Subscription | null + isLoading: boolean + error: string | null + fetchSubscription: () => Promise + setSubscription: (subscription: Subscription) => void + clearSubscription: () => void +} + +export const useUserSubscriptionStore = create((set, get) => ({ + subscription: null, + isLoading: false, + error: null, + + fetchSubscription: async () => { + set({ isLoading: true, error: null }) + + try { + const response = await fetch('/api/user/subscription') + + if (!response.ok) { + throw new Error('Failed to fetch subscription') + } + + const subscription = await response.json() + set({ subscription, isLoading: false }) + } catch (error) { + set({ + error: error instanceof Error ? error.message : 'Unknown error', + isLoading: false + }) + } + }, + + setSubscription: (subscription) => { + set({ subscription, error: null }) + }, + + clearSubscription: () => { + set({ subscription: null, error: null }) + }, +})) +``` + +### **3. Free Trial Store** +Handles free trial state for non-authenticated users (development only). + +```typescript +// lib/store/freeTrialStore.ts +interface FreeTrialStore { + queriesUsed: number + maxQueries: number + hasFreeTrial: () => boolean + incrementQueries: () => void + resetTrial: () => void + getRemainingQueries: () => number +} + +export const useFreeTrialStore = create()( + persist( + (set, get) => ({ + queriesUsed: 0, + maxQueries: 0, // Disabled in production, configurable in development + + hasFreeTrial: () => { + const { queriesUsed, maxQueries } = get() + return queriesUsed < maxQueries + }, + + incrementQueries: () => { + set((state) => ({ + queriesUsed: Math.min(state.queriesUsed + 1, state.maxQueries) + })) + }, + + resetTrial: () => { + set({ queriesUsed: 0 }) + }, + + getRemainingQueries: () => { + const { queriesUsed, maxQueries } = get() + return Math.max(0, maxQueries - queriesUsed) + }, + }), + { + name: 'free-trial-storage', + } + ) +) +``` + +### **4. Cookie Consent Store** +Manages GDPR compliance state. + +```typescript +// lib/store/cookieConsentStore.ts +interface CookieConsentStore { + hasConsented: boolean + showBanner: boolean + acceptedCategories: string[] + giveConsent: (categories: string[]) => void + withdrawConsent: () => void + isFirstVisit: () => boolean +} + +export const useCookieConsentStore = create()( + persist( + (set, get) => ({ + hasConsented: false, + showBanner: true, + acceptedCategories: [], + + giveConsent: (categories) => { + set({ + hasConsented: true, + showBanner: false, + acceptedCategories: categories + }) + + // Set analytics cookies if consented + if (categories.includes('analytics')) { + // Enable Google Analytics + if (typeof window !== 'undefined' && window.gtag) { + window.gtag('consent', 'update', { + analytics_storage: 'granted' + }) + } + } + }, + + withdrawConsent: () => { + set({ + hasConsented: false, + showBanner: true, + acceptedCategories: [] + }) + + // Disable analytics + if (typeof window !== 'undefined' && window.gtag) { + window.gtag('consent', 'update', { + analytics_storage: 'denied' + }) + } + }, + + isFirstVisit: () => { + return !get().hasConsented && get().showBanner + }, + }), + { + name: 'cookie-consent', + } + ) +) +``` + +### **5. Session Store** +Manages session identifiers and temporary state. + +```typescript +// lib/store/sessionStore.ts +interface SessionStore { + sessionId: string + tempData: any + setSessionId: (id: string) => void + setTempData: (data: any) => void + clearSession: () => void + generateNewSession: () => string +} + +export const useSessionStore = create((set, get) => ({ + sessionId: '', + tempData: null, + + setSessionId: (id: string) => { + set({ sessionId: id }) + }, + + setTempData: (data: any) => { + set({ tempData: data }) + }, + + clearSession: () => { + set({ sessionId: '', tempData: null }) + }, + + generateNewSession: () => { + const newId = crypto.randomUUID() + set({ sessionId: newId }) + return newId + }, +})) +``` + +## ๐Ÿ”„ State Synchronization + +### **Server-Client Sync** +```typescript +// lib/hooks/useServerSync.ts +export function useServerSync() { + const { checkCredits } = useCredits() + const { fetchSubscription } = useUserSubscriptionStore() + const { data: session } = useSession() + + const syncAll = useCallback(async () => { + if (session) { + await Promise.all([ + checkCredits(), + fetchSubscription(), + ]) + } + }, [session, checkCredits, fetchSubscription]) + + // Sync on session change + useEffect(() => { + syncAll() + }, [session, syncAll]) + + // Periodic sync every 5 minutes + useEffect(() => { + const interval = setInterval(syncAll, 5 * 60 * 1000) + return () => clearInterval(interval) + }, [syncAll]) + + return { syncAll } +} +``` + +### **Optimistic Updates** +```typescript +// lib/hooks/useOptimisticUpdate.ts +export function useOptimisticUpdate( + initialData: T, + updateFn: (data: T) => Promise +) { + const [data, setData] = useState(initialData) + const [isLoading, setIsLoading] = useState(false) + const [error, setError] = useState(null) + + const update = useCallback(async (optimisticData: T) => { + // Immediately update UI + setData(optimisticData) + setIsLoading(true) + setError(null) + + try { + // Send to server + const serverData = await updateFn(optimisticData) + setData(serverData) + } catch (err) { + // Revert on error + setData(initialData) + setError(err as Error) + } finally { + setIsLoading(false) + } + }, [initialData, updateFn]) + + return { data, isLoading, error, update } +} +``` + +## ๐ŸŽฏ State Management Best Practices + +### **1. Provider Hierarchy** +```typescript +// components/ClientLayout.tsx +export default function ClientLayout({ children }: { children: ReactNode }) { + return ( + + + + + + {children} + + + + + + ) +} +``` + +### **2. Selective Re-renders** +```typescript +// Use selectors to prevent unnecessary re-renders +const useSelectedChatMessages = () => { + return useChatHistoryStore((state) => state.messages) +} + +const useMessageById = (id: string) => { + return useChatHistoryStore((state) => + state.messages.find(msg => msg.id === id) + ) +} +``` + +### **3. State Validation** +```typescript +// lib/utils/stateValidation.ts +export function validateChatMessage(message: any): ChatMessage | null { + if (!message || typeof message !== 'object') return null + + if (!message.text || !message.sender) return null + + if (message.sender !== 'user' && message.sender !== 'ai') return null + + return { + text: message.text, + sender: message.sender, + agent: message.agent || undefined, + id: message.id || undefined, + timestamp: message.timestamp || new Date().toISOString() + } +} +``` + +### **4. State Debugging** +```typescript +// lib/utils/stateDebug.ts +export function createStoreLogger(storeName: string) { + return (config: any) => (set: any, get: any, api: any) => { + const loggedSet = (...args: any[]) => { + console.group(`๐Ÿ”„ ${storeName} State Update`) + console.log('Previous:', get()) + set(...args) + console.log('Current:', get()) + console.groupEnd() + } + + return config(loggedSet, get, api) + } +} + +// Usage +export const useChatHistoryStore = create()( + createStoreLogger('ChatHistory')( + persist( + (set) => ({ + // store implementation + }), + { name: 'chat-history' } + ) + ) +) +``` \ No newline at end of file diff --git a/auto-analyst-frontend/docs/stripe-integration.md b/auto-analyst-frontend/docs/stripe-integration.md deleted file mode 100644 index 78c72a4d..00000000 --- a/auto-analyst-frontend/docs/stripe-integration.md +++ /dev/null @@ -1,382 +0,0 @@ -# Stripe Integration - -This document covers the complete Stripe integration in Auto-Analyst, including payment processing, subscription management, and webhook handling. - -## Overview - -Auto-Analyst uses Stripe for: -- 7-day free trial with payment authorization -- Subscription billing (monthly/yearly) -- Payment processing -- Subscription lifecycle management -- Real-time event synchronization via webhooks - -## Environment Variables - -```bash -# Required Stripe Configuration -STRIPE_SECRET_KEY=sk_test_... # or sk_live_... for production -STRIPE_WEBHOOK_SECRET=whsec_... # Webhook endpoint secret - -# Client-side Stripe Configuration -NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_... # or pk_live_... for production -``` - -## Stripe Configuration - -### API Version -```typescript -const stripe = new Stripe(process.env.STRIPE_SECRET_KEY, { - apiVersion: '2025-02-24.acacia', -}) -``` - -### Required Stripe Products - -#### 1. Standard Plan -```json -{ - "name": "Standard Plan", - "prices": [ - { - "amount": 1500, // $15.00 - "currency": "usd", - "interval": "month" - } - ] -} -``` - -#### 2. Pro Plan (if available) -```json -{ - "name": "Pro Plan", - "prices": [ - { - "amount": 2500, // $25.00 - "currency": "usd", - "interval": "month" - } - ] -} -``` - -## Trial System Implementation - -### 7-Day Trial Flow - -1. **Subscription Creation** (`/api/checkout-sessions`) -2. **Payment Method Setup** (Setup intent for payment authorization) -3. **Trial Activation** (`/api/trial/start`) -4. **Trial Access** (500 credits immediately) -5. **Auto-Conversion** (After 7 days, Stripe charges automatically) - -### Subscription Creation Code -```typescript -// app/api/checkout-sessions/route.ts -// Auto-Analyst creates subscription directly, not via checkout sessions -const subscription = await stripe.subscriptions.create({ - customer: customerId, - items: [{ price: priceId }], - trial_end: trialEndTimestamp, // 7 days from now - expand: ['latest_invoice.payment_intent'], - payment_behavior: 'default_incomplete', - payment_settings: { - save_default_payment_method: 'on_subscription', - }, - metadata: { - userId: userId, - planName, - interval, - isTrial: 'true' - } -}) -``` - -## Subscription States - -### State Diagram -``` -[Trial Start] โ†’ [trialing] โ†’ [active] (payment successful) - โ†“ - [canceled] (trial canceled) - โ†“ - [subscription deleted] -``` - -### Status Mapping -```typescript -interface SubscriptionStatus { - stripe: 'trialing' | 'active' | 'canceled' | 'past_due' | 'unpaid' - redis: 'trialing' | 'active' | 'canceling' | 'canceled' | 'past_due' - display: 'Trial' | 'Active' | 'Canceling' | 'Canceled' | 'Past Due' -} -``` - -## Key Stripe Objects - -### Customer -```typescript -interface StripeCustomer { - id: string // cus_ABC123 - email: string - created: number - metadata: { - userId: string // Our internal user ID - } -} -``` - -### Subscription -```typescript -interface StripeSubscription { - id: string // sub_DEF456 - customer: string // cus_ABC123 - status: SubscriptionStatus - trial_start?: number // Unix timestamp - trial_end?: number // Unix timestamp - current_period_start: number // Unix timestamp - current_period_end: number // Unix timestamp - cancel_at_period_end: boolean // true if canceling - canceled_at?: number // Unix timestamp - metadata: { - userId: string - userEmail: string - isTrial: string - } -} -``` - -### Payment Intent -```typescript -interface StripePaymentIntent { - id: string // pi_GHI789 - customer: string // cus_ABC123 - status: PaymentIntentStatus - amount: number - metadata: { - userId: string - isTrial: string - } -} -``` - -## Subscription Management - -### Creating Trial Subscription -```typescript -// Direct subscription creation with trial -const subscription = await stripe.subscriptions.create({ - customer: customerId, - items: [{ price: priceId }], - trial_end: trialEndTimestamp, // 7 days from now - payment_behavior: 'default_incomplete', - metadata: { userId, isTrial: 'true' } -}) -``` - -### Canceling Subscription -```typescript -// app/api/trial/cancel/route.ts -if (subscription.status === 'trialing') { - // Immediate cancellation for trials - await stripe.subscriptions.cancel(subscriptionId, { - prorate: false - }) -} else { - // Cancel at period end for active subscriptions - await stripe.subscriptions.update(subscriptionId, { - cancel_at_period_end: true - }) -} -``` - -### Retrieving Subscription Data -```typescript -const subscription = await stripe.subscriptions.retrieve(subscriptionId, { - expand: ['latest_invoice', 'customer'] -}) -``` - -## Payment Processing - -### Authorization vs. Capture - -#### Trial Authorization -- **Authorization Only**: Payment method validated and authorized -- **No Immediate Charge**: User not charged until trial ends -- **Future Payment**: Stripe automatically charges after 7 days - -#### Active Subscription -- **Immediate Charge**: Regular subscription payments charged immediately -- **Recurring Billing**: Automatic monthly/yearly charges - -### Payment Method Validation -```typescript -// Validate payment method exists for trial -const paymentMethods = await stripe.paymentMethods.list({ - customer: customerId, - type: 'card' -}) - -if (paymentMethods.data.length === 0) { - throw new Error('No payment method found') -} -``` - -## Error Handling - -### Common Stripe Errors -```typescript -try { - await stripe.subscriptions.create(subscriptionData) -} catch (error) { - switch (error.code) { - case 'resource_missing': - // Customer or price not found - break - case 'card_declined': - // Payment method declined - break - case 'authentication_required': - // 3D Secure required - break - default: - // Generic error handling - } -} -``` - -### Webhook Error Handling -```typescript -// app/api/webhooks/route.ts -export async function POST(request: NextRequest) { - let event: Stripe.Event - - try { - const body = await request.text() - const signature = request.headers.get('stripe-signature')! - - event = stripe.webhooks.constructEvent( - body, - signature, - process.env.STRIPE_WEBHOOK_SECRET! - ) - } catch (err) { - console.error('Webhook signature verification failed:', err) - return NextResponse.json({ error: 'Invalid signature' }, { status: 400 }) - } - - // Process event... -} -``` - -## Testing - -### Test Cards -```typescript -// Successful payment -'4242424242424242' - -// Payment requires authentication -'4000002500003155' - -// Payment declined -'4000000000000002' - -// Insufficient funds -'4000000000009995' -``` - -### Test Scenarios - -#### 1. Successful Trial -```bash -# Create checkout session -POST /api/checkout-sessions -โ†’ Redirects to Stripe checkout -โ†’ User enters test card 4242424242424242 -โ†’ checkout.session.completed webhook fires -โ†’ User gets trial access -``` - -#### 2. Trial Cancellation -```bash -# Cancel during trial -POST /api/trial/cancel -โ†’ Stripe subscription canceled -โ†’ customer.subscription.deleted webhook fires -โ†’ Credits set to 0 -``` - -#### 3. Failed Payment Authorization -```bash -# Use declined card -โ†’ payment_intent.payment_failed webhook fires -โ†’ Trial access prevented -``` - -## Monitoring and Logs - -### Key Metrics -- Trial conversion rate -- Payment success rate -- Churn rate -- Revenue metrics - -### Logging Strategy -```typescript -console.log(`Trial started for user ${userId}`) -console.log(`Payment succeeded for subscription ${subscriptionId}`) -console.error(`Payment failed for user ${userId}:`, error) -``` - -### Stripe Dashboard -- Monitor payment failures -- Track subscription metrics -- Review webhook deliveries -- Analyze customer lifecycle - -## Security Best Practices - -1. **Never expose secret keys** in client-side code -2. **Validate webhook signatures** for all incoming events -3. **Use HTTPS** for all webhook endpoints -4. **Implement idempotency** for webhook processing -5. **Log security events** for auditing -6. **Rate limit** API endpoints -7. **Validate user permissions** before subscription operations - -## Production Considerations - -### Webhook Reliability -```typescript -// Implement retry logic for failed webhook processing -const maxRetries = 3 -let retryCount = 0 - -while (retryCount < maxRetries) { - try { - await processWebhook(event) - break - } catch (error) { - retryCount++ - if (retryCount === maxRetries) { - throw error - } - await new Promise(resolve => setTimeout(resolve, 1000 * retryCount)) - } -} -``` - -### Database Consistency -- Use transactions for critical operations -- Implement eventual consistency for webhook data -- Add data validation and sanitization -- Monitor for data discrepancies - -### Performance -- Cache frequently accessed Stripe data -- Use webhook data to update local cache -- Implement request deduplication -- Monitor API rate limits \ No newline at end of file diff --git a/auto-analyst-frontend/docs/system/authentication.md b/auto-analyst-frontend/docs/system/authentication.md new file mode 100644 index 00000000..86b12e1c --- /dev/null +++ b/auto-analyst-frontend/docs/system/authentication.md @@ -0,0 +1,248 @@ +# Authentication System Documentation + +Auto-Analyst uses NextAuth.js for comprehensive authentication management. + +## Authentication Overview + +### Supported Providers +- **Google OAuth** - Primary authentication method +- **Credentials** - Admin temporary login +- **Guest Mode** - Limited trial access + +### NextAuth Configuration +```typescript +// app/api/auth/[...nextauth]/route.ts +const handler = NextAuth({ + providers: [ + GoogleProvider({ + clientId: process.env.GOOGLE_CLIENT_ID, + clientSecret: process.env.GOOGLE_CLIENT_SECRET, + }), + CredentialsProvider({ + name: "Temporary Login", + credentials: { + password: { label: "Temporary Code", type: "password" }, + }, + async authorize(credentials) { + if (credentials?.password === process.env.NEXT_PUBLIC_ANALYTICS_ADMIN_PASSWORD) { + return { + id: "admin", + name: "Administrator", + email: "admin@example.com", + } + } + return null + } + }) + ], + callbacks: { + async signIn({ user }) { + // Save user profile to Redis on first login + await profileUtils.saveUserProfile(user.id, { + email: user.email, + name: user.name, + image: user.image, + joinedDate: new Date().toISOString(), + role: 'Free' + }) + return true + } + } +}) +``` + +## Session Management + +### Client-Side Sessions +```typescript +// Using useSession hook +const { data: session, status } = useSession() + +if (status === "loading") return +if (status === "unauthenticated") return + +// Authenticated user +return +``` + +### Server-Side Sessions +```typescript +// API routes authentication +export async function GET(request: NextRequest) { + const token = await getToken({ req: request }) + + if (!token?.sub) { + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + } + + // Proceed with authenticated request +} +``` + +## User Profile Management + +### Profile Storage (Redis) +```typescript +// lib/redis.ts +export const profileUtils = { + async saveUserProfile(userId: string, profile: UserProfile) { + await redis.hset(KEYS.USER_PROFILE(userId), { + email: profile.email, + name: profile.name || '', + image: profile.image || '', + joinedDate: profile.joinedDate || new Date().toISOString(), + role: profile.role || 'Free' + }) + }, + + async getUserProfile(userId: string) { + return await redis.hgetall(KEYS.USER_PROFILE(userId)) + } +} +``` + +### Profile Context +```typescript +// lib/contexts/user-context.tsx +const UserProvider = ({ children }) => { + const { data: session } = useSession() + const [userProfile, setUserProfile] = useState(null) + + useEffect(() => { + if (session?.user) { + fetchUserProfile(session.user.id) + } + }, [session]) + + return ( + + {children} + + ) +} +``` + +## Authentication Flow + +### 1. Login Process +``` +User clicks "Sign in with Google" + โ†“ +Google OAuth consent + โ†“ +NextAuth callback processing + โ†“ +Profile saved to Redis + โ†“ +Session created + โ†“ +User redirected to dashboard +``` + +### 2. Logout Process +```typescript +const handleLogout = async () => { + // Clear local state + clearMessages() + clearCredits() + + // Sign out from NextAuth + await signOut({ callbackUrl: '/' }) +} +``` + +## Protected Routes + +### Page Protection +```typescript +// app/admin/page.tsx +export default function AdminPage() { + const { data: session, status } = useSession() + + if (status === "loading") return + + if (!session?.user?.isAdmin) { + redirect('/login') + } + + return +} +``` + +### API Protection +```typescript +// Middleware for protected API routes +export async function authMiddleware(request: NextRequest) { + const token = await getToken({ req: request }) + + if (!token) { + return new Response('Unauthorized', { status: 401 }) + } + + // Add user info to request + request.userId = token.sub +} +``` + +## Security Features + +### CSRF Protection +- Built-in NextAuth CSRF protection +- Secure cookie configuration +- SameSite cookie policies + +### Session Security +```typescript +// nextauth configuration +export default NextAuth({ + session: { + strategy: "jwt", + maxAge: 30 * 24 * 60 * 60, // 30 days + }, + jwt: { + secret: process.env.NEXTAUTH_SECRET, + }, + cookies: { + sessionToken: { + name: "next-auth.session-token", + options: { + httpOnly: true, + sameSite: "lax", + path: "/", + secure: process.env.NODE_ENV === "production", + }, + }, + }, +}) +``` + +## Guest User Support + +### Anonymous Access +```typescript +// lib/store/freeTrialStore.ts +const useFreeTrialStore = create( + persist( + (set, get) => ({ + queriesUsed: 0, + maxQueries: 5, + + hasFreeTrial: () => get().queriesUsed < get().maxQueries, + }), + { name: 'free-trial-storage' } + ) +) +``` + +### Guest Session Handling +```typescript +// Guest users get temporary session IDs +const getGuestId = () => { + let guestId = localStorage.getItem('guestUserId') + if (!guestId) { + guestId = `guest-${Date.now()}` + localStorage.setItem('guestUserId', guestId) + } + return guestId +} +``` \ No newline at end of file diff --git a/auto-analyst-frontend/docs/system/middleware.md b/auto-analyst-frontend/docs/system/middleware.md new file mode 100644 index 00000000..f4ecd87b --- /dev/null +++ b/auto-analyst-frontend/docs/system/middleware.md @@ -0,0 +1,305 @@ +# Middleware Configuration + +This document explains the Next.js middleware setup for Auto-Analyst frontend. + +## Overview + +The middleware handles route protection, particularly for admin routes, using Next.js edge runtime capabilities. + +## Middleware Implementation + +### Current Configuration + +```typescript +// middleware.ts +import { NextResponse } from 'next/server'; +import type { NextRequest } from 'next/server'; + +export function middleware(request: NextRequest) { + // Get the pathname + const { pathname } = request.nextUrl; + + // Check if it's an admin route + if (pathname.startsWith('/admin')) { + const token = request.cookies.get('authToken')?.value; + + // If there's no token or it's not valid, redirect to login + if (!token) { + const url = new URL('/login', request.url); + url.searchParams.set('redirect', pathname); + return NextResponse.redirect(url); + } + } + + return NextResponse.next(); +} + +// Configure which paths should trigger this middleware +export const config = { + matcher: ['/admin/:path*'], +}; +``` + +## Route Protection + +### Protected Routes + +Currently, the middleware protects: + +- `/admin/*` - All admin dashboard routes + +### Authentication Method + +**Current Implementation:** +- Checks for `authToken` cookie (basic implementation) +- Redirects to `/login` with redirect parameter if no token found + + +```typescript +import { getToken } from 'next-auth/jwt' + +export async function middleware(request: NextRequest) { + const { pathname } = request.nextUrl; + + if (pathname.startsWith('/admin')) { + const token = await getToken({ + req: request, + secret: process.env.NEXTAUTH_SECRET + }); + + if (!token?.isAdmin) { + const url = new URL('/login', request.url); + url.searchParams.set('redirect', pathname); + return NextResponse.redirect(url); + } + } + + return NextResponse.next(); +} +``` + +## Middleware Patterns + +### 1. Route Protection + +```typescript +// Protect multiple route patterns +export const config = { + matcher: [ + '/admin/:path*', + '/dashboard/:path*', + '/settings/:path*' + ], +}; +``` + +### 2. API Route Protection + +```typescript +if (pathname.startsWith('/api/admin')) { + const apiKey = request.headers.get('X-Admin-API-Key'); + + if (apiKey !== process.env.ADMIN_API_KEY) { + return new Response('Unauthorized', { status: 401 }); + } +} +``` + +### 3. Conditional Redirects + +```typescript +// Redirect authenticated users away from login +if (pathname === '/login') { + const token = request.cookies.get('next-auth.session-token'); + + if (token) { + return NextResponse.redirect(new URL('/chat', request.url)); + } +} +``` + +## Best Practices + +### Performance Considerations + +1. **Keep middleware lightweight** - Runs on every request +2. **Use efficient checks** - Avoid heavy computations +3. **Cache where possible** - Store frequently accessed data + +### Security Guidelines + +1. **Validate tokens properly** - Use NextAuth token verification +2. **Handle edge cases** - Invalid tokens, expired sessions +3. **Log security events** - Track unauthorized access attempts + +### Error Handling + +```typescript +export function middleware(request: NextRequest) { + try { + // Protection logic + } catch (error) { + console.error('Middleware error:', error); + // Allow request to proceed rather than blocking + return NextResponse.next(); + } +} +``` + +## Advanced Middleware Features + +### CORS Headers + +```typescript +export function middleware(request: NextRequest) { + const response = NextResponse.next(); + + // Add CORS headers + response.headers.set('Access-Control-Allow-Origin', '*'); + response.headers.set('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE'); + + return response; +} +``` + +### Request Logging + +```typescript +export function middleware(request: NextRequest) { + const start = Date.now(); + const response = NextResponse.next(); + + // Log request in development + if (process.env.NODE_ENV === 'development') { + console.log(`${request.method} ${request.nextUrl.pathname} - ${Date.now() - start}ms`); + } + + return response; +} +``` + +### Rate Limiting + +```typescript +const requestCounts = new Map(); + +export function middleware(request: NextRequest) { + const ip = request.ip || 'unknown'; + const current = requestCounts.get(ip) || 0; + + if (current > 100) { // 100 requests per window + return new Response('Rate limit exceeded', { status: 429 }); + } + + requestCounts.set(ip, current + 1); + + // Reset counter after 1 minute + setTimeout(() => requestCounts.delete(ip), 60000); + + return NextResponse.next(); +} +``` + +## Configuration Options + +### Matcher Patterns + +```typescript +export const config = { + matcher: [ + // Match all paths except static files + '/((?!_next/static|_next/image|favicon.ico).*)', + + // Match specific patterns + '/admin/:path*', + '/api/protected/:path*', + + // Exclude specific paths + '/((?!api|_next/static|_next/image|favicon.ico).*)', + ], +}; +``` + +### Conditional Execution + +```typescript +export const config = { + matcher: [ + { + source: '/admin/:path*', + has: [ + { + type: 'header', + key: 'authorization', + }, + ], + }, + ], +}; +``` + +## Integration with NextAuth + +### Session-based Protection + +```typescript +import { withAuth } from 'next-auth/middleware' + +export default withAuth( + function middleware(request) { + // Additional middleware logic + }, + { + callbacks: { + authorized: ({ token, req }) => { + // Admin routes require admin role + if (req.nextUrl.pathname.startsWith('/admin')) { + return token?.role === 'admin' + } + + // Protected routes require any valid token + return !!token + }, + }, + } +) + +export const config = { + matcher: ['/admin/:path*', '/dashboard/:path*'] +} +``` + +## Troubleshooting + +### Common Issues + +1. **Infinite redirects**: Check redirect logic carefully +2. **Middleware not triggering**: Verify matcher configuration +3. **Performance issues**: Avoid heavy operations in middleware + +### Debug Middleware + +```typescript +export function middleware(request: NextRequest) { + console.log('Middleware triggered for:', request.nextUrl.pathname); + console.log('Cookies:', request.cookies.toString()); + console.log('Headers:', Object.fromEntries(request.headers)); + + // Your middleware logic + + return NextResponse.next(); +} +``` + +## Future Enhancements + +Potential improvements to the middleware system: + +1. **Enhanced Authentication**: Better token validation +2. **Rate Limiting**: Request throttling per user/IP +3. **Security Headers**: CSP, HSTS, etc. +4. **Analytics**: Request tracking and monitoring +5. **A/B Testing**: Feature flag support +6. **Geolocation**: Region-based routing + + \ No newline at end of file diff --git a/auto-analyst-frontend/docs/system/model-registry.md b/auto-analyst-frontend/docs/system/model-registry.md new file mode 100644 index 00000000..d0219a89 --- /dev/null +++ b/auto-analyst-frontend/docs/system/model-registry.md @@ -0,0 +1,388 @@ +# Model Registry Management + +This guide explains how to add, modify, and manage AI models in the Auto-Analyst platform using the synchronized model registry files. + +## Overview + +Auto-Analyst uses two synchronized model registry files to maintain consistency between frontend and backend: + +- **Frontend**: `lib/model-registry.ts` - TypeScript configuration for UI and client-side logic +- **Backend**: `src/utils/model_registry.py` - Python configuration for API and server-side operations + +> **Important**: Both files must be updated simultaneously to maintain consistency across the platform. + +## Model Registry Structure + +Both registries contain the same core information: + +- **Providers** - Supported AI providers (OpenAI, Anthropic, GROQ, Gemini) +- **Model Costs** - Token pricing for each model +- **Model Tiers** - Credit cost groupings (1, 3, 5, 20 credits) +- **Model Metadata** - Display names and context windows + +## Adding a New Model + +### Step 1: Update Frontend Registry (`model-registry.ts`) + +Add the model to the appropriate provider in `MODEL_COSTS`: + +```typescript +// lib/model-registry.ts +export const MODEL_COSTS = { + openai: { + // ... existing models + "gpt-5": { input: 0.005, output: 0.015 }, // New model + }, + anthropic: { + // ... existing models + "claude-4-haiku": { input: 0.0001, output: 0.0005 }, // New model + } +} +``` + +Add display metadata: + +```typescript +export const MODEL_METADATA: Record = { + // ... existing models + "gpt-5": { displayName: "GPT-5", contextWindow: 256000 }, + "claude-4-haiku": { displayName: "Claude 4 Haiku", contextWindow: 300000 }, +} +``` + +Add to appropriate tier: + +```typescript +export const MODEL_TIERS = { + "tier3": { + "name": "Premium", + "credits": 5, + "models": [ + // ... existing models + "gpt-5" + ] + }, + "tier1": { + "name": "Basic", + "credits": 1, + "models": [ + // ... existing models + "claude-4-haiku" + ] + } +} +``` + +### Step 2: Update Backend Registry (`model_registry.py`) + +Mirror the same changes in the Python file: + +```python +# src/utils/model_registry.py +MODEL_COSTS = { + "openai": { + # ... existing models + "gpt-5": {"input": 0.005, "output": 0.015}, # New model + }, + "anthropic": { + # ... existing models + "claude-4-haiku": {"input": 0.0001, "output": 0.0005}, # New model + } +} +``` + +Add metadata: + +```python +MODEL_METADATA = { + # ... existing models + "gpt-5": {"display_name": "GPT-5", "context_window": 256000}, + "claude-4-haiku": {"display_name": "Claude 4 Haiku", "context_window": 300000}, +} +``` + +Add to tiers: + +```python +MODEL_TIERS = { + "tier3": { + "name": "Premium", + "credits": 5, + "models": [ + # ... existing models + "gpt-5" + ] + }, + "tier1": { + "name": "Basic", + "credits": 1, + "models": [ + # ... existing models + "claude-4-haiku" + ] + } +} +``` + +### Step 3: Test the Changes + +**Frontend Testing:** +```typescript +// Test in browser console +import { getModelCreditCost, getDisplayName } from '@/lib/model-registry' + +console.log(getModelCreditCost('gpt-5')) // Should return 5 +console.log(getDisplayName('claude-4-haiku')) // Should return "Claude 4 Haiku" +``` + +**Backend Testing:** +```python +# Test in Python shell +from src.utils.model_registry import get_credit_cost, get_display_name + +print(get_credit_cost('gpt-5')) # Should return 5 +print(get_display_name('claude-4-haiku')) # Should return "Claude 4 Haiku" +``` + +## Adding a New Provider + +### Step 1: Add Provider Constants + +**Frontend (`model-registry.ts`):** +```typescript +export const PROVIDERS = { + openai: "OpenAI", + anthropic: "Anthropic", + groq: "GROQ", + gemini: "Google Gemini", + newprovider: "New Provider" // Add new provider +}; +``` + +**Backend (`model_registry.py`):** +```python +PROVIDERS = { + "openai": "OpenAI", + "anthropic": "Anthropic", + "groq": "GROQ", + "gemini": "Google Gemini", + "newprovider": "New Provider" # Add new provider +} +``` + +### Step 2: Add Provider Models + +**Frontend:** +```typescript +export const MODEL_COSTS = { + // ... existing providers + newprovider: { + "new-model-1": { input: 0.001, output: 0.002 }, + "new-model-2": { input: 0.003, output: 0.006 } + } +} +``` + +**Backend:** +```python +MODEL_COSTS = { + # ... existing providers + "newprovider": { + "new-model-1": {"input": 0.001, "output": 0.002}, + "new-model-2": {"input": 0.003, "output": 0.006} + } +} +``` + +### Step 3: Update UI Configuration + +Add to `MODEL_PROVIDERS_UI` in frontend: + +```typescript +export const MODEL_PROVIDERS_UI = [ + // ... existing providers + { + name: 'newprovider', + models: Object.keys(MODEL_COSTS.newprovider).map(id => ({ + id, + name: MODEL_METADATA[id]?.displayName || id + })), + displayName: PROVIDERS.newprovider + } +]; +``` + +## Credit Tier Guidelines + +Choose the appropriate tier based on model capability and cost: + +| Tier | Credits | Typical Models | Use Case | +|------|---------|----------------|----------| +| **Tier 1** | 1 credit | Basic models, fast responses | Quick queries, code completion | +| **Tier 2** | 3 credits | Standard models, good quality | General conversation, analysis | +| **Tier 3** | 5 credits | Premium models, high quality | Complex reasoning, detailed analysis | +| **Tier 4** | 20 credits | Top-tier models, best quality | Advanced reasoning, critical tasks | + +### Example Tier Assignments: + +```typescript +// Low-cost, fast models โ†’ Tier 1 +"claude-3-5-haiku-latest": Tier 1 (1 credit) +"llama3-8b-8192": Tier 1 (1 credit) + +// Mid-range models โ†’ Tier 2-3 +"gpt-4o-mini": Tier 2 (3 credits) +"claude-3-5-sonnet-latest": Tier 3 (5 credits) + +// Premium, expensive models โ†’ Tier 4 +"o1-pro": Tier 4 (20 credits) +"claude-3-opus-latest": Tier 4 (20 credits) +``` + +## Pricing Research + +When adding models, research current pricing from provider documentation: + +### OpenAI Pricing +- Visit: https://openai.com/api/pricing/ +- Look for input/output token costs per 1K tokens + +### Anthropic Pricing +- Visit: https://www.anthropic.com/pricing +- Convert per-token to per-1K-token pricing + +### GROQ Pricing +- Visit: https://groq.com/pricing/ +- Usually very competitive pricing + +### Google Gemini Pricing +- Visit: https://cloud.google.com/vertex-ai/generative-ai/pricing +- Check for Gemini model pricing + +## Validation and Testing + +### 1. Sync Validation + +Create a script to validate sync between files: + +```typescript +// scripts/validate-model-sync.ts +import { MODEL_COSTS as FRONTEND_COSTS } from '../lib/model-registry' + +// Compare with backend costs (would need to import Python data) +function validateSync() { + // Implementation to compare frontend and backend registries +} +``` + +### 2. Credit Cost Testing + +Test credit calculations: + +```typescript +// Test credit costs match expectations +const testModels = ['gpt-4o', 'claude-3-5-sonnet-latest', 'o1-pro'] +testModels.forEach(model => { + const cost = getModelCreditCost(model) + console.log(`${model}: ${cost} credits`) +}) +``` + +### 3. UI Testing + +Verify models appear correctly in: +- Model selection dropdown +- Credit cost display +- Tier indicators +- Provider grouping + +## Backend Integration + +### Credit Deduction + +The backend uses the registry for credit calculations: + +```python +# In chat processing +model_name = request.model +credit_cost = get_credit_cost(model_name) + +# Deduct credits from user account +success = deduct_user_credits(user_id, credit_cost) +if not success: + return {"error": "Insufficient credits"} +``` + +### Model Validation + +Validate requested models exist: + +```python +from src.utils.model_registry import MODEL_COSTS + +def validate_model(model_name: str, provider: str) -> bool: + provider_models = MODEL_COSTS.get(provider, {}) + return model_name in provider_models +``` + +## Maintenance Best Practices + +### 1. Regular Updates +- Monitor provider pricing changes +- Update costs monthly or when providers announce changes +- Test thoroughly after updates + +### 2. Version Control +- Always update both files in the same commit +- Include tests for new models +- Document breaking changes + +### 3. Documentation +- Update this documentation when adding new providers +- Maintain changelog of model additions/removals +- Document any special model requirements + +## Troubleshooting + +### Common Issues + +1. **Model not appearing in UI** + - Check `MODEL_PROVIDERS_UI` configuration + - Verify model is in `MODEL_METADATA` + - Clear browser cache + +2. **Incorrect credit costs** + - Verify model is in correct tier + - Check tier credit amounts + - Validate backend synchronization + +3. **Frontend/Backend mismatch** + - Compare both registry files + - Run validation scripts + - Check recent commits for partial updates + +### Debug Commands + +```typescript +// Frontend debugging +console.log('All tiers:', MODEL_TIERS) +console.log('Model tier:', getModelTier('gpt-4o')) +console.log('Credit cost:', getModelCreditCost('gpt-4o')) +``` + +```python +# Backend debugging +print("All tiers:", MODEL_TIERS) +print("Model tier:", get_model_tier('gpt-4o')) +print("Credit cost:", get_credit_cost('gpt-4o')) +``` + +## Future Enhancements + +Potential improvements to the model registry system: + +1. **Automated Sync Validation** - CI/CD checks for registry consistency +2. **Dynamic Pricing** - API-based pricing updates +3. **A/B Testing** - Different credit costs for user segments +4. **Usage Analytics** - Track model popularity and adjust tiers +5. **Cost Optimization** - Automatic tier adjustments based on usage patterns \ No newline at end of file diff --git a/auto-analyst-frontend/docs/system/redis-schema.md b/auto-analyst-frontend/docs/system/redis-schema.md new file mode 100644 index 00000000..004a1890 --- /dev/null +++ b/auto-analyst-frontend/docs/system/redis-schema.md @@ -0,0 +1,242 @@ +# Redis Schema Documentation + +Auto-Analyst uses Redis for caching, session management, and user data storage. + +## Redis Connection + +### Configuration +```typescript +// lib/redis.ts +import { Redis } from '@upstash/redis' + +const redis = new Redis({ + url: process.env.UPSTASH_REDIS_REST_URL, + token: process.env.UPSTASH_REDIS_REST_TOKEN, +}) +``` + +### Key Naming Convention +```typescript +export const KEYS = { + USER_PROFILE: (userId: string) => `user:${userId}:profile`, + USER_SUBSCRIPTION: (userId: string) => `user:${userId}:subscription`, + USER_CREDITS: (userId: string) => `user:${userId}:credits`, +} +``` + +## Data Schemas + +### 1. User Profile +```typescript +// Key: user:{userId}:profile +interface UserProfile { + email: string + name: string + image: string + joinedDate: string + role: string + lastLogin?: string +} + +// Redis Hash +await redis.hset(KEYS.USER_PROFILE(userId), { + email: "user@example.com", + name: "John Doe", + image: "https://...", + joinedDate: "2024-01-01", + role: "Free" +}) +``` + +### 2. User Credits +```typescript +// Key: user:{userId}:credits +interface UserCredits { + total: string + used: string + resetDate: string + lastUpdate: string + isTrialCredits?: string + paymentIntentId?: string +} + +// Redis Hash +await redis.hset(KEYS.USER_CREDITS(userId), { + total: "500", + used: "150", + resetDate: "2024-02-01", + lastUpdate: "2024-01-15T10:30:00Z" +}) +``` + +### 3. User Subscription +```typescript +// Key: user:{userId}:subscription +interface UserSubscription { + plan: string + status: string + interval: string + amount: string + stripeSubscriptionId?: string + stripeCustomerId?: string + nextBilling?: string + cancelAtPeriodEnd?: string +} + +// Redis Hash +await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + plan: "pro", + status: "active", + interval: "month", + amount: "29.99", + stripeSubscriptionId: "sub_...", + nextBilling: "2024-02-01" +}) +``` + +## Utility Functions + +### Credit Management +```typescript +export const creditUtils = { + // Get remaining credits + async getRemainingCredits(userId: string): Promise { + const creditsHash = await redis.hgetall(KEYS.USER_CREDITS(userId)) + if (!creditsHash?.total) return 0 + + const total = parseInt(creditsHash.total) + const used = parseInt(creditsHash.used || '0') + + return Math.max(0, total - used) + }, + + // Deduct credits + async deductCredits(userId: string, amount: number): Promise { + const creditsHash = await redis.hgetall(KEYS.USER_CREDITS(userId)) + if (!creditsHash?.total) return false + + const total = parseInt(creditsHash.total) + const used = parseInt(creditsHash.used || '0') + const remaining = total - used + + if (remaining < amount) return false + + await redis.hset(KEYS.USER_CREDITS(userId), { + used: (used + amount).toString(), + lastUpdate: new Date().toISOString() + }) + + return true + }, + + // Reset credits (monthly/yearly) + async resetUserCredits(userId: string): Promise { + const subscription = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) + if (!subscription?.plan) return false + + const planCredits = getPlanCredits(subscription.plan) + const resetDate = subscription.interval === 'year' + ? getOneYearFromToday() + : getOneMonthFromToday() + + await redis.hset(KEYS.USER_CREDITS(userId), { + total: planCredits.toString(), + used: '0', + resetDate, + lastUpdate: new Date().toISOString() + }) + + return true + } +} +``` + +### Subscription Management +```typescript +export const subscriptionUtils = { + // Get user subscription data + async getUserSubscriptionData(userId: string) { + const [profile, subscription, credits] = await Promise.all([ + redis.hgetall(KEYS.USER_PROFILE(userId)), + redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)), + redis.hgetall(KEYS.USER_CREDITS(userId)) + ]) + + return { + profile, + subscription: subscription || { plan: 'free', status: 'active' }, + credits: { + used: parseInt(credits?.used || '0'), + total: parseInt(credits?.total || '0'), + remaining: Math.max(0, parseInt(credits?.total || '0') - parseInt(credits?.used || '0')) + } + } + }, + + // Check if subscription is active + async isSubscriptionActive(userId: string): Promise { + const subscription = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) + return subscription?.status === 'active' && subscription?.plan !== 'free' + } +} +``` + +## Session Storage + +### NextAuth Sessions +NextAuth.js handles session storage automatically, but we extend it with Redis for additional user data. + +### Guest Sessions +```typescript +// For non-authenticated users +const getGuestSession = () => { + let guestId = localStorage.getItem('guestUserId') + if (!guestId) { + guestId = `guest-${Date.now()}` + localStorage.setItem('guestUserId', guestId) + } + return guestId +} +``` + +## Data Lifecycle + +### User Registration +1. User signs in with Google +2. Profile saved to Redis +3. Default credits allocated (if applicable) +4. Free plan subscription created + +### Subscription Changes +1. Stripe webhook received +2. Subscription data updated in Redis +3. Credits allocated based on new plan +4. Reset date calculated + +### Credit Usage +1. User sends chat message +2. Model credit cost calculated +3. Credits deducted from Redis +4. UI updated with new balance + +## Performance Optimization + +### Batch Operations +```typescript +// Update multiple fields atomically +await redis.hset(KEYS.USER_CREDITS(userId), { + used: newUsed.toString(), + lastUpdate: new Date().toISOString(), + lastAction: 'chat_message' +}) +``` + +### Connection Pooling +- Upstash Redis handles connection pooling +- Built-in retry logic for failed requests +- Automatic failover for high availability + +### Caching Strategy +- User data cached in Redis for fast access +- Frontend caches credit balance locally +- Periodic sync between client and server \ No newline at end of file diff --git a/auto-analyst-frontend/docs/system/webhooks.md b/auto-analyst-frontend/docs/system/webhooks.md new file mode 100644 index 00000000..b4224236 --- /dev/null +++ b/auto-analyst-frontend/docs/system/webhooks.md @@ -0,0 +1,247 @@ +# Webhooks Documentation + +Auto-Analyst processes webhooks from Stripe to handle subscription and payment events. + +## Webhook Overview + +### Supported Events +- `invoice.payment_succeeded` - Successful payment processing +- `customer.subscription.created` - New subscription created +- `customer.subscription.updated` - Subscription plan changes +- `customer.subscription.deleted` - Subscription cancellation +- `payment_intent.succeeded` - One-time payment success + +### Webhook Endpoint +```typescript +// app/api/webhooks/route.ts +export async function POST(request: Request) { + const body = await request.text() + const sig = request.headers.get('stripe-signature') + + let event: Stripe.Event + + try { + event = stripe.webhooks.constructEvent(body, sig, process.env.STRIPE_WEBHOOK_SECRET) + } catch (err) { + return new Response(`Webhook signature verification failed`, { status: 400 }) + } + + // Process event + await processWebhookEvent(event) + + return new Response('Webhook processed successfully', { status: 200 }) +} +``` + +## Event Processing + +### Payment Success Handler +```typescript +async function handlePaymentSucceeded(invoice: Stripe.Invoice) { + const customerId = invoice.customer as string + const subscriptionId = invoice.subscription as string + + // Get subscription details + const subscription = await stripe.subscriptions.retrieve(subscriptionId) + const userId = subscription.metadata.userId + + if (!userId) { + console.error('No userId found in subscription metadata') + return + } + + // Update user subscription in Redis + await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + plan: subscription.items.data[0].price.lookup_key, + status: subscription.status, + interval: subscription.items.data[0].price.recurring?.interval, + amount: (subscription.items.data[0].price.unit_amount / 100).toString(), + stripeSubscriptionId: subscription.id, + stripeCustomerId: customerId, + nextBilling: new Date(subscription.current_period_end * 1000).toISOString() + }) + + // Allocate credits based on plan + await allocateCreditsForPlan(userId, subscription.items.data[0].price.lookup_key) +} +``` + +### Subscription Deletion Handler +```typescript +async function handleSubscriptionDeleted(subscription: Stripe.Subscription) { + const userId = subscription.metadata.userId + + if (!userId) return + + // Downgrade to free plan + await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + plan: 'free', + status: 'canceled', + interval: 'month', + amount: '0', + canceledAt: new Date().toISOString() + }) + + // Set credits to 0 + await redis.hset(KEYS.USER_CREDITS(userId), { + total: '0', + used: '0', + resetDate: '', + lastUpdate: new Date().toISOString() + }) +} +``` + +### Subscription Update Handler +```typescript +async function handleSubscriptionUpdated(subscription: Stripe.Subscription) { + const userId = subscription.metadata.userId + + if (!userId) return + + // Update subscription details + await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + plan: subscription.items.data[0].price.lookup_key, + status: subscription.status, + cancelAtPeriodEnd: subscription.cancel_at_period_end.toString(), + nextBilling: new Date(subscription.current_period_end * 1000).toISOString() + }) + + // If plan changed, reallocate credits + if (subscription.status === 'active') { + await allocateCreditsForPlan(userId, subscription.items.data[0].price.lookup_key) + } +} +``` + +## Credit Allocation + +### Plan-Based Credit Assignment +```typescript +async function allocateCreditsForPlan(userId: string, planName: string) { + const creditAmounts = { + 'trial': 500, + 'standard': 500, + 'pro': 999999, // Unlimited + 'enterprise': 999999 + } + + const credits = creditAmounts[planName] || 0 + const resetDate = planName.includes('yearly') + ? getOneYearFromToday() + : getOneMonthFromToday() + + await redis.hset(KEYS.USER_CREDITS(userId), { + total: credits.toString(), + used: '0', + resetDate, + lastUpdate: new Date().toISOString() + }) +} +``` + +## Security + +### Webhook Verification +```typescript +function verifyWebhookSignature(body: string, signature: string): boolean { + try { + stripe.webhooks.constructEvent(body, signature, process.env.STRIPE_WEBHOOK_SECRET) + return true + } catch (error) { + console.error('Webhook signature verification failed:', error) + return false + } +} +``` + +### Idempotency +```typescript +// Prevent duplicate processing +const processedEvents = new Set() + +async function processWebhookEvent(event: Stripe.Event) { + if (processedEvents.has(event.id)) { + console.log(`Event ${event.id} already processed`) + return + } + + // Process event + await handleEvent(event) + + // Mark as processed + processedEvents.add(event.id) + + // Clean up old events (prevent memory leaks) + if (processedEvents.size > 1000) { + processedEvents.clear() + } +} +``` + +## Error Handling + +### Retry Logic +```typescript +async function processWithRetry(event: Stripe.Event, maxRetries = 3) { + for (let attempt = 1; attempt <= maxRetries; attempt++) { + try { + await processEvent(event) + return + } catch (error) { + console.error(`Attempt ${attempt} failed:`, error) + + if (attempt === maxRetries) { + // Log to error tracking service + console.error(`Failed to process event after ${maxRetries} attempts:`, event.id) + throw error + } + + // Exponential backoff + await new Promise(resolve => setTimeout(resolve, Math.pow(2, attempt) * 1000)) + } + } +} +``` + +### Dead Letter Queue +```typescript +async function handleFailedWebhook(event: Stripe.Event, error: Error) { + // Store failed event for manual processing + await redis.lpush('failed_webhooks', JSON.stringify({ + eventId: event.id, + eventType: event.type, + error: error.message, + timestamp: new Date().toISOString(), + data: event.data + })) +} +``` + +## Monitoring + +### Webhook Logs +```typescript +function logWebhookEvent(event: Stripe.Event, status: 'success' | 'error', error?: Error) { + console.log({ + eventId: event.id, + eventType: event.type, + status, + timestamp: new Date().toISOString(), + error: error?.message + }) +} +``` + +### Health Checks +```typescript +// Monitor webhook processing health +export async function GET() { + const failedCount = await redis.llen('failed_webhooks') + + return Response.json({ + status: failedCount > 10 ? 'unhealthy' : 'healthy', + failedWebhooks: failedCount, + lastProcessed: await redis.get('last_webhook_processed') + }) +} \ No newline at end of file diff --git a/auto-analyst-frontend/docs/trial-system.md b/auto-analyst-frontend/docs/trial-system.md deleted file mode 100644 index 856f4fda..00000000 --- a/auto-analyst-frontend/docs/trial-system.md +++ /dev/null @@ -1,614 +0,0 @@ -# Trial System - -This document covers the complete 7-day trial system implementation in Auto-Analyst, including payment authorization, access management, and conversion logic. - -## Overview - -Auto-Analyst uses a 7-day free trial with Stripe subscription trials. Users must authorize a payment method to access the trial but are not charged until the trial period ends. - -## Trial Flow Architecture - -### High-Level Flow -``` -[Start Trial] โ†’ [Payment Auth] โ†’ [Trial Access] โ†’ [Auto-Convert] OR [Cancel] - โ†“ โ†“ โ†“ โ†“ โ†“ -[Pricing Page] โ†’ [Stripe Checkout] โ†’ [500 Credits] โ†’ [Paid Plan] โ†’ [Access Removed] -``` - -### Detailed Flow -1. User clicks "Start 7-Day Trial" on pricing page -2. Redirected to Stripe checkout with trial subscription -3. Payment method authorization required (no charge) -4. `checkout.session.completed` webhook fired -5. User calls `/api/trial/start` to activate trial -6. Immediate access with 500 Standard plan credits -7. After 7 days: Stripe automatically charges OR user cancels - -## Implementation Components - -### 1. Subscription Creation (Not Checkout Session) - -**File**: `app/api/checkout-sessions/route.ts` - -```typescript -export async function POST(request: NextRequest) { - const { priceId, userId, planName, interval, promoCode } = await request.json() - - // Create or retrieve customer - let customerId = await getOrCreateCustomer(userId) - - // Calculate trial end date - const trialEndTimestamp = TrialUtils.getTrialEndTimestamp() - - // Create subscription with trial period - const subscription = await stripe.subscriptions.create({ - customer: customerId, - items: [{ price: priceId }], - trial_end: trialEndTimestamp, - expand: ['latest_invoice.payment_intent'], - payment_behavior: 'default_incomplete', - payment_settings: { - save_default_payment_method: 'on_subscription', - }, - metadata: { - userId: userId || 'anonymous', - planName, - interval, - priceId, - isTrial: 'true', - trialEndDate: TrialUtils.getTrialEndDate(), - }, - // Apply coupon if provided - ...(couponId && { coupon: couponId }) - }) - - // Create setup intent for payment method collection if needed - let clientSecret = subscription.latest_invoice?.payment_intent?.client_secret - - if (!clientSecret && subscription.status === 'trialing') { - const setupIntent = await stripe.setupIntents.create({ - customer: customerId, - usage: 'off_session', - metadata: { - subscription_id: subscription.id, - is_trial_setup: 'true', - userId: userId || 'anonymous', - isTrial: 'true', - planName, - interval, - }, - }) - clientSecret = setupIntent.client_secret - } - - return NextResponse.json({ - subscriptionId: subscription.id, - clientSecret: clientSecret, - trialEnd: subscription.trial_end, - isTrialSetup: !subscription.latest_invoice?.payment_intent - }) -} -``` - -### 2. Trial Activation - -**File**: `app/api/trial/start/route.ts` - -```typescript -export async function POST(request: NextRequest) { - const { subscriptionId, planName, interval, amount } = await request.json() - const token = await getToken({ req: request }) - const userId = token.sub - - // Retrieve and validate subscription - const subscription = await stripe.subscriptions.retrieve(subscriptionId) - - if (subscription.status !== 'trialing') { - return NextResponse.json( - { error: 'Subscription is not in trial status' }, - { status: 400 } - ) - } - - // Validate payment method is attached - const hasPaymentMethod = await validatePaymentMethod(subscription) - - if (!hasPaymentMethod) { - return NextResponse.json( - { error: 'Payment method setup required. Please complete payment method verification.' }, - { status: 400 } - ) - } - - // Store customer mapping for webhooks - await redis.set(`stripe:customer:${subscription.customer}`, userId) - - // Store subscription data - const now = new Date() - const trialEndDate = TrialUtils.getTrialEndDate(now) - - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - plan: planName, - planType: 'STANDARD', - status: 'trialing', - amount: amount ? amount.toString() : '15', - interval: interval || 'month', - purchaseDate: now.toISOString(), - renewalDate: trialEndDate, - lastUpdated: now.toISOString(), - stripeCustomerId: subscription.customer as string, - stripeSubscriptionId: subscription.id, - trialEndDate: trialEndDate - }) - - // Initialize trial credits - await creditUtils.initializeTrialCredits(userId, { - total: TrialUtils.getTrialCredits(), - resetDate: CreditConfig.getNextResetDate() - }) - - return NextResponse.json({ - success: true, - trialStarted: true, - credits: TrialUtils.getTrialCredits(), - trialEndDate: trialEndDate, - subscriptionId: subscription.id - }) -} -``` - -### 3. Payment Method Validation - -```typescript -async function validatePaymentMethod(customerId: string): Promise { - try { - // Check for attached payment methods - const paymentMethods = await stripe.paymentMethods.list({ - customer: customerId, - type: 'card' - }) - - if (paymentMethods.data.length > 0) return true - - // Check customer's default payment method - const customer = await stripe.customers.retrieve(customerId) - if (customer.invoice_settings?.default_payment_method) return true - - // Check setup intents for this customer - const setupIntents = await stripe.setupIntents.list({ - customer: customerId, - limit: 1 - }) - - return setupIntents.data.some(si => si.status === 'succeeded') - } catch (error) { - console.error('Payment method validation error:', error) - return false - } -} -``` - -### 4. Trial Cancellation - -**File**: `app/api/trial/cancel/route.ts` - -```typescript -export async function POST(request: NextRequest) { - const token = await getToken({ req: request }) - const userId = token.sub - - const subscriptionData = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) - const subscriptionId = subscriptionData.stripeSubscriptionId - - if (!subscriptionId) { - return NextResponse.json( - { error: 'No subscription found' }, - { status: 404 } - ) - } - - const subscription = await stripe.subscriptions.retrieve(subscriptionId) - - if (subscription.status === 'trialing') { - // Immediate cancellation for trials - await stripe.subscriptions.cancel(subscriptionId, { - prorate: false - }) - - // Mark as trial cancellation - await redis.hset(KEYS.USER_CREDITS(userId), { - trialCanceled: 'true' - }) - - return NextResponse.json({ - success: true, - canceled: true, - creditsRemoved: true, - message: 'Trial canceled successfully' - }) - } else { - // Cancel at period end for active subscriptions - await stripe.subscriptions.update(subscriptionId, { - cancel_at_period_end: true - }) - - const periodEnd = new Date(subscription.current_period_end * 1000) - - return NextResponse.json({ - success: true, - canceledAtPeriodEnd: true, - accessUntil: periodEnd.toISOString(), - message: 'Subscription will cancel at period end' - }) - } -} -``` - -## Trial States - -### State Management - -```typescript -interface TrialState { - status: 'trialing' | 'active' | 'canceled' - trialStart?: string // ISO timestamp - trialEnd?: string // ISO timestamp - hasPaymentMethod: boolean - creditsGranted: boolean - autoConvertEnabled: boolean -} -``` - -### State Transitions - -```mermaid -graph TD - A[Start Trial] --> B{Payment Auth?} - B -->|Success| C[Trialing] - B -->|Failed| D[No Access] - C --> E{7 Days Later} - E -->|Auto-Convert| F[Active Subscription] - E -->|User Cancels| G[Canceled] - C -->|User Cancels| G - F -->|User Cancels| H[Cancel at Period End] - G --> I[Access Removed] - H --> I -``` - -### Status Synchronization - -**Redis Storage**: -```typescript -// user:${userId}:subscription -{ - status: 'trialing', - displayStatus: 'trialing', - stripeSubscriptionStatus: 'trialing', - trialStartDate: '2024-01-15T10:30:00.000Z', - trialEndDate: '2024-01-22T10:30:00.000Z' -} -``` - -**Stripe to Redis Mapping**: -```typescript -const statusMapping = { - 'trialing': { status: 'trialing', displayStatus: 'Trial' }, - 'active': { status: 'active', displayStatus: 'Active' }, - 'canceled': { status: 'canceled', displayStatus: 'Canceled' }, - 'past_due': { status: 'past_due', displayStatus: 'Past Due' } -} -``` - -## Webhook Integration - -### Trial Conversion Webhook - -**Event**: `invoice.payment_succeeded` - -```typescript -// When trial ends and first payment succeeds -if (invoice.billing_reason === 'subscription_create') { - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'active', - stripeSubscriptionStatus: 'active', - displayStatus: 'active', - trialToActiveDate: new Date().toISOString(), - trialEndedAt: new Date().toISOString() - }) - - // Refresh credits for new active status - await subscriptionUtils.refreshCreditsIfNeeded(userId) -} -``` - -### Trial Cancellation Webhook - -**Event**: `customer.subscription.deleted` - -```typescript -await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'canceled', - stripeSubscriptionStatus: 'canceled', - displayStatus: 'canceled', - canceledAt: new Date().toISOString(), - subscriptionDeleted: 'true' -}) - -// Zero out credits for canceled trials -await creditUtils.setZeroCredits(userId) -``` - -## Payment Authorization - -### Stripe Configuration - -```typescript -// No immediate charge during trial -const subscription = await stripe.subscriptions.create({ - customer: customerId, - items: [{ price: priceId }], - trial_period_days: 7, - - // Payment method will be charged after trial - payment_behavior: 'default_incomplete', - expand: ['latest_invoice.payment_intent'] -}) -``` - -### Authorization Validation - -```typescript -// Ensure payment method is authorized before granting access -const paymentIntent = subscription.latest_invoice?.payment_intent - -if (paymentIntent?.status !== 'succeeded') { - throw new Error('Payment authorization required') -} -``` - -## Credit Management - -### Trial Credit Allocation - -```typescript -// Initialize 500 credits for trial users -await creditUtils.initializeTrialCredits(userId, { - total: 500, - resetDate: CreditConfig.getNextResetDate(), - isTrialCredits: 'true', - paymentIntentId: session.payment_intent.id -}) -``` - -### Credit Preservation Logic - -```typescript -// Credits preserved during successful trial conversion -// Only zeroed for genuine cancellations - -function shouldZeroCredits(creditsData: any): boolean { - // Genuine trial cancellation (user canceled before payment) - if (creditsData.trialCanceled === 'true') return true - - // Subscription deleted (not converted to paid) - if (creditsData.subscriptionDeleted === 'true') return true - - return false -} -``` - -## Frontend Integration - -### Trial Start Button - -```typescript -// components/pricing/TrialButton.tsx -function TrialButton({ priceId, planType }: TrialButtonProps) { - const [isLoading, setIsLoading] = useState(false) - - const startTrial = async () => { - setIsLoading(true) - - try { - // Create checkout session - const response = await fetch('/api/checkout-sessions', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ priceId, planType }) - }) - - const { url } = await response.json() - - // Redirect to Stripe checkout - window.location.href = url - } catch (error) { - console.error('Trial start failed:', error) - setIsLoading(false) - } - } - - return ( - - ) -} -``` - -### Checkout Success Handler - -```typescript -// app/checkout/success/page.tsx -'use client' - -export default function CheckoutSuccess() { - const searchParams = useSearchParams() - const sessionId = searchParams.get('session_id') - - useEffect(() => { - if (sessionId) { - startTrial(sessionId) - } - }, [sessionId]) - - const startTrial = async (sessionId: string) => { - try { - const response = await fetch('/api/trial/start', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ sessionId }) - }) - - if (response.ok) { - router.push('/chat?from=trial&refresh=true') - } else { - const error = await response.json() - console.error('Trial activation failed:', error) - } - } catch (error) { - console.error('Trial start error:', error) - } - } -} -``` - -## Error Handling - -### Common Error Scenarios - -1. **Payment Method Declined** - ```typescript - // Webhook: payment_intent.payment_failed - // Action: Prevent trial access, show error message - ``` - -2. **Checkout Session Expired** - ```typescript - // API Response: 400 Bad Request - // Message: "Checkout session expired" - ``` - -3. **Trial Already Started** - ```typescript - // API Response: 400 Bad Request - // Message: "Trial already active for this user" - ``` - -4. **3D Secure Required** - ```typescript - // Webhook: payment_intent.requires_action - // Action: Redirect user to complete authentication - ``` - -### Error Recovery - -```typescript -// Automatic retry for failed payment authorization -const retryPaymentSetup = async (customerId: string) => { - const setupIntent = await stripe.setupIntents.create({ - customer: customerId, - payment_method_types: ['card'], - usage: 'off_session' - }) - - return setupIntent.client_secret -} -``` - -## Testing - -### Test Scenarios - -1. **Successful Trial Flow** - ```bash - # Use test card: 4242424242424242 - # Expected: Trial access granted immediately - ``` - -2. **Declined Payment** - ```bash - # Use test card: 4000000000000002 - # Expected: Trial access denied - ``` - -3. **3D Secure Authentication** - ```bash - # Use test card: 4000002500003155 - # Expected: Additional authentication required - ``` - -4. **Trial Cancellation** - ```bash - # Cancel within 7 days - # Expected: Immediate access removal, no charge - ``` - -5. **Trial Conversion** - ```bash - # Wait 7 days (or use Stripe CLI) - # Expected: Automatic payment, continued access - ``` - -### Stripe CLI Testing - -```bash -# Trigger trial end event -stripe trigger invoice.payment_succeeded \ - --add invoice:billing_reason=subscription_create - -# Trigger trial cancellation -stripe trigger customer.subscription.deleted -``` - -## Monitoring - -### Key Metrics - -- **Trial Conversion Rate**: % of trials that convert to paid -- **Trial Cancellation Rate**: % of trials canceled before conversion -- **Payment Authorization Success**: % of successful payment setups -- **Time to Trial Start**: Average time from signup to trial access - -### Logging - -```typescript -// Trial events logging -console.log('Trial started:', { - userId, - trialEndDate, - creditsGranted: 500, - paymentMethodVerified: true -}) - -console.log('Trial converted:', { - userId, - conversionDate: new Date().toISOString(), - totalTrialDays: 7 -}) - -console.log('Trial canceled:', { - userId, - cancellationDate: new Date().toISOString(), - daysCanceled: daysIntoTrial -}) -``` - -## Security Considerations - -### Payment Authorization Validation -- Always verify payment method exists before granting access -- Validate Stripe webhook signatures -- Check session completion status -- Prevent duplicate trial activations - -### User Access Control -- Trial access only with valid subscription -- Credit limits enforced server-side -- Session-based authentication required -- No client-side access control bypass - -### Data Protection -- Encrypt sensitive payment data -- Log security events -- Monitor for unusual patterns -- Secure API endpoints with rate limiting \ No newline at end of file diff --git a/auto-analyst-frontend/docs/user-flows/cancellation-flows.md b/auto-analyst-frontend/docs/user-flows/cancellation-flows.md new file mode 100644 index 00000000..8c6bd96b --- /dev/null +++ b/auto-analyst-frontend/docs/user-flows/cancellation-flows.md @@ -0,0 +1,308 @@ +# Cancellation Flows Documentation + +Auto-Analyst provides multiple cancellation options for users to manage their subscriptions. + +## Cancellation Types + +### 1. Immediate Cancellation +- Subscription ends immediately +- Credits removed instantly +- User downgraded to free tier +- No refund processed + +### 2. End-of-Period Cancellation +- Subscription continues until next billing cycle +- Credits remain available until period end +- Automatic downgrade on next billing date +- Default cancellation method + +### 3. Trial Cancellation +- 2-day trial can be canceled anytime (configurable in credits-config.ts) +- No charges if canceled before trial ends +- User reverts to free tier limits + +## Implementation + +### Frontend Cancellation Component +```typescript +// components/CancellationFlow.tsx +const CancellationFlow = () => { + const [cancellationType, setCancellationType] = useState<'immediate' | 'end-of-period'>('end-of-period') + const [isProcessing, setIsProcessing] = useState(false) + + const handleCancellation = async () => { + setIsProcessing(true) + + try { + const response = await fetch('/api/user/cancel-subscription', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + type: cancellationType, + reason: selectedReason + }) + }) + + if (response.ok) { + // Show success message + toast.success('Subscription canceled successfully') + // Redirect to dashboard + router.push('/dashboard') + } + } catch (error) { + toast.error('Failed to cancel subscription') + } finally { + setIsProcessing(false) + } + } + + return ( + + + + + + ) +} +``` + +### API Route Handler +```typescript +// app/api/user/cancel-subscription/route.ts +export async function POST(request: NextRequest) { + const token = await getToken({ req: request }) + if (!token?.sub) { + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + } + + const { type, reason } = await request.json() + const userId = token.sub + + try { + // Get user's Stripe subscription + const subscription = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) + const stripeSubscriptionId = subscription.stripeSubscriptionId + + if (!stripeSubscriptionId) { + return NextResponse.json({ error: 'No active subscription' }, { status: 400 }) + } + + if (type === 'immediate') { + // Cancel immediately + await stripe.subscriptions.cancel(stripeSubscriptionId) + await handleImmediateCancellation(userId) + } else { + // Cancel at period end + await stripe.subscriptions.update(stripeSubscriptionId, { + cancel_at_period_end: true + }) + await handleEndOfPeriodCancellation(userId) + } + + // Log cancellation reason + await logCancellation(userId, type, reason) + + return NextResponse.json({ success: true }) + } catch (error) { + console.error('Cancellation error:', error) + return NextResponse.json({ error: 'Cancellation failed' }, { status: 500 }) + } +} +``` + +## Cancellation Logic + +### Immediate Cancellation +```typescript +async function handleImmediateCancellation(userId: string) { + // Update subscription status + await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + status: 'canceled', + canceledAt: new Date().toISOString(), + cancelAtPeriodEnd: 'false' + }) + + // Remove credits immediately + await redis.hset(KEYS.USER_CREDITS(userId), { + total: '0', + used: '0', + resetDate: '', + lastUpdate: new Date().toISOString() + }) + + // Send cancellation email + await sendCancellationEmail(userId, 'immediate') +} +``` + +### End-of-Period Cancellation +```typescript +async function handleEndOfPeriodCancellation(userId: string) { + // Mark for cancellation at period end + await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + cancelAtPeriodEnd: 'true', + canceledAt: new Date().toISOString() + }) + + // Credits remain until period end + // Webhook will handle final cancellation + + // Send confirmation email + await sendCancellationEmail(userId, 'end-of-period') +} +``` + +### Trial Cancellation +```typescript +async function handleTrialCancellation(userId: string) { + // Get trial subscription + const subscription = await redis.hgetall(KEYS.USER_SUBSCRIPTION(userId)) + + if (subscription.status === 'trialing') { + // Cancel trial immediately + await stripe.subscriptions.cancel(subscription.stripeSubscriptionId) + + // Revert to free plan + await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { + plan: 'free', + status: 'canceled', + trialCanceled: 'true' + }) + + // Remove trial credits + await redis.hset(KEYS.USER_CREDITS(userId), { + total: '0', + used: '0', + isTrialCredits: 'false' + }) + } +} +``` + +## User Experience + +### Cancellation UI Flow +```typescript +// Step 1: Cancellation reasons +const CancellationReasons = ({ onSelect }) => { + const reasons = [ + 'Too expensive', + 'Not using it enough', + 'Found a better alternative', + 'Technical issues', + 'Other' + ] + + return ( +
+

Why are you canceling?

+ {reasons.map(reason => ( + + ))} +
+ ) +} + +// Step 2: Retention offers +const RetentionOffers = ({ onAccept, onDecline }) => ( +
+

Wait! Before you go...

+
+

50% off your next month

+

Continue with your current plan for half price

+ +
+ +
+) + +// Step 3: Final confirmation +const CancellationConfirmation = ({ plan, onConfirm }) => ( +
+

Are you sure?

+

Your {plan} subscription will be canceled.

+

You'll lose access to premium features.

+ +
+) +``` + +### Post-Cancellation Experience +```typescript +const PostCancellation = ({ cancellationType }) => ( +
+

Subscription Canceled

+ + {cancellationType === 'immediate' ? ( +

Your subscription has been canceled immediately.

+ ) : ( +

Your subscription will end at the current billing period.

+ )} + +
+

What happens now?

+
    +
  • You can still access premium features until {endDate}
    • +
  • Your data will be preserved for 30 days
    • +
  • You can reactivate anytime
    • +
+
+ + +
+) +``` + +## Analytics & Tracking + +### Cancellation Analytics +```typescript +// Track cancellation metrics +const logCancellation = async (userId: string, type: string, reason: string) => { + await redis.lpush('cancellation_analytics', JSON.stringify({ + userId, + type, + reason, + timestamp: new Date().toISOString(), + plan: subscription.plan, + daysActive: calculateDaysActive(subscription.createdAt) + })) +} + +// Admin dashboard metrics +const getCancellationMetrics = async () => { + const cancellations = await redis.lrange('cancellation_analytics', 0, -1) + + return { + totalCancellations: cancellations.length, + reasonBreakdown: calculateReasonBreakdown(cancellations), + averageDaysActive: calculateAverageLifetime(cancellations), + retentionOfferSuccess: calculateRetentionSuccess(cancellations) + } +} +``` + +### Retention Strategies +- Discount offers for price-sensitive users +- Feature education for underutilization +- Technical support for issues +- Pause subscription option +- Win-back campaigns for canceled users \ No newline at end of file diff --git a/auto-analyst-frontend/docs/webhooks.md b/auto-analyst-frontend/docs/webhooks.md deleted file mode 100644 index 272e5db2..00000000 --- a/auto-analyst-frontend/docs/webhooks.md +++ /dev/null @@ -1,433 +0,0 @@ -# Webhooks - -This document covers all Stripe webhooks implemented in Auto-Analyst, their purposes, and data synchronization logic. - -## Overview - -Auto-Analyst uses Stripe webhooks to maintain real-time synchronization between Stripe and our Redis database. All webhooks are processed through a single endpoint: `/api/webhooks/route.ts` - -## Webhook Configuration - -### Endpoint URL -``` -https://your-domain.com/api/webhooks -``` - -### Required Events -The following events must be configured in your Stripe dashboard: - -```typescript -const requiredEvents = [ - 'checkout.session.completed', - 'customer.subscription.updated', - 'customer.subscription.deleted', - 'customer.subscription.trial_will_end', - 'invoice.payment_succeeded', - 'invoice.payment_failed', - 'payment_intent.payment_failed', - 'payment_intent.canceled', - 'setup_intent.setup_failed', - 'payment_intent.requires_action' -] -``` - -## Webhook Security - -### Signature Verification -```typescript -export async function POST(request: NextRequest) { - const signature = request.headers.get('stripe-signature') - - if (!signature) { - return NextResponse.json({ error: 'No Stripe signature found' }, { status: 400 }) - } - - const rawBody = await getRawBody(request.body as unknown as Readable) - - try { - event = stripe.webhooks.constructEvent(rawBody, signature, webhookSecret) - } catch (err: any) { - console.error(`โš ๏ธ Webhook signature verification failed.`, err.message) - return NextResponse.json({ error: `Webhook Error: ${err.message}` }, { status: 400 }) - } - - // Process event... -} -``` - -### Environment Variables -```bash -STRIPE_WEBHOOK_SECRET=whsec_... # From Stripe Dashboard -``` - -## Webhook Events - -### 1. `checkout.session.completed` - -**Purpose**: Logs successful checkout completion (legacy - now handled by trial flow) - -**Processing Logic**: -```typescript -case 'checkout.session.completed': { - const session = event.data.object as Stripe.Checkout.Session - console.log(`Checkout session completed: ${session.id} - handled by trial flow`) - return NextResponse.json({ received: true }) -} -``` - -**Redis Updates**: None (logging only) - -### 2. `customer.subscription.updated` - -**Purpose**: Synchronizes subscription status changes - -**Processing Logic**: -```typescript -case 'customer.subscription.updated': { - const subscription = event.data.object as Stripe.Subscription - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - const updateData: any = { - status: subscription.status, - lastUpdated: new Date().toISOString(), - stripeSubscriptionStatus: subscription.status - } - - // Handle specific status transitions - if (currentStatus === 'trialing' && subscription.status === 'active') { - updateData.trialEndedAt = new Date().toISOString() - updateData.trialToActiveDate = new Date().toISOString() - } - - if (subscription.status === 'canceled') { - updateData.canceledAt = new Date().toISOString() - } - - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), updateData) - break -} -``` - -**Redis Updates**: -- Subscription status -- Status transition timestamps -- Stripe synchronization data - -### 3. `customer.subscription.deleted` - -**Purpose**: Handles subscription cancellation and cleanup - -**Processing Logic**: -```typescript -case 'customer.subscription.deleted': { - const subscription = event.data.object as Stripe.Subscription - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - // Set credits to 0 immediately when subscription is canceled - await creditUtils.setZeroCredits(userId) - - // Update subscription data to reflect cancellation - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - plan: 'No Active Plan', - planType: 'NONE', - status: 'canceled', - amount: '0', - interval: 'month', - lastUpdated: now.toISOString(), - canceledAt: now.toISOString(), - stripeCustomerId: '', - stripeSubscriptionId: '' - }) - - // Mark credits as subscription deleted - await redis.hset(KEYS.USER_CREDITS(userId), { - total: '0', - used: '0', - resetDate: '', - lastUpdate: now.toISOString(), - subscriptionDeleted: 'true' - }) - - break -} -``` - -**Redis Updates**: -- Subscription status โ†’ 'canceled' -- Credits โ†’ 0 -- Plan โ†’ 'No Active Plan' -- Cleanup Stripe IDs - -### 4. `customer.subscription.trial_will_end` - -**Purpose**: Logs upcoming trial expiration (for potential email notifications) - -**Processing Logic**: -```typescript -case 'customer.subscription.trial_will_end': { - const subscription = event.data.object as Stripe.Subscription - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - // Optional: Send reminder email about trial ending - // You can add email notification logic here - - return NextResponse.json({ received: true }) -} -``` - -**Redis Updates**: None (logging only) - -### 5. `invoice.payment_succeeded` - -**Purpose**: Handles successful payments and trial conversions - -**Processing Logic**: -```typescript -case 'invoice.payment_succeeded': { - const invoice = event.data.object as Stripe.Invoice - - if (invoice.subscription) { - const subscription = await stripe.subscriptions.retrieve(invoice.subscription as string) - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - if (invoice.billing_reason === 'subscription_cycle') { - // Trial ended, first payment successful - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'active', - lastUpdated: new Date().toISOString(), - trialEndedAt: new Date().toISOString(), - lastPaymentDate: new Date().toISOString(), - stripeSubscriptionStatus: subscription.status - }) - } else if (invoice.billing_reason === 'subscription_create') { - // Initial subscription creation payment - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: subscription.status, - lastUpdated: new Date().toISOString(), - initialPaymentDate: new Date().toISOString(), - stripeSubscriptionStatus: subscription.status - }) - } - } - - return NextResponse.json({ received: true }) -} -``` - -**Redis Updates**: -- Status activation -- Payment timestamps -- Trial conversion tracking - -### 6. `invoice.payment_failed` - -**Purpose**: Handles failed recurring payments - -**Processing Logic**: -```typescript -case 'invoice.payment_failed': { - const invoice = event.data.object as Stripe.Invoice - - if (invoice.subscription && invoice.billing_reason === 'subscription_cycle') { - const subscription = await stripe.subscriptions.retrieve(invoice.subscription as string) - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - // Set credits to 0 and mark subscription as past_due - await creditUtils.setZeroCredits(userId) - - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'past_due', - lastUpdated: new Date().toISOString(), - paymentFailedAt: new Date().toISOString() - }) - } - - return NextResponse.json({ received: true }) -} -``` - -**Redis Updates**: -- Status โ†’ 'past_due' -- Credits โ†’ 0 -- Payment failure timestamp - -### 7. Payment Protection Events - -These events prevent trial access when payment authorization fails: - -#### `payment_intent.payment_failed` -```typescript -case 'payment_intent.payment_failed': { - const paymentIntent = event.data.object as Stripe.PaymentIntent - - if (paymentIntent.metadata?.isTrial === 'true') { - const userId = paymentIntent.metadata?.userId - - if (userId) { - // Prevent trial access by ensuring credits remain at 0 - await creditUtils.setZeroCredits(userId) - - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'payment_failed', - lastUpdated: new Date().toISOString(), - paymentFailedAt: new Date().toISOString(), - failureReason: 'Payment authorization failed during trial signup' - }) - } - } - break -} -``` - -#### `payment_intent.canceled` -```typescript -case 'payment_intent.canceled': { - const paymentIntent = event.data.object as Stripe.PaymentIntent - - if (paymentIntent.metadata?.isTrial === 'true') { - const userId = paymentIntent.metadata?.userId - - if (userId) { - await creditUtils.setZeroCredits(userId) - - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'canceled', - lastUpdated: new Date().toISOString(), - canceledAt: new Date().toISOString(), - cancelReason: 'Payment intent canceled during trial signup' - }) - } - } - break -} -``` - -#### `setup_intent.setup_failed` -```typescript -case 'setup_intent.setup_failed': { - const setupIntent = event.data.object as Stripe.SetupIntent - - if (setupIntent.metadata?.is_trial_setup === 'true') { - const subscriptionId = setupIntent.metadata?.subscription_id - - if (subscriptionId) { - const subscription = await stripe.subscriptions.retrieve(subscriptionId) - const userId = await getUserIdFromCustomerId(subscription.customer as string) - - // Cancel the trial subscription since setup failed - await stripe.subscriptions.cancel(subscriptionId) - - // Ensure user doesn't get trial access - await creditUtils.setZeroCredits(userId) - - await redis.hset(KEYS.USER_SUBSCRIPTION(userId), { - status: 'setup_failed', - lastUpdated: new Date().toISOString(), - setupFailedAt: new Date().toISOString(), - failureReason: 'Payment method setup failed during trial signup' - }) - } - } - break -} -``` - -#### `payment_intent.requires_action` -```typescript -case 'payment_intent.requires_action': { - const paymentIntent = event.data.object as Stripe.PaymentIntent - - if (paymentIntent.metadata?.isTrial === 'true') { - const userId = paymentIntent.metadata?.userId - console.log(`Trial payment requires 3D Secure authentication for user ${userId}`) - - // Don't grant trial access until authentication is complete - } - break -} -``` - -## Helper Functions - -### User ID Resolution -```typescript -async function getUserIdFromCustomerId(customerId: string): Promise { - try { - const userId = await redis.get(`stripe:customer:${customerId}`) - return userId - } catch (error) { - console.error('Error getting userId from Redis:', error) - return null - } -} -``` - -### Error Handling -```typescript -try { - // Process webhook event - await processWebhookEvent(event) - return NextResponse.json({ received: true }) -} catch (error: any) { - console.error('Webhook error:', error) - return NextResponse.json({ error: error.message || 'Webhook handler failed' }, { status: 500 }) -} -``` - -## Webhook Testing - -### Local Development -```bash -# Install Stripe CLI -stripe listen --forward-to localhost:3000/api/webhooks - -# Trigger test events -stripe trigger customer.subscription.updated -stripe trigger invoice.payment_succeeded -``` - -### Test Event Data -```typescript -// Mock subscription update -{ - "id": "evt_test_webhook", - "object": "event", - "type": "customer.subscription.updated", - "data": { - "object": { - "id": "sub_test123", - "customer": "cus_test123", - "status": "active", - "trial_end": 1643723400 - } - } -} -``` - -## Monitoring - -### Webhook Delivery -- Check Stripe Dashboard โ†’ Webhooks โ†’ Endpoint logs -- Monitor delivery success/failure rates -- Review retry attempts - -### Error Tracking -```typescript -// Log all webhook events for debugging -console.log('Webhook received:', { - type: event.type, - id: event.id, - created: event.created, - livemode: event.livemode -}) -``` - -## Best Practices - -1. **Idempotency**: Handle duplicate events gracefully -2. **Fast Response**: Return 200 quickly, process asynchronously if needed -3. **Error Handling**: Return 5xx for retriable errors, 4xx for permanent failures -4. **Logging**: Log all events for debugging and monitoring -5. **Validation**: Verify event structure before processing -6. **Security**: Always verify webhook signatures -7. **Data Consistency**: Use atomic operations for Redis updates \ No newline at end of file diff --git a/auto-analyst-frontend/lib/redis.ts b/auto-analyst-frontend/lib/redis.ts index 3efb7ffb..c292c9b3 100644 --- a/auto-analyst-frontend/lib/redis.ts +++ b/auto-analyst-frontend/lib/redis.ts @@ -362,7 +362,7 @@ export const subscriptionUtils = { if (isGenuineCancellation) { await creditUtils.setZeroCredits(userId); - console.log(`[Credits] Set zero credits for genuinely canceled user ${userId} (status: ${subscriptionData.status})`); + // console.log(`[Credits] Set zero credits for genuinely canceled user ${userId} (status: ${subscriptionData.status})`); return true; } else { console.log(`[Credits] Skipping credit reset for user ${userId} - appears to be successful trial conversion, not cancellation`); diff --git a/auto-analyst-frontend/lib/utils/logger.ts b/auto-analyst-frontend/lib/utils/logger.ts index 282b9f0b..405abb3a 100644 --- a/auto-analyst-frontend/lib/utils/logger.ts +++ b/auto-analyst-frontend/lib/utils/logger.ts @@ -6,7 +6,7 @@ */ // Set this to true to disable all non-error logs -const DISABLE_LOGS = false; +const DISABLE_LOGS = true; const logger = { log: (...args: any[]) => { diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..92be07b1 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,281 @@ +# Auto-Analyst Documentation + +Welcome to the Auto-Analyst project documentation. This directory contains comprehensive guides for understanding, developing, and deploying the entire Auto-Analyst platform. + +## ๐Ÿ—๏ธ Project Overview + +Auto-Analyst is an AI-powered analytics platform that enables users to analyze data through natural language conversations. The platform consists of a Next.js frontend and a Python FastAPI backend, integrated with various AI models and cloud services. + +``` +Auto-Analyst Platform Architecture +โ”œโ”€โ”€ Frontend (Next.js 14) +โ”‚ โ”œโ”€โ”€ Chat Interface with AI Agents +โ”‚ โ”œโ”€โ”€ Credit Management System +โ”‚ โ”œโ”€โ”€ User Authentication & Authorization +โ”‚ โ”œโ”€โ”€ Admin Dashboard & Analytics +โ”‚ โ””โ”€โ”€ Stripe Payment Integration +โ”œโ”€โ”€ Backend (Python FastAPI) +โ”‚ โ”œโ”€โ”€ AI Agent Management +โ”‚ โ”œโ”€โ”€ Code Execution Engine +โ”‚ โ”œโ”€โ”€ Data Processing Pipeline +โ”‚ โ”œโ”€โ”€ Database Integration (PostgreSQL/SQLite) +โ”‚ โ””โ”€โ”€ Analytics & Usage Tracking +โ””โ”€โ”€ Infrastructure + โ”œโ”€โ”€ Redis (Caching & Sessions) + โ”œโ”€โ”€ Stripe (Payments & Subscriptions) + โ”œโ”€โ”€ Cloud Storage (AWS S3/Vercel Blob) + โ””โ”€โ”€ AI Models (OpenAI, Anthropic, Google, etc.) +``` + +## ๐Ÿ“ Documentation Structure + +### **๐Ÿ“– Main Documentation** (`/docs/`) +- **[Backend Documentation](./backend.md)** - Python FastAPI backend API overview and route categories +- **[Frontend Documentation](./frontend.md)** - Next.js frontend overview and component structure + +### **๐ŸŽจ Frontend Specific** (`/auto-analyst-frontend/docs/`) +- **[Frontend Documentation Hub](../auto-analyst-frontend/docs/README.md)** - Comprehensive frontend guides +- **[Development Environment Setup](../auto-analyst-frontend/docs/development/environment-setup.md)** - Complete environment configuration +- **[Default Agents System](../auto-analyst-frontend/docs/features/default-agents.md)** - AI agent setup and configuration +- **[Credit Configuration Guide](../auto-analyst-frontend/docs/billing/credit-configuration.md)** - Centralized credit and trial management +- **[Trial System Architecture](../auto-analyst-frontend/docs/billing/trial-system.md)** - 2-day trial implementation +- **[Model Registry](../auto-analyst-frontend/docs/system/model-registry.md)** - AI model management +- **[Middleware Guide](../auto-analyst-frontend/docs/system/middleware.md)** - Route protection and request handling + +### **๐Ÿ”ง Backend Specific** (`/auto-analyst-backend/docs/`) +- **[Getting Started Guide](../auto-analyst-backend/docs/getting_started.md)** - Backend setup and deployment +- **[API Endpoints Reference](../auto-analyst-backend/docs/api/endpoints.md)** - Complete API documentation +- **[Architecture Overview](../auto-analyst-backend/docs/architecture/architecture.md)** - Backend system design +- **[Development Workflow](../auto-analyst-backend/docs/development/development_workflow.md)** - Backend development guide +- **[Database Schema](../auto-analyst-backend/docs/system/database-schema.md)** - Data models and relationships +- **[Shared DataFrame System](../auto-analyst-backend/docs/system/shared_dataframe.md)** - Session data management + +## ๐Ÿš€ Quick Start Guide + +### **For New Developers** + +1. **Clone the Repository** + ```bash + git clone https://github.com/your-org/Auto-Analyst.git + cd Auto-Analyst-CS + ``` + +2. **Backend Setup** + ```bash + cd auto-analyst-backend + python -m venv venv + source venv/bin/activate # On Windows: venv\Scripts\activate + pip install -r requirements.txt + ``` + +3. **Frontend Setup** + ```bash + cd auto-analyst-frontend + npm install + ``` + +4. **Environment Configuration** + - Copy `.env.example` to `.env.local` in both directories + - Configure API keys, database URLs, and service credentials + - See [Frontend Environment Setup](../auto-analyst-frontend/docs/development/environment-setup.md) + +5. **Start Development Servers** + ```bash + # Terminal 1 - Backend + cd auto-analyst-backend && uvicorn app:app --reload + + # Terminal 2 - Frontend + cd auto-analyst-frontend && npm run dev + ``` + +### **For Product Managers** + +1. **Understanding the Platform** + - Review [Frontend Documentation](./frontend.md) for user-facing features + - Check [Backend Documentation](./backend.md) for API capabilities + - Understand the [Database Schema](../auto-analyst-backend/docs/system/database-schema.md) for data structure + +2. **Key Business Features** + - **Credit System**: Usage-based billing with model tiers (1-20 credits per query) + - **Trial System**: 2-day free trial with 500 credits (configurable) + - **Subscription Management**: Stripe-powered recurring billing + - **Multi-Agent AI**: Specialized agents for different data tasks + +3. **Configuration Management** + - **Credit Configuration**: Centralized in [credits-config.ts](../auto-analyst-frontend/docs/billing/credit-configuration.md) + - **Trial Settings**: Adjustable duration and credit allocation + - **Model Pricing**: Different AI models with varying credit costs + +### **For DevOps Engineers** + +1. **Deployment Options** + - **Frontend**: Vercel, AWS Amplify, Docker + - **Backend**: AWS ECS, Docker, HuggingFace Spaces + - **Database**: PostgreSQL (production), SQLite (development) + - **Caching**: Redis/Upstash + +2. **Infrastructure Setup** + - Configure monitoring and logging + - Set up CI/CD pipelines + - Environment variable management + - Database migrations and backups + +3. **Security Considerations** + - API key rotation and management + - Database security and access control + - Rate limiting and DDoS protection + - SSL/TLS certificate management + +## ๐ŸŽฏ Key Features + +### **๐Ÿค– AI-Powered Analytics** +- **Multi-Agent System**: Specialized AI agents for different data analysis tasks +- **Natural Language Interface**: Chat-based interaction with data +- **Code Generation**: Automatic Python code generation for analysis +- **Real-time Execution**: Live code execution with results visualization +- **Deep Analysis**: Comprehensive multi-agent analysis with detailed reporting + +### **๐Ÿ’ณ Business Model** +- **Credit-Based Pricing**: Pay-per-query model with different AI model tiers +- **Subscription Plans**: Monthly/yearly plans with credit allocations +- **Free Trial**: 2-day trial with 500 credits (configurable) +- **Enterprise Features**: Custom solutions and dedicated support + +### **๐Ÿ” Authentication & Security** +- **Google OAuth**: Primary authentication method via NextAuth.js +- **Session Management**: Redis-based session storage +- **Admin Dashboard**: Analytics and user management interface +- **API Security**: Rate limiting and authentication tokens + +### **๐Ÿ“Š Analytics & Monitoring** +- **Usage Tracking**: Detailed analytics on user behavior and feature usage +- **Cost Analysis**: Real-time cost tracking per model and user +- **Performance Metrics**: Response times and success rates +- **Business Intelligence**: Revenue and subscription analytics + +## ๐Ÿ”ง Development Workflow + +### **Frontend Development** +```bash +cd auto-analyst-frontend +npm run dev # Start development server +npm run build # Build for production +npm run lint # Run ESLint +npm run type-check # TypeScript checking +``` + +### **Backend Development** +```bash +cd auto-analyst-backend +uvicorn app:app --reload # Start with hot reload +python -m pytest # Run tests +python scripts/init_db.py # Initialize database +``` + +### **Full Stack Development** +```bash +# Use Docker Compose for full environment +docker-compose up -d + +# Or run both servers simultaneously +npm run dev:all # If configured in root package.json +``` + +## ๐Ÿ—๏ธ Architecture Decisions + +### **Frontend Architecture** +- **Next.js 14**: App Router for modern React patterns +- **TypeScript**: Full type safety across the application +- **Tailwind CSS**: Utility-first styling with custom components +- **Zustand**: Lightweight state management for complex UI state +- **NextAuth.js**: Authentication framework with Google OAuth + +### **Backend Architecture** +- **FastAPI**: Modern Python web framework with automatic OpenAPI +- **SQLAlchemy**: Database ORM with migration support +- **DSPy**: AI agent framework for model interactions and orchestration +- **Pydantic**: Data validation and serialization +- **Async/Await**: Non-blocking I/O for better performance + +### **Data Architecture** +- **PostgreSQL**: Primary database for production +- **Redis**: Caching, session storage, and real-time data +- **S3/Blob Storage**: File uploads and static assets +- **Real-time Updates**: WebSocket connections for live data + +## ๐Ÿ”— Integration Points + +### **AI Model Providers** +- **OpenAI**: GPT-4, GPT-4o-mini models +- **Anthropic**: Claude Sonnet, Claude Haiku +- **Google**: Gemini models +- **Groq**: High-speed inference + +### **Payment Processing** +- **Stripe**: Subscription management and payments +- **Webhooks**: Real-time payment event processing +- **Credit System**: Usage tracking and billing automation + +### **Third-Party Services** +- **Vercel**: Frontend hosting and edge functions +- **Upstash**: Managed Redis service +- **HuggingFace**: AI model hosting and deployment +- **AWS**: Cloud infrastructure and storage + +## ๐Ÿ“ˆ Monitoring & Analytics + +### **Application Monitoring** +- **Error Tracking**: Comprehensive error logging and reporting +- **Performance Metrics**: Response time monitoring and optimization +- **Usage Analytics**: User behavior tracking and feature usage +- **Cost Tracking**: Real-time cost analysis per feature and user + +### **Business Metrics** +- **User Acquisition**: Registration and trial conversion tracking +- **Revenue Tracking**: Subscription and usage revenue analytics +- **Feature Usage**: Most popular AI agents and features +- **Support Metrics**: User satisfaction and support ticket analytics + +## ๐Ÿค Contributing + +### **Code Standards** +- Follow TypeScript/Python best practices +- Use consistent naming conventions +- Write comprehensive tests for new features +- Document new features and APIs thoroughly + +### **Documentation Standards** +- Keep documentation up-to-date with code changes +- Use clear, concise language with practical examples +- Organize documentation logically with proper linking +- Include configuration examples and troubleshooting guides + +### **Development Process** +1. Fork the repository and create a feature branch +2. Make changes with appropriate tests +3. Update relevant documentation +4. Submit a pull request with clear description +5. Address code review feedback promptly + +## ๐Ÿ“ž Support & Resources + +### **Internal Resources** +- **Development Team**: Technical implementation questions and code reviews +- **Product Team**: Feature requirements and business logic discussions +- **DevOps Team**: Infrastructure and deployment issues + +### **External Documentation** +- [Next.js Documentation](https://nextjs.org/docs) - Frontend framework +- [FastAPI Documentation](https://fastapi.tiangolo.com/) - Backend framework +- [Stripe API Reference](https://stripe.com/docs/api) - Payment processing +- [Redis Documentation](https://redis.io/docs) - Caching and sessions + +### **Community** +- GitHub Issues for bug reports and feature requests +- Internal Slack channels for development discussions +- Code review process for quality assurance +- Documentation updates and improvements + +--- + +This documentation serves as the central hub for understanding and working with the Auto-Analyst platform. For specific implementation details, refer to the frontend and backend documentation in their respective directories. \ No newline at end of file diff --git a/docs/backend.md b/docs/backend.md index 51ff9ffb..0510539c 100644 --- a/docs/backend.md +++ b/docs/backend.md @@ -1,70 +1,121 @@ -## **Auto-Analyst API Overview** +# **Auto-Analyst Backend API Overview** -The **Auto-Analyst** application provides a structured API for data analysis, AI-powered insights, and real-time analytics. The API is categorized into three main sections, each documented separately for better modularity: +The **Auto-Analyst** backend provides a comprehensive API for data analysis, AI-powered insights, and real-time analytics. The API is organized into specialized route categories, each documented separately for better modularity: -1. **[Core Application Routes](/auto-analyst-backend/docs/routes/core.md)** โ€“ Handles data management, session control, and model configurations. -2. **[Chat & AI Analysis Routes](/auto-analyst-backend/docs/routes/chats.md)** โ€“ Provides AI-driven data insights and supports interaction with multiple specialized AI agents. -3. **[Analytics & WebSocket Routes](/auto-analyst-backend/docs/routes/analytics.md)** โ€“ Manages real-time updates, tracking, and logging of AI model usage. +1. **[Core Application Routes](auto-analyst-backend/docs/api/routes/core.md)** โ€“ Data management, session control, model configurations, and basic AI analysis. +2. **[Chat Management Routes](auto-analyst-backend/docs/api/routes/chats.md)** โ€“ Chat sessions, message handling, and user management. +3. **[Code Execution Routes](auto-analyst-backend/docs/api/routes/code.md)** โ€“ Python code execution, editing, fixing, and cleaning operations. +4. **[Deep Analysis Routes](auto-analyst-backend/docs/api/routes/deep_analysis.md)** โ€“ Advanced multi-agent analysis and comprehensive reporting. +5. **[Analytics & Monitoring Routes](auto-analyst-backend/docs/api/routes/analytics.md)** โ€“ Real-time dashboards, usage tracking, and system monitoring. --- -### **1. Core Application Routes ([auto-analyst-backend/docs/routes/core.md](/auto-analyst-backend/docs/routes/core.md))** +## **1. Core Application Routes** -These routes handle **data management, session handling, and model settings**. +**Purpose**: Foundation for data management and basic AI analysis +**Documentation**: [auto-analyst-backend/docs/api/routes/core.md](auto-analyst-backend/docs/api/routes/core.md) -- **Data Management** - - `POST /upload_dataframe`: Uploads a CSV dataset for analysis. - - `GET /api/default-dataset`: Retrieves the default dataset for the session. - - `POST /reset-session`: Resets the session to use the default dataset. +**Key Features**: +- **Data Management**: Upload CSV/Excel files, preview datasets, reset sessions +- **AI Analysis**: Query processing with specialized agents (data_viz, sk_learn, statistical_analytics, preprocessing) +- **Model Settings**: Configure AI providers, models, temperature, and token limits +- **Session Management**: Track user interactions and maintain analysis context -- **Model Settings** - - `GET /api/model-settings`: Retrieves the current AI model settings. - - `POST /settings/model`: Updates model configurations, including provider, temperature, and token limits. +**Available AI Agents**: +- `data_viz_agent`: Creates visualizations using Plotly +- `sk_learn_agent`: Performs machine learning analysis with Scikit-learn +- `statistical_analytics_agent`: Conducts statistical analysis using StatsModels +- `preprocessing_agent`: Handles data preprocessing and transformation -- **Session Management** - - Sessions track user interactions, datasets, and configurations. - - Managed using `session_id` (via query parameters or headers). - - Admin authentication requires an API key (`X-Admin-API-Key`). +--- + +## **2. Chat Management Routes** + +**Purpose**: Handle chat sessions, messages, and user interactions +**Documentation**: [auto-analyst-backend/docs/api/routes/chats.md](auto-analyst-backend/docs/api/routes/chats.md) + +**Key Features**: +- **Chat Sessions**: Create, retrieve, update, and delete chat conversations +- **Message Management**: Add messages to chats with timestamp tracking +- **User Management**: Create and manage user accounts with email-based identification +- **Cleanup Operations**: Remove empty chats while preserving model usage records + +--- + +## **3. Code Execution Routes** + +**Purpose**: Execute, edit, and manage Python code for data analysis +**Documentation**: [auto-analyst-backend/docs/api/routes/code.md](auto-analyst-backend/docs/api/routes/code.md) + +**Key Features**: +- **Code Execution**: Run Python code against session datasets with safety measures +- **AI-Powered Editing**: Modify code based on user instructions using AI +- **Error Fixing**: Automatically fix code errors using DSPy refinement +- **Code Cleaning**: Format and organize imports with proper structure +- **Safety Features**: Remove blocking calls, handle isolated namespaces + +--- + +## **4. Deep Analysis Routes** + +**Purpose**: Advanced multi-agent analysis with comprehensive reporting +**Documentation**: [auto-analyst-backend/docs/api/routes/deep_analysis.md](auto-analyst-backend/docs/api/routes/deep_analysis.md) + +**Key Features**: +- **Multi-Agent Orchestration**: Coordinates multiple AI agents for comprehensive analysis +- **Template Integration**: Uses user's active templates and agent preferences +- **Streaming Progress**: Real-time updates during analysis execution +- **Report Generation**: Creates detailed HTML reports with visualizations +- **Credit Tracking**: Monitors token usage, costs, and credits consumed + +**Analysis Flow**: +1. Question Generation (20%) โ†’ Planning (40%) โ†’ Agent Execution (60%) +2. Code Synthesis (80%) โ†’ Code Execution (85%) โ†’ Synthesis (90%) โ†’ Conclusion (100%) --- -### **2. Chat & AI Analysis Routes ([auto-analyst-backend/docs/routes/chats.md](/auto-analyst-backend/docs/routes/chats.md))** +## **5. Analytics & Monitoring Routes** -These routes provide **AI-powered insights and query handling** using specialized agents. +**Purpose**: Real-time monitoring, usage tracking, and system analytics +**Documentation**: [auto-analyst-backend/docs/api/routes/analytics.md](auto-analyst-backend/docs/api/routes/analytics.md) -- **AI Analysis** - - `POST /chat/{agent_name}`: Processes a query using a specified AI agent. - - `POST /chat`: Executes a query across multiple AI agents and streams responses. - - `POST /execute_code`: Executes Python code for advanced data analysis and visualization. +**Key Features**: +- **Dashboard Analytics**: Comprehensive usage statistics and performance metrics +- **User Analytics**: Activity tracking, session management, and user behavior +- **Model Analytics**: Performance metrics, success rates, and cost analysis +- **Real-Time Updates**: WebSocket endpoints for live dashboard updates +- **Admin Authentication**: Secure access to analytics data with API key validation -- **Available AI Agents** - - `data_viz_agent`: Creates visualizations using Plotly. - - `sk_learn_agent`: Performs machine learning analysis with Scikit-learn. - - `statistical_analytics_agent`: Conducts statistical analysis using StatsModels. - - `preprocessing_agent`: Handles data preprocessing and transformation. +**WebSocket Endpoints**: +- `/analytics/dashboard/realtime` - Live dashboard updates +- `/analytics/realtime` - Real-time user analytics -- **Agent Integration Flow** - - Queries are dispatched based on intent. - - Responses are formatted in Markdown and streamed. - - Usage metrics are tracked for model optimization. +--- + +## **Authentication & Security** + +- **Session Management**: Sessions track user interactions via `session_id` +- **Admin Access**: Analytics routes require API key authentication (`X-Admin-API-Key`) +- **User Context**: Optional `user_id` parameters for access control and tracking +- **Error Handling**: Standardized HTTP responses (400, 401, 403, 404, 500) --- -### **3. Analytics & WebSocket Routes ([auto-analyst-backend/docs/routes/analytics.md](/auto-analyst-backend/docs/routes/analytics.md))** +## **Common Request Patterns** -These routes handle **real-time updates, logging, and error tracking**. +- **Query Parameters**: `session_id`, `user_id`, `chat_id` for context management +- **Headers**: `X-Admin-API-Key` for admin routes, `X-Force-Refresh` for data uploads +- **Streaming Responses**: Real-time updates for chat and deep analysis operations +- **File Uploads**: Support for CSV and Excel files with metadata + +--- -- **Real-Time Updates** - - WebSocket endpoints: - - `/analytics/dashboard/realtime` (for dashboard updates). - - `/analytics/realtime` (for live user updates). - - Active connections are managed and updated when new data is available. +## **Integration Flow** -- **Event Handling & Broadcasting** - - `broadcast_dashboard_update()`: Sends model usage stats to connected clients. - - `broadcast_user_update()`: Updates users on live data analysis. +1. **Data Upload** โ†’ Upload CSV/Excel via Core Routes +2. **Chat Creation** โ†’ Initialize conversation via Chat Routes +3. **AI Analysis** โ†’ Query agents via Core Routes or trigger Deep Analysis +4. **Code Execution** โ†’ Run/edit/fix code via Code Routes +5. **Monitoring** โ†’ Track usage and performance via Analytics Routes -- **Error Handling & Logging** - - **Try-Except blocks** ensure robust error management. - - HTTP error responses (400, 401, 403, 404, 500) are standardized. - - Logging tracks system events and AI interactions. +For detailed endpoint specifications, request/response examples, and implementation details, refer to the individual route documentation files. diff --git a/docs/db_schema.md b/docs/db_schema.md deleted file mode 100644 index 9acc8be4..00000000 --- a/docs/db_schema.md +++ /dev/null @@ -1,93 +0,0 @@ -# Schema Design Explanation - -The database schema is designed to support the application's functionality by efficiently managing user data, chat sessions, messages, and model usage statistics. Each model is structured to ensure data integrity and facilitate quick access to relevant information, enabling seamless interactions within the Auto-Analyst application. - -## Database Configuration - -The application supports both PostgreSQL and SQLite databases, configured through the `DATABASE_URL` environment variable. PostgreSQL-specific configurations include connection pooling and connection recycling, while SQLite configurations include foreign key constraint enforcement. - -## Database Schema - -The application uses the following database models: - -### User -The User table stores user information and maintains relationships with chats and usage records. - -```python -class User(Base): - __tablename__ = 'users' - - user_id = Column(Integer, primary_key=True, autoincrement=True) - username = Column(String, unique=True, nullable=False) - email = Column(String, unique=True, nullable=False) - created_at = Column(DateTime, default=datetime.utcnow) - chats = relationship("Chat", back_populates="user", cascade="all, delete-orphan") - usage_records = relationship("ModelUsage", back_populates="user") -``` - -### Chat -The Chat table stores chat sessions and maintains relationships with users, messages, and usage records. Messages are automatically deleted when a chat is deleted (cascade delete). - -```python -class Chat(Base): - __tablename__ = 'chats' - - chat_id = Column(Integer, primary_key=True, autoincrement=True) - user_id = Column(Integer, ForeignKey('users.user_id', ondelete="CASCADE"), nullable=True) - title = Column(String, default='New Chat') - created_at = Column(DateTime, default=datetime.utcnow) - user = relationship("User", back_populates="chats") - messages = relationship("Message", back_populates="chat", cascade="all, delete-orphan") - usage_records = relationship("ModelUsage", back_populates="chat") -``` - -### Message -The Message table stores individual messages within chats. Messages are automatically deleted when their associated chat is deleted. - -```python -class Message(Base): - __tablename__ = 'messages' - - message_id = Column(Integer, primary_key=True, autoincrement=True) - chat_id = Column(Integer, ForeignKey('chats.chat_id', ondelete="CASCADE"), nullable=False) - sender = Column(String, nullable=False) # 'user' or 'ai' - content = Column(Text, nullable=False) - timestamp = Column(DateTime, default=datetime.utcnow) - chat = relationship("Chat", back_populates="messages") -``` - -### ModelUsage -The ModelUsage table tracks AI model usage metrics for analytics and billing purposes. When a user or chat is deleted, the usage records are preserved but their references are set to NULL. - -```python -class ModelUsage(Base): - __tablename__ = 'model_usage' - - usage_id = Column(Integer, primary_key=True) - user_id = Column(Integer, ForeignKey('users.user_id', ondelete="SET NULL"), nullable=True) - chat_id = Column(Integer, ForeignKey('chats.chat_id', ondelete="SET NULL"), nullable=True) - model_name = Column(String(100), nullable=False) - provider = Column(String(50), nullable=False) - prompt_tokens = Column(Integer, default=0) - completion_tokens = Column(Integer, default=0) - total_tokens = Column(Integer, default=0) - query_size = Column(Integer, default=0) # Size in characters - response_size = Column(Integer, default=0) # Size in characters - cost = Column(Float, default=0.0) # Cost in USD - timestamp = Column(DateTime, default=datetime.utcnow) - is_streaming = Column(Boolean, default=False) - request_time_ms = Column(Integer, default=0) # Request processing time in milliseconds - user = relationship("User", back_populates="usage_records") - chat = relationship("Chat", back_populates="usage_records") -``` - -## Database Management - -The database is managed using SQLAlchemy, which provides an Object-Relational Mapping (ORM) layer. The database initialization and session management are handled in `init_db.py`, which provides the following key functions: - -- `init_db()`: Creates all database tables -- `get_session()`: Returns a new database session -- `get_db()`: Provides a database session with proper error handling and cleanup -- `is_postgres_db()`: Checks if the application is using PostgreSQL - -The models defined in this schema are utilized for storing and retrieving data in the database, with proper relationship management and cascade operations to maintain data integrity. diff --git a/docs/frontend.md b/docs/frontend.md index 5fa15272..cf29ae6c 100644 --- a/docs/frontend.md +++ b/docs/frontend.md @@ -1,232 +1,246 @@ -# Auto-Analyst Frontend Architecture Documentation +# Auto-Analyst Frontend Overview -## 1. Overall Architecture -The Auto-Analyst frontend is built using **Next.js** with a combination of **App Router** and **Pages Router** patterns. The application follows a component-based architecture with a clear separation of concerns: +## 1. Frontend Architecture +The Auto-Analyst frontend is built using **Next.js 14** with the **App Router** pattern. The application follows a component-based architecture with clear separation of concerns between UI, business logic, and data management. -- **App Directory**: Utilizes Next.js 13+ App Router for newer features. -- **Pages Directory**: Uses traditional Next.js Pages Router for some analytics features. -- **Components**: Reusable UI elements organized by feature and functionality. -- **State Management**: Combination of local state and custom stores. -- **UI Framework**: Tailwind CSS with custom components. +### Tech Stack +- **Framework**: Next.js 14 with App Router +- **Language**: TypeScript for full type safety +- **Styling**: Tailwind CSS with shadcn/ui components +- **Authentication**: NextAuth.js with Google OAuth +- **State Management**: Zustand stores + React Context +- **Data Fetching**: React Query + custom API clients +- **Payment Integration**: Stripe for subscriptions and billing --- -## 2. Directory Structure +## 2. Key Frontend Features + +### ๐Ÿค– AI Chat Interface +- **Real-time messaging** with AI agents via WebSocket +- **Code execution** with syntax highlighting and live results +- **File upload** support for CSV/Excel data analysis +- **Message persistence** with Redis-backed chat history +- **Multi-agent** conversations with specialized AI assistants + +### ๐Ÿ” Authentication & Security +- **Google OAuth** primary authentication +- **Session management** with NextAuth.js and Redis +- **Admin dashboard** access with temporary login +- **Route protection** via Next.js middleware +- **Guest mode** with limited trial access + +### ๐Ÿ’ณ Credit & Billing System +- **Real-time credit tracking** with usage monitoring +- **Tier-based pricing** (1-20 credits per AI model) +- **2-day trial** with 500 credits for new users +- **Stripe integration** for subscription management +- **Usage analytics** and cost breakdown + +### ๐Ÿ“Š Admin Dashboard +- **User analytics** and platform statistics +- **Cost analysis** per model and user +- **Real-time metrics** and performance monitoring +- **User management** and activity tracking + +--- + +## 3. Component Architecture + +### Core Components ``` -auto-analyst-frontend/ -โ”œโ”€โ”€ app/ # Next.js App Router pages & layouts -โ”‚ โ”œโ”€โ”€ admin/ # Admin-related pages -โ”‚ โ”œโ”€โ”€ analytics/ # Analytics pages (App Router version) -โ”‚ โ”œโ”€โ”€ api/ # API routes for server-side operations -โ”‚ โ”œโ”€โ”€ chat/ # Chat interface pages -โ”‚ โ”œโ”€โ”€ contact/ # Contact form pages -โ”‚ โ”œโ”€โ”€ enterprise/ # Enterprise features pages -โ”‚ โ”œโ”€โ”€ login/ # Authentication pages -โ”‚ โ””โ”€โ”€ layout.tsx # Root layout for App Router -โ”œโ”€โ”€ components/ # Reusable components -โ”‚ โ”œโ”€โ”€ admin/ # Admin dashboard components -โ”‚ โ”œโ”€โ”€ analytics/ # Analytics visualization components -โ”‚ โ”œโ”€โ”€ chat/ # Chat-related components -โ”‚ โ”œโ”€โ”€ ui/ # General UI components (shadcn/ui based) -โ”‚ โ””โ”€โ”€ [feature].tsx # Feature-specific components -โ”œโ”€โ”€ config/ # Configuration files -โ”œโ”€โ”€ lib/ # Utility functions and libraries -โ”‚ โ”œโ”€โ”€ api/ # API client functions -โ”‚ โ”œโ”€โ”€ store/ # State management stores -โ”‚ โ””โ”€โ”€ utils/ # Utility functions -โ”œโ”€โ”€ pages/ # Traditional Next.js Pages Router -โ”‚ โ””โ”€โ”€ analytics/ # Analytics dashboard pages -โ”œโ”€โ”€ public/ # Static assets -โ”œโ”€โ”€ styles/ # Global styles -โ””โ”€โ”€ types/ # TypeScript type definitions +components/ +โ”œโ”€โ”€ ui/ # shadcn/ui base components +โ”œโ”€โ”€ chat/ # Chat interface components +โ”‚ โ”œโ”€โ”€ ChatInterface.tsx # Main chat container +โ”‚ โ”œโ”€โ”€ ChatInput.tsx # User input handling +โ”‚ โ”œโ”€โ”€ ChatWindow.tsx # Message display +โ”‚ โ””โ”€โ”€ MessageContent.tsx # Message rendering +โ”œโ”€โ”€ admin/ # Admin dashboard components +โ”œโ”€โ”€ analytics/ # Analytics visualization +โ””โ”€โ”€ landing/ # Landing page sections +``` + +### Page Structure (App Router) +``` +app/ +โ”œโ”€โ”€ page.tsx # Landing page +โ”œโ”€โ”€ chat/page.tsx # Main chat interface +โ”œโ”€โ”€ admin/page.tsx # Admin dashboard +โ”œโ”€โ”€ analytics/ # Analytics pages +โ”œโ”€โ”€ login/page.tsx # Authentication +โ”œโ”€โ”€ api/ # API routes (Next.js) +โ””โ”€โ”€ layout.tsx # Root layout ``` --- -## 3. Key Component Breakdown - -### 3.1 Core Components - -#### Chat Interface Components -| Component | Purpose | Location | -|-------------------|---------------------------------|------------| -| `ChatInterface.tsx` | Main container for chat functionality | [View File](/auto-analyst-frontend/components/ChatInterface.tsx) | -| `ChatInput.tsx` | User input area for chat | [View File](/auto-analyst-frontend/components/ChatInput.tsx) | -| `ChatWindow.tsx` | Display area for chat messages | [View File](/auto-analyst-frontend/components/ChatWindow.tsx) | -| `MessageContent.tsx` | Renders individual messages | [View File](/auto-analyst-frontend/components/chat/MessageContent.tsx) | -| `CodeBlocker.tsx` | Syntax highlighting for code blocks | [View File](/auto-analyst-frontend/components/chat/CodeBlocker.tsx) | -| `LoadingIndicator.tsx` | Loading state visualization | [View File](/auto-analyst-frontend/components/chat/LoadingIndicator.tsx) | -| `AgentHint.tsx` | Agent hint component | [View File](/auto-analyst-frontend/components/chat/AgentHint.tsx) | -| `FreeTrialOverlay.tsx` | Free trial overlay component | [View File](/auto-analyst-frontend/components/chat/FreeTrialOverlay.tsx) | -| `WelcomeSection.tsx` | Welcome section for landing page | [View File](/auto-analyst-frontend/components/WelcomeSection.tsx) | -| `Sidebar.tsx` | Sidebar component | [View File](/auto-analyst-frontend/components/Sidebar.tsx) | -| `PlotlyChart.tsx` | Plotly chart component | [View File](/auto-analyst-frontend/components/PlotlyChart.tsx) | - -#### Analytics Dashboard Components -| Component | Purpose | Location | -|-------------------|--------------------------------|------------| -| `AnalyticsLayout.tsx` | Layout wrapper for analytics pages | [View File](/auto-analyst-frontend/components/analytics/AnalyticsLayout.tsx) | -| `Charts.tsx` | Visualization components | [View File](/auto-analyst-frontend/components/admin/Charts.tsx) | -| `UsageTable.tsx` | Tabular data presentation | [View File](/auto-analyst-frontend/components/admin/UsageTable.tsx) | -| `DateRangePicker.tsx` | Date range selection for filtering | [View File](/auto-analyst-frontend/components/admin/DateRangePicker.tsx) | - -#### UI Components -| Component | Purpose | Location | -|-----------|---------|----------| -| `alert.tsx` | Alert component | [View File](/auto-analyst-frontend/components/ui/alert.tsx) | -| `button.tsx` | Reusable button component | [View File](/auto-analyst-frontend/components/ui/button.tsx) | -| `calendar.tsx` | Calendar component | [View File](/auto-analyst-frontend/components/ui/calendar.tsx) | -| `card.tsx` | Card container component | [View File](/auto-analyst-frontend/components/ui/card.tsx) | -| `CopyButton.tsx` | Copy button component | [View File](/auto-analyst-frontend/components/ui/CopyButton.tsx) | -| `dialog.tsx` | Modal dialog component | [View File](/auto-analyst-frontend/components/ui/dialog.tsx) | -| `input.tsx` | Input component | [View File](/auto-analyst-frontend/components/ui/input.tsx) | -| `popover.tsx` | Popover component | [View File](/auto-analyst-frontend/components/ui/popover.tsx) | -| `scroll-area.tsx` | Scrollable area component | [View File](/auto-analyst-frontend/components/ui/scroll-area.tsx) | -| `table.tsx` | Data table component | [View File](/auto-analyst-frontend/components/ui/table.tsx) | -| `tabs.tsx` | Tab navigation component | [View File](/auto-analyst-frontend/components/ui/tabs.tsx) | -| `textarea.tsx` | Textarea component | [View File](/auto-analyst-frontend/components/ui/textarea.tsx) | -| `tooltip.tsx` | Tooltip component | [View File](/auto-analyst-frontend/components/ui/tooltip.tsx) | - -#### Landing Page Components -| Component | Purpose | Location | -|-----------|---------|----------| -| `LandingPage.tsx` | Main landing page layout | [View File](/auto-analyst-frontend/components/LandingPage.tsx) | -| `HeroSection.tsx` | Hero section for landing page | [View File](/auto-analyst-frontend/components/HeroSection.tsx) | -| `FeatureSection.tsx` | Features showcase section | [View File](/auto-analyst-frontend/components/FeatureSection.tsx) | -| `TestimonialsSection.tsx` | Customer testimonials | [View File](/auto-analyst-frontend/components/TestimonialsSection.tsx) | +## 4. State Management + +### Global State +- **Credit Context**: Real-time credit tracking and billing state +- **Deep Analysis Context**: Long-running analysis progress +- **Session Store**: User authentication and preferences +- **Chat History Store**: Message persistence and retrieval + +### Local State +- Component-level state with React hooks +- Form state management with controlled components +- UI state (modals, dropdowns, loading states) --- -### 3.2 Pages Structure +## 5. API Integration -#### App Router Pages (New Architecture) -``` -app/ -โ”œโ”€โ”€ page.tsx # Landing page -โ”œโ”€โ”€ chat/ # Chat application -โ”‚ โ””โ”€โ”€ page.tsx # Main chat interface -โ”œโ”€โ”€ analytics/ # Analytics overview -โ”‚ โ””โ”€โ”€ page.tsx # Analytics summary page -โ”œโ”€โ”€ admin/ # Admin section -โ”‚ โ””โ”€โ”€ page.tsx # Admin dashboard -โ”œโ”€โ”€ enterprise/ # Enterprise features -โ”‚ โ””โ”€โ”€ page.tsx # Enterprise page -โ””โ”€โ”€ layout.tsx # Root layout with navigation -``` +### Communication Patterns +1. **Direct Backend Calls**: Real-time chat and data operations +2. **Next.js API Routes**: Authentication, payments, and Redis operations +3. **WebSocket**: Live chat streaming and real-time updates -#### Pages Router (Legacy) +### API Client Structure ``` -pages/ -โ”œโ”€โ”€ _app.tsx # App wrapper for Pages Router -โ””โ”€โ”€ analytics/ # Detailed analytics pages - โ”œโ”€โ”€ costs.tsx # Cost analysis page - โ”œโ”€โ”€ dashboard.tsx # Main analytics dashboard - โ”œโ”€โ”€ models.tsx # Model performance analysis - โ””โ”€โ”€ users.tsx # User activity analysis +lib/api/ +โ”œโ”€โ”€ auth.ts # Authentication API calls +โ”œโ”€โ”€ analytics.ts # Admin analytics data +โ””โ”€โ”€ chat.ts # Chat and AI interactions ``` --- -## 4. State Management +## 6. Environment Configuration + +### Required Variables +```bash +# Core Configuration +NEXT_PUBLIC_API_URL=http://localhost:8000 +NEXTAUTH_URL=http://localhost:3000 +NEXTAUTH_SECRET=your-secret-key -### 4.1 Local Component State -- Uses React's `useState` and `useEffect` for local UI state. +# Authentication +GOOGLE_CLIENT_ID=your-google-client-id +GOOGLE_CLIENT_SECRET=your-google-client-secret -### 4.2 Custom Stores -Custom state management is implemented in `/lib/store/`: -| Store | Purpose | -|-------|---------| -| `chatHistoryStore.ts` | Manages chat history | -| `cookieConsentStore.ts` | Manages cookie consent | -| `freeTrialStore.ts` | Tracks trial usage | -| `sessionStore.ts` | Manages user authentication state | +# Redis & Caching +UPSTASH_REDIS_REST_URL=your-redis-url +UPSTASH_REDIS_REST_TOKEN=your-redis-token + +# Stripe Payments +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_... +STRIPE_SECRET_KEY=sk_test_... +STRIPE_WEBHOOK_SECRET=whsec_... +``` --- -## 5. Key Features Implementation +## 7. Development Workflow -### 5.1 Chat Interface -- **Handles:** message history, API communication, streaming response, and error handling. -- **Features:** syntax highlighting, loading indicators, Markdown rendering. +### Available Scripts +```bash +npm run dev # Start development server +npm run build # Build for production +npm run start # Start production server +npm run lint # ESLint code checking +npm run type-check # TypeScript validation +``` + +### Development Environment +1. **Frontend Server**: `http://localhost:3000` +2. **Backend API**: `http://localhost:8000` +3. **Hot Reload**: Automatic code reloading +4. **Error Overlay**: Development error reporting + +--- + +## 8. Performance & Optimization -### 5.2 Analytics Dashboard -- **Dashboard Features:** - - Summary statistics, usage trends, real-time updates. -- **Model Analysis:** - - Model performance, cost tracking, user insights. +### Next.js Features +- **Automatic code splitting** by route +- **Image optimization** with Next.js Image component +- **Bundle analysis** for size optimization +- **Edge runtime** support for API routes -### 5.3 Authentication -- **Components:** - - `AuthProvider.tsx`: Authentication context. - - `lib/api/auth.ts`: API integrations. - - `sessionStore.ts`: Session persistence. +### Caching Strategy +- **Redis caching** for user sessions and data +- **Browser caching** for static assets +- **SWR/React Query** for API response caching --- -## 6. UI Design Patterns +## 9. Security Implementation -### 6.1 Component Library -- Built on `shadcn/ui` components. -- **Ensures:** consistency, accessibility, responsiveness. +### Frontend Security +- **Environment variable** protection (NEXT_PUBLIC_ prefix for client-side) +- **API route protection** with authentication middleware +- **XSS protection** via input sanitization +- **CSRF protection** built into NextAuth.js -### 6.2 Layouts -| Layout File | Purpose | -|-------------|---------| -| `app/layout.tsx` | Root layout (App Router) | -| `components/layout.tsx` | Main layout | -| `components/analytics/AnalyticsLayout.tsx` | Analytics layout | -| `components/admin/AdminLayout.tsx` | Admin layout | +### Authentication Flow +``` +User Login โ†’ Google OAuth โ†’ NextAuth.js โ†’ Session Creation โ†’ Redis Storage +โ†“ +Route Protection โ†’ Middleware Check โ†’ API Authentication โ†’ Backend Access +``` --- -## 7. API Integration -API clients are located in `/lib/api/`: -| Module | Purpose | -|--------|---------| -| `analytics.ts` | Fetch analytics data | -| `auth.ts` | Handle authentication | +## 10. Deployment Options -Configuration in `/config/api.ts` manages: -- Base URL -- Environment variables -- API versioning +### Vercel (Recommended) +- **Automatic deployments** from GitHub +- **Environment variables** configuration +- **Preview deployments** for pull requests +- **Edge functions** for global performance + +### Alternative Platforms +- **AWS Amplify** with Terraform configuration +- **Docker containers** for custom hosting +- **Static export** for CDN deployment --- -## 8. Development and Build Process -| Command | Purpose | -|---------|---------| -| `npm run dev` | Start the development server | -| `npm run build` | Build the production app | -| `npm start` | Start the production server | +## ๐Ÿ“– Detailed Documentation + +For comprehensive frontend documentation, see: -Build artifacts are stored in `.next/` (ignored in `.gitignore`). +### **[Frontend Documentation Hub](../auto-analyst-frontend/docs/README.md)** + +**Architecture & Development** +- [Environment Setup Guide](../auto-analyst-frontend/docs/development/environment-setup.md) +- [Component Architecture](../auto-analyst-frontend/docs/architecture/) +- [API Integration](../auto-analyst-frontend/docs/communication/) + +**System Configuration** +- [Authentication System](../auto-analyst-frontend/docs/system/authentication.md) +- [Middleware Configuration](../auto-analyst-frontend/docs/system/middleware.md) +- [Model Registry](../auto-analyst-frontend/docs/system/model-registry.md) + +**Business Logic** +- [Credit Configuration](../auto-analyst-frontend/docs/billing/credit-configuration.md) +- [Trial System](../auto-analyst-frontend/docs/billing/trial-system.md) +- [Stripe Integration](../auto-analyst-frontend/docs/billing/stripe-integration.md) --- -## 9. Environment Configuration -Environment variables are stored in `.env.local`: -| Variable | Purpose | -|----------|---------| -| `NEXT_PUBLIC_API_URL` | Backend API endpoint | -| `NEXT_PUBLIC_ADMIN_PASSWORD` | Admin password | -| `NEXT_PUBLIC_ADMIN_EMAIL` | Admin email | -| `NEXT_PUBLIC_FREE_TRIAL_LIMIT` | Free trial limit (Production: 2, Development: 20000)| -| `AUTH_SECRET` | Authentication secret | -| `ANALYTICS_CONFIG` | Analytics settings | -| `NEXTAUTH_URL` | Authentication URL (http://localhost:3000) | -| `NEXTAUTH_SECRET` | Authentication secret | -| `GOOGLE_CLIENT_ID` | Google OAuth client ID | -| `GOOGLE_CLIENT_SECRET` | Google OAuth client secret | -| `SMTP_HOST` | SMTP server host | -| `SMTP_PORT` | SMTP server port | -| `SMTP_USER` | SMTP server username | -| `SMTP_PASS` | SMTP server password | -| `ADMIN_API_KEY` | Admin API key | - -For security, `.env.local` is not committed to Git. +## ๐Ÿ”ง Quick Start + +```bash +# Clone and setup +git clone +cd Auto-Analyst-CS/auto-analyst-frontend + +# Install dependencies +npm install + +# Configure environment +cp .env.example .env.local +# Edit .env.local with your configuration + +# Start development +npm run dev +``` + +Visit `http://localhost:3000` to see the application. --- -## 10. Conclusion -This documentation provides an in-depth overview of the Auto-Analyst frontend architecture, covering components, state management, API integration, and development workflows. ๐Ÿš€ \ No newline at end of file +This overview provides the essential information about the Auto-Analyst frontend. For detailed implementation guides, component documentation, and development workflows, refer to the [comprehensive frontend documentation](../auto-analyst-frontend/docs/README.md). \ No newline at end of file diff --git a/docs/redis-setup.md b/docs/redis-setup.md deleted file mode 100644 index b71d66e5..00000000 --- a/docs/redis-setup.md +++ /dev/null @@ -1,158 +0,0 @@ -# Redis (Upstash) Schema for Subscription Management - -This project uses **Redis (Upstash)** to store user subscription data, credits, and related metadata. The data is stored using **hash-based storage** for efficient retrieval. - -## **Key Structure** - -The Redis keys follow a structured naming convention for organization: - -- **User Subscription Data:** `user:{userId}:subscription` -- **User Credit Data:** `user:{userId}:credits` -- **User Profile Data:** `user:{userId}:profile` ---- - -## ๐Ÿ“Œ **Schema Details** - -### **1๏ธโƒฃ User Subscription Data** -Stored as a **hash** at `user:{userId}:subscription` - -| Field Name | Type | Description | -|------------------------|--------|-------------| -| `planType` | String | Subscription tier (`FREE`, `STANDARD`, `PRO`) | -| `amount` | String | Price of the subscription (e.g., `"15"`, `"29"`) | -| `purchaseDate` | String | ISO date when the subscription was purchased | -| `interval` | String | Billing cycle (`month` or `year`) | -| `status` | String | Subscription status (`active`, `inactive`) | -| `renewalDate` | String | ISO date for the next renewal | -| `stripeCustomerId` | String | Stripe customer ID (if applicable) | -| `stripeSubscriptionId`| String | Stripe subscription ID (if applicable) | -| `nextMonthlyReset` | String | (For yearly plans) Next monthly credit reset date | - -โœ… **Example Data:** -``` -HGETALL user:12345:subscription -{ - "planType": "PRO", - "amount": "29", - "purchaseDate": "2025-03-01T12:00:00Z", - "interval": "month", - "status": "active", - "renewalDate": "2025-04-01", - "stripeCustomerId": "cus_abc123", - "stripeSubscriptionId": "sub_xyz789" -} -``` - ---- - -### **2๏ธโƒฃ User Credit Data** -Stored as a **hash** at `user:{userId}:credits` - -| Field Name | Type | Description | -|-------------|--------|-------------| -| `used` | String | Number of credits used | -| `total` | String | Total credits available | -| `resetDate` | String | Next credit reset date (ISO format) | -| `lastUpdate` | String | Last update timestamp (ISO format) | - -โœ… **Example Data:** -``` -HGETALL user:12345:credits -{ - "used": "150", - "total": "500", - "resetDate": "2025-04-01", - "lastUpdate": "2025-03-20T14:30:00Z" -} -``` - -### **3๏ธโƒฃ User Profile Data** -Stored as a **hash** at `user:{userId}:profile` - -| Field Name | Type | Description | -|-------------|--------|-------------| -| `email` | String | User's email address | -| `name` | String | User's name | -| `image` | String | User's image | -| `joinedDate` | String | User's joined date | -| `role` | String | User's role | - - - -โœ… **Example Data:** -``` -HGETALL user:12345:profile -{ - "email": "user@example.com", - "name": "John Doe", - "image": "https://example.com/image.jpg", - "joinedDate": "2025-03-01", - "role": "PRO" -} -``` - ---- - -## ๐Ÿ›  **Operations Performed in API** - -1. **Fetching User Subscription Data:** - - Retrieves subscription details from `user:{userId}:subscription` - - Determines the **plan type** (`FREE`, `STANDARD`, `PRO`) - - Calculates **renewal date** based on `purchaseDate` & `interval` - - Manages **monthly resets** for yearly subscriptions - -2. **Fetching User Credits:** - - Retrieves credit usage from `user:{userId}:credits` - - Resets monthly credits if applicable - - Ensures correct handling of **yearly plans** - -3. **Handling Yearly Subscriptions:** - - Adds `nextMonthlyReset` for yearly plans - - Ensures credits reset monthly even in yearly billing cycles - ---- - -## **How to Access Data in Redis (Upstash)** - -Run the following commands to manually check data in Redis: - -๐Ÿ”น **Check Subscription Data for a User** -```sh -HGETALL user:12345:subscription -``` - -๐Ÿ”น **Check Credit Data for a User** -```sh -HGETALL user:12345:credits -``` - -๐Ÿ”น **Update Subscription Plan** -```sh -HSET user:12345:subscription planType "STANDARD" -``` - -๐Ÿ”น **Reset User Credits** -```sh -HSET user:12345:credits used "0" total "500" -``` - -๐Ÿ”น **Update User Profile** -```sh -HSET user:12345:profile name "John Doe" -``` - -๐Ÿ”น **Delete User Profile** -```sh -DEL user:12345:profile -``` - -๐Ÿ”น **Delete User Subscription** -```sh -DEL user:12345:subscription -``` - -๐Ÿ”น **Delete User Credits** -```sh -DEL user:12345:credits -``` - diff --git a/test_default_agents.py b/test_default_agents.py deleted file mode 100644 index 0519ecba..00000000 --- a/test_default_agents.py +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file