[Feature] Enable mode-specific model and provider configuration for optimized performance

[Avtrkrb]

Description

Currently, Nanocoder uses a single provider and model configuration across all development modes (normal, auto-accept, plan). This limitation prevents users from optimizing their setup for different phases of development. I propose implementing mode-specific model and provider configuration that allows users to define different AI models for different development modes, enabling optimal resource utilization and performance across the development lifecycle.

Use Case:

• Planning Phase: Use powerful, sophisticated models (like GLM-4.7, Claude Opus, GPT-4o) for research and planning
• Execution Phase: Use efficient local models (20B-30B range) for faster, cost-effective implementation
• Hardware Constraints: Users with limited hardware can leverage powerful cloud models for planning while using local models for execution
• Cost Optimization: Use free models for planning, local models for execution to minimize costs

Proposed Solution:

Configuration Schema Extension

Add modeProviders section to agents.config.json:

{
  "nanocoder": {
    "providers": [
      {
        "name": "OpenRouter",
        "baseUrl": "https://openrouter.ai/api/v1",
        "apiKey": "your-openrouter-api-key-here",
        "models": ["openai/gpt-4o", "anthropic/claude-3-opus"]
      },
      {
        "name": "Z.ai",
        "baseUrl": "https://api.z.ai/api/paas/v4/",
        "apiKey": "your-z.ai-api-key",
        "models": ["glm-4.7", "glm-4.5", "glm-4.5-air"]
      },
      {
        "name": "Local Ollama",
        "baseUrl": "http://localhost:11434/v1",
        "models": ["llama3.2", "qwen2.5-coder", "deepseek-coder:6.7b"]
      }
    ],
    "modeProviders": {
      "plan": {
        "provider": "Z.ai",
        "model": "glm-4.7",
        "temperature": 0.7
      },
      "normal": {
        "provider": "Local Ollama",
        "model": "deepseek-coder:6.7b",
        "temperature": 0.3
      },
      "auto-accept": {
        "provider": "Local Ollama", 
        "model": "deepseek-coder:6.7b",
        "temperature": 0.1
      }
    },
    "mcpServers": [...]
  }
}

Implementation Approach

1. Configuration System Enhancement

   • Extend AppConfig interface with modeProviders property in /source/types/config.ts
   • Update configuration validation and loading logic in /source/config/index.ts
   • Add fallback logic for missing mode-specific configurations
   • Maintain backward compatibility with existing configurations

2. Client Factory Modification

   • Enhance createLLMClient() in /source/client-factory.ts to accept optional mode parameter
   • Implement mode-aware provider selection logic
   • Add connection validation for mode-specific providers
   • Add graceful fallback to global provider if mode-specific provider is unavailable

3. Mode Handler Updates

   • Modify useModeHandlers in /source/hooks/useModeHandlers.tsx to handle automatic provider switching
   • Update mode transition logic to include provider switching when modes change
   • Maintain chat history clearing behavior appropriately during provider switches
   • Add error handling for provider connection failures during mode transitions

4. Setup Wizard Enhancement

   • Add mode-specific provider selection screens to /source/wizard/config-wizard.tsx
   • Implement guided configuration for different modes in wizard steps
   • Add validation and testing for mode-specific providers during setup
   • Update UI to show mode-specific provider recommendations

5. State Management Updates

   • Update useAppState in /source/hooks/useAppState.tsx to track mode-specific provider settings
   • Modify provider switching logic to respect mode preferences
   • Ensure smooth transitions between modes without disrupting user experience
   • Add state management for tracking current mode's provider configuration

6. App State Integration

   • Update mode context switching to trigger provider reinitialization
   • Add provider switching logic to mode transition handlers
   • Ensure proper cleanup of old client connections when switching providers
   • Maintain tool manager compatibility during provider switches

Technical Architecture Details

Core Components to Modify:

• /source/client-factory.ts - Enhanced client creation with mode awareness
• /source/hooks/useModeHandlers.tsx - Mode transition with provider switching
• /source/hooks/useAppState.tsx - State management for mode-specific providers
• /source/wizard/config-wizard.tsx - Enhanced setup wizard
• /source/types/config.ts - Extended configuration schema
• /source/config/index.ts - Configuration loading with mode-specific logic

Key Functions to Update:

• createLLMClient(provider?: string, mode?: DevelopmentMode) - Enhanced with mode parameter
• handleProviderSelect(selectedProvider: string, mode?: DevelopmentMode) - Mode-aware provider selection
• handleModelSelect(selectedModel: string, mode?: DevelopmentMode) - Mode-aware model selection
• enterConfigWizardMode() - Enhanced to handle mode-specific configurations

Error Handling:

• Provider connection validation during mode transitions
• Graceful fallback when mode-specific provider is unavailable
• Clear user notifications for provider switching failures
• Automatic retry logic for failed provider connections

Performance Considerations:

• Efficient provider switching without full app restart
• Connection pooling for multiple providers
• Memory management for multiple client instances
• Caching of provider configurations to minimize validation overhead

Benefits

1. Optimized Performance: Powerful models for planning, efficient local models for execution
2. Cost Optimization: Use free/cheap models for planning, local models for execution
3. Hardware Flexibility: High-end hardware for planning, modest for execution
4. Enhanced User Experience: Automatic optimization based on development phase

Backward Compatibility

• Existing configurations will continue to work without changes
• Default behavior remains unchanged if modeProviders is not specified
• Fallback to global provider settings if mode-specific configuration is missing
• Clear migration path for existing users with automatic configuration upgrade suggestions

Acceptance Criteria

• Mode-specific providers can be configured in agents.config.json
• Providers automatically switch when modes change (Shift+Tab)
• Fallback to global provider works when mode-specific provider is unavailable
• Setup wizard guides users through mode-specific configuration
• Existing configurations remain backward compatible
• All tests pass with new configuration schema
• Performance is maintained during mode transitions
• Provider switching is seamless without disrupting ongoing conversations
• Error handling provides clear feedback when providers are unavailable

Implementation Notes (optional)

Security Considerations:

• Maintain existing security model with mode-based approval requirements
• Ensure provider switching doesn't bypass security validations
• Validate all mode-specific configurations before activation

Testing Strategy:

• Unit tests for mode-specific provider selection logic
• Integration tests for mode transitions with provider switching
• Configuration validation tests for new schema
• Backward compatibility tests with existing configurations

Performance Impact:

• Minimal overhead for provider switching (connection reuse where possible)
• Efficient configuration caching to avoid repeated validation
• Optimized for typical development workflows with mode transitions

User Experience:

• Clear visual indicators when providers switch during mode changes

• Smooth transitions without disrupting user workflow

• Intuitive setup wizard for mode-specific configurations

• Helpful error messages when providers are misconfigured

• I have searched existing issues to ensure this is not a duplicate

• This feature aligns with the project's goals (local-first AI assistance)

[self-sasi]

Hey @Avtrkrb, could you assign me to this?

[Avtrkrb]

Hey @Avtrkrb, could you assign me to this?

Go for it 👍

[miloslavnosek]

This is like the only thing that I am missing so far on this otherwise awesome project. Is this feature being implemented?