Skip to content

taugroup/Weatherman_updated

BranchesTags

Repository files navigation

🌩️ NimbusNews: AI-Powered Weather Report Automation

NimbusNews Architecture Diagram Architecture Overview: From Weather Charts to AI-Generated Video Reports

📖 Overview

NimbusNews automates weather reporting by converting meteorological charts into engaging video summaries. Designed for researchers, meteorologists, and weather stations, it combines cutting-edge AI technologies to deliver professional weather reports in minutes.

Key Technologies:

  • GPT-4o / Gemini 2.5 Pro (via TAMU AI Chat) for advanced chart analysis
  • Llama 3.2 Vision (11B) for alternative vision processing
  • Deepgram AI for natural text-to-speech conversion
  • Wav2Lip for realistic lip-syncing
  • Cloudflare R2 for scalable cloud storage

🏆 Winner of TamuHack 2025

🛠️ Features

Component Technology Purpose
Vision Analysis TAMU AI Chat (Gemini 2.5 Pro / Llama 3.2) Analyze weather charts
Text Processing NLP models Generate concise summaries
Audio Generation Deepgram AI (Aura Asteria voice) Convert text to speech
Video Synthesis Wav2Lip neural network Create lip-synced videos
Cloud Storage Cloudflare R2 Store inputs and outputs
Web Interface Streamlit User-friendly dashboard

🌪️ Sample Input

500 hPa Weather Chart
GFS Forecast 4-Panel Chart (500 hPa heights, 250 hPa winds, 700 hPa thicknesses)

🚀 Quick Start

Prerequisites

Before starting, ensure you have:

Automated Installation (Recommended)

macOS/Linux

# 1. Clone the repository
git clone https://github.com/Maitreya123/Weatherman_upd.git
cd Weatherman_upd

# 2. Run automated setup (installs everything)
chmod +x setup.sh
./setup.sh

# 3. Configure your API keys
nano .env
# Add your TAMU_CHAT_API_KEY, DEEPGRAM_API_KEY, and R2 credentials

# 4. Start the system
./start.sh

# 5. Open your browser
# Navigate to: http://localhost:8501

Windows

REM 1. Clone the repository
git clone https://github.com/Maitreya123/Weatherman_upd.git
cd Weatherman_upd

REM 2. Run automated setup (installs everything)
setup.bat

REM 3. Configure your API keys
notepad .env
REM Add your TAMU_CHAT_API_KEY, DEEPGRAM_API_KEY, and R2 credentials

REM 4. Start the system
start.bat

REM 5. Open your browser
REM Navigate to: http://localhost:8501

What the Setup Does

The automated setup script:

  1. ✅ Checks Python 3.10+ and ffmpeg installation
  2. ✅ Creates two virtual environments (weatherman_env, wav2lipmodern_env)
  3. ✅ Installs all Python dependencies
  4. ✅ Creates necessary directories (images/, audio/, video/, logs/)
  5. ✅ Generates startup scripts (start.sh, stop.sh, status.sh)
  6. ✅ Creates .env file from template

Time: 5-10 minutes (depending on internet speed)

⚙️ Configuration

Environment Variables

Edit the .env file with your API credentials:

# TAMU AI Chat API Key (get from https://chat.tamu.ai/)
TAMU_CHAT_API_KEY=sk-your-actual-key-here
TAMU_CHAT_BASE_URL=https://chat-api.tamu.ai/openai

# Deepgram API Key (get from https://console.deepgram.com/)
DEEPGRAM_API_KEY=your-deepgram-key-here

# Cloudflare R2 Storage Credentials
R2_ACCESS_KEY_ID=your-r2-access-key-here
R2_SECRET_ACCESS_KEY=your-r2-secret-key-here
R2_ENDPOINT_URL=https://your-account-id.r2.cloudflarestorage.com
BUCKET_NAME=weather-man

# Optional: Model preference
MODEL_NAME=protected.gemini-2.5-pro

⚠️ Important:

  • Replace all placeholder values with your actual API keys
  • Never commit .env to version control
  • Keep your API keys secure

Getting API Keys

TAMU AI Chat:

  1. Visit https://chat.tamu.ai/
  2. Sign in with Texas A&M credentials
  3. Navigate to API settings
  4. Generate new API key

Deepgram:

  1. Visit https://console.deepgram.com/
  2. Create free account
  3. Go to API Keys section
  4. Create new API key

Cloudflare R2:

  1. Visit https://dash.cloudflare.com/
  2. Navigate to R2 Object Storage
  3. Create a bucket (e.g., "weather-man")
  4. Generate API tokens with R2 read/write permissions
  5. Note your account ID for the endpoint URL

🎮 Using the System

Starting the System

# macOS/Linux
./start.sh

# Windows
start.bat

This starts:

  • Wav2Lip Server on http://127.0.0.1:5001
  • Streamlit App on http://localhost:8501

Checking Status

# macOS/Linux
./status.sh

# Windows
status.bat

Stopping the System

# macOS/Linux
./stop.sh

# Windows
stop.bat

Using the Web Interface

  1. Open browser to http://localhost:8501

  2. Upload or Select Image:

    • Upload a weather chart (PNG/JPG)
    • Or select from previously uploaded images in R2 storage

  3. Select AI Model:

    • protected.gemini-2.5-pro (Recommended - most accurate)
    • llama-3.2-11b-vision-instruct (Faster alternative)

  4. Click "Analyze image":

    • Vision AI analyzes the weather chart
    • Text is cleaned and summarized
    • Audio is generated with natural voice
    • Video is created with lip-sync

  5. View Results:

    • Read detailed analysis and summary
    • Listen to audio report
    • Watch generated video report
    • Download or save to R2 storage

Features

Generate Report Tab:

  • Upload weather charts
  • Select AI models
  • Generate analysis, audio, and video
  • Save reports to cloud storage

Video Reports Tab:

  • Browse previously generated videos
  • View saved reports from R2 storage
  • Download or share reports

Weather Forecast Tab:

  • Search weather by location
  • View current conditions
  • See daily forecasts

About Tab:

  • System information
  • Technology stack details

📊 AI Workflow Pipeline

┌─────────────┐
│ Weather     │
│ Chart       │
│ (PNG/JPG)   │
└──────┬──────┘
       │
       ▼
┌─────────────────────────────────┐
│   Vision Analysis               │
│   • Gemini 2.5 Pro              │
│   • Llama 3.2 11B Vision        │
└──────┬──────────────────────────┘
       │
       ▼
┌─────────────────────────────────┐
│   Text Processing               │
│   • Clean raw analysis          │
│   • Generate summary            │
└──────┬──────────────────────────┘
       │
       ▼
┌─────────────────────────────────┐
│   Audio Generation              │
│   • Deepgram TTS                │
│   • Aura Asteria voice          │
└──────┬──────────────────────────┘
       │
       ▼
┌─────────────────────────────────┐
│   Video Synthesis               │
│   • Wav2Lip lip-sync            │
│   • Base video + audio          │
└──────┬──────────────────────────┘
       │
       ▼
┌─────────────────────────────────┐
│   Cloud Storage                 │
│   • Upload to R2                │
│   • Available for viewing       │
└─────────────────────────────────┘

Typical Processing Time: 40-120 seconds per report

🔧 Manual Installation (Advanced)

If you prefer manual setup or the automated script fails:

1. Create Virtual Environments

# Main application environment
python3 -m venv weatherman_env
source weatherman_env/bin/activate  # macOS/Linux
# or: weatherman_env\Scripts\activate  # Windows

pip install -r requirements.txt
deactivate

# Wav2Lip environment
python3 -m venv wav2lipmodern_env
source wav2lipmodern_env/bin/activate  # macOS/Linux
# or: wav2lipmodern_env\Scripts\activate  # Windows

cd Wav2LipModern
pip install -r requirements.txt
cd ..
deactivate

2. Download Wav2Lip Checkpoints

mkdir -p Wav2LipModern/checkpoints
cd Wav2LipModern/checkpoints

# Download Wav2Lip model weights (~150MB)
wget https://github.com/Rudrabha/Wav2Lip/releases/download/v1.0/Wav2Lip.pth

cd ../..

3. Create Directories

mkdir -p images audio video logs prompts

4. Configure Environment

cp .env.example .env
nano .env  # Add your API keys

5. Start Manually

Terminal 1 - Wav2Lip Server:

source wav2lipmodern_env/bin/activate
python wav2lip_server.py

Terminal 2 - Streamlit App:

source weatherman_env/bin/activate
streamlit run app.py --server.port 8501

🐛 Troubleshooting

Common Issues

1. Python Version Error

Error: Python 3.10+ required

Solution: Install Python 3.10 or higher from python.org

2. ffmpeg Not Found

Error: ffmpeg not found

Solution:

  • macOS: brew install ffmpeg
  • Linux: sudo apt-get install ffmpeg
  • Windows: Download from ffmpeg.org and add to PATH

3. API Key Invalid

Error: Invalid API key or authentication failed

Solution:

  • Verify API key in .env file
  • Get new key from respective service
  • Restart the system after updating

4. Port Already in Use

Error: Address already in use: Port 8501

Solution:

# macOS/Linux
lsof -ti:8501 | xargs kill -9
lsof -ti:5001 | xargs kill -9

# Windows
netstat -ano | findstr :8501
taskkill /PID <PID> /F

5. Face Detection Failed

Error: Face detection failed

Solution:

  • Verify video/weather_woman_base.mp4 exists
  • Check Wav2Lip checkpoints are downloaded
  • Ensure video has clear, visible face

Viewing Logs

Check logs for detailed error information:

# macOS/Linux
tail -f logs/streamlit.log
tail -f logs/wav2lip_server.log
tail -f logs/app.log

# Windows
type logs\streamlit.log
type logs\wav2lip_server.log
type logs\app.log

Getting Help

If issues persist:

  1. Check logs in logs/ directory
  2. Review ARCHITECTURE.md for technical details
  3. Verify all prerequisites are installed
  4. Ensure API keys are valid
  5. Open GitHub issue with error details

📁 Project Structure

Weatherman_upd/
├── app.py                      # Main Streamlit application
├── wav2lip_server.py          # Video processing server
├── config.py                   # Configuration management
├── logger.py                   # Logging system
├── deepgram_audio.py          # Audio generation
├── weather_data.py            # Weather API integration
│
├── Agents/
│   ├── vision_agent.py        # Vision analysis
│   └── reform_agent.py        # Text processing
│
├── TamuAI/
│   └── tamu_ai_client.py      # TAMU API client
│
├── Wav2LipModern/             # Lip-sync engine
│   ├── run.py
│   ├── inference.py
│   └── checkpoints/           # Model weights
│
├── images/                     # Input weather charts
├── audio/                      # Generated audio files
├── video/                      # Base & output videos
├── prompts/                    # AI prompts
├── logs/                       # Application logs
│
├── setup.sh / setup.bat       # Automated setup
├── start.sh / start.bat       # Start system
├── stop.sh / stop.bat         # Stop system
├── status.sh / status.bat     # Check status
│
├── .env                        # API keys (create from .env.example)
├── .env.example               # Environment template
├── requirements.txt           # Python dependencies
└── README.md                  # This file

🏗️ System Architecture

For detailed technical documentation, see ARCHITECTURE.md

Key Components:

  • Frontend: Streamlit web interface (Port 8501)
  • Vision Layer: VisionAgent + TAMUAIClient
  • Text Layer: ReformAgent for summarization
  • Audio Layer: Deepgram integration
  • Video Layer: Wav2Lip server (Port 5001)
  • Storage Layer: Cloudflare R2 integration

Two Virtual Environments:

  • weatherman_env: Main application
  • wav2lipmodern_env: Video processing

🔒 Security Best Practices

  1. Never commit .env file to version control
  2. Rotate API keys regularly
  3. Use environment variables for all secrets
  4. Restrict R2 bucket permissions to necessary operations only
  5. Keep dependencies updated for security patches
  6. Use HTTPS in production deployments
  7. Implement rate limiting for API endpoints

🚀 Production Deployment

For production use:

  1. Use production WSGI server for Flask:

    pip install gunicorn
gunicorn -w 4 -b 127.0.0.1:5001 wav2lip_server:app

  2. Set up reverse proxy (nginx) for Streamlit

  3. Configure firewall rules to restrict access

  4. Set up monitoring and alerting

  5. Use systemd services (Linux) for auto-restart

  6. Implement backup strategy for R2 storage

See ARCHITECTURE.md for detailed deployment instructions.

📈 Performance

Typical Processing Times:

  • Image Analysis: 5-15 seconds
  • Text Summarization: 3-8 seconds
  • Audio Generation: 2-5 seconds
  • Video Processing: 30-90 seconds
  • Total: ~40-120 seconds per report

System Requirements:

  • Minimum: 4 cores, 8GB RAM, 10GB storage
  • Recommended: 8+ cores, 16GB RAM, 50GB SSD, GPU (optional)

🤝 Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes with tests
  4. Submit a pull request
  5. Follow PEP 8 style guidelines

📜 License

MIT License - See LICENSE file for details

👥 Developers

Dheeraj Mudireddy

Praneet Surabhi

🏆 Acknowledgments

  • TamuHack 2025 - Winner 🏆
  • Texas A&M University - TAMU AI Chat API access
  • Deepgram - Text-to-speech technology
  • Cloudflare - R2 storage infrastructure
  • Wav2Lip Team - Lip-sync technology

📞 Support

Repository: github.com/Maitreya123/Weatherman_upd

Issues: Open a GitHub issue with:

  • Error message
  • Log excerpts
  • System information
  • Steps to reproduce

Last Updated: February 2026

About

No description, website, or topics provided.

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages