GSC Data Dive 📊

Go beyond basic Google Search Console metrics. Find actionable SEO insights faster and drive growth.

GSC Data Dive is a powerful web application that transforms your Google Search Console data into actionable SEO insights. Unlike basic GSC reports, it reveals keyword position distributions, identifies cannibalization issues, and provides comprehensive performance analytics to help you optimize your search rankings.

This repository is a self-hosted application, not a public hosted SaaS. To use it, you need your own Google Cloud OAuth credentials and your own deployment target for the frontend and Netlify functions.

🚀 Features

📈 Advanced Position Tracking

• True Position Distribution: See actual keyword position ranges (1-3, 4-10, 11-20, etc.) instead of misleading averages
• Multiple Visualization Views: Switch between Stacked Area, Small Multiples, and Percentage views
• Historical Position Changes: Track ranking improvements and declines over time
• Interactive Charts: Powered by Chart.js and Recharts for smooth, responsive visualizations

🔍 Keyword Analysis

• Top Keywords Performance: Identify your best-performing search terms
• Position Change Tracking: Monitor which keywords are gaining or losing rankings
• Keyword-Level Metrics: Deep dive into clicks, impressions, CTR, and position data

🏁 Cannibalization Detection

• Automated Issue Detection: Identifies when multiple pages compete for the same keywords
• Severity Classification: High, medium, and low priority cannibalization issues
• Page-Level Insights: See which URLs are competing and their performance metrics
• Export Capabilities: Download cannibalization reports for further analysis

📊 Comprehensive Analytics

• Performance Summary: Overall site metrics with comparison periods
• Top Pages Analysis: Track your highest-performing landing pages
• Date Range Comparisons: Compare current performance against previous periods
• Device & Country Filtering: Segment data by device type and geographic location

📤 Data Export

• CSV Export: Export position data, keyword reports, and cannibalization findings
• Custom Date Ranges: Flexible date selection for analysis periods
• In-App Exports: Export the datasets surfaced by the current dashboard views

📅 Data Availability & Performance

Historical Data Access

• Available Range: Up to 16 months of historical data (Google Search Console API limit)
• Data Freshness: Typically 2-3 days delayed
• Recommended Query Size: 30-90 days for optimal performance

Performance Considerations

• Small Ranges (7-30 days): Instant loading
• Medium Ranges (30-90 days): 5-15 seconds
• Large Ranges (90-365 days): 1-2 minutes (data fetched in 30-day chunks)
• Maximum Range (485 days/16 months): 2-3 minutes

Browser Caching

Fetched data is cached client-side to improve repeat loads. The app uses sessionStorage for short-lived cache entries, stores selected site/date preferences in localStorage, and can persist aggregated report data in IndexedDB for up to 7 days when enhanced fetching is enabled. Server-side Netlify functions may also keep short-lived in-memory cache entries for repeated requests. Use the "Clear Cache" button to remove browser-side cached data.

Google API Limitations

Per Google's Search Console API documentation:

• Large date ranges consume more API quota
• Queries with page + query dimensions may include sampled data
• Maximum 50,000 rows per day per search type

🛠️ Technology Stack

Frontend

• React 18 - Modern UI framework
• React Router - Client-side routing
• Chart.js & Recharts - Data visualization
• Tailwind CSS - Utility-first styling
• Vite - Fast build tool and dev server

Backend & Deployment

• Netlify Functions - Serverless backend
• Google Search Console API - Data source
• Google OAuth 2.0 - Secure authentication
• Node.js - Runtime for serverless functions

Security Features

• CSRF Protection - Cross-site request forgery prevention
• Content Security Policy - Restrictive default CSP in the app and matching Netlify response headers
• Secure Token Management - HttpOnly cookies for session management
• Best-Effort Rate Limiting - Serverless request throttling to reduce abuse

🚀 Quick Start

Prerequisites

• Node.js 18+ and npm
• Google Cloud Console project with Search Console API enabled
• Netlify account (for deployment)
• Netlify CLI available through npx or installed globally if preferred

If you are evaluating or adopting this project, plan to deploy and operate your own copy.

Installation

1. Clone the repository

   git clone https://github.com/vesivanov/GSC-data-dive.git
   cd GSC-data-dive

2. Install dependencies

   npm install

3. Environment Configuration

   Copy the example file and fill in your Google OAuth credentials:

   VITE_GOOGLE_CLIENT_ID=your_google_client_id
   GOOGLE_CLIENT_ID=your_google_client_id
   GOOGLE_CLIENT_SECRET=your_google_client_secret
   
   ALLOWED_ORIGINS=http://localhost:8888,http://localhost:5117,https://your-site.netlify.app
    
   VITE_USE_ENHANCED_FETCH=false

   Or:

   cp .env.example .env

   Environment variables used by this project:

   • VITE_GOOGLE_CLIENT_ID: Google OAuth client ID exposed to the browser
   • GOOGLE_CLIENT_ID: Google OAuth client ID used by Netlify functions
   • GOOGLE_CLIENT_SECRET: Google OAuth client secret used only on the server
   • ALLOWED_ORIGINS: Comma-separated allowlist for browser origins
   • VITE_USE_ENHANCED_FETCH: Enables the bundled report-fetch path with browser/server caching
   • NODE_ENV: Runtime mode for cookie security and local development behavior

4. Start the frontend

   npm run dev

   The Vite app will be available at http://localhost:5117

5. Run the full app with Netlify Functions

   npx netlify dev

   If npx cannot resolve the Netlify CLI in your environment, install it globally with npm install -g netlify-cli and rerun the command. For local auth and API calls, use the Netlify Dev URL, typically http://localhost:8888. Running npm run dev alone is not enough to exercise the full OAuth and serverless flow.

Google Cloud Setup

1. Create a Google Cloud Project

   • Go to Google Cloud Console
   • Create a new project or select existing one

2. Enable Search Console API

   • Navigate to APIs & Services > Library
   • Search for "Google Search Console API"
   • Click "Enable"

3. Create OAuth 2.0 Credentials

   • Go to APIs & Services > Credentials
   • Click "Create Credentials" > "OAuth 2.0 Client IDs"
   • Set application type to "Web application"
   • Add authorized JavaScript origins: http://localhost:8888 and http://localhost:5117
   • Add your production origin after deployment

4. Configure OAuth Consent Screen

   • Set up the consent screen with your app details
   • Add the Search Console readonly scope: https://www.googleapis.com/auth/webmasters.readonly

What To Expect

• This project is designed for self-hosting, not one-click consumer signup
• You are responsible for your own Google Cloud project, OAuth configuration, and deployment
• Google Search Console data quality can vary for large ranges and page/query-heavy reports
• This repo does not ship with a managed database or background job system

📁 Project Structure

src/
├── components/          # Reusable UI components
│   ├── Dashboard.jsx    # Main dashboard component
│   ├── PositionGraph.jsx # Position tracking visualizations
│   ├── TopKeywords.jsx  # Keyword performance component
│   └── ...
├── pages/              # Route-based page components
│   ├── Homepage.jsx    # Landing page
│   ├── Login.jsx       # Authentication page
│   ├── Dashboard.jsx   # Main analytics dashboard
│   ├── CannibalizationReport.jsx # Cannibalization analysis
│   └── AIDataAnalyst.jsx # Prioritized insights workspace
├── context/            # React Context providers
│   ├── AuthContext.jsx # Authentication state management
│   ├── GscDataContext.jsx # GSC data management
│   └── ...
├── services/           # API and external service integrations
│   └── gscApi.js       # Google Search Console API
├── utils/              # Utility functions
│   ├── validation.js   # Input validation
│   ├── csrf.js         # CSRF protection
│   └── ...
└── styles/             # CSS styling
    ├── app.css         # Global styles
    └── components.css  # Component-specific styles

netlify/
└── functions/          # Serverless backend functions
    ├── google-auth.cjs # OAuth flow handling
    ├── gsc-api.cjs     # Search Console API proxy
    └── gsc-report-bundle.cjs # Unified reporting bundle

🔧 Available Scripts

• npm run dev - Start development server
• npm run build - Build for production
• npm run preview - Preview production build locally
• npm start - Alternative dev server command
• npm test -- --runInBand - Run the test suite
• npm run lint - Run ESLint
• npm run typecheck - Run TypeScript checks
• npm run tailwind:build-test - Build Tailwind CSS for testing

🔐 Security Features

• CSRF Protection: All state-changing requests require valid CSRF tokens
• Secure Cookie Management: Authentication tokens stored in HttpOnly cookies
• Content Security Policy: The app ships with a restrictive default CSP and matching Netlify headers
• Best-Effort Rate Limiting: API endpoints apply lightweight serverless throttling to reduce abuse
• Input Validation: All user inputs sanitized and validated

🌐 Deployment

Netlify Deployment

1. Connect your repository to Netlify

2. Set environment variables in the Netlify dashboard:

   • VITE_GOOGLE_CLIENT_ID
   • GOOGLE_CLIENT_ID
   • GOOGLE_CLIENT_SECRET
   • ALLOWED_ORIGINS
   • VITE_USE_ENHANCED_FETCH

   Example production value:

   ALLOWED_ORIGINS=https://your-site.netlify.app,https://www.yourdomain.com

3. Configure build settings:

   • Build command: npm run build
   • Publish directory: dist
   • Functions directory: netlify/functions

The netlify.toml file contains all necessary configuration for deployment.

🧰 Common Setup Issues

• Google login popup opens but authentication fails: Verify that VITE_GOOGLE_CLIENT_ID, GOOGLE_CLIENT_ID, and GOOGLE_CLIENT_SECRET all belong to the same Google Cloud OAuth application.
• Invalid origin responses from Netlify functions: Make sure the browser origin you are using is included in ALLOWED_ORIGINS.
• OAuth works in Vite but not through the full app: Run npx netlify dev and use the Netlify Dev URL for end-to-end local testing.
• Cookies are not being set in production: Confirm you are serving the app over HTTPS and that your deployed frontend origin matches your configured OAuth origin and allowed origin list.
• Large reports feel slow or incomplete: This is usually a Google Search Console API limitation rather than a frontend rendering problem. Try smaller date ranges first.

⚠️ Limitations

• Search volume is not provided by Google Search Console and is not included in this app
• Large date ranges and page/query detail reports may return sampled or truncated data from Google
• Rate limiting is lightweight and intended to reduce abuse, not replace a dedicated API gateway
• The default deployment model assumes Netlify functions; adapting to another backend platform will require work

📊 Data Sources

• Google Search Console API: Primary data source for search performance metrics
• Real-time Processing: Data fetched and processed on-demand
• Secure API Calls: All GSC requests proxied through secure Netlify functions

🎨 UI/UX Features

• Responsive Design: Optimized for desktop, tablet, and mobile devices
• Interactive Charts: Hover effects, tooltips, and drill-down capabilities
• Modern Styling: Clean, professional interface with Tailwind CSS
• Performance Optimized: Lazy loading, code splitting, and efficient rendering

🔮 Roadmap

• Insights Workspace: Prioritized opportunities from the current GSC dataset
• Advanced Filtering: More granular data segmentation options
• Custom Dashboards: User-configurable dashboard layouts
• Scheduled Reports: Automated email reports
• Multi-site Management: Support for multiple GSC properties
• API Integration: Connect with other SEO tools

🤝 Contributing

See CONTRIBUTING.md for setup, quality checks, and pull request expectations.

🛡️ Project Policies

• SECURITY.md
• CODE_OF_CONDUCT.md

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.