MCP server for Googlebot vs AI crawler parity analysis — compare crawl behaviour from Nginx logs with GSC data to detect visibility gaps
-
Updated
Mar 28, 2026 - JavaScript
#
crawl-analysis
Here are 6 public repositories matching this topic...
Audit Googlebot crawl behaviour from raw nginx access logs. A local-only Streamlit dashboard with seven analysis views — crawl budget, orphans, status waste, stale pages, performance, bot verification, parameter traps.
python nginx log-analysis seo data-analysis googlebot search-engine-optimization streamlit technical-seo crawl-analysis
-
Updated
May 19, 2026 - Python
Multi-page Looker Studio dashboard built on Screaming Frog crawl data — surfacing indexability issues, crawl budget waste, and technical SEO problems in one interactive report.
-
Updated
May 19, 2026
SQL warehouse for search console, crawl, index coverage, freshness drift, and URL-level observability diagnostics.
python sql warehouse data-modeling indexation technical-seo analytics-engineering crawl-analysis search-observability
-
Updated
Jul 20, 2026 - HTML
Turn crawl and performance data into a ranked SEO action list, helping teams focus time and budget on the pages and fixes that matter most. Runs locally with no uploads.
python open-source seo web-analytics pandas pytest data-analysis prioritization google-search-console seo-tools privacy-first pydantic screaming-frog local-first technical-seo core-web-vitals steamlit ga4 content-optimization crawl-analysis
-
Updated
Jul 20, 2026 - Python
Modular Toolkit for PageRank Calculation — pipeable R functions for URL cleaning, redirect resolution, edge dedup, and igraph-backed PageRank on website link graphs
redirects r graph-algorithms seo pagerank rstats r-package network-analysis igraph centrality r-lang screaming-frog technical-seo web-graph url-canonicalization internal-linking link-graph site-architecture crawl-analysis link-equity
-
Updated
Jul 26, 2026 - R
Improve this page
Add a description, image, and links to the crawl-analysis topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the crawl-analysis topic, visit your repo's landing page and select "manage topics."