WebFetch Domain Access Issues

[kaolacheng]

WebFetch Domain Access Issues

This document summarizes the domain access limitations encountered when using Claude Code's WebFetch tool. These issues affect research, technical investigation, and general web browsing capabilities.

---

Issue #1: Academic Websites Blocked (Submitted as GitHub Issue)

Title:

Problem

The WebFetch tool is unable to access several major academic and research-focused websites, preventing paper search and technical research within Claude Code.

Affected Domains

• scholar.google.com — Google Scholar academic search
• arxiv.org — Preprint server for CS/ML/AI papers
• Potentially other academic domains (e.g., IEEE Xplore, ACM Digital Library)

Error Message

Error: Unable to verify if domain scholar.google.com is safe to fetch. 
This may be due to network restrictions or enterprise security policies blocking claude.ai.

Reproduction Steps

1. Attempt to use WebFetch with URL https://scholar.google.com/scholar?q=sparse+matrix+multiplication
2. Receive "Unable to verify if domain is safe to fetch" error

Impact

• Unable to search for academic papers or research papers related to the project
• Cannot access arXiv preprints for latest ML/AI research
• Google Scholar searches are blocked, limiting literature review capabilities

Steps to Reproduce

1. Use WebFetch tool to access any URL on scholar.google.com or arxiv.org
2. Observe error message about domain safety verification failure

Expected Behavior

The WebFetch tool should be able to access common academic resource sites without restrictions, as these are widely used and trusted educational/research resources.

---

Issue #2: GitHub Domain Blocked (Potentially)

Note: This may require investigation to confirm if it's a WebFetch limitation or specific to certain endpoints.

Affected Domains

• github.com — Code hosting platform

Impact

• Cannot fetch content from GitHub repositories, pull requests, issues via WebFetch
• Limits ability to read documentation hosted on GitHub wikis or raw files

---

Suggested Fixes

1. Add domain allowlist — Explicitly whitelisting trusted educational/research domains (scholar.google.com, arxiv.org, etc.) in the WebFetch safety verification logic
2. Reduce overly strict filtering — Review what constitutes a "safe" fetch and whether academic domains should be treated as inherently safe
3. Provide configuration option — Allow users to manually add allowed domains via config or environment variable

---

Issue tracker: DdogezD/claudium

---

Generated on 2025-04-30 by Claude Code assistant