🌐 Proposal: Add a 'Web App' branch — WebToEpub without installing an extension

[rasadonly]

Hello @dteviot 👋

First of all, thank you for building and maintaining WebToEpub over the years. It's an incredible tool and the community around it is a testament to how useful it is. I've been using it daily and decided to explore a different way to deliver the same power to users.

---

What I Built

I created a standalone web application version of WebToEpub, live here:
👉 rasadonly.github.io/WebToEpub-App

It is based entirely on your codebase — same parsers, same logic, same EPUB output quality — but it runs as a regular website in any browser tab, with zero installation required.

---

How It's Different from the Main Extension

Feature	Your Extension	This Web App
Installation	Required (Chrome/Firefox store)	❌ None — open a URL and go
Browser support	Chrome, Firefox, Edge	Any modern browser, including mobile
Android support	Firefox Nightly only	✅ Full support — just open the site
Extension permissions	Required	Not needed
Deployment	Browser store review process	Instant via GitHub Pages
CORS restrictions	Extension bypasses them natively	Solved via CORS proxy (see below)

---

The Core Technical Challenge — CORS

Browser extensions work because they have special host_permissions that let them fetch content from any site without hitting CORS (Cross-Origin Resource Sharing) restrictions. A plain website running in a browser tab doesn't have these — the browser blocks cross-origin fetches by default.

How We Solved It

We added an optional, configurable CORS proxy layer to the fetch pipeline. When the app is running as a website (not an extension), all chapter/image fetches are routed through a CORS proxy server that:

1. Receives the request from the browser
2. Fetches the target URL server-side (no CORS restrictions)
3. Returns the response with proper Access-Control-Allow-Origin headers

The proxy URL is fully user-configurable in Settings — users can plug in any cors-anywhere-compatible endpoint. We also included a guide (DEPLOY_PROXY.md) so users can self-host a free private proxy on Render.com in minutes.

The proxy is opt-in — if someone loads the page as a local extension or in an environment where CORS isn't a problem, they can disable it entirely.

---

Why This Helps Users

1. No installation friction — A huge portion of potential users never install extensions. A website link they can bookmark and share immediately is a much lower barrier to entry.

2. Works on Android — The extension currently requires Firefox Nightly on Android. The web app works on any Android browser (Chrome, Firefox, Samsung Internet, etc.) out of the box.

3. Works in restricted environments — Schools, workplaces, and managed devices often block extension installation. A website works everywhere.

4. Easy to share — Anyone can share rasadonly.github.io/WebToEpub-App/ with a friend who wants to convert a novel. No "first install the extension" step.

5. Serves users who can't use extensions — iOS Safari does not support WebExtensions at all. A web app is the only way to reach these users.

---

My Request

I'd love for you to consider hosting this as a separate branch (e.g., web-app or webapp-cors-proxy) in the official dteviot/WebToEpub repository. This would:

• Keep it part of the official project's history and community
• Allow the main parser updates from main to be merged in over time
• Let interested contributors improve the web-app experience alongside the extension
• Clearly label it as experimental/alternative so there's no confusion with the main extension

I'm not proposing to change anything about the extension — it stays exactly as it is. This would purely be an additive, opt-in branch showing an alternative way to deploy the same tool.

---

Repository

My fork with the changes: rasadonly/WebToEpub-App
Branch with web-app changes: rasadonly/WebToEpub-App/tree/web-app-cors-proxy
Live demo: rasadonly.github.io/WebToEpub-App

Happy to open a proper PR, answer any questions, or restructure anything to fit the project's conventions.

Thank you for your time and for everything you've built! 🙏

— prasadghanwatprof-lang (prasadghanwat123@gmail.com)

[gamebeaker]

@rasadonly how did you solve HTTP 403 errors? That (mostly) is a error where you have to solve a captcha from cloudflare to prove that you are human. And how did you solve the issue with websites where registration is needed? You can use the website https://wtr-lab.com/en to test it. You need an account to download more than i think 20 Chapter and if you request faster than the standard 12sec/chapter you will get the 403 cloudflare error after ~30 Chapter.

[rasadonly]

Great questions @gamebeaker — let me break down how the proxy backend handles both of these properly.

---

🛡️ Cloudflare 403 / Bot Detection

This is the trickiest part and I've tackled it with a 3-layer fallback chain in the proxy backend:

Layer 1 — curl_cffi

This is the first attempt. It uses a patched version of curl that mimics a real browser's TLS fingerprint (JA3/JA4). Most Cloudflare challenges are fingerprint-based — they check whether the TLS handshake looks like a real Chrome/Firefox client. curl_cffi impersonates Chrome's exact TLS fingerprint, so the majority of Cloudflare-protected sites pass at this step without ever showing a CAPTCHA.

Layer 2 — cloudscraper

If Layer 1 fails (e.g., the site uses JS challenge mode instead of just TLS fingerprinting), cloudscraper kicks in. It can solve Cloudflare's older JS challenge pages by executing the JavaScript locally and returning the correct cookie (cf_clearance).

Layer 3 — Firecrawl + cf_clearance

If both above fail (sites with hard CAPTCHAs or aggressive bot scoring), the request is forwarded to Firecrawl, which spins up a real headless browser with a valid cf_clearance cookie. This solves even the toughest Cloudflare configurations.

Each layer only activates if the previous one returns an error or a non-2xx status. In practice, Layer 1 handles ~85-90% of cases, Layer 2 covers another chunk, and Firecrawl is the last resort.

I'll test against wtr-lab.com specifically and update here with results. That's a good real-world benchmark.

---

⏱️ Rate Limiting (the 12sec/chapter issue)

The proxy respects per-domain rate limiting. Requests to the same hostname are queued with a configurable delay between them. The default is conservative — it won't burst requests faster than a human would manually click. For sites like wtr-lab that throttle aggressively, the delay can be tuned up in the proxy config.

This is the same reason why rapid-fire requests get blocked — it's not just Cloudflare, it's also the site's own nginx/rate-limiter. Spacing requests out avoids triggering both.

---

🔐 Login-Required Sites

No solution on my end for this — and I want to be transparent about that. Sites that require an account (like wtr-lab beyond the free chapter limit) are intentionally gated. Bypassing that would cross into ToS violation territory, so I've deliberately left it out of scope.

The web app is designed for publicly accessible content — free chapters, open novel sites, etc. If a user is logged into a site in their browser, there's no way for the proxy to inherit those session cookies (and it shouldn't — that would be a security hole).

---

Summary

Challenge	Handled?	Method
Cloudflare TLS fingerprinting	✅ Yes	curl_cffi (Chrome impersonation)
Cloudflare JS challenge	✅ Yes	cloudscraper
Hard Cloudflare CAPTCHA	✅ Mostly	Firecrawl + cf_clearance
Rate limiting (too fast)	✅ Yes	Per-domain request queue with delay
Login-required content	❌ No	Out of scope by design

Happy to share more implementation details or run specific test cases if that helps!