FanMTL Novel Scraper

A lightweight Python Web scraper designed to download and compile web novels from FanMTL into a single, clean text file (.txt). It automatically cleans out scripts, ads, and unnecessary styling while supporting seamless resume capability if execution is interrupted.

---

Features

• Ad & Junk Cleaning: Automatically strips out script, div, iframe, and ins elements from the content area to keep the output text clean.
• Resume Capability: Reads the target text file to detect already-downloaded chapters and skips them automatically.
• Customizable Parameters: Easily set custom target URLs, output file names, request delays, and max chapter limits via command-line arguments.
• Rate Limiting: Built-in polite delay between requests to avoid hitting rate limits or triggering aggressive blocks.

---

Requirements

• Python 3.7+
• requests
• beautifulsoup4

Installation

1. Clone or download this repository.
2. Install dependencies:

pip install requests beautifulsoup4

---

Usage

1. Default Run

To run the script with default configuration (Greek Mythology: The Spiritual Dominator):

python scraper.py

2. Custom Command-Line Arguments

You can pass several flags to scrape a different novel or customize execution:

python scraper.py --url <STARTING_URL> --output <OUTPUT_FILE> --delay <SECONDS> --max <MAX_CHAPTERS>

Argument Options

Option	Short	Type	Default	Description
--url	-	String	Default FanMTL URL	Full URL of the starting chapter.
--output	-	String	Greek_Mythology_The_Spiritual_Dominator.txt	File path for saving the scraped content.
--delay	-	Float	0.5	Pause time (in seconds) between HTTP requests.
--max	-	Integer	None (All)	Maximum number of chapters to scrape in one run.

---

Examples

• Scrape 50 chapters of a specific novel:

python scraper.py --url "https://www.fanmtl.com/novel/some-novel-slug_1.html" --output "my_novel.txt" --max 50

• Increase request delay to 2 seconds:

python scraper.py --delay 2.0

---

Output Format

The scraped output is written cleanly as standard plain text (.txt):

Chapter 1: Chapter 1 - The Awakening
====================================

Paragraph text line 1...

Paragraph text line 2...

--------------------------------------------------

Chapter 2: Chapter 2 - The Journey
====================================

...

---

Disclaimer

This script is intended strictly for personal, offline reading purposes. Please respect FanMTL's terms of service and avoid setting the delay to zero to prevent placing unnecessary load on their servers.