Skip to content

How to Use ScrapeWeb

Jump to bottom
15930883 edited this page Apr 17, 2025 · 2 revisions

🕵️‍♂️ What ScrapeWeb Does

ScrapeWeb is a desktop application that helps you extract titles and links from blog-style pages by specifying CSS selectors. It outputs your results in JSON and/or Markdown formats for further use.

📘 Step-by-Step: Using ScrapeWeb

1. Open ScrapeWeb

Launch the application (ScrapeWeb.exe) from the dist/win-unpacked directory or your extracted portable app.

2. Enter the Blog URL

Paste the URL of the blog or article list page (e.g. https://example.com/blog).

3. Find the Right CSS Selectors

🔎 A. Open the Browser’s DevTools

  1. Right-click the blog post title or link in your browser
  2. Click "Inspect" (in Chrome, Edge, Firefox, etc.)
  3. In the Elements tab, hover over the highlighted HTML

🧠 B. Copy the CSS Selector

  • Right-click the element in the DevTools
  • Choose: Copy → Copy selector

✅ This gives you something like:

#main-content > div.post > h2.title

You can paste this into ScrapeWeb’s Title Selector field.

💡 Tips:

  • You can simplify selectors:
    Instead of:
    #main-content > div.post > h2.title
    Use:
    .post-title or h2.title if consistent across posts.

  • The Link Selector can often just be a, assuming the title or post container contains a link.

4. Set the Output Format

Choose from:

  • JSON
  • Markdown
  • Both

5. Choose an Output Folder

Click Browse to select where the files should be saved.

6. Scrape the Data

Click Go Scrape — the app will process the blog page and extract:

  • Titles
  • Links
  • Page metadata (title, description, etc.)

7. View Results

Click See Scrapes to open the folder where files were written.

🌒 Dark Mode and Preferences

  • Use the Dark Mode button (top-right) to switch themes
  • The app remembers your last used theme and URLs

🧰 Examples

Example 1: Cybersecurity Blog

URL: https://www.cm-alliance.com/cybersecurity-blog

  • Title Selector: .blogv2-postbox-title
  • Link Selector: a

Example 2: Splunk Threat Blog

URL: https://www.splunk.com/en_us/blog/author/secmrkt-research.html

  • Title Selector: .title h4
  • Link Selector: a

📎 Common Troubleshooting

Issue Suggestion
Nothing is extracted Double-check your CSS selectors
Output file missing Ensure output folder is selected
“Scrape Now” button does nothing Check console logs or recheck selector formatting

Clone this wiki locally