Skip to content

PRO100CHOK/threads-net-posts-profiles-scraper-python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Threads.net Posts & Profiles Scraper — Python Example

Scrape Threads.net posts, profiles, and replies in Python. Run keyword searches, pull profile metadata, fetch every reply to a viral thread, and build mention-monitoring pipelines — without a Threads developer account.

Apify Actor Python 3.10+ License: MIT

Working Python code that turns Meta's Threads platform into a queryable data source via the Threads Scraper Apify actor. Five scrape modes (search, profile, post, user_posts, post_replies) cover the entire surface area of Threads — keyword search, account discovery, profile metadata, a user's timeline, individual post details, and the reply tree under any post.

Why this exists

Meta has not opened a public Threads API. The only official integration is a closed-beta access program for "select developer partners," which leaves brand-monitoring teams, social-listening tools, and academic researchers with no way to ingest Threads data. Threads is currently the fastest-growing X/Twitter alternative (200M+ monthly active users in 2026), so missing it means missing where conversations about your brand are happening today.

This actor handles the residential IP rotation, Threads' graphql endpoints, cursor-based pagination, and the optional cookie-based login flow needed to paginate past the first ~20 results on profile timelines. Your Python script just provides a list of keywords, usernames, or post URLs and reads the results.

Use cases

  • Brand-mention monitoring — search for your brand name daily and push new mentions into Slack / a help desk.
  • Competitive social listening — watch your competitors' Threads handles for new product announcements before they hit the press.
  • Customer-support triage — pull replies under your support post to triage and respond.
  • Influencer discovery — search for niche topics and identify the most-followed accounts posting about them.
  • Viral content analysis — pull every reply to a viral thread to mine sentiment and gather UGC ideas.
  • Academic research — collect a public dataset of Threads conversations around a topic for sociolinguistic or political-science research.

Requirements

  • Python 3.10+
  • A free Apify account
  • No Threads / Meta developer account required
  • (Optional) Threads session cookies for richer pagination on private/follower-only data

Quick start

git clone https://github.com/pro100chok/threads-net-posts-profiles-scraper-python.git
cd threads-net-posts-profiles-scraper-python
pip install -r requirements.txt
cp .env.example .env
# paste your APIFY_API_TOKEN
python main.py

main.py searches Threads for posts mentioning two keywords (claude code, anthropic) and dumps the matching posts to JSON + CSV with author, text, likes, replies, and timestamps. Edit QUERIES to track your own brand or topic.

Five scrape modes

The action parameter switches between five surfaces of Threads:

action What you pass in What comes back
search queries Posts (or accounts, if serp_type=accounts) matching keyword.
profile usernames Profile metadata — followers, bio, verification, link in bio.
user_posts usernames Latest posts on a user's timeline.
post posts (URLs) Full data for a specific post.
post_replies posts (URLs) All replies (with optional pagination via cookies).

How it works

The actor calls Threads' internal GraphQL endpoints through residential US proxies. For each mode it builds the right query payload, paginates through the cursor-based response, parses each post / profile / reply into a flat schema, and pushes it to the run's dataset. Your Python script reads via the official apify-client. No browser automation needed on your side.

For mode = user_posts or post_replies, Threads' public endpoint only returns the first ~20 results before requiring authentication. You can pass threadsCookies (a JSON string of session cookies exported from EditThisCookie or similar) to paginate further. Without cookies you still get the first 20 items per query.

Example: brand mention monitor

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_API_TOKEN"])

run = client.actor("pro100chok/threads-scraper-usage").call(run_input={
    "action": "search",
    "queries": ["claude code", "anthropic"],
    "maxItems": 50,
})

for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    user = (post.get("user") or {}).get("username")
    print(f"@{user} ({post['likeCount']}❤): {post['text'][:120]}")

Example output (search mode)

{
  "query": "claude code",
  "url": "https://www.threads.net/@dev_dan/post/D9XyZaBcDef",
  "user": {
    "username": "dev_dan",
    "fullName": "Dan",
    "isVerified": false,
    "followerCount": 1820
  },
  "text": "Claude Code just refactored 800 lines of legacy code in one shot. Wild.",
  "likeCount": 245,
  "replyCount": 18,
  "repostCount": 7,
  "publishedAt": "2026-05-14T11:02:33Z",
  "media": [
    { "type": "image", "url": "https://scontent-...jpg" }
  ]
}

Input parameters

Parameter Type Used by Description
action string all One of search, profile, post, user_posts, post_replies.
queries string[] search Search keywords, one per line.
serp_type string search default for posts, accounts for account discovery.
usernames string[] profile / user_posts Threads handles with or without @.
posts string[] post / post_replies Threads post URLs.
maxItems integer all Cap per query/username/post. Use 0 for unlimited (subject to login).
threadsCookies string optional JSON cookie blob for an authenticated session — enables deep pagination.
proxy object all Residential US proxy recommended; default rotates per request.

More examples

File Demonstrates
examples/01_basic_usage.py Single keyword search.
examples/02_profile_details.py Profile metadata for a username list.
examples/03_post_replies_thread.py Pull replies to a specific post.
examples/04_export_to_csv.py Multi-keyword scan + engagement score + pandas export.
examples/05_export_to_google_sheets.py Append mentions to a shared Sheet for community moderators.

FAQ

How does pricing work? The actor is charged per dataset item (post or profile). Apify's free $5/month covers a few thousand records.

Do I need a Threads account? No. Public posts, profiles, and replies are returned without authentication. If you want deep pagination on profile timelines or reply threads, you can optionally pass a threadsCookies JSON blob from your own logged-in browser session.

Is there a rate limit? Threads occasionally rate-limits aggressive scrapers, even residential IPs. The actor includes per-session backoff and proxy rotation, which keeps successful-run rates above 99% in normal usage. If you need very high volume, set maxItems per-query reasonably and stagger runs.

Will my account get banned if I use cookies? The actor sends cookies only to Threads' own endpoints (no third party) — same surface a normal logged-in browser hits. Risk is low for read-only scraping in typical volumes, but use a throwaway account if you need high volumes or are scraping sensitive segments.

Can I get posts older than 30 days? Threads search currently caps at roughly the last month for keyword searches without an authenticated session. With cookies you can paginate further back, but Threads' own UI doesn't expose deep historical search reliably.

What about Threads' own Replies-on-Instagram cross-posts? The actor returns Threads-native posts. Cross-posts from Instagram show up if they are visible in the Threads UI for the post / profile in question.

Can I run it on a schedule? Yes. Use Apify Schedules — cron-style triggers, daily / hourly / weekly runs, results push to your dataset on each fire.

Do you also have a flat-rate version? Yes — Threads Scraper monthly offers unlimited usage for $20/month. If you scrape more than ~10,000 posts/month, that plan is cheaper than pay-per-event.

Related actors

See all my actors at apify.com/pro100chok.

License

MIT — see LICENSE.


Built on top of the Threads Scraper Apify actor.