Skip to content

johnsaurabh/NewsAggregator

Repository files navigation

AI Morning Brief

A serverless news aggregator that wakes up at 7 AM, reads 90+ tech articles, and emails you the 15 that actually matter — curated by Claude AI.

Python AWS Lambda Made with Claude AI License: MIT

┌──────────────────────────────────────────────────────────────────┐
│  YOUR AI MORNING BRIEF                    Wednesday, April 2     │
│  ─────────────────────────────────────────────────────    [15]   │
│  90 articles reviewed  ·  15 selected  ·  Curated by Claude AI  │
│                                                                  │
│  ◆ LLMs & foundation models          — 4 stories                │
│  ⬡ Cybersecurity                     — 3 stories                │
│  ▲ Developer tools & open source     — 3 stories                │
│  ● Groundbreaking research           — 3 stories                │
│  ★ Must-know products & launches     — 2 stories                │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ◆  LLMs & FOUNDATION MODELS                        4 stories   │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  [1]  Meta releases Llama 4 Scout: 109B MoE model        │  │
│  │       Meta's Llama 4 Scout uses a mixture-of-experts...   │  │
│  │       Meta AI Blog                    Read original →     │  │
│  ├────────────────────────────────────────────────────────────┤  │
│  │  [2]  Claude 3.7 Sonnet sets new SWE-bench record        │  │
│  │       Anthropic's latest model scores 70.3% on...        │  │
│  │       Anthropic                       Read original →     │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                  │
│  ⬡  CYBERSECURITY                                   3 stories   │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  [1]  Critical zero-day in OpenSSH (CVE-2025-XXXX)       │  │
│  │       A pre-auth remote code execution flaw affects...    │  │
│  │       Krebs on Security               Read original →     │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                  │
│  ─────────────────────────────────────────────────────────────  │
│  Delivered every morning at 7 AM  ·  Powered by Claude & Lambda │
└──────────────────────────────────────────────────────────────────┘

What this is

If you work in tech, you already know the problem: AI is moving faster than any human can track. New model releases, novel attack techniques, papers that shift how we think about reasoning — it's a firehose, and by 9 AM you have seventeen tabs open and have learned nothing useful. Most people's solution is to follow a dozen newsletters, each of which covers 20% of what you care about and 80% of what you don't.

This project takes a different approach. A Lambda function fires every morning before you wake up, pulls from 19 curated RSS feeds and 5 NewsAPI query streams, deduplicates them against a 7-day history in DynamoDB, and ships the whole pile to Claude Haiku. Claude acts as a senior tech editor: it discards crypto, gaming, PR fluff, and "researchers say AI could..." non-stories, then writes technically sharp, developer-focused summaries of what remains. You get one email. Clean layout. 15 stories, five categories, no noise.

What makes this different from a keyword filter or a simple RSS aggregator is the editorial layer. Claude doesn't just match strings — it evaluates relevance, filters duplicates from multiple sources, assesses technical novelty, and writes summaries that tell you both what happened and why it matters to someone who builds software. The entire prompt is a single, editable string, which means you can retune it for any reader profile in five minutes.


Example Output

Email View

Email

Categorized Stories

Categories

Full Digest

Full


How it works

EventBridge (7 AM UTC)
      │
      ▼
Lambda Orchestrator (Python 3.12, 256 MB, 10 min timeout)
      │
      ├──► RSS Feeds (19 sources: HN, HuggingFace, OpenAI, DeepMind,
      │              BAIR, Krebs, BleepingComputer, ArsTechnica + more)
      │
      └──► NewsAPI (5 targeted queries: LLMs, AI research,
                    cybersecurity, AI security, dev tools)
                    │
                    ▼
      DynamoDB  ── seen this URL in the last 7 days? skip it.
                    │
                    ▼
      Claude AI (Haiku) ── rank, filter, summarise up to 120 articles
                    │
                    ├──► S3 (archive daily digest as static HTML)
                    │
                    ▼
      AWS SES ── one clean email lands in your inbox

EventBridge triggers the Lambda on a cron schedule — no servers to maintain, no always-on costs.
feedparser + requests pull raw articles from 19 RSS feeds and 5 NewsAPI queries.
DynamoDB holds a rolling 7-day window of seen article URLs, with TTL-based auto-expiry.
Claude Haiku receives up to 120 articles and returns a ranked, categorised JSON digest.
S3 stores each day's HTML as a static archive, keyed by date.
SES delivers the formatted email from your verified sender address.


The AI filtering layer

The Claude prompt — the heart of the system — is written for a specific reader: a software developer who cares deeply about LLMs, cybersecurity, and tools that change how software is built. Claude acts as a senior tech editor, not a keyword filter.

What gets in: New LLM model releases and meaningful updates. AI research papers with genuine technical novelty. Real cybersecurity events — breaches, CVEs, novel attack techniques, significant patches. Developer tools that meaningfully change how software is built. Occasionally, a product launch so significant it captures the entire industry's attention.

What gets discarded immediately: Crypto and web3. Gaming. Social media drama. Pure business/finance stories with no technical angle. Clickbait. Opinion pieces with no new information. PR announcements dressed up as news. Duplicate coverage of the same story — Claude keeps the most technically detailed source.

What the summaries look like: Each selected article gets 3–4 sentences. Sentences 1–2 describe what happened with technical specificity — the model name, the CVE number, the technique. Sentence 3 explains why it matters to a developer building in or around AI and security. Sentence 4 covers business or broader impact, but only if it's genuinely significant.

The prompt is a single string constant (CLAUDE_PROMPT in lambda_function.py) that you can edit to match any reader profile — a security researcher, a product manager, a startup founder. Change the interests, change the filtering rules, change the summary style. It takes five minutes.


Tech stack

Component Technology Why
Orchestration AWS Lambda (Python 3.12) Zero idle cost, scales to zero, handles the 10-min run time comfortably
Scheduler Amazon EventBridge Native cron for Lambda, free tier covers this entirely
RSS fetching feedparser 6.0.11 Battle-tested, handles malformed feeds gracefully
News API NewsAPI.org Targeted keyword queries across thousands of sources
HTTP client requests 2.32.3 Standard, timeout-aware
Deduplication Amazon DynamoDB (on-demand) Sub-millisecond lookups, TTL auto-expiry, no servers
AI curation Claude Haiku (claude-haiku-4-5-20251001) Best cost/quality ratio for structured extraction; Haiku is fast and cheap
AI SDK anthropic 0.49.0 Official Python SDK
Email template Custom HTML (email_template.py) Inline-CSS, table-based layout for client compatibility
Email delivery Amazon SES $0 for <62K emails/month from Lambda; reliable deliverability
Archive Amazon S3 Static HTML stored per day, accessible as a simple web page
Secrets AWS Secrets Manager Single JSON secret, no env vars in Lambda config

Project structure

news-aggregator/
├── lambda_function.py    # Main Lambda handler: fetch, dedup, rank, archive, send
├── email_template.py     # HTML email builder — table-based, inline CSS
├── deploy.sh             # One-command deploy: packages, zips, creates or updates Lambda
├── setup-secrets.sh      # One-time secret setup in AWS Secrets Manager
├── requirements.txt      # Python dependencies (installed into the Lambda package)
└── .gitignore            # Excludes deployment artefacts, secrets, OS files

Setup & deployment

Prerequisites

  • AWS CLI configured (aws configure)
  • Python 3.12
  • An AWS account with billing enabled
  • A NewsAPI key (free at newsapi.org) — optional but recommended
  • An Anthropic API key

1. Clone the repo

git clone https://github.com/johnsaurabh/NewsAggregator.git
cd ai-morning-brief

2. Verify your sender email in SES

Before SES will send email, your sender address must be verified:

aws ses verify-email-identity --email-address you@yourdomain.com --region us-east-1

Check your inbox and click the verification link. If your AWS account is still in SES sandbox mode, you must also verify the recipient address the same way.

3. Create the DynamoDB table

aws dynamodb create-table \
  --table-name news-seen-articles \
  --attribute-definitions AttributeName=article_url,AttributeType=S \
  --key-schema AttributeName=article_url,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --region us-east-1

# Enable TTL so old entries expire automatically after 7 days
aws dynamodb update-time-to-live \
  --table-name news-seen-articles \
  --time-to-live-specification "Enabled=true,AttributeName=expires_at" \
  --region us-east-1

4. Store your secrets

The script prompts for each value — nothing is hardcoded:

bash setup-secrets.sh

When prompted, enter:

ANTHROPIC_API_KEY  →  your Anthropic key (sk-ant-...)
NEWSAPI_KEY        →  your NewsAPI key (or leave blank)
RECIPIENT_EMAIL    →  where the brief should land
SENDER_EMAIL       →  your SES-verified sender address
S3_BUCKET          →  bucket name for archiving (or leave blank)

The secrets are stored in AWS Secrets Manager under the key news-aggregator/keys as:

{
  "ANTHROPIC_API_KEY": "sk-ant-...",
  "NEWSAPI_KEY": "...",
  "RECIPIENT_EMAIL": "you@example.com",
  "SENDER_EMAIL": "brief@yourdomain.com",
  "S3_BUCKET": "your-bucket-name"
}

5. Deploy

bash deploy.sh

This installs dependencies into a package/ folder, zips everything, creates the IAM role and Lambda function (or updates them if they already exist), and sets the timeout to 10 minutes and memory to 256 MB.

6. Test it manually

aws lambda invoke \
  --function-name news-aggregator \
  --region us-east-1 \
  out.json && cat out.json

A successful run returns:

{"status": "success", "articles_reviewed": 90, "articles_sent": 15, "archive_url": "..."}

7. Schedule the daily run with EventBridge

The cron expression below fires at 7 AM UTC every day. Adjust to your timezone (e.g. cron(0 7 * * ? *) for UTC, cron(0 12 * * ? *) for UTC+5).

aws events put-rule \
  --name daily-news-trigger \
  --schedule-expression "cron(0 7 * * ? *)" \
  --state ENABLED \
  --region us-east-1

aws events put-targets \
  --rule daily-news-trigger \
  --targets "Id=NewsLambda,Arn=<YOUR_LAMBDA_ARN>" \
  --region us-east-1

Get your Lambda ARN:

aws lambda get-function --function-name news-aggregator --region us-east-1 \
  --query 'Configuration.FunctionArn' --output text

Grant EventBridge permission to invoke your function:

aws lambda add-permission \
  --function-name news-aggregator \
  --statement-id EventBridgeInvoke \
  --action lambda:InvokeFunction \
  --principal events.amazonaws.com \
  --source-arn <YOUR_RULE_ARN> \
  --region us-east-1

Customisation

The entire editorial layer is a single string: CLAUDE_PROMPT in lambda_function.py (line 128).

To retune the brief for a different reader, edit three sections of that prompt:

READER PROFILE — who this person is and what they care about. Change this and everything downstream changes.

FILTERING RULES — what to include and what to discard. Add or remove domains of interest. Raise or lower the bar for the "must-know" category.

CATEGORIES — the five topic buckets. You can rename them, merge them, or replace them entirely. The email template reads these dynamically so it adapts automatically.

Examples of profiles you could build in under 10 minutes:

  • A biotech researcher: swap LLMs for CRISPR/drug discovery, keep the "groundbreaking research" category
  • A startup founder: focus on product launches, funding rounds with technical substance, market-moving AI announcements
  • A security engineer: expand the cybersecurity categories, add red team research, remove the AI-only focus

Cost breakdown

Running once a day with typical usage:

Service Usage Monthly cost
AWS Lambda 30 invocations × ~60s × 256 MB ~$0.00 (free tier)
Amazon DynamoDB ~300 reads + ~450 writes/month ~$0.01
Amazon SES 30 emails/month $0.00 (free tier)
AWS Secrets Manager 1 secret $0.40
Claude Haiku ~25K input + 4K output tokens/day ~$1.10
Amazon S3 30 small HTML files ~$0.01
NewsAPI 5 requests/day (free plan) $0.00
EventBridge 1 scheduled rule $0.00
Total ~$1.52/month

The dominant cost is Claude. Using claude-haiku-4-5-20251001 rather than Sonnet or Opus keeps this under $2/month while delivering sharp editorial quality.


Roadmap

  • Web dashboard to browse the daily archive
  • Slack / Telegram delivery option
  • Multiple reader profiles (different prompts, different recipients)
  • Weekly digest mode (one email per week, higher bar for inclusion)
  • Feedback loop — thumbs up/down on stories to tune the prompt over time

Contributing & License

MIT licensed. Fork it, tune the prompt, point it at different feeds, send it to Slack instead of email — it's designed to be modified.

If you build something interesting on top of this, open an issue and share it.

About

Serverless AI pipeline — aggregates 19 sources, LLM-filters for signal, delivers curated intelligence daily by email. ~$1.52/month on AWS.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors