Skip to content

feat: add podcast-transcript-fetcher skill - #48

Merged
farizanjum merged 8 commits into
mainfrom
feat/podcast-transcript-fetcher
Jul 1, 2026
Merged

feat: add podcast-transcript-fetcher skill#48
farizanjum merged 8 commits into
mainfrom
feat/podcast-transcript-fetcher

Conversation

@farizanjum

@farizanjum farizanjum commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the podcast-transcript-fetcher skill to the OpenDirectory registry with Tier 2 (RSS+Groq Whisper) as the recommended approach.

What's included

  • scripts/get_transcript.py - Core CLI with 3 tiers (best-effort free sources, recommended RSS+Whisper, premium Taddy API)
  • scripts/podcasts.json - Registry of 5 podcasts with RSS feeds and transcript sources
  • references/podcasts.md - Detailed source documentation
  • Cross-podcast search (--search, --guest)
  • Batch transcription (--last N) with parallel workers (--parallel N)
  • Search-to-transcribe pipeline (--search --transcribe)
  • Parallel transcription via ThreadPoolExecutor (--parallel flag, default 1)
  • Multi-key GROQ API support (GROQ_API_KEY, GROQ_API_KEY_2, ...) with round-robin assignment
  • RSS caching for performance

Cleanup changes

  • Removed broken free sources: youtubetranscript.com (returned HTML), Spoken.md API (paid, no key), YouTube transcript handler (dead code)
  • Tier 2 (RSS+Groq Whisper) is now the recommended approach - fast, free, and works for all 5 podcasts
  • Tier 1 sources are best-effort (limited to GitHub archive for Lenny's)
  • Tier 3 (Taddy API) remains as commercial/premium option for large-scale needs
  • Updated README and SKILL.md to reflect the new tier recommendations

Fixes included

  • find_episode() returns None on miss instead of crashing with episodes[0]
  • Search pipeline deduplication uses correct 'link' field instead of 'url'
  • Search pipeline filename index uses idx+1 instead of len(picked) (prevents collision)
  • podcast_filter in search pipeline resolves via find_podcast() so aliases work
  • try_tier1 github_archive no longer early-returns
  • try_tier2 cleans up temp directory in finally block
  • _transcribe_groq() prints explicit error when GROQ_API_KEY is missing
  • podcasts.json description corrected to 'podcast-transcript-fetcher'

Testing

  • The Cheeky Pint flows directly to Tier 2 (RSS+Groq Whisper) - confirmed working
  • 20VC Tier 1 runs cleanly with no errors or warnings
  • All broken free source code paths removed - no orphan references
  • LSP diagnostics clean (only pre-existing Windows stdout.reconfigure warning)

Category

Research - helps find and retrieve podcast transcript content for analysis.

Copilot AI review requested due to automatic review settings July 1, 2026 08:16
@greptile-apps

greptile-apps Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a new podcast-transcript-fetcher skill to the OpenDirectory registry. The main changes are:

  • New skill metadata, README, SKILL.md, and environment example.
  • A Python CLI for podcast lookup, RSS search, batch transcription, Groq/local Whisper transcription, and Taddy fallback.
  • Podcast source documentation and a five-podcast registry with RSS feeds and transcript source metadata.
  • Repository and CLI registry updates to include the new skill.

Confidence Score: 5/5

The changes are additive and focused on a new skill entry, documentation, and a standalone CLI.

No blocking correctness or security issues were identified in the reviewed changes, and the modified registry and skill files appear consistent with the described behavior.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex registered the podcast-transcript-fetcher skill (version 1.1.0) under skills/podcast-transcript-fetcher, added its skill files, and updated the README to reflect a total registry count of 62, documenting five supported podcasts and including Tier 2, RSS, Groq, Whisper, and fetch/search/analyze terms.
  • T-Rex enabled and exercised the podcast-transcript-fetcher CLI entrypoint, populated the registry with five podcasts, and verified search and alias behaviors across multiple commands, including identical results with cached RSS files and guest alias behavior, with direct alias checks passing and find_episode returning None when not found.
  • T-Rex ran the transcription workflow harness and demonstrated the missing-key error path, along with deduplication and batch processing details for three Cheeky Pint episodes, frontmatter samples, parallel worker settings, and round-robin key usage, aided by a deterministic monkeypatch harness.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (7): Last reviewed commit: "fix(try_tier2): save fallback audio to p..." | Re-trigger Greptile

@greptile-apps

greptile-apps Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

T-Rex pricing update — T-Rex was free through June 2026. Effective July 1, 2026, T-Rex adds 2 credits on top of the standard 1-credit review (3 total). T-Rex settings

Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new OpenDirectory skill, podcast-transcript-fetcher, including a Python CLI plus supporting registry/docs, and registers it in the repo-wide skill registry + root README skill list.

Changes:

  • Introduces podcast-transcript-fetcher skill documentation and metadata (SKILL.md, README.md, package.json, .env.example).
  • Adds a transcript-fetching CLI (scripts/get_transcript.py) and a podcast registry (scripts/podcasts.json).
  • Updates repo registries (packages/cli/registry.json, root README.md) to include the new skill and increment skill count.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
skills/podcast-transcript-fetcher/SKILL.md Skill instructions and usage docs for the new transcript fetcher.
skills/podcast-transcript-fetcher/scripts/podcasts.json Podcast registry (RSS feeds + transcript source metadata).
skills/podcast-transcript-fetcher/scripts/get_transcript.py Core CLI implementing RSS search/batch + Groq/local transcription + Taddy fallback.
skills/podcast-transcript-fetcher/references/podcasts.md Reference documentation for source availability and access methods.
skills/podcast-transcript-fetcher/README.md Skill README with install/usage/how-it-works overview.
skills/podcast-transcript-fetcher/package.json Skill metadata (name/version/category/description).
skills/podcast-transcript-fetcher/.env.example Example environment variables for API keys.
README.md Updates global skill count and adds the new skill to the All Skills table.
packages/cli/registry.json Registers the new skill in the CLI registry.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
Comment thread skills/podcast-transcript-fetcher/SKILL.md
Comment thread skills/podcast-transcript-fetcher/SKILL.md Outdated
Comment thread skills/podcast-transcript-fetcher/references/podcasts.md
Comment thread skills/podcast-transcript-fetcher/references/podcasts.md
Comment thread skills/podcast-transcript-fetcher/scripts/podcasts.json Outdated
… and search pipeline fixes

- Add ThreadPoolExecutor parallel transcription via --parallel N flag
- Add _transcribe_job() helper for independent per-episode processing
- Add _get_api_keys() to scan GROQ_API_KEY, GROQ_API_KEY_2, ... from env
- Add _pick_key() round-robin key assignment across workers
- Refactor _transcribe_groq() to accept optional api_key parameter
- Rewrite batch_transcribe() and _run_search_pipeline() with concurrent.futures
- Fix search pipeline deduplication: ep.get('url') -> ep.get('link')
- Fix search pipeline filename index collision: idx + 1 instead of len(picked)
- Fix podcast_filter resolution in search pipeline via find_podcast()
- Fix try_tier1 github_archive fallthrough (no longer early-returns None)
- Fix try_tier2 temp dir cleanup with try/finally block
- Fix find_episode() returning episodes[0] on miss -> returns None
- Update .env.example with multi-key docs (GROQ_API_KEY_2, ...)
- Add parallel transcription section to SKILL.md, README.md, podcasts.md
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
…sources, Tier 2 recommended

- Remove broken free sources: youtubetranscript.com, Spoken.md API, YouTube transcript handler
- Tier 2 (RSS+Groq Whisper) is now the recommended approach - fast, free, and reliable
- Tier 1 sources are best-effort (limited to GitHub archive for Lenny's)
- Tier 3 Taddy API remains as commercial/premium option
- Update README, SKILL.md, and registry to reflect new tier structure
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py
…h, batch last-N ordering, search dedup

- find_podcast: use key=lambda for sort to avoid TypeError when scores
  are tied (Python 3 can't compare dicts as tiebreaker)
- batch_transcribe: sort episodes by pub_date descending before selecting
  the most recent N, so order doesn't vary with RSS feed ordering
- search pipeline dedup: track both GUID and link independently with
  separate seen sets to catch duplicates sharing the same URL
@farizanjum

farizanjum commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

Addressed all 3 Greptile issues:

  1. find_podcast crash: scored.sort() now uses key=lambda x: x[0] instead of bare tuple comparison, avoiding the TypeError when token overlap scores tie. Also returns None early for empty/blank queries.

  2. batch_transcribe last-N: episodes are now sorted by pub_date descending before selecting the most recent N, so ordering is consistent regardless of RSS feed ordering (newest-first or oldest-first).

  3. Search dedup: _run_search_pipeline now tracks both GUID and link independently with separate seen_guids and seen_links sets, catching duplicates that share the same URL but have different GUIDs.

All verified: syntax clean, LSP clean, search/alias/lookup working correctly.

…dir leak

When download_audio() failed, the function returned None before
entering the try/finally block, leaving tmp_dir on disk forever.
Moved the download call inside try so finally always cleans up.
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py
…doc references

batch_transcribe:
- Replace confusing indices/total-i math with enumerate(reversed(selected), start=1)
  per Greptile suggestion - no more off-by-one risk
- Add ISO 8601 date parsing fallback in _parse_pub_date so sort works with
  non-RFC-2822 date formats (T-Rex tests use ISO dates)

Cleanup stale references:
- podcasts.json: fix description from 'podcast-transcript' to 'podcast-transcript-fetcher'
- SKILL.md: remove youtubetranscript.com and Spoken.md references (sources removed)
- references/podcasts.md: remove dead Spoken.md API section from Cheeky Pint
Comment thread skills/podcast-transcript-fetcher/scripts/get_transcript.py Outdated
… cleanup

When no transcription backend is available (no GROQ_API_KEY and no
faster-whisper), the code printed 'Audio saved at: {tmp_path}' but
the finally block immediately deleted tmp_dir. Users could never
actually use the file. Now copies audio to output/{podcast}/audio/
before cleanup so the recovery message points to a real file.
@farizanjum
farizanjum merged commit 9e2f73c into main Jul 1, 2026
2 checks passed
@farizanjum
farizanjum deleted the feat/podcast-transcript-fetcher branch July 1, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants