🛰️ jobs-sensor

Multi-source hiring intelligence that reads what companies accidentally tell you in their job posts.

Every "Senior Shopify Developer wanted" is a company announcing its roadmap, its tech stack, and its pain, in public, for free. Most people don't read it. jobs-sensor does: it picks up that signal across job boards, applicant-tracking systems, and a paid historical API, then merges it into one clean, deduped picture of who's hiring for what.

The discipline that makes it worth running: it does the free reconnaissance first and only spends paid credits on companies that already look worth it. Map the terrain before you commit forces.

---

Data sources: free recon first, paid intel last

Source	Cost	What you get
JobSpy	FREE	Current listings from Indeed, LinkedIn, Glassdoor, ZipRecruiter
ATS endpoints	FREE	Direct JSON straight from Greenhouse, Lever, Ashby, SmartRecruiters, Workable
TheirStack	1 credit/job	5+ years of historical postings, NLP-extracted tech stack, firmographics

Architecture: a waterfall with a bouncer

Domain input (single or batch)
  |
Phase 1 -- FREE RECON  (costs nothing, always runs)
  |-- JobSpy: "Does this company have jobs? What kind?"
  +-- ATS: Direct query if the ATS platform is known
  |
Phase 2 -- QUALIFICATION GATES  (the bouncer)
  should_ping():      any engineering or ecommerce jobs found?
  should_deep_pull(): 2+ jobs in last 30 days AND ecommerce tech in stack?
  |
Phase 3 -- THEIRSTACK  (paid, on-demand, only if the bouncer nods)
  1-credit ping: firmographics + historical tech stack
  Deep pull:     full job descriptions + NLP tech extraction
  |
Persist to PostgreSQL (strict dedupe, multi-source merge)
  |
hiring_signals VIEW (aggregated per domain, actionability scoring)

The qualification gates are the whole point. Credits are money; this pipeline refuses to spend them on a company that free recon already proved is a dead end. No engineering jobs found? The gate stays shut and you spent $0.00 finding that out. Decisions over decoration.

Quick start

# Install
pip install -e ".[dev]"

# Set env
export DATABASE_URL='postgresql://...'        # Required
export THEIRSTACK_API_KEY='...'               # Optional (only for paid tiers)

# Apply schema
psql $DATABASE_URL -f schema/001_unified_schema.sql

# Free recon (no API key, no credits, no excuses)
python3 -m jobs_sensor.cli --domain example.com --recon-only

# Full waterfall: free recon -> qualify -> ping -> deep pull
python3 -m jobs_sensor.cli --domain example.com

# Give JobSpy a hand with the real company name
python3 -m jobs_sensor.cli --domain example.com --company-name "Example Co"

# Batch scan with a hard credit ceiling (paranoia encouraged)
python3 -m jobs_sensor.cli --batch domains.txt --limit 5 --credit-cap 50

# How many credits do I have left?
python3 -m jobs_sensor.cli --budget

# Dry run: estimate credits, call nothing
python3 -m jobs_sensor.cli --domain example.com --dry-run

Qualification gates, plainly

• Gate 1 (recon → ping): Did free recon surface any engineering or ecommerce jobs? If no, stop. For free.
• Gate 2 (ping → deep pull): Does the company have 2+ jobs in the last 30 days and ecommerce tech in its stack? Only then is a full, credit-burning deep pull worth it.

Data model highlights

Table	Purpose
job_signals	Canonical postings, one row per unique job, multi-source via a sources[] array
job_raw	Raw per-source payloads (keep the receipts)
company_signals	Firmographic cache per domain (derived, not gospel)
company_sources	ATS platform slug per domain
source_runs	Audit trail, every API call logged with credit tracking
hiring_signals (VIEW)	Aggregated intelligence with actionability scoring

Dedupe that survives job refreshes: the same role found by JobSpy and TheirStack merges into one row: sources: ['jobspy','theirstack'], source_count: 2. Fingerprint is MD5(domain | normalized_title | normalized_location), deliberately no date, so a reposted job doesn't masquerade as a new one.

Titles normalize carefully, too: C++, C#, and .NET survive; everything else gets scrubbed. (Ask us how we learned that the hard way.)

Swap the vertical

The whole thing keys off keyword lists in config/ecommerce.yaml. Hunting fintech signals instead of ecommerce? Swap the config, keep the machine.

License

FSL-1.1-ALv2 (Functional Source License). Use it, learn from it, contribute to it, just don't resell it as a competing product. Each version converts to Apache 2.0 after two years, because we're not monsters.