Add public read-only Data API (v1) over the AISafety.com directories#417
Open
NoahLloyd wants to merge 14 commits into
Open
Add public read-only Data API (v1) over the AISafety.com directories#417NoahLloyd wants to merge 14 commits into
NoahLloyd wants to merge 14 commits into
Conversation
Initial index for sitewide LLM assistant research. Frames the assistant as a directory navigator (not a general AI safety tutor), since the site is structured as curated resource lists rather than editorial content. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Maps the ~10 Airtable tables and hand-written pages that would feed a RAG pipeline. Proposes one-document-per-Airtable-record granularity (~900-1200 docs total) with structured metadata for filtering, and calls out edge cases: Publish?/Hide? filters, magic rows on /map, inactive orgs, job staleness, and the 4x4 donation-guide tree. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Frames the assistant as a directory navigator: routes, recommends, walks the donation tree, and disambiguates listings. Hand-off rules for conceptual AI safety questions (route to aisafety.info), explicit no-go list (career coaching, news, application drafting), and a behaviour checklist for evals. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Compares pure-prompt, RAG-vector, RAG-hybrid (tools + vector rerank), and fine-tuning. Recommends hybrid: tool calls expose the same filters the resource pages already have (type, location, experience, etc.), with vector similarity ranking within the filtered subset. Inference on Vercel Edge; index refresh hooks into existing check-rebuild cron. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Compares floating widget, dedicated /ask page, and inline-on-resource- page filter parsing. Recommends /ask page as the primary surface (reuses existing design tokens — text-field, button-primary, container-default — and the FeaturedCard pattern for citations), with floating widget deferred and inline-filter U3 as a v1.5 follow-up. Covers naming, state (client-only via useChat), streaming/tool-call UX, accessibility, and rate limiting. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Concrete picks for model, embeddings, vector store, runtime, SDK, rate limit, observability, and index build pipeline. Also flags open questions that need decisions before implementation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pulls together the analysis in 01-05 into a single readable brief with a v0 -> v1 -> v1.5 -> v2 plan, cost ballpark, risks table, decision log, and explicit open questions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
06-evaluation.md spec: 30-query eval set, hybrid programmatic + LLM-as-judge scoring, harness layout, launch gate at >=80% pass-rate per row. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
07-prompt-injection.md enumerates realistic attack scenarios against the Airtable-backed corpus (the main vector) and the chat surface, then specs layered mitigations: system-prompt clauses, tool-result sandboxing, pre-index sanitisation, output filtering, tool-loop budget, rate limit, and moderation queue. Includes a mitigation matrix and explicit non-goals. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
08-cost-model.md walks through per-turn token math, the three traffic regimes (quiet/busy/peak), prompt-caching impact, optimisation levers in order of impact, and concrete spending-cap numbers. Headline: chat-time output tokens dominate everything else; a quiet-to-busy first-month ramp projects ~$370. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Exposes the existing Airtable-backed directories as 10 JSON endpoints under
/api/v1 (communities, organizations, events, jobs, funding, courses, advisors,
media-channels, founder-resources, projects), plus a self-describing index, a
generated OpenAPI spec, and a /developers docs page. Responses use a wrapped
{ data, meta } envelope with query-param filtering, free-text q search, open
CORS, absolute image URLs, and CC-BY-4.0 licensing. Reuses the existing
src/lib/data getX() functions and adds a new events normalizer (events was
previously an Airtable iframe embed); no shared-code changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Author
|
Sorry about the messy commit list. Please ignore the AI-research / project-idea commits, all the changes are reverted, but still accidentally included the commits as part of the PR |
…blic API - Remove `sort` from the communities and founder-resources endpoints (and their data shapes); ordering still comes from the Airtable query sort. - Strip `email` from the projects endpoint at the route layer so it can't be bulk-harvested; the /projects page still renders it for human visitors. - Remove code/doc text that named internal Airtable fields (Internal notes, submitter email, etc.) so the source no longer advertises them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Member
|
This PR was reviewed to confirm no non-public Airtable fields are exposed. The design is sound — each endpoint explicitly selects its output fields, so internal fields such as
Type-check, lint, and a production build all pass. 🤖 Written by Claude Code |
Member
|
LGTM now |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a public, read-only JSON Data API (v1) over the existing Airtable-backed AISafety.com directories: 10 collection endpoints under
/api/v1plus a self-describing index, a generated OpenAPI spec, and an on-brand/developersdocs page. Every endpoint reuses the existingsrc/lib/datafunctions and returns a wrapped{ data, meta }envelope with query-param filtering, free-textqsearch, open CORS, absolute image URLs, and CC-BY-4.0 licensing. A neweventsnormalizer is added (events was previously an Airtable iframe embed), and there are no changes to shared data-layer code. Verified with type-check, lint, and a full production build.🤖 Generated with Claude Code