diff --git a/.claude/agent-memory/cdisc-frontend-dev/project_foundation.md b/.claude/agent-memory/cdisc-frontend-dev/project_foundation.md index ed075af..b091d7f 100644 --- a/.claude/agent-memory/cdisc-frontend-dev/project_foundation.md +++ b/.claude/agent-memory/cdisc-frontend-dev/project_foundation.md @@ -11,6 +11,7 @@ The app uses a Flask application factory pattern in `app.py`. `db` and `migrate` - `ingestion` → `/ingestion` - `bc` → `/bc` - `ncit` → `/ncit` +- `loinc` → `/loinc` - `specializations` → `/specializations` - `governance` → `/governance` - `audit` → `/audit` @@ -22,8 +23,10 @@ Route files live in `routes/.py`. Each exports `bp = Blueprint('', _ - `specialization.py` — `DatasetSpecialization` - `governance.py` — `GovernanceRecord` - `audit.py` — `AuditLog` +- `ingestion.py` — `IngestionRecord` (staging rows for the upload queue) -`models/__init__.py` imports `db` from `app`. Models import `db` directly from `app` (not from `models`). +All models import `db` from `extensions` (`from extensions import db`) — never +from `app`. `extensions.py` holds the shared `db`/`migrate` singletons. Database: SQLite at `sqlite:///cdisc_curation.db` by default. Tables are auto-created via `db.create_all()` inside `create_app()`. @@ -34,14 +37,16 @@ Virtual environment: `.venv/` — use `.venv/bin/pip` and `.venv/bin/python` for **Services** (`services/`): - `ingestion.py` — `parse_xlsx`, `parse_csv`, `parse_json`, `deduplicate`, `map_fields`, `validate_bc` - `ncit_api.py` — `NCItApiClient` with `search_concept(term, size)`, `get_concept(ncit_code)`, `get_preferred_name(ncit_code)` +- `loinc_api.py` — `LoincApiClient.search(term)` against NLM Clinical Tables (optional Basic Auth) - `export.py` — `export_json(bc_list)`, `export_xlsx(bc_list)` (returns BytesIO), `export_odm_xml(bc_list)` (returns str) -- `cdisc_api.py` — `CDISCApiClient` (stub for CDISC Library API) +- `cdisc_api.py` — `CDISCApiClient`, a full CDISC Library REST client with a stale-tolerant in-memory cache (not a stub) **Route implementations are complete** (as of 2026-03-27). All 24 URL rules register successfully. Key route decisions: - `bc.export` is a static path `/bc/export` — must be defined before `bc.detail` (`/bc/`) to avoid Flask treating "export" as a bc_id. - `governance` blueprint has no `/` route — board is at `/governance/board`. - `ncit.index` at `/ncit/` redirects to `ncit.mapping`. -- Ingestion queue stored in Flask `session`, capped at 100 records. +- Ingestion queue stored in the `IngestionRecord` DB table (NOT the Flask + session — session storage was replaced to avoid cookie overflow). - `_save_decs()` in `bc.py` does a full delete-then-reinsert of DECs on every save. **Why:** Routes were stubs; wired up 2026-03-27 to connect templates to real model queries and service calls. diff --git a/.claude/agents/cdisc-concept-explorer.md b/.claude/agents/cdisc-concept-explorer.md new file mode 100644 index 0000000..f7fc28f --- /dev/null +++ b/.claude/agents/cdisc-concept-explorer.md @@ -0,0 +1,67 @@ +--- +name: cdisc-concept-explorer +description: "Use this agent when work in the cdisc-concept-curation project needs live CDISC Library data: searching Biomedical Concepts by name or category, comparing a locally curated BC against the published Library version, finding dataset specializations for a BC, or recommending which published concept a curated draft should align with.\n\n\nContext: The user is curating a draft BC and wants to check for an existing published equivalent.\nuser: \"Is there already a published CDISC BC for 'Systolic Blood Pressure' that our draft duplicates?\"\nassistant: \"I'll use the cdisc-concept-explorer agent to search the CDISC Library API and compare candidates against the draft.\"\n\nDuplicate detection against the live Library is this agent's core job in the curation workflow.\n\n\n\n\nContext: The user wants dataset specializations for a BC in the review queue.\nuser: \"What SDTM dataset specializations exist for C64796?\"\nassistant: \"Let me launch the cdisc-concept-explorer agent to query the Library's specializations endpoint for C64796.\"\n\nSpecialization lookup requires live API access — use the cdisc-concept-explorer agent.\n\n" +tools: Read, Bash, WebFetch, ToolSearch, Write, Edit +model: sonnet +memory: project +--- + +You are an expert CDISC standards specialist with deep knowledge of the CDISC +Biomedical Concepts (BC) library, controlled terminology, and clinical trial +data standards. You support the **cdisc-concept-curation** project — a Flask +app where draft BCs move through ingest → SME review → governance approval → +publish. + +## Your Core Mission +Help users search, evaluate, and compare CDISC Biomedical Concepts from the +live CDISC Library API, especially to (a) detect duplicates between locally +curated drafts and published concepts, (b) enrich drafts with authoritative +metadata, and (c) find dataset specializations tied to a BC. + +## Environment & API Access +- Base URL: `https://library.cdisc.org/api/cosmos/v2` (the app's + `services/cdisc_api.py` uses `https://api.library.cdisc.org/api/cosmos/v2` — + both hosts serve the same API) +- **Primary auth header**: `api-key: ` using `CDISC_API_KEY` env var +- Fallback header (only if `CDISC_API_KEY` is unset): + `Ocp-Apim-Subscription-Key` using `CDISC_SUBSCRIPTION_KEY` +- Key endpoints (details, response shapes, and quirks are documented in your + memory file `reference_api_endpoints.md` — consult it first): + - GET /mdr/bc/biomedicalconcepts — search all BCs + - GET /mdr/bc/biomedicalconcepts/{id} — one BC's full detail + - GET /mdr/bc/categories — list categories + - GET /mdr/specializations/datasetspecializations?biomedicalconcept={id} + +## Project Integration Points +- Local drafts live in the `biomedical_concepts` table + (`models/bc.py: BiomedicalConcept`, PK = NCIt C-code `bc_id`). +- The app's own Library client is `services/cdisc_api.py: CDISCApiClient` + (`get_biomedical_concepts()`, `get_bc(id)`, `check_duplicate(short_name)`). + Prefer reading through it when reasoning about app behavior; use curl for + ad-hoc exploration. +- The `/bc/library/` route renders a published BC for comparison. + +## CRITICAL: API-First Policy +- You MUST attempt the CDISC Library API before using any other source. +- If neither `CDISC_API_KEY` nor `CDISC_SUBSCRIPTION_KEY` is set → STOP and + tell the user to set one. Never return training-data C-codes. +- If the API errors → STOP and report the HTTP status. Do not substitute + training-data values. +- Training knowledge MAY be used only to suggest search terms, never for BC + identifiers or C-codes. + +## Output Format +### Search Results — candidate BCs with key details +### Recommendation — primary pick with rationale, alternatives with when-to-prefer +### Curation Notes — duplicate risk vs local drafts, metadata worth copying into the draft, deprecation flags + +## Quality Standards +- Never guess a BC identifier — verify against the API +- Prefer official CDISC terminology over informal names +- Flag deprecated concepts and newer package versions +- If no exact match exists, say so and recommend the closest fit + +**Update your agent memory** as you discover BC mappings, API response +quirks, category coverage, and duplicate-detection patterns in this project. +`reference_api_endpoints.md` in your memory directory already documents +verified endpoint shapes — keep it current. diff --git a/.claude/agents/cdisc-frontend-dev.md b/.claude/agents/cdisc-frontend-dev.md index 5a870c3..a99125a 100644 --- a/.claude/agents/cdisc-frontend-dev.md +++ b/.claude/agents/cdisc-frontend-dev.md @@ -23,6 +23,9 @@ Your sole responsibility is front-end development for the CDISC biomedical conce - Modify server-side data pipelines or API logic beyond what is needed to wire up a front-end view - Make architectural decisions about data storage, authentication systems, or infrastructure +**Brand Compliance (mandatory)** +Before creating or updating any UI — HTML templates, CSS, or visual/JS-driven presentation — invoke the `cdisc-brand-guidelines` skill and apply it: CDISC Blue (`#134678`), Purple (`#553278`), Orange (`#D57E00`), Green (`#286040`) as primary colors, Arial as the font, and the documented heading/table/accent conventions. This applies to net-new pages and edits to existing ones alike. Do not introduce ad hoc colors, fonts, or styling that conflict with the brand guidelines. + **Technical Standards & Best Practices** 1. **HTML**: Write semantic HTML5. Use appropriate elements (`
`, `
`, `
`, ` {% endif %} + + + +{% endblock %} + +{% block extra_js %} + {% endblock %} diff --git a/templates/dashboard.html b/templates/dashboard.html index 4964918..b06c72b 100644 --- a/templates/dashboard.html +++ b/templates/dashboard.html @@ -206,9 +206,17 @@

{% endif %} - {% if api_bc_count > 50 %} -

Showing 50 of {{ api_bc_count }} BCs.

- {% endif %} +
+ +
+ + +
+
@@ -256,9 +264,17 @@

{% endif %} - {% if api_spec_count > 50 %} -

Showing 50 of {{ api_spec_count }} specializations.

- {% endif %} +
+ +
+ + +
+
@@ -324,21 +340,53 @@

Quick Actions

{% endblock %} diff --git a/templates/governance.html b/templates/governance.html index 8af0866..85fb318 100644 --- a/templates/governance.html +++ b/templates/governance.html @@ -27,7 +27,7 @@

Governance Workflow Board

{{ columns.published | length if columns else 0 }}
-
Published
+
Ready to Publish
@@ -62,12 +62,7 @@

Governance Workflow Board

aria-label="Advance {{ bc.short_name }} to SME Review"> Advance - + {% endfor %} @@ -165,7 +160,7 @@

Governance Workflow Board

- Published + Ready to Publish {{ columns.published | length if columns else 0 }} @@ -181,11 +176,11 @@

Governance Workflow Board

{% if bc.ncit_code %}{{ bc.ncit_code }}{% endif %} {% if bc.package_date %} · Published {{ bc.package_date }}{% endif %}
- Published + Ready to Publish
{% endfor %} {% else %} -
No published BCs yet.
+
No BCs ready for publication yet.
{% endif %} @@ -207,7 +202,42 @@

Governance Actions

View Full Audit Trail + + + + + + + {% endblock %} diff --git a/templates/library_bc_detail.html b/templates/library_bc_detail.html index 9108639..ff26470 100644 --- a/templates/library_bc_detail.html +++ b/templates/library_bc_detail.html @@ -53,28 +53,54 @@

Definition

{{ bc.definition | default('No definition available.') }}
- - {% if bc.coding %} + + {% set loinc_coding = bc.coding | selectattr('systemName', 'equalto', 'LOINC') | list if bc.coding else [] %}
-

External Coding

- {% for c in bc.coding %} -
+

LOINC

+ {% if loinc_coding %} + {% set lc = loinc_coding[0] %} +
- -
{{ c.system | default('—') }}
+ +
{{ lc.code | default('—') }}
-
- -
{{ c.systemName | default('—') }}
+
+ +
{{ loinc_data.get('LONG_COMMON_NAME') or lc.system | default('—') }}
-
- -
{{ c.code | default('—') }}
+
+ {% if loinc_data %} + {% set loinc_fields = [ + ('LONG_COMMON_NAME', 'Long Common Name'), + ('SHORTNAME', 'Short Name'), + ('COMPONENT', 'Component'), + ('PROPERTY', 'Property'), + ('METHOD_TYP', 'Method Type'), + ('units', 'Units'), + ('datatype', 'Data Type'), + ('CONSUMER_NAME', 'Consumer Name'), + ('RELATEDNAMES2', 'Related Names'), + ('AnswerLists', 'Answer Lists'), + ('isCopyrighted', 'Is Copyrighted'), + ('containsCopyrighted', 'Contains Copyrighted'), + ('EXTERNAL_COPYRIGHT_NOTICE', 'Copyright Notice'), + ('EXTERNAL_COPYRIGHT_LINK', 'Copyright Link'), + ] %} +
+ {% for key, label in loinc_fields %} + {% if loinc_data.get(key) %} +
+ {{ label }} + {{ loinc_data[key] }}
+ {% endif %} + {% endfor %}
- {% endfor %} + {% endif %} + {% else %} +

No LOINC code assigned in the CDISC Library for this concept.

+ {% endif %}
- {% endif %}
@@ -88,7 +114,7 @@

Metadata

{% endif %} {% if bc.href %}
- +
{{ bc.href }}
{% endif %} diff --git a/templates/specializations.html b/templates/specializations.html index ff8ebc4..7de78de 100644 --- a/templates/specializations.html +++ b/templates/specializations.html @@ -21,7 +21,7 @@

Dataset Specializations

- {% if editing_spec %}Edit Specialization{% else %}New Specialization{% endif %} + {% if edit_spec %}Edit Specialization{% else %}New Specialization{% endif %}

@@ -30,19 +30,43 @@

novalidate> {{ form.hidden_tag() if form else '' }} +
+
+ + +
+
+
+
+
@@ -53,12 +77,12 @@

+ {% if not edit_spec or edit_spec.domain == 'SDTM' %}checked{% endif %}>
+ {% if edit_spec and edit_spec.domain == 'CDASH' %}checked{% endif %}>
@@ -68,7 +92,7 @@

@@ -86,9 +110,9 @@

Variables

- - {% if editing_spec and editing_spec.variables %} - {% for var in editing_spec.variables %} + + {% if edit_spec and edit_spec.variables %} + {% for var in edit_spec.variables %} All Specializations
- Edit -
{{ form.hidden_tag() if form else '' }} - -