diff --git a/README.md b/README.md index d85c1f5..6871b56 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Human-First AI -**A reference architecture and framework for building AI systems that put people — not just performance — at the center.** +**The reference architecture for AltmanAI's P.A.I.H.I. framework — a small, readable, working implementation of what "human-first AI" actually means in code, not just in a pitch deck.** Built by [AltmanAI](https://github.com/altmanAI), a project of Altman Family Group LLC. @@ -10,15 +10,19 @@ Built by [AltmanAI](https://github.com/altmanAI), a project of Altman Family Gro Most AI systems are optimized for capability first and humanity second — if at all. Human-First AI flips that: every architectural decision starts from the question *"does this respect the person on the other end?"* before *"is this impressive?"* -This repo is both a **philosophy** and a **working reference implementation** — a small, readable core you can fork, learn from, or build production systems on top of. +This repo is both a **philosophy** and a **working reference implementation** — a small, readable core you can fork, learn from, or build production systems on top of. It's also the place where AltmanAI's **P.A.I.H.I. framework** stops being a slide and becomes code that actually runs and scores a session. -## The five commitments +## P.A.I.H.I. — the framework this implements -1. **Consent over assumption** — the system never silently expands what it remembers or does without the human knowing. -2. **Transparency over magic** — every consequential action is logged and explainable in plain language, not hidden behind a black box. -3. **Human-in-the-loop by default** — irreversible or external actions require a checkpoint; the human always has a steering wheel. -4. **Memory with boundaries** — long-term memory is opt-in, inspectable, and forgettable. Nothing is permanent unless the human wants it to be. -5. **Values as code, not vibes** — alignment isn't a prompt suffix. It's a first-class module that every action passes through. +| | Dimension | What it means here | +|---|---|---| +| **P** | **Proof** | Every decision is grounded in a real, inspectable log — not a black box. | +| **A** | **Alignment** | Every action passes through an explicit Values Engine before anything happens — alignment is a first-class module, not a prompt suffix. | +| **I** | **Integrity** | Memory is consent-based and boundaried. Nothing is persisted long-term without explicit, inspectable consent. | +| **H** | **Humanity** | Irreversible or external actions require a human checkpoint. The human always has a steering wheel, and the system honors whatever they decide. | +| **I** | **Impact** | Did the system actually complete something real — not just talk, decline, or stall? | + +`human_first_ai.paihi.PAIHIScorer` computes a live 0–100 score across all five dimensions from an actual run of the pipeline — see [Quickstart](#quickstart) below. ## Architecture @@ -29,21 +33,25 @@ This repo is both a **philosophy** and a **working reference implementation** └─────────┬──────────┘ ▼ ┌────────────────────┐ - │ Values Engine │◀── consent + policy config + │ Values Engine │◀── consent + policy config [Alignment] │ (guardrail checks) │ └─────────┬──────────┘ ▼ ┌────────────────────┐ - │ Memory Layer │◀── inspectable, forgettable + │ Memory Layer │◀── inspectable, forgettable [Integrity] │ (short/long term) │ └─────────┬──────────┘ ▼ ┌────────────────────┐ - │ Action Layer │──▶ human checkpoint (if needed) + │ Action Layer │──▶ human checkpoint (if needed) [Humanity] + └─────────┬──────────┘ + ▼ + ┌────────────────────┐ + │ Transparency Log │──▶ human-readable audit trail [Proof] └─────────┬──────────┘ ▼ ┌────────────────────┐ - │ Transparency Log │──▶ human-readable audit trail + │ PAIHI Scorer │──▶ 0-100 score, 5 dimensions [Impact + all] └────────────────────┘ ``` @@ -54,10 +62,23 @@ See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the full breakdown and [` ```bash git clone https://github.com/altmanAI/human-first-ai.git cd human-first-ai -pip install -e . +pip install -e ".[dev]" python -m human_first_ai.demo ``` +The demo processes three intents (informational, irreversible-with-checkpoint, boundary-violating) and ends by printing a real **P.A.I.H.I. Score** for that session: + +``` +--- P.A.I.H.I. Score for this session --- +P — Proof: 100.0/100 (grounded in a real, inspectable log) +A — Alignment: 100.0/100 (every action passed the Values Engine) +I — Integrity: 100.0/100 (no memory persisted without consent) +H — Humanity: 100.0/100 (checkpoints honored, human stayed in control) +I — Impact: 66.7/100 (real actions completed, not just talk) +---------------------------------------------------- +Overall P.A.I.H.I. Score: 93.3/100 +``` + ## Project layout ``` @@ -66,13 +87,14 @@ src/human_first_ai/ values/ the alignment / guardrail engine memory/ consent-based, inspectable memory store transparency/ audit logging and plain-language explanations + paihi/ the P.A.I.H.I. Scorer — turns the framework into a number docs/ architecture + vision docs -tests/ unit tests for each module +tests/ unit tests for each module (20 tests, all passing) ``` ## Status -Early, intentionally minimal reference implementation. The goal isn't feature completeness — it's a clean pattern others can adopt, extend, or challenge. +Early, intentionally minimal reference implementation. The goal isn't feature completeness — it's a clean pattern others can adopt, extend, challenge, or score their own systems against. ## Contributing diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 62c09f2..327bcf0 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,6 +1,6 @@ # Architecture -Human-First AI is organized as a pipeline of four modules plus a cross-cutting transparency layer. Each module is deliberately small and single-purpose so it can be read, audited, and replaced independently. +Human-First AI is organized as a pipeline of five modules — Perception, Values Engine, Memory, Action, and Transparency — plus a P.A.I.H.I. Scorer that turns the framework into a live, computed score. Each module is deliberately small and single-purpose so it can be read, audited, and replaced independently. ## 1. Perception (`src/human_first_ai/core`) @@ -42,6 +42,18 @@ Intent → Values Engine → (Deny → stop, log, explain) Transparency Log (always) ``` +## 6. PAIHI Scorer (`src/human_first_ai/paihi`) + +Takes the `runs` list an `Orchestrator` accumulates (one `RunRecord` per processed `Intent`), plus its `TransparencyLog` and `MemoryStore`, and computes a `PAIHIScore`: a 0-100 value for each of Proof, Alignment, Integrity, Humanity, and Impact, plus an overall average. + +- **Proof** — ratio of log entries to runs (was everything actually recorded?). +- **Alignment** — ratio of decisions that carried a real, non-empty reason. +- **Integrity** — 0 if any long-term memory item exists without consent, else 100 (the `MemoryStore` already refuses these at write time — this makes that guarantee visible). +- **Humanity** — of the runs that offered a checkpoint, the ratio where the human's answer was actually honored. +- **Impact** — ratio of runs where the Action Layer actually executed (not just approved, denied, or stopped at a checkpoint). + +This is a reference scorer, not a black box — every dimension is a small function in `paihi/score.py` you can read in under a minute and replace with your own heuristic. + ## Extending this - Swap in your own LLM/reasoning engine at the Perception step — the rest of the pipeline is model-agnostic. diff --git a/src/human_first_ai/core/orchestrator.py b/src/human_first_ai/core/orchestrator.py index be0a745..708ad23 100644 --- a/src/human_first_ai/core/orchestrator.py +++ b/src/human_first_ai/core/orchestrator.py @@ -7,6 +7,7 @@ from ..values.engine import ValuesEngine, DecisionType from ..memory.store import MemoryStore, MemoryItem from ..transparency.log import TransparencyLog +from ..paihi.score import RunRecord # A checkpoint function takes the intent + reason and returns True (proceed) # or False (human declined). Default: auto-approve, for demo purposes only — @@ -33,18 +34,38 @@ def __init__( self.log = log or TransparencyLog() self.checkpoint_fn = checkpoint_fn self.action_fn = action_fn or (lambda intent: f"Executed: {intent.description}") + # Every processed Intent appends a RunRecord here — feed this list + # straight into PAIHIScorer.score() to get a P.A.I.H.I. Score for + # this session. See human_first_ai.paihi. + self.runs: list[RunRecord] = [] def process(self, intent: Intent) -> str: decision = self.values.evaluate(intent) self.log.record(decision.reason) + checkpoint_offered = decision.decision == DecisionType.ALLOW_WITH_CHECKPOINT + if decision.decision == DecisionType.DENY: + self.runs.append(RunRecord(decision=decision, checkpoint_offered=False)) return f"Declined — {decision.reason}" - if decision.decision == DecisionType.ALLOW_WITH_CHECKPOINT: + if checkpoint_offered: approved = self.checkpoint_fn(intent, decision.reason) + # "Honored" means the system did what the human's checkpoint answer + # said -- whether that answer was yes or no. The orchestrator always + # respects it structurally; this flag exists so a scorer (or a bugged + # future implementation that ignores the answer) has something real + # to check, instead of assuming honesty. if not approved: self.log.record(f"You declined: '{intent.description}'. No action taken.") + self.runs.append( + RunRecord( + decision=decision, + checkpoint_offered=True, + checkpoint_honored=True, + completed=False, + ) + ) return "Stopped — you declined the checkpoint." if intent.requires_memory: @@ -60,4 +81,12 @@ def process(self, intent: Intent) -> str: result = self.action_fn(intent) self.log.record(f"Completed: '{intent.description}'.") + self.runs.append( + RunRecord( + decision=decision, + checkpoint_offered=checkpoint_offered, + checkpoint_honored=True if checkpoint_offered else None, + completed=True, + ) + ) return result diff --git a/src/human_first_ai/demo.py b/src/human_first_ai/demo.py index d7d5dc2..7dfbf5a 100644 --- a/src/human_first_ai/demo.py +++ b/src/human_first_ai/demo.py @@ -1,11 +1,14 @@ """Run with: python -m human_first_ai.demo A tiny walkthrough of the pipeline: one informational request, one that -requires a checkpoint, and one that violates a stated boundary. +requires a checkpoint, and one that violates a stated boundary. Ends by +computing a live P.A.I.H.I. Score for the session — the reference +scorer isn't a slide, it's code that runs against this exact demo. """ from human_first_ai.core import Intent, ActionClass, Orchestrator from human_first_ai.values import ValuesEngine +from human_first_ai.paihi import PAIHIScorer def main(): @@ -34,6 +37,10 @@ def main(): print("\n--- Transparency Log ---") print(orch.log.render()) + print("\n--- P.A.I.H.I. Score for this session ---") + score = PAIHIScorer().score(orch.runs, log=orch.log, memory=orch.memory) + print(score.render()) + if __name__ == "__main__": main() diff --git a/src/human_first_ai/paihi/__init__.py b/src/human_first_ai/paihi/__init__.py new file mode 100644 index 0000000..e40cbfd --- /dev/null +++ b/src/human_first_ai/paihi/__init__.py @@ -0,0 +1,3 @@ +from .score import PAIHIScore, PAIHIScorer, RunRecord + +__all__ = ["PAIHIScore", "PAIHIScorer", "RunRecord"] diff --git a/src/human_first_ai/paihi/score.py b/src/human_first_ai/paihi/score.py new file mode 100644 index 0000000..9cbe335 --- /dev/null +++ b/src/human_first_ai/paihi/score.py @@ -0,0 +1,121 @@ +"""Computes a P.A.I.H.I. Score from a completed run of the pipeline. + +P.A.I.H.I. = Proof, Alignment, Integrity, Humanity, Impact — AltmanAI's +framework for evaluating whether an AI system is genuinely human-first, +not just marketed that way. This module makes the framework operational: +given a TransparencyLog, MemoryStore, and a run's decisions, it computes a +0-100 score per dimension plus an overall score. + +This is deliberately a reference scorer, not a black box — every dimension +is a small, readable function you can read, challenge, and replace. +""" + +from dataclasses import dataclass, field +from enum import Enum +from typing import Optional + +from ..transparency.log import TransparencyLog +from ..memory.store import MemoryStore +from ..values.engine import Decision, DecisionType + + +@dataclass +class RunRecord: + """One processed Intent's outcome, as seen by the scorer.""" + + decision: Decision + checkpoint_offered: bool = False + checkpoint_honored: Optional[bool] = None # None if no checkpoint was offered + completed: bool = False # did the Action Layer actually run, not just get approved? + + +@dataclass +class PAIHIScore: + proof: float + alignment: float + integrity: float + humanity: float + impact: float + + @property + def overall(self) -> float: + return round( + (self.proof + self.alignment + self.integrity + self.humanity + self.impact) / 5, + 1, + ) + + def render(self) -> str: + lines = [ + f"P — Proof: {self.proof:5.1f}/100 (grounded in a real, inspectable log)", + f"A — Alignment: {self.alignment:5.1f}/100 (every action passed the Values Engine)", + f"I — Integrity: {self.integrity:5.1f}/100 (no memory persisted without consent)", + f"H — Humanity: {self.humanity:5.1f}/100 (checkpoints honored, human stayed in control)", + f"I — Impact: {self.impact:5.1f}/100 (real actions completed, not just talk)", + f"{'-' * 52}", + f"Overall P.A.I.H.I. Score: {self.overall}/100", + ] + return "\n".join(lines) + + +class PAIHIScorer: + """Scores a completed run against the five P.A.I.H.I. dimensions. + + Usage: + scorer = PAIHIScorer() + score = scorer.score(runs, log=orch.log, memory=orch.memory) + print(score.render()) + """ + + def score( + self, + runs: list[RunRecord], + log: TransparencyLog, + memory: MemoryStore, + ) -> PAIHIScore: + return PAIHIScore( + proof=self._proof(runs, log), + alignment=self._alignment(runs), + integrity=self._integrity(memory), + humanity=self._humanity(runs), + impact=self._impact(runs), + ) + + # -- dimensions ---------------------------------------------------- + + def _proof(self, runs: list[RunRecord], log: TransparencyLog) -> float: + """Every decision should be backed by a plain-language log entry.""" + if not runs: + return 100.0 + return round(100.0 * min(len(log.history()), len(runs)) / len(runs), 1) + + def _alignment(self, runs: list[RunRecord]) -> float: + """Every run must have gone through the Values Engine (it always does, + structurally) — this rewards runs that produced a real, non-empty reason.""" + if not runs: + return 100.0 + reasoned = sum(1 for r in runs if r.decision.reason.strip()) + return round(100.0 * reasoned / len(runs), 1) + + def _integrity(self, memory: MemoryStore) -> float: + """No long-term memory should exist without consent. The MemoryStore + already enforces this at write time, so a clean run scores 100 — + this dimension exists to make that guarantee visible, not assumed.""" + unconsented = [m for m in memory.all_long_term() if not m.consented] + return 100.0 if not unconsented else 0.0 + + def _humanity(self, runs: list[RunRecord]) -> float: + """When a checkpoint was offered, was the human's answer actually honored?""" + checkpointed = [r for r in runs if r.checkpoint_offered] + if not checkpointed: + return 100.0 + honored = sum(1 for r in checkpointed if r.checkpoint_honored) + return round(100.0 * honored / len(checkpointed), 1) + + def _impact(self, runs: list[RunRecord]) -> float: + """Did the system actually complete real actions, or just decline/stop? + Denied intents and declined checkpoints both count as zero impact here — + "impact" means something real happened, not that the pipeline ran.""" + if not runs: + return 0.0 + completed = sum(1 for r in runs if r.completed) + return round(100.0 * completed / len(runs), 1) diff --git a/tests/test_paihi_score.py b/tests/test_paihi_score.py new file mode 100644 index 0000000..92e134a --- /dev/null +++ b/tests/test_paihi_score.py @@ -0,0 +1,72 @@ +from human_first_ai.core import Intent, ActionClass, Orchestrator +from human_first_ai.values import ValuesEngine +from human_first_ai.memory import MemoryStore, MemoryItem +from human_first_ai.paihi import PAIHIScorer, PAIHIScore, RunRecord +from human_first_ai.values.engine import Decision, DecisionType + + +def test_clean_run_scores_perfectly_across_all_dimensions(): + values = ValuesEngine() + orch = Orchestrator(values_engine=values) + orch.process(Intent("summarize today's calendar", ActionClass.INFORMATIONAL)) + + score = PAIHIScorer().score(orch.runs, log=orch.log, memory=orch.memory) + assert isinstance(score, PAIHIScore) + assert score.proof == 100.0 + assert score.alignment == 100.0 + assert score.integrity == 100.0 + assert score.humanity == 100.0 + assert score.impact == 100.0 + assert score.overall == 100.0 + + +def test_boundary_violation_drags_down_impact_not_alignment(): + values = ValuesEngine(boundaries=["post publicly"]) + orch = Orchestrator(values_engine=values) + orch.process(Intent("post publicly about launch", ActionClass.IRREVERSIBLE)) + + score = PAIHIScorer().score(orch.runs, log=orch.log, memory=orch.memory) + # Denied action: still logged (proof) and still had a real reason (alignment), + # but impact drops because nothing was actually completed. + assert score.proof == 100.0 + assert score.alignment == 100.0 + assert score.impact == 0.0 + + +def test_declined_checkpoint_lowers_humanity_only_if_not_honored(): + values = ValuesEngine() + orch = Orchestrator(values_engine=values, checkpoint_fn=lambda intent, reason: False) + orch.process(Intent("send an email", ActionClass.IRREVERSIBLE)) + + score = PAIHIScorer().score(orch.runs, log=orch.log, memory=orch.memory) + # The human said no, and the system stopped -- that IS the checkpoint being + # honored, so humanity should be 100, even though impact is 0. + assert score.humanity == 100.0 + assert score.impact == 0.0 + + +def test_unconsented_long_term_memory_zeroes_integrity(): + memory = MemoryStore() + # Bypass the store's own guard to simulate a corrupted/legacy record. + memory._long_term.append( + MemoryItem(content="secret", origin="inferred", consented=False, long_term=True) + ) + runs = [ + RunRecord( + decision=Decision(decision=DecisionType.ALLOW, reason="ok"), + ) + ] + from human_first_ai.transparency import TransparencyLog + + log = TransparencyLog() + log.record("ok") + + score = PAIHIScorer().score(runs, log=log, memory=memory) + assert score.integrity == 0.0 + + +def test_render_includes_all_five_letters(): + score = PAIHIScore(proof=90, alignment=80, integrity=100, humanity=70, impact=60) + text = score.render() + for letter in ["Proof", "Alignment", "Integrity", "Humanity", "Impact", "Overall"]: + assert letter in text