Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,19 @@ permissions:

jobs:
checks:
name: lint + type + test (py${{ matrix.python-version }})
name: lint + type + test (py3.14)
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# 3.14 is the target runtime; 3.13 is the supported floor
# (docs/spikes/python-3.14-compat.md).
python-version: ["3.13", "3.14"]
steps:
- uses: actions/checkout@v5

# Single version by design. The workspace pins requires-python = ">=3.14"
# (#82), so a 3.13 leg could not resolve the environment even to report a
# compatibility signal — the floor and the matrix have to agree. 3.14 is
# also what both role images run, so CI and production share an interpreter.
- uses: astral-sh/setup-uv@v6
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
python-version: "3.14"

- name: Sync workspace (locked)
run: uv sync --locked
Expand Down
12 changes: 10 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,24 @@ COMPOSE ?= docker compose -f deploy/compose/docker-compose.yml --env-file .env
# Engine replica count for `make scale`.
N ?= 2

.PHONY: help sync lint format type test check up down destroy migrate seed logs ps scale init-env secrets
.PHONY: help sync hooks lint format type test check up down destroy migrate seed logs ps scale init-env secrets

help: ## List the available targets
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-12s\033[0m %s\n", $$1, $$2}'

# ─── Workspace ────────────────────────────────────────────────────────────────

sync: ## Install/refresh the workspace environment
sync: ## Install/refresh the workspace environment (and install git hooks)
uv sync
$(MAKE) hooks

# Hooks are per-clone: .pre-commit-config.yaml is committed, .git/hooks is not.
# Without this the gitleaks hook — the one that stops a secret reaching history in
# the first place — is silently absent from every fresh clone. CI runs the same
# checks either way, but by then the secret is already committed.
hooks: ## Install the pre-commit hooks into this clone
uv run pre-commit install --install-hooks

lint: ## Ruff lint + formatting check
uv run ruff check .
Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,18 @@ See [`ARCHITECTURE.md`](./ARCHITECTURE.md) for the full picture and
Needs [uv](https://docs.astral.sh/uv/) and Docker.

```bash
make sync # install the workspace, and the git hooks that keep secrets out of history
make init-env # .env from .env.example, with a master key and sealed pepper generated
make up # build, start api + engine + postgres + redis, wait for health, migrate
make seed # optional: a disabled demo source to click around
make scale N=3 # more engine replicas
make down # stop; `make destroy` also drops the data volume
```

Run `make sync` in a fresh clone before anything else: git hooks live in `.git/hooks`, which is
not part of a checkout, so until it runs the pre-commit gitleaks hook is not installed and a
secret can be committed locally. (`make hooks` installs them on their own.)

The console is then at <http://localhost:8000/> and the OpenAPI docs at `/docs`. Signing in needs
OIDC configured in `.env`; the first person to sign in lands as a viewer unless
`ICEBERG_BOOTSTRAP_ADMIN_SUBJECT` names them.
Expand Down
2 changes: 1 addition & 1 deletion apps/api/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "iceberg-api"
version = "0.1.0"
description = "FastAPI control plane: REST API, HTMX UI, scheduler, notifications. Sole writer of record."
requires-python = ">=3.13"
requires-python = ">=3.14"
dependencies = [
# Only the api role owns the schema and runs migrations (docs/deployment.md).
"alembic>=1.18.5",
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/iceberg_api/auth/dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def get_db_session() -> Iterator[Session]:
yield from session_dependency()


def get_secret_store(settings: "SettingsDep") -> SecretStore:
def get_secret_store(settings: SettingsDep) -> SecretStore:
"""The secret store as a dependency (ADR 0007), overridable in tests."""
return build_secret_store(settings)

Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/iceberg_api/auth/oidc.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ class ProviderMetadata:
jwks_uri: str

@classmethod
def from_document(cls, document: dict[str, Any]) -> "ProviderMetadata":
def from_document(cls, document: dict[str, Any]) -> ProviderMetadata:
try:
return cls(
issuer=document["issuer"],
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/iceberg_api/findings/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ def _trimmed(cls, value: str) -> str:
return trimmed

@model_validator(mode="after")
def _expiry_in_the_future(self) -> "SuppressionCreate":
def _expiry_in_the_future(self) -> SuppressionCreate:
if self.expires_at is None:
return self
# A naive timestamp is read as UTC, the same assumption every other
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/iceberg_api/pagination.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def encode(self) -> str:
return base64.urlsafe_b64encode(payload.encode()).decode().rstrip("=")

@classmethod
def decode(cls, raw: str) -> "Cursor":
def decode(cls, raw: str) -> Cursor:
try:
padded = raw + "=" * (-len(raw) % 4)
payload: dict[str, Any] = json.loads(base64.urlsafe_b64decode(padded))
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/iceberg_api/sources/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ class ScheduleUpdate(BaseModel):
enabled: bool | None = None

@model_validator(mode="after")
def _valid_cron(self) -> "ScheduleUpdate":
def _valid_cron(self) -> ScheduleUpdate:
if self.cron is not None and not croniter.is_valid(self.cron.strip()):
raise ValueError("cron must be a valid 5-field expression, e.g. '0 3 * * *'")
return self
Expand Down
2 changes: 1 addition & 1 deletion apps/api/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def app_fixture(
db_engine: Engine,
session: Session,
secret_store: EnvKeyBackend,
dispatcher: "RecordingDispatcher",
dispatcher: RecordingDispatcher,
) -> FastAPI:
"""The real app, with settings, the database, and the provider overridden."""
# background=False: the tests drive the scheduler and reclaim directly.
Expand Down
2 changes: 1 addition & 1 deletion apps/engine/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "iceberg-engine"
version = "0.1.0"
description = "Dramatiq scanner worker: connector fetch, detection, redaction. No DB credentials."
requires-python = ">=3.13"
requires-python = ">=3.14"
dependencies = [
"iceberg-core",
"iceberg-detect",
Expand Down
4 changes: 2 additions & 2 deletions apps/engine/src/iceberg_engine/api_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ def _request(
self.sleep(delay)
try:
return self._send(method, url, json)
except (LeaseRefused, AuthenticationFailed):
except LeaseRefused, AuthenticationFailed:
# The API's considered answer. Asking again changes nothing.
raise
except RetryableApiError as exc:
Expand Down Expand Up @@ -276,6 +276,6 @@ def _parse_cancelled_ids(raw: object) -> list[uuid.UUID]:
for value in raw:
try:
parsed.append(uuid.UUID(str(value)))
except (ValueError, AttributeError):
except ValueError, AttributeError:
logger.warning("heartbeat_bad_cancelled_id", value=str(value)[:64])
return parsed
2 changes: 1 addition & 1 deletion apps/engine/src/iceberg_engine/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ def _submit(
lease.task_id, report.as_submission(lease.idempotency_key, pack.version)
)
return True
except (LeaseRefused, AuthenticationFailed):
except LeaseRefused, AuthenticationFailed:
# On the results route a 403/409 means the lease is gone, not a bad token:
# it worked at lease time moments ago. The task is no longer ours to report.
log.info("scan_task_lease_lost_before_report", status=report.status)
Expand Down
19 changes: 17 additions & 2 deletions docs/spikes/python-3.14-compat.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ metadata for **all 34 packages in the lock**:
`greenlet`, `markupsafe`, `pydantic-core`).
- No package's `Requires-Python` excludes 3.14.

Live execution on 3.14 is delegated to the **CI matrix** (issue #19), which runs the full
lint/type/test suite on both 3.13 and 3.14 on every PR — so 3.14 is continuously proven, not
Live execution on 3.14 was delegated to the **CI matrix** (issue #19), which ran the full
lint/type/test suite on both 3.13 and 3.14 on every PR — so 3.14 was continuously proven, not
assumed.

## Decision
Expand All @@ -42,6 +42,21 @@ assumed.
once the M0 containers epic lands a dev image with 3.14 baked in.
- CI treats **3.14 as primary** and 3.13 as the compatibility leg.

## Resolved — floor raised (issue #82, 2026-07-31)

The condition set above was met: #80 landed both role images on `python:3.14-slim`, and its CI
run was the first in this repo to actually execute the 3.14 leg end to end. So 3.14 is now the
runtime *and* a proven test target, and the fallback exists to protect an environment that no
longer has to be catered for.

- The workspace and all five members pin `requires-python = ">=3.14"`; ruff targets `py314`
and mypy checks against `3.14`.
- The 3.13 CI leg is **dropped, not kept as a compatibility check**. With the floor at `>=3.14`
a 3.13 leg cannot resolve the environment, so it could only ever fail — the floor and the
matrix have to agree. CI now runs a single 3.14 job, the same interpreter the images ship.

The consequence, stated plainly: contributors and images on 3.13 are no longer supported.

## Reproducing

The spike project is not committed (scratch work). To re-run: `uv init`, add the dependencies
Expand Down
2 changes: 1 addition & 1 deletion packages/connectors/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "iceberg-connectors"
version = "0.1.0"
description = "Connector interface and source connectors (Confluence MVP; Jira/SMB later)."
requires-python = ">=3.13"
requires-python = ">=3.14"
dependencies = [
"iceberg-core",
"structlog>=25.5.0",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ def _body(
return {**page, **detail}, body_text(detail)

def _page_units(
self, text: str, page_id: str, context: "_PageContext", outcome: FetchOutcome
self, text: str, page_id: str, context: _PageContext, outcome: FetchOutcome
) -> Iterator[ContentUnit]:
"""The page body, if it has any text."""
if not text:
Expand All @@ -208,7 +208,7 @@ def _comment_units(
self,
client: ConfluenceClient,
page_id: str,
context: "_PageContext",
context: _PageContext,
outcome: FetchOutcome,
) -> Iterator[ContentUnit]:
"""Footer and inline comments, each its own unit.
Expand Down Expand Up @@ -243,9 +243,9 @@ def _attachment_units(
self,
client: ConfluenceClient,
page_id: str,
context: "_PageContext",
context: _PageContext,
outcome: FetchOutcome,
sandbox: "_LazySandbox",
sandbox: _LazySandbox,
) -> Iterator[ContentUnit]:
"""Text-extractable attachments, downloaded and parsed in a child process.

Expand Down Expand Up @@ -377,7 +377,7 @@ class _SpaceContext:
name: Any
site: str

def for_page(self, page: dict[str, Any], page_id: str) -> "_PageContext":
def for_page(self, page: dict[str, Any], page_id: str) -> _PageContext:
webui = _webui(page)
return _PageContext(
space=self,
Expand Down Expand Up @@ -459,7 +459,7 @@ def _email(connection: dict[str, Any]) -> str | None:
def _as_int(value: Any) -> int | None:
try:
return int(value)
except (TypeError, ValueError):
except TypeError, ValueError:
return None


Expand Down
2 changes: 1 addition & 1 deletion packages/connectors/src/iceberg_connectors/protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def as_payload(self) -> dict[str, Any]:
return {"label": self.label, "params": self.params}

@classmethod
def from_payload(cls, payload: dict[str, Any]) -> "TaskSpec":
def from_payload(cls, payload: dict[str, Any]) -> TaskSpec:
"""Rebuild a spec from a lease. Tolerates a bare params mapping.

A spec written by an older engine may have no ``label``; that is cosmetic,
Expand Down
2 changes: 1 addition & 1 deletion packages/connectors/src/iceberg_connectors/sandbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ def run[ResultT](
def close(self) -> None:
self._discard_pool()

def __enter__(self) -> "ExtractionSandbox":
def __enter__(self) -> ExtractionSandbox:
return self

def __exit__(self, *_exc: object) -> None:
Expand Down
2 changes: 1 addition & 1 deletion packages/connectors/tests/confluence_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ def _authorized(self, request: httpx2.Request) -> bool:
return False
try:
decoded = base64.b64decode(header.removeprefix("Basic ")).decode()
except (ValueError, UnicodeDecodeError):
except ValueError, UnicodeDecodeError:
return False
return decoded == f"{EMAIL}:{API_TOKEN}"

Expand Down
2 changes: 1 addition & 1 deletion packages/core/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "iceberg-core"
version = "0.1.0"
description = "Shared models, config, secret-store, fingerprinting, and redaction utilities."
requires-python = ">=3.13"
requires-python = ">=3.14"
dependencies = [
"cryptography>=49.0.0",
"prometheus-client>=0.25.0",
Expand Down
4 changes: 2 additions & 2 deletions packages/core/src/iceberg_core/redaction.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def __post_init__(self) -> None:
if self.start < 0 or self.end <= self.start:
raise ValueError(f"invalid span: [{self.start}, {self.end})")

def overlaps(self, other: "Span") -> bool:
def overlaps(self, other: Span) -> bool:
return self.start < other.end and other.start < self.end


Expand All @@ -78,7 +78,7 @@ class RedactionPolicy:
min_length_to_reveal: int = 16

@classmethod
def from_strategy_name(cls, name: str, **overrides: int) -> "RedactionPolicy":
def from_strategy_name(cls, name: str, **overrides: int) -> RedactionPolicy:
"""Build a policy from a rule pack's ``redaction:`` value.

Unknown names raise: a typo in a rule pack must fail at load time, not
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/iceberg_core/secrets/env_key.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ def __init__(self, master_key: bytes, *, pepper_ref: str | None = None) -> None:
self._pepper_ref = pepper_ref

@classmethod
def from_settings(cls, settings: SecretStoreSettings) -> "EnvKeyBackend":
def from_settings(cls, settings: SecretStoreSettings) -> EnvKeyBackend:
if settings.master_key is None:
raise SecretStoreConfigError(f"ICEBERG_MASTER_KEY is not set; {_GENERATE_HINT}")
return cls(
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/iceberg_core/suppression.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def as_payload(self) -> dict[str, str]:
return {"id": str(self.id), "scope": self.scope.value, "pattern": self.pattern}

@classmethod
def from_payload(cls, payload: dict[str, str]) -> "SuppressionRule":
def from_payload(cls, payload: dict[str, str]) -> SuppressionRule:
"""Rebuild a rule an engine received in its lease.

Raises rather than skipping an unparseable entry: an engine that quietly
Expand Down
2 changes: 1 addition & 1 deletion packages/detect/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "iceberg-detect"
version = "0.1.0"
description = "Detection engine and versioned rule packs (regex + entropy + keyword proximity)."
requires-python = ">=3.13"
requires-python = ">=3.14"
dependencies = [
"iceberg-core",
# Rule packs are YAML shipped inside the engine image (ADR 0008).
Expand Down
2 changes: 1 addition & 1 deletion packages/detect/src/iceberg_detect/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ def detect(
snippet = redact_snippet(
text, candidate.span, policy=candidate.rule.redaction, other_spans=neighbours
)
except (RedactionError, ValueError):
except RedactionError, ValueError:
# Redaction failing is a bug in the masking logic, and the only safe
# response is to drop the finding: a raw snippet must never be the
# fallback (ADR 0004).
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "icebergsst"
version = "0.1.0"
description = "IcebergSST — secret scanning for non-git enterprise sources (workspace root)"
requires-python = ">=3.13"
requires-python = ">=3.14"
dependencies = [
"iceberg-api",
"iceberg-engine",
Expand Down Expand Up @@ -39,7 +39,7 @@ dev = [

[tool.ruff]
line-length = 100
target-version = "py313"
target-version = "py314"

[tool.ruff.lint]
select = [
Expand All @@ -63,7 +63,7 @@ select = [

[tool.mypy]
strict = true
python_version = "3.13"
python_version = "3.14"
files = ["apps", "packages"]
warn_unreachable = true
pretty = true
Expand Down
Loading
Loading