Skip to content

docs: Add production operations runbook #42

Description

@Dahkenangnon

Problem

Create comprehensive operational documentation for production deployments in docs/runbooks/.

Proposed shape

1. Key Rotation (docs/runbooks/key-rotation.md)

  • Step-by-step JWKS rollover procedure
  • Key state transitions (pending → active → retiring → revoked)
  • Using /federation_historical_keys endpoint
  • Timeline recommendations
  • Rollback procedures

2. Trust Anchor Migration (docs/runbooks/trust-anchor-migration.md)

  • Updating authority_hints in Entity Configuration
  • Re-resolving trust chains
  • Coordination with superior authorities
  • Zero-downtime migration strategies

3. Cache Invalidation (docs/runbooks/cache-invalidation.md)

  • When to invalidate cached Entity Configurations
  • Trust Chain cache expiration policies
  • Cache warming strategies
  • Manual cache clearing procedures

4. Trust Mark Revocation (docs/runbooks/trust-mark-revocation.md)

  • Immediate revocation procedures
  • Updating Trust Mark Status endpoint
  • Notifying affected parties
  • Audit trail requirements

5. Monitoring and Metrics (docs/runbooks/monitoring.md)

  • Key metrics to track:
    • Trust Chain resolution success/failure rates
    • Cache hit rates
    • Entity Statement expiration events
    • Federation endpoint response times
  • Alerting thresholds
  • Dashboard examples

6. Failure Modes (docs/runbooks/failure-modes.md)

  • Authority unavailable scenarios
  • Expired Entity Statement handling
  • Clock skew detection and mitigation
  • Network timeout recovery
  • Conflicting trust path resolution

Format

Each runbook should include:

  • Problem description
  • Step-by-step procedures
  • Code examples where applicable
  • Troubleshooting guide
  • Related endpoints/APIs

Alternatives considered (optional)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions