From 5afe12d66a2f4694b0a6fa6671fd8963e3b37cd2 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Tue, 14 Apr 2026 16:25:23 +0200 Subject: [PATCH 01/16] Audit Logging OTEP Signed-off-by: Hilmar Falkenberg --- oteps/0267-audit-logging.md | 287 ++++++++++++++++++++++++++++++++++++ 1 file changed, 287 insertions(+) create mode 100644 oteps/0267-audit-logging.md diff --git a/oteps/0267-audit-logging.md b/oteps/0267-audit-logging.md new file mode 100644 index 00000000000..ccc902ffd5d --- /dev/null +++ b/oteps/0267-audit-logging.md @@ -0,0 +1,287 @@ +# Audit Logging Signal + +Introduce a dedicated Audit Logging signal to OpenTelemetry that guarantees +lossless, tamper-evident delivery of security-relevant events and satisfies +compliance requirements such as ISO 27001. + + + + +## Motivation + +### The problem with re-using the existing Log signal + +OpenTelemetry Logs are designed for general-purpose observability. They share +the same pipeline as traces and metrics, which means they are subject to: + +- **Sampling** – processors and exporters may drop records for performance + reasons. +- **Back-pressure shedding** – SDK queues may overflow and silently discard + records. +- **Transformation** – processors may modify, redact, or aggregate records + before export. + +These behaviours are intentional and desirable for observability logs. They are +**incompatible** with audit logging, where every record MUST be delivered +exactly once to the designated audit sink, in order, without modification. + +Furthermore, compliance frameworks such as ISO 27001, SOC 2, PCI-DSS and HIPAA +require: + +- Guaranteed delivery to a tamper-protected sink. +- Proof of record integrity (i.e. the record was not altered in transit). +- Separation of audit records from operational logs. +- Clock synchronisation across all emitting services. + +None of these guarantees are made today by any OTel signal. + +### Use cases + +| Actor | Audit event | Compliance driver | +|-----------------|-------------------------------|------------------------------| +| End user | Successful / failed login | ISO 27001 A.8.15, SOC 2 CC6 | +| End user | Access to sensitive data | GDPR Art. 30, HIPAA §164.312 | +| Operator | Configuration change | ISO 27001 A.8.15 | +| Service account | Elevated-privilege action | ISO 27001 A.5.18 | +| Service | Outbound call to external API | PCI-DSS Req. 10 | +| Runtime | Antivirus / IDs rule fired | ISO 27001 A.8.16 | + +### Why a new signal and not a new Log feature flag? + +A feature flag (e.g. `audit=true` on a `LogRecord`) still shares the existing +pipeline and therefore inherits all the behaviours listed above. A dedicated +signal allows: + +- A purpose-built SDK pipeline with no sampling, no transformation, and + at-least-once delivery semantics. +- A distinct OTLP service name / endpoint so the audit sink can be isolated + from the observability backend. +- A different data model with mandatory integrity metadata. + +## Explanation + +### Signal overview + +The new signal is called **Audit Logging**. It is modelled structurally after +the existing Log signal and reuses the `LogRecord` data model as its transport +layer, adding a small set of mandatory fields. + +``` +Application code + │ + ▼ + AuditLogger.emit(AuditRecord) + │ + ▼ + AuditProvider (SDK-level, no sampling, no dropping) + │ └── AttributeProcessor (enrich only, never redact/drop) + ▼ + AuditExporter ──► Audit sink (e.g. OpenSearch SIEM, Splunk, S3 WORM) + │ + └── returns SHA-256 receipt hash ◄──────┐ + │ + emit() return value (delivery proof) +``` + +### AuditProvider + +`AuditProvider` is the entry point, analogous to `LoggerProvider`. It: + +- Holds a reference to a configured `AuditExporter`. +- Provides named `AuditLogger` instances. +- MUST NOT expose a sampling configuration option. +- MUST guarantee at-least-once delivery: records are queued durably (or + synchronously) until the exporter acknowledges receipt. + +### AuditLogger + +`AuditLogger` is obtained from `AuditProvider` and is scoped to an +instrumentation scope (library name + version). It exposes a single method: + +``` +receipt = AuditLogger.emit(AuditRecord) → AuditReceipt +``` + +`emit` is **synchronous by default**: it MUST block until the exporter +acknowledges the record, or return an error. An asynchronous variant MAY be +provided, but MUST still guarantee delivery and MUST NOT silently discard on +failure. + +### AuditRecord data model + +`AuditRecord` is quite similar to `LogRecord` with the following mandatory fields: + +| Field | Type | Description | +|---------------------|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------| +| `Timestamp` | `fixed64` | Nanoseconds since UNIX epoch (UTC). MUST be set. `<= ObservedTimestamp <= now(UTC)` | +| `ObservedTimestamp` | `fixed64` | Nanoseconds since UNIX epoch when the SDK observed the event. Used for clock skew detection. MUST be set. `>= Timestamp <= now(UTC)` | +| `EventName` | `string` | Semantic name of the audit event (e.g. `user.login.success`). MUST be set. | +| `Actor` | `AnyValue` | Identity that performed the action (user ID, service account, …). MUST be set. | +| `ActorType` | `enum` | `USER`, `SERVICE`, `SYSTEM`. MUST be set. | +| `Outcome` | `enum` | `SUCCESS`, `FAILURE`, `UNKNOWN`. MUST be set. | +| `Resource` | `AnyValue` | The target resource (file, endpoint, table, …). SHOULD be set. | +| `Action` | `string` | Verb describing what was done (e.g. `READ`, `WRITE`, `DELETE`, `LOGIN`). MUST be set. | +| `SourceIP` | `string` | Source network address, if applicable. | +| `Body` | `AnyValue` | Free-form text or protobuf with additional details about the event. | +| `Attributes` | `map` | Arbitrary key-value pairs for additional context. | +| `Signature` | `bytes` | Digital signature of the audit record for integrity verification. | +| `Algorithm` | `string` | Algorithm used for the digital signature (e.g., `RS256`, `ES256`). | +| `Certificate` | `bytes` | Digital public certificate used for the signature verification. | + +### AuditReceipt + +`AuditReceipt` is returned by `emit` once the record has been acknowledged by +the exporter/sink: + +| Field | Type | Description | +|-----------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------| +| `RecordId` | `string` | Unique, stable identifier assigned by the sink. | +| `IntegrityHash` | `string` | SHA-256 of the canonical serialization of the `AuditRecord`, computed by the sink after successful write. Used for integrity verification. | +| `SinkTimestamp` | `fixed64` | Nanoseconds since UNIX epoch when the sink persisted the record. | + +The `IntegrityHash` is computed by the **sink**, not the SDK, so it reflects the +record as actually persisted – not as emitted. The emitting service MAY store +the receipt for its own tamper-detection purposes. The emitting service SHOULD compute the same hash locally and compare it to the receipt to verify integrity. A mismatch indicates the record was altered in transit or by the sink, which should trigger an alert (e.g. log an error, raise an incident). + +### OTLP transport + +Audit records are exported using a dedicated OTLP endpoint (distinct from the +standard `/v1/logs` endpoint), tentatively `/v1/audit`. This allows: + +- Network-level isolation (firewall rules, separate credentials). +- Independent QoS policies (no back-pressure shedding). +- Clear separation in the sink's data model. + +The payload is the standard `ExportLogsServiceRequest` proto with an additional +`audit` boolean flag in the `ResourceLogs` message. + +## Internal details + +### No sampling, no dropping + +The SDK MUST reject any attempt to configure a sampler on the `AuditProvider`. +The SDK MUST use an unbounded (or disk-backed) queue. If the in-memory queue is +full, the SDK MUST either block the caller or persist the record to a local +durable buffer rather than silently drop it. + +### Processor restrictions + +Only **additive** processors (enrich, tag, forward) are permitted in the audit +pipeline. Processors that delete attributes, filter records, or aggregate +records MUST be rejected at configuration time. + +### Clock synchronisation + +The SDK SHOULD warn at startup if the system clock is not synchronised (e.g. +NTP offset > 1 second). The `ObservedTimestamp` field provides a second +timestamp from the SDK's perspective, which allows the sink to detect skewed +emitters. + +### Interaction with the existing Log signal + +Audit records MUST NOT be routed through the standard `LoggerProvider` +pipeline. The two signals are independent. An instrumentation library MAY emit +both an audit record (for compliance) and a regular log record (for +observability) for the same event. + +### Failure handling + +If the exporter cannot reach the sink: + +1. The SDK MUST buffer the record (in memory or on disk, depending on + configuration). +2. The SDK MUST retry with exponential back-off. +3. If the buffer is exhausted (disk full, etc.) the SDK MUST surface a + hard error to the caller rather than silently drop the record. +4. The SDK MUST expose a metric (`audit.records.dropped`) that counts + any records lost due to unrecoverable errors. + +## Trade-offs and mitigations + +| Trade-off | Mitigation | +|------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------| +| Synchronous `emit` adds latency to the calling thread | An async variant with a durable local queue mitigates this; the queue size and flush interval are configurable. | +| A separate pipeline increases SDK complexity | The audit pipeline deliberately reuses `LogRecord` as its wire format, so OTLP exporters can be reused with minimal changes. | +| SHA-256 receipt requires a round-trip to the sink | The receipt is optional for callers that do not need proof-of-delivery; a fire-and-forget async mode MAY omit it. | +| Disk-backed queue introduces a dependency on localStorage | This is opt-in; the default is an in-memory queue with blocking back-pressure. | + +## Prior art and alternatives + +### Alternative 1 – Existing Log signal with a dedicated exporter + +Tag `LogRecord`s with `audit=true` and route them to a dedicated exporter via a +filtering processor. **Rejected**: still subject to SDK queue overflow and +requires consumers to filter records; does not provide integrity receipts. + +### Alternative 2 – Out-of-band audit library + +Bypass OTel entirely and use a dedicated audit library (e.g. OWASP Enterprise +Security API Logger). **Rejected**: loses OTel's context propagation, semantic +conventions, and OTLP transport; introduces a second observability stack. + +### Alternative 3 – W3C Audit Vocabulary / CEF + +Map to an existing audit format. **Not rejected**: the `AuditRecord` semantic +conventions SHOULD map cleanly to Common Event Format (CEF) and the W3C audit +vocabulary. This OTEP does not prescribe a wire format beyond OTLP. + +### Prior art in OTel + +- [OTEP 0092 – Logs Vision](logs/0092-logs-vision.md): defines the overall + logging architecture on which this proposal builds. +- [OTEP 0097 – Log Data Model](logs/0097-log-data-model.md): `AuditRecord` + extends the `LogRecord` data model defined here. +- [OTEP 0202 – Events and Logs API](0202-events-and-logs-api.md): the + `AuditLogger`/`AuditProvider` hierarchy mirrors the `Logger`/`LoggerProvider` + hierarchy introduced here. + +## Open questions + +1. **OTLP endpoint path** – Should audit records share `/v1/logs` with a + distinguishing flag, or use a dedicated `/v1/audit` path? A dedicated + path makes routing and access control simpler but requires changes to all + OTLP receivers. + +2. **Receipt hash computation** – Should the hash be computed by the SDK + (guaranteeing what was sent) or by the sink (guaranteeing what was stored)? + The current proposal says sink, but this requires a synchronous + acknowledgement that may not be supported by all exporters. + +3. **Mandatory vs. optional receipt** – Should `emit` always block for a + receipt, or should this be configurable? Blocking guarantees delivery proof + but impacts throughput. + +4. **Retention metadata** – Should the `AuditRecord` carry a `RetentionPolicy` + field (e.g. `keep_for: 7_years`) so the sink can enforce retention + automatically? This is application-domain logic that may be out of scope for + OTel. + +5. **Redaction / PII** – ISO 27001 requires protecting logs from unauthorised + access but does not require storing PII in plain text. How should the signal + interact with attribute-level encryption or redaction at the SDK layer? + +6. **Multi-sink fan-out** – Some compliance frameworks require logs to be + written to two independent sinks simultaneously. Should this be a first-class + configuration option of `AuditProvider`? + +## Prototypes + +No prototype exists yet. A reference implementation is planned for: + +- **Java** – extending the existing `io.opentelemetry:opentelemetry-sdk-logs` + module. +- **Go** – extending `go.opentelemetry.io/otel/sdk/log`. + +## Future possibilities + +- **Semantic conventions** for common audit event names + (`user.login.success`, `resource.delete`, `config.change`, ...) following the + same pattern as existing OTel semantic conventions. +- **Integrity chain** – each record carries the hash of the previous record + (blockchain-style) so gaps in the audit trail are detectable without a + centralized sequence counter. +- **Declarative configuration** support so that audit pipeline parameters + (sink URL, queue depth, retry policy, retention) can be specified in the + [OTel configuration file schema](https://github.com/open-telemetry/opentelemetry-configuration). +- **Collector support** – a dedicated `auditlog` receiver/exporter pair in + the OpenTelemetry Collector that enforces the no-drop guarantee end-to-end. From 6e9be5a806ca8820f606a205ae776aa37ecd90c5 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Wed, 15 Apr 2026 15:44:05 +0200 Subject: [PATCH 02/16] No partial success, no Instrumentation Scope Durability at the sink is out of scope Signed-off-by: Hilmar Falkenberg --- oteps/0267-audit-logging.md | 42 +++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/oteps/0267-audit-logging.md b/oteps/0267-audit-logging.md index ccc902ffd5d..788af778cfe 100644 --- a/oteps/0267-audit-logging.md +++ b/oteps/0267-audit-logging.md @@ -155,6 +155,48 @@ standard `/v1/logs` endpoint), tentatively `/v1/audit`. This allows: The payload is the standard `ExportLogsServiceRequest` proto with an additional `audit` boolean flag in the `ResourceLogs` message. +#### No partial success + +The OTLP receiver MUST NOT respond with a `partial_success` for an audit log +export request. If the receiver cannot process one or more audit records in a +batch, it MUST reject the **entire** batch and return an error. The exporter +MUST treat any `partial_success` response as a hard failure, retain all +records, and retry the full batch. + +This constraint ensures that no audit record can be silently lost by a receiver +that processes only part of a batch and acknowledges the rest. + +#### Instrumentation Scope is not applicable + +The `InstrumentationScope` field present in the standard `ScopeLogs` message +has no meaning for audit logging. Audit records are not emitted by +instrumentation libraries – they are emitted by application or system code +acting on behalf of an actor. Exporters MUST leave the `InstrumentationScope` +field empty (or populated with only the SDK name/version as a technical marker) +and receivers MUST NOT use it for routing, filtering, or processing audit +records. + +The three OTLP envelope layers that **do** apply to audit logging are: + +| OTLP layer | Role in audit logging | +|--------------------|----------------------------------------------------------------| +| `Resource` | Identifies the emitting service/host – reused without change. | +| `LogRecord` (body) | Carries the `AuditRecord` payload. | +| `Attributes` | Key-value context (user ID, IP, outcome, …) – reused as-is. | + +#### Durability at the sink is out of scope + +This OTEP does not prescribe how the audit sink (e.g. OpenSearch, Splunk, +S3 WORM, a SIEM) stores, retains, or protects records once they have been +successfully acknowledged. Requirements such as write-once/read-many storage, +retention periods, or geographic replication are deployment concerns that depend +on the specific sink technology and the applicable compliance framework. + +The boundary of the OTel audit signal ends at the moment the sink acknowledges +receipt (and, optionally, returns an `AuditReceipt`). Everything beyond that +point – including long-term durability, tamper-proof storage, and retention +enforcement – is outside the scope of this specification. + ## Internal details ### No sampling, no dropping From 5dd875e69147b2e5819d27d8bb1018b9c7d60f83 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Thu, 23 Apr 2026 17:17:38 +0200 Subject: [PATCH 03/16] audit spec 1 Signed-off-by: Hilmar Falkenberg --- specification/audit/README.md | 158 ++++++++++ specification/audit/api.md | 205 +++++++++++++ specification/audit/data-model.md | 484 ++++++++++++++++++++++++++++++ specification/audit/sdk.md | 466 ++++++++++++++++++++++++++++ 4 files changed, 1313 insertions(+) create mode 100644 specification/audit/README.md create mode 100644 specification/audit/api.md create mode 100644 specification/audit/data-model.md create mode 100644 specification/audit/sdk.md diff --git a/specification/audit/README.md b/specification/audit/README.md new file mode 100644 index 00000000000..24b12485dbc --- /dev/null +++ b/specification/audit/README.md @@ -0,0 +1,158 @@ + + +# OpenTelemetry Audit Logging + +**Status**: [Development](../document-status.md) + +
+Table of Contents + + + +- [OpenTelemetry Audit Logging](#opentelemetry-audit-logging) + - [Introduction](#introduction) + - [Why a Dedicated Audit Logging Signal?](#why-a-dedicated-audit-logging-signal) + - [Compliance Use Cases](#compliance-use-cases) + - [Signal Overview](#signal-overview) + - [Relationship to the Log Signal](#relationship-to-the-log-signal) + - [Specifications](#specifications) + - [References](#references) + + + +
+ +## Introduction + +Audit logging is a compliance-critical signal that records +security-relevant events for regulatory and accountability purposes. +Compliance frameworks such as ISO 27001, SOC 2, PCI-DSS, and HIPAA +require that audit records be delivered without loss, modification, or +sampling to a tamper-protected audit sink. + +The OpenTelemetry Audit Logging signal provides a purpose-built pipeline +with the delivery and integrity guarantees that compliance frameworks +demand. It extends OpenTelemetry's unified observability approach to +the domain of security and compliance auditing. + +## Why a Dedicated Audit Logging Signal? + +The existing OpenTelemetry [Log signal](../logs/README.md) is designed +for general-purpose observability. Its pipeline is intentionally subject +to behaviours that are incompatible with audit logging: + +- **Sampling** – processors and exporters may drop records for + performance reasons. +- **Back-pressure shedding** – SDK queues may overflow and silently + discard records. +- **Transformation** – processors may modify, redact, or aggregate + records before export. + +For audit logging, every record MUST be delivered to the designated audit +sink without modification or loss. A feature flag on a `LogRecord` cannot +fulfil this requirement because it still shares the existing pipeline and +inherits all the above behaviours. + +A dedicated Audit Logging signal provides: + +- A purpose-built SDK pipeline with no sampling, no transformation, and + at-least-once delivery semantics. +- A distinct OTLP endpoint (`/v1/audit`) so the audit sink can be + isolated from the observability backend, enabling independent access + control and QoS policies. +- A dedicated data model with mandatory integrity fields. +- Clear separation of audit records from operational logs, as required + by ISO 27001 Annex A and similar frameworks. + +## Compliance Use Cases + +The following table illustrates representative audit events and the +compliance drivers that motivate them: + +| Actor | Audit event | Compliance driver | +|-----------------|-------------------------------|------------------------------| +| End user | Successful / failed login | ISO 27001 A.8.15, SOC 2 CC6 | +| End user | Access to sensitive data | GDPR Art. 30, HIPAA §164.312 | +| Operator | Configuration change | ISO 27001 A.8.15 | +| Service account | Elevated-privilege action | ISO 27001 A.5.18 | +| Service | Outbound call to external API | PCI-DSS Req. 10 | +| Runtime | Antivirus / IDS rule fired | ISO 27001 A.8.16 | + +## Signal Overview + +The Audit Logging signal is modelled structurally after the Log signal. +It reuses the OTLP `LogRecord` as its wire-format transport and adds a +small set of mandatory fields that satisfy compliance requirements. + +``` +Application code + │ + ▼ + AuditLogger.emit(AuditRecord) ──► returns AuditReceipt + │ + ▼ + AuditProvider (no sampling, no dropping) + │ └── AuditRecordProcessor (enrich only – never redact or drop) + ▼ + AuditExporter ──► Audit sink (OpenSearch, Splunk, S3 WORM, SIEM, …) + │ + └── returns AuditReceipt + (RecordId + IntegrityHash + SinkTimestamp) +``` + +The principal API components are: + +- **AuditProvider** – the entry point of the API. It provides named + `AuditLogger` instances and MUST NOT expose a sampler configuration + option. See [Audit Logging API](./api.md#auditprovider). +- **AuditLogger** – emits `AuditRecord`s and returns an `AuditReceipt` + once the sink has acknowledged the record. + See [Audit Logging API](./api.md#auditlogger). + +The principal data components are: + +- **AuditRecord** – the audit event payload containing mandatory fields + for actor, action, outcome, timestamps, and optional integrity + metadata. See [Audit Record Data Model](./data-model.md#auditrecord). +- **AuditReceipt** – proof-of-delivery returned by the sink, containing + a unique record identifier and a SHA-256 integrity hash. + See [Audit Record Data Model](./data-model.md#auditreceipt). + +## Relationship to the Log Signal + +The Audit Logging signal is **independent** of the Log signal. Audit +records MUST NOT be routed through the standard `LoggerProvider` +pipeline. + +An application or instrumentation library MAY emit both an `AuditRecord` +(for compliance) and a regular `LogRecord` (for observability) for the +same event. The two records travel through independent pipelines to +independent backends. + +The OTLP wire format reuses the `LogRecord` protobuf message as the +payload carrier. The following OTLP envelope layers apply: + +| OTLP layer | Role in audit logging | +|------------------------|--------------------------------------------------------| +| `Resource` | Identifies the emitting service / host – reused as-is. | +| `LogRecord` (body) | Carries the `AuditRecord` payload. | +| `Attributes` | Key-value context (actor, outcome, …) – reused as-is. | +| `InstrumentationScope` | **Not applicable.** MUST be left empty by exporters. | + +## Specifications + +- [Audit Logging API](./api.md) +- [Audit Logging SDK](./sdk.md) +- [Audit Record Data Model](./data-model.md) + +## References + +- [OTEP 0267 – Audit Logging Signal](../../oteps/0267-audit-logging.md) +- [OTEP 0092 – OpenTelemetry Logs Vision](../../oteps/logs/0092-logs-vision.md) +- [OTEP 0097 – Log Data Model](../../oteps/logs/0097-log-data-model.md) +- [OTEP 0202 – Events and Logs API](../../oteps/0202-events-and-logs-api.md) diff --git a/specification/audit/api.md b/specification/audit/api.md new file mode 100644 index 00000000000..b94af43ec16 --- /dev/null +++ b/specification/audit/api.md @@ -0,0 +1,205 @@ + + +# Audit Logging API + +**Status**: [Development](../document-status.md) + +
+Table of Contents + + + +- [AuditProvider](#auditprovider) + * [AuditProvider operations](#auditprovider-operations) + + [Get an AuditLogger](#get-an-auditlogger) +- [AuditLogger](#auditlogger) + * [Emit an AuditRecord](#emit-an-auditrecord) +- [Optional and required parameters](#optional-and-required-parameters) +- [Concurrency requirements](#concurrency-requirements) +- [References](#references) + + + +
+ +The Audit Logging API provides application code with a means to emit +[`AuditRecord`s](./data-model.md#auditrecord) that are guaranteed to +reach the configured audit sink without loss or modification. + +The API consists of these main components: + +- [**AuditProvider**](#auditprovider) – the entry point of the API. + Provides access to `AuditLogger` instances. +- [**AuditLogger**](#auditlogger) – emits `AuditRecord`s and returns + an [`AuditReceipt`](./data-model.md#auditreceipt). + +```mermaid +graph TD + A[AuditProvider] -->|Get| B(AuditLogger) + B -->|Emit| C(AuditRecord) + C -->|returns| D(AuditReceipt) +``` + +Unlike the [Logs API](../logs/api.md), the Audit Logging API: + +- Does **not** provide an `Enabled` check: audit records are always + emitted regardless of configuration, because dropping audit records + is prohibited. +- Does **not** accept a sampling-related configuration: the pipeline + MUST NOT sample or drop records. +- Has `emit` return an [`AuditReceipt`](./data-model.md#auditreceipt) + as proof-of-delivery once the sink acknowledges the record. + +## AuditProvider + +`AuditLogger`s are obtained from an `AuditProvider`. + +The `AuditProvider` is expected to be accessed from a central place. +The API SHOULD provide a way to set and access a global default +`AuditProvider`. + +The API MUST provide a No-op `AuditProvider` implementation that is +used when no SDK is installed. The No-op provider MUST return a No-op +`AuditLogger` whose `emit` method completes without error and returns a +No-op `AuditReceipt`. + +### AuditProvider operations + +The `AuditProvider` MUST provide the following function: + +- Get an `AuditLogger` + +#### Get an AuditLogger + +This API MUST accept the following parameter: + +- `name` (required): A string identifying the component, library, or + subsystem that is emitting audit records (for example, + `"com.example.auth"` or `"payment-service"`). The name is used as a + diagnostic label and is stored as an SDK-internal marker. It MUST NOT + be empty. If an empty string is provided, the SDK SHOULD log a + warning and use a fallback name rather than failing. + + Note: unlike `LoggerProvider.GetLogger`, this `name` is NOT mapped + to an OTLP `InstrumentationScope`. See + [README – OTLP Envelope Layers](./README.md#relationship-to-the-log-signal). + +This API MAY accept the following optional parameters: + +- `version` (optional): A string specifying the version of the + emitting component (for example, `"1.2.3"`). +- `schema_url` (optional): A Schema URL to be recorded in emitted + records for semantic convention versioning. + +The term *identical* applied to `AuditLogger`s describes instances +where all parameters are equal. The term *distinct* describes instances +where at least one parameter has a different value. + +## AuditLogger + +The `AuditLogger` is responsible for emitting `AuditRecord`s to the +audit pipeline. + +The `AuditLogger` MUST provide a function to: + +- [Emit an `AuditRecord`](#emit-an-auditrecord) + +The `AuditLogger` MUST NOT provide an `Enabled` or sampling-related +function. Conditional emission is the sole responsibility of +application code; the SDK MUST NOT silently suppress records based on +any sampling or filtering configuration. + +### Emit an AuditRecord + +The effect of calling this API is to submit an `AuditRecord` to the +audit pipeline and to block until the audit sink acknowledges receipt, +returning an `AuditReceipt`. + +The API MUST accept the following parameters: + +- [`Timestamp`](./data-model.md#field-timestamp) (required): the time + at which the auditable action occurred. +- [`EventName`](./data-model.md#field-eventname) (required): the + semantic name of the audit event. +- [`Actor`](./data-model.md#field-actor) (required): the identity that + performed the action. +- [`ActorType`](./data-model.md#field-actortype) (required): the type + of the actor (`USER`, `SERVICE`, or `SYSTEM`). +- [`Action`](./data-model.md#field-action) (required): the verb + describing what was done. +- [`Outcome`](./data-model.md#field-outcome) (required): the result of + the action (`SUCCESS`, `FAILURE`, or `UNKNOWN`). + +The API MUST accept the following optional parameters: + +- [`ObservedTimestamp`](./data-model.md#field-observedtimestamp) + (optional): if not set, the SDK MUST set this to the current time. +- [`TargetResource`](./data-model.md#field-targetresource) (optional): + the object upon which the action was performed. +- [`SourceIP`](./data-model.md#field-sourceip) (optional): the source + network address. +- [`Body`](./data-model.md#field-body) (optional): free-form additional + event details. +- [`Attributes`](./data-model.md#field-attributes) (optional): + arbitrary key-value context pairs. +- [`Signature`](./data-model.md#field-signature) (optional): digital + signature over the record. +- [`Algorithm`](./data-model.md#field-algorithm) (optional): the + signature algorithm; MUST be set if `Signature` is set. +- [`Certificate`](./data-model.md#field-certificate) (optional): the + public-key certificate for signature verification. + +**Return value**: The API MUST return an +[`AuditReceipt`](./data-model.md#auditreceipt) when the audit sink +acknowledges that the record has been persisted. The receipt contains +a `RecordId`, an `IntegrityHash`, and a `SinkTimestamp`. + +**Delivery semantics**: `emit` is synchronous by default. It MUST block +the calling thread until the exporter receives a successful +acknowledgement from the audit sink. This guarantees at-least-once +delivery and provides the caller with a `RecordId` that can be logged +or stored for audit trail completeness. + +An asynchronous variant MAY be provided by the SDK. An asynchronous +`emit`: + +- MUST still guarantee at-least-once delivery through an internal + durable buffer. +- MUST NOT silently discard records on failure. +- MAY return a future, promise, or channel through which the + `AuditReceipt` will be delivered once the sink acknowledges the + record. + +**Error handling**: If the audit sink cannot be reached and the SDK's +retry budget is exhausted, `emit` MUST surface a hard error to the +caller (for example, by raising an exception or returning an error +code). It MUST NOT return silently or return a partial / empty receipt. + +## Optional and required parameters + +Required parameters are those not marked optional in the +[Emit an AuditRecord](#emit-an-auditrecord) section. The API MUST be +structured to require these parameters (for example, as positional +arguments or a required record type). + +For each optional parameter, the API MUST be structured to accept it, +but MUST NOT require the caller to provide it. + +## Concurrency requirements + +For languages that support concurrent execution, the Audit Logging API +provides the following guarantees: + +- **AuditProvider** – all methods MUST be documented as safe for + concurrent use. +- **AuditLogger** – `emit` MUST be documented as safe for concurrent + use. Concurrent `emit` calls MUST be serialized internally by the SDK + so that records are enqueued in the order they are received. + +## References + +- [OTEP 0267 – Audit Logging Signal](../../oteps/0267-audit-logging.md) +- [OTEP 0202 – Events and Logs API](../../oteps/0202-events-and-logs-api.md) diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md new file mode 100644 index 00000000000..c9edf46bc54 --- /dev/null +++ b/specification/audit/data-model.md @@ -0,0 +1,484 @@ + + +# Audit Record Data Model + +**Status**: [Development](../document-status.md) + +
+Table of Contents + + + +- [Audit Record Data Model](#audit-record-data-model) + - [Design Notes](#design-notes) + - [Requirements](#requirements) + - [Relationship to LogRecord](#relationship-to-logrecord) + - [OTLP Envelope Layers](#otlp-envelope-layers) + - [AuditRecord Definition](#auditrecord-definition) + - [Field: `Timestamp`](#field-timestamp) + - [Field: `ObservedTimestamp`](#field-observedtimestamp) + - [Field: `EventName`](#field-eventname) + - [Actor Fields](#actor-fields) + - [Field: `Actor`](#field-actor) + - [Field: `ActorType`](#field-actortype) + - [Field: `Action`](#field-action) + - [Field: `Outcome`](#field-outcome) + - [Field: `TargetResource`](#field-targetresource) + - [Field: `SourceIP`](#field-sourceip) + - [Field: `Body`](#field-body) + - [Field: `Attributes`](#field-attributes) + - [Integrity Fields](#integrity-fields) + - [Field: `Signature`](#field-signature) + - [Field: `Algorithm`](#field-algorithm) + - [Field: `Certificate`](#field-certificate) + - [AuditReceipt Definition](#auditreceipt-definition) + - [Field: `RecordId`](#field-recordid) + - [Field: `IntegrityHash`](#field-integrityhash) + - [Field: `SinkTimestamp`](#field-sinktimestamp) + - [Example AuditRecords](#example-auditrecords) + - [Successful user login](#successful-user-login) + - [Failed privileged configuration change](#failed-privileged-configuration-change) + - [References](#references) + + + +
+ +## Design Notes + +### Requirements + +The Audit Record Data Model was designed to satisfy the following +requirements: + +- Every audit record MUST carry sufficient information to identify the + actor, the action performed, the target of the action, and the + outcome. This is the minimum required by ISO 27001 Annex A (event + logging) and equivalent compliance frameworks. + +- The data model MUST include timestamps that allow clock-skew detection + across distributed services (ISO 27001 – clock synchronisation). + +- The data model MUST support optional digital signatures so that + records can be verified for integrity after delivery. + +- The data model MUST be efficiently representable in OTLP by reusing + the existing `LogRecord` proto message as a transport container. + +- The data model MUST be extensible via arbitrary key-value attributes + to accommodate application-specific compliance requirements. + +### Relationship to LogRecord + +`AuditRecord` is transported as an OTLP `LogRecord`. The mandatory +audit-specific fields are stored in dedicated `LogRecord` fields where +a natural mapping exists, and in the `Attributes` map otherwise. The +`Body` field carries the free-form `AuditRecord.Body` payload. + +The `SeverityNumber` and `SeverityText` fields of the underlying +`LogRecord` MUST NOT be used for audit records. Severity is not a +meaningful concept for audit events; `Outcome` serves a similar purpose. + +### OTLP Envelope Layers + +The following OTLP envelope layers apply to audit records: + +| OTLP layer | Role in audit logging | +|------------------------|--------------------------------------------------------| +| `Resource` | Identifies the emitting service / host – reused as-is. | +| `LogRecord` | Carries the `AuditRecord` payload (see mapping below). | +| `Attributes` | Key-value context – reused as-is. | +| `InstrumentationScope` | **Not applicable.** MUST be left empty. | + +## AuditRecord Definition + +An `AuditRecord` is a single security-relevant event emitted by +application or system code on behalf of an actor. + +The following table provides a summary of all fields. Detailed +descriptions follow. + +| Field | Type | Req. | Description | +|---------------------|-------------------------|--------|------------------------------------------------| +| `Timestamp` | `fixed64` | MUST | Event time, ns since UNIX epoch (UTC). | +| `ObservedTimestamp` | `fixed64` | MUST | SDK observation time, ns since UNIX epoch. | +| `EventName` | `string` | MUST | Semantic name of the audit event. | +| `Actor` | `AnyValue` | MUST | Identity that performed the action. | +| `ActorType` | `enum` | MUST | `USER`, `SERVICE`, or `SYSTEM`. | +| `Action` | `string` | MUST | Verb describing what was done. | +| `Outcome` | `enum` | MUST | `SUCCESS`, `FAILURE`, or `UNKNOWN`. | +| `TargetResource` | `AnyValue` | SHOULD | The object acted upon. | +| `SourceIP` | `string` | MAY | Source network address. | +| `Body` | `AnyValue` | MAY | Free-form additional event details. | +| `Attributes` | `map` | MAY | Arbitrary key-value context. | +| `Signature` | `bytes` | MAY | Digital signature of the record. | +| `Algorithm` | `string` | MAY | Signature algorithm (e.g. `ES256`). | +| `Certificate` | `bytes` | MAY | Public certificate for signature verification. | + +### Field: `Timestamp` + +| Property | Value | +|------------|----------------------------------| +| Type | `fixed64` | +| Required | MUST be set | +| Constraint | `Timestamp <= ObservedTimestamp` | + +The time at which the audit event occurred, expressed as nanoseconds +since the UNIX epoch (UTC). The application MUST set this field to the +time the auditable action actually took place. + +If the application cannot determine the precise event time, it SHOULD +use the current time at the moment of the `emit` call and document this +in the `Body` or `Attributes`. + +The clock used for `Timestamp` MUST be a wall-clock source +synchronised via NTP or an equivalent protocol. The SDK SHOULD warn at +startup if the system clock offset exceeds one second. + +### Field: `ObservedTimestamp` + +| Property | Value | +|------------|----------------------------------------------------| +| Type | `fixed64` | +| Required | MUST be set | +| Constraint | `ObservedTimestamp >= Timestamp` and `<= now(UTC)` | + +The time at which the OpenTelemetry SDK observed the event, expressed +as nanoseconds since the UNIX epoch (UTC). The SDK MUST set this field +to the current time when `emit` is called if the application does not +provide it. + +`ObservedTimestamp` enables the audit sink to detect clock skew between +the emitting service and the SDK host, and between the SDK host and the +sink. A significant difference between `Timestamp` and +`ObservedTimestamp` SHOULD be treated as a clock synchronisation +warning. + +### Field: `EventName` + +| Property | Value | +|----------|--------------------------------| +| Type | `string` | +| Required | MUST be set; MUST NOT be empty | + +A short, dot-separated semantic name that uniquely identifies the type +of audit event. `EventName` MUST be stable across releases and SHOULD +follow a hierarchical naming convention, for example: + +- `user.login.success` +- `user.login.failure` +- `resource.file.read` +- `config.change` +- `privilege.escalation` + +Semantic conventions for common `EventName` values SHOULD be defined +in the [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/). + +### Actor Fields + +Actor fields identify the entity that performed the auditable action. +Together, `Actor` and `ActorType` provide the minimum identity context +required by ISO 27001 Annex A item 1 (user IDs). + +#### Field: `Actor` + +| Property | Value | +|----------|-------------| +| Type | `AnyValue` | +| Required | MUST be set | + +The identity of the entity that performed the action. This MAY be: + +- A user ID (string or integer) +- A service account name +- A fully qualified system identifier +- A structured object (e.g. `{ "id": "u123", "email": "a@example.com" }`) + +If the actor identity cannot be determined (for example, an +unauthenticated request), `Actor` MUST still be set to a value +indicating the unknown state (e.g. `"anonymous"` or `"unknown"`). + +#### Field: `ActorType` + +| Property | Value | +|----------|-----------------------------| +| Type | `enum` | +| Required | MUST be set | +| Values | `USER`, `SERVICE`, `SYSTEM` | + +Classifies the kind of entity that performed the action: + +| Value | Meaning | +|-----------|--------------------------------------------------------| +| `USER` | A human user, identified by a user account. | +| `SERVICE` | An automated service, daemon, or service account. | +| `SYSTEM` | The operating system or a privileged system component. | + +In times of AI agents and autonomous systems, the distinction between `USER` +and `SERVICE` may become blurred. The `ActorType` values are intended to be +broad categories rather than rigid classifications, and the SDK MUST NOT +enforce any particular semantics beyond what is described in the table above. +Applications SHOULD choose the most appropriate `ActorType` based on the +context of the action and the identity of the actor. Use `SERVICE` for +non-human actors that operate autonomously, and `USER` for human-initiated +actions, even if they are performed by an AI agent on behalf of a user. + +### Field: `Action` + +| Property | Value | +|----------|--------------------------------| +| Type | `string` | +| Required | MUST be set; MUST NOT be empty | + +A short verb or verb phrase that describes what the actor did. +Applications SHOULD use uppercase verbs to distinguish actions from +event names. Common values include: + +`LOGIN`, `LOGOUT`, `READ`, `WRITE`, `CREATE`, `UPDATE`, `DELETE`, +`EXECUTE`, `APPROVE`, `REJECT`, `EXPORT`, `IMPORT`. + +Applications MAY define domain-specific action verbs. Action names +SHOULD be stable across releases. + +### Field: `Outcome` + +| Property | Value | +|----------|---------------------------------| +| Type | `enum` | +| Required | MUST be set | +| Values | `SUCCESS`, `FAILURE`, `UNKNOWN` | + +The result of the auditable action: + +| Value | Meaning | +|-----------|--------------------------------------------------------------| +| `SUCCESS` | The action completed successfully. | +| `FAILURE` | The action was attempted but did not complete successfully. | +| `UNKNOWN` | The outcome could not be determined at the time of emission. | + +ISO 27001 Annex A items 5 and 6 require logging both successful and +unsuccessful access attempts. Using `UNKNOWN` SHOULD be avoided except +for fire-and-forget actions where acknowledgement is not possible. + +### Field: `TargetResource` + +| Property | Value | +|----------|---------------| +| Type | `AnyValue` | +| Required | SHOULD be set | + +The object upon which the action was performed. This MAY be: + +- A file path (string) +- A database table or row identifier +- A REST endpoint path +- A structured object with type and identifier fields + +If the action was not directed at a specific resource (for example, a +system startup event), this field MAY be omitted. + +Note: this field identifies the *target* of the auditable action and is +distinct from the OTLP `Resource` envelope, which identifies the +*emitting service or host*. + +### Field: `SourceIP` + +| Property | Value | +|----------|------------| +| Type | `string` | +| Required | MAY be set | + +The network address of the source of the auditable action. This field +SHOULD be set for network-initiated actions (for example, a login +attempt over HTTPS) and MAY be omitted for local or intra-process +actions. + +The value SHOULD conform to the IPv4 dotted-decimal notation +(`203.0.113.42`) or the IPv6 full notation (`2001:db8::1`). Port +numbers MAY be appended using the `host:port` convention. + +### Field: `Body` + +| Property | Value | +|----------|------------| +| Type | `AnyValue` | +| Required | MAY be set | + +Free-form additional information about the audit event. The value MAY +be a string, a byte buffer, or a structured object. Applications SHOULD +store information that does not fit naturally into the named fields here. + +The `Body` MUST NOT be used to store fields that duplicate the named +mandatory fields (`Actor`, `Action`, `Outcome`, etc.). + +### Field: `Attributes` + +| Property | Value | +|----------|-------------------------| +| Type | `map` | +| Required | MAY be set | + +Arbitrary key-value pairs that provide additional context about the +audit event. Applications SHOULD use the OpenTelemetry +[attribute naming conventions](../common/attribute-naming.md). + +Examples of useful attributes: + +- `session.id` – the authenticated session identifier +- `request.id` – a correlation identifier for the originating request +- `tenant.id` – the tenant in a multi-tenant system +- `geolocation.country` – country of origin for network requests +- `tls.version` – the TLS protocol version used + +Attribute keys MUST be unique within an `AuditRecord`. If the same key +is set multiple times, the last value MUST be used. + +### Integrity Fields + +The integrity fields enable a relying party to verify that an +`AuditRecord` was not altered after emission. They are OPTIONAL but +SHOULD be set in environments where tamper evidence is required by the +applicable compliance framework. + +#### Field: `Signature` + +| Property | Value | +|----------|------------| +| Type | `bytes` | +| Required | MAY be set | + +A digital signature over the canonical serialization of the +`AuditRecord`. The signature MUST cover all mandatory fields plus any +`Attributes` and `Body` that are present at emission time. + +If `Signature` is set, `Algorithm` MUST also be set. + +#### Field: `Algorithm` + +| Property | Value | +|----------|-----------------------------------| +| Type | `string` | +| Required | MUST be set if `Signature` is set | + +The algorithm used to compute `Signature`. The value SHOULD be a +registered JWA algorithm identifier, for example `RS256`, `ES256`, or +`EdDSA`. + +#### Field: `Certificate` + +| Property | Value | +|----------|------------| +| Type | `bytes` | +| Required | MAY be set | + +The public-key certificate (DER-encoded X.509) corresponding to the +signing key, included so that relying parties can verify `Signature` +without an out-of-band key lookup. + +Alternatively, Key ID, Subject Key Identifier (SKI), Issuer + SerialNumber or +other key reference MAY be included as an attribute instead of the full +certificate, but this is less self-contained and requires additional +configuration on the relying party side to resolve the key reference. + +If `Certificate` is omitted, the relying party MUST obtain the public +key through a separately configured trust anchor. + +## AuditReceipt Definition + +An `AuditReceipt` is returned by `AuditLogger.emit` once the audit +sink has successfully persisted the record. It serves as proof-of-delivery +and enables integrity verification by the emitting application. + +| Field | Type | Required | Description | +|-----------------|-----------|-------------|--------------------------------------------------------------| +| `RecordId` | `string` | MUST be set | Unique identifier assigned by the sink. | +| `IntegrityHash` | `string` | MUST be set | SHA-256 of the record as persisted by the sink. | +| `SinkTimestamp` | `fixed64` | MUST be set | Nanoseconds since UNIX epoch when the sink wrote the record. | + +### Field: `RecordId` + +A unique, stable identifier for the persisted record, assigned by the +audit sink. The format is sink-specific (for example, a UUID, a +sequential integer, or a content-addressed hash). + +The emitting application MAY store `RecordId` for later retrieval or +cross-referencing. + +### Field: `IntegrityHash` + +The SHA-256 hash of the canonical serialization of the `AuditRecord` +as it was written to persistent storage, computed by the sink. The hash +is returned to the emitting application so that it can verify that the +record was not altered between emission and persistence. + +The `IntegrityHash` is computed by the **sink**, not by the SDK, so it +reflects the record as actually stored – not merely as transmitted. + +The emitting application SHOULD compute the same hash locally +immediately after calling `emit` and compare it to the received +`IntegrityHash`. A mismatch indicates that the record was altered in +transit or by the sink and SHOULD trigger an alert (for example, log an +error or raise an incident). + +### Field: `SinkTimestamp` + +Nanoseconds since the UNIX epoch (UTC) at which the audit sink +persisted the record. This value MAY differ from `ObservedTimestamp` +due to buffering or network latency, and MAY be used to detect +abnormally long delivery times. + +## Example AuditRecords + +### Successful user login + +```json +{ + "Timestamp": 1714041600000000000, + "ObservedTimestamp": 1714041600001000000, + "EventName": "user.login.success", + "Actor": "u8472", + "ActorType": "USER", + "Action": "LOGIN", + "Outcome": "SUCCESS", + "TargetResource": "/api/auth/session", + "SourceIP": "203.0.113.42", + "Attributes": { + "session.id": "sess-abc123", + "tls.version": "TLSv1.3" + } +} +``` + +### Failed privileged configuration change + +```json +{ + "Timestamp": 1714041700000000000, + "ObservedTimestamp": 1714041700002000000, + "EventName": "config.change", + "Actor": "svc-deployer", + "ActorType": "SERVICE", + "Action": "UPDATE", + "Outcome": "FAILURE", + "TargetResource": { + "type": "kubernetes/configmap", + "namespace": "production", + "name": "app-secrets" + }, + "Body": "Authorization denied: missing role 'config-writer'", + "Attributes": { + "request.id": "req-xyz789", + "k8s.namespace": "production" + } +} +``` + +## References + +- [OTEP 0267 – Audit Logging Signal](../../oteps/0267-audit-logging.md) +- [OTEP 0097 – Log Data Model](../../oteps/logs/0097-log-data-model.md) +- [ISO 27001:2022 Annex A – Audit Logging controls](https://www.iso.org/standard/27001) +- [Common Event Format (CEF) specification](https://www.microfocus.com/documentation/arcsight/arcsight-smartconnectors-8.4/cef-implementation-standard/) diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md new file mode 100644 index 00000000000..b0b0e888784 --- /dev/null +++ b/specification/audit/sdk.md @@ -0,0 +1,466 @@ + + +# Audit Logging SDK + +**Status**: [Development](../document-status.md) + +
+Table of Contents + + + +- [Audit Logging SDK](#audit-logging-sdk) + - [AuditProvider](#auditprovider) + - [AuditProvider Creation](#auditprovider-creation) + - [AuditLogger Creation](#auditlogger-creation) + - [Configuration](#configuration) + - [No Sampling](#no-sampling) + - [Shutdown](#shutdown) + - [ForceFlush](#forceflush) + - [AuditLogger](#auditlogger) + - [Emit an AuditRecord](#emit-an-auditrecord) + - [AuditRecord Queue](#auditrecord-queue) + - [Durability guarantees](#durability-guarantees) + - [Back-pressure handling](#back-pressure-handling) + - [AuditRecordProcessor](#auditrecordprocessor) + - [Processor restrictions](#processor-restrictions) + - [AuditRecordProcessor operations](#auditrecordprocessor-operations) + - [OnEmit](#onemit) + - [Shutdown](#shutdown-1) + - [ForceFlush](#forceflush-1) + - [Built-in processors](#built-in-processors) + - [Simple processor](#simple-processor) + - [Signing processor (Sig)](#signing-processor-sig) + - [AuditRecordExporter](#auditrecordexporter) + - [AuditRecordExporter operations](#auditrecordexporter-operations) + - [Export](#export) + - [ForceFlush](#forceflush-2) + - [Shutdown](#shutdown-2) + - [OTLP transport requirements](#otlp-transport-requirements) + - [Dedicated endpoint](#dedicated-endpoint) + - [No partial success](#no-partial-success) + - [InstrumentationScope](#instrumentationscope) + - [Failure handling](#failure-handling) + - [Clock synchronisation](#clock-synchronisation) + - [Interaction with the Log signal](#interaction-with-the-log-signal) + - [Observability](#observability) + - [Concurrency requirements](#concurrency-requirements) + - [References](#references) + + + +
+ +The Audit Logging SDK is the implementation of the +[Audit Logging API](./api.md). It provides the guarantee that every +emitted `AuditRecord` is delivered to the configured audit sink without +loss, modification, or sampling. + +All language implementations of OpenTelemetry that support the Audit +Logging signal MUST provide an SDK conforming to this specification. + +## AuditProvider + +A `AuditProvider` MUST provide a way to associate a +[Resource](../resource/sdk.md) with all `AuditRecord`s produced by any +`AuditLogger` obtained from it. + +### AuditProvider Creation + +The SDK SHOULD allow the creation of multiple independent +`AuditProvider` instances. Although most applications will only need one. + +Each `AuditProvider` instance MUST have its own independent queue, +exporter pipeline, and failure counter. + +### AuditLogger Creation + +`AuditLogger` instances MUST only be created through an +`AuditProvider`. + +The `AuditProvider` MUST implement the +[Get an AuditLogger](./api.md#get-an-auditlogger) operation. The +`name` and optional `version` / `schema_url` parameters are stored +internally on the created `AuditLogger` for diagnostic purposes. + +If an invalid `name` (null or empty string) is provided, a working +`AuditLogger` MUST be returned rather than null or an exception. The +`name` SHOULD retain the invalid value and the SDK SHOULD emit a +warning. + +### Configuration + +The `AuditProvider` owns the following configuration: + +- One or more [AuditRecordProcessor](#auditrecordprocessor) instances. +- The [AuditRecord Queue](#auditrecord-queue) parameters. +- The retry and back-pressure policy (see + [Failure handling](#failure-handling)). + +Configuration MUST be applied to all already-returned `AuditLogger`s +when it is changed after provider creation (that is, adding a processor +after the provider is created MUST affect all existing loggers). + +### No Sampling + +The `AuditProvider` MUST NOT accept or expose any sampler or +sampling-rate configuration. Any attempt to configure sampling on the +audit pipeline MUST be rejected at configuration time with an +explanatory error. + +The SDK MUST NOT apply any record filtering, dropping, or aggregation +that reduces the number of `AuditRecord`s reaching the exporter. + +### Shutdown + +`Shutdown` MUST be called at most once per `AuditProvider` instance. + +Before returning, `Shutdown` MUST: + +1. Call `ForceFlush` to ensure all buffered records are exported. +2. Call `Shutdown` on every registered + `AuditRecordProcessor`. + +`Shutdown` SHOULD complete or abort within a configurable timeout. +After `Shutdown` returns, subsequent calls to `GetAuditLogger` SHOULD +return an error or throw an exception. + +`Shutdown` SHOULD provide a way for the caller to determine whether it +succeeded, failed, or timed out. + +### ForceFlush + +`ForceFlush` requests that all buffered `AuditRecord`s be exported +immediately. + +`ForceFlush` MUST invoke `ForceFlush` on all registered +`AuditRecordProcessor` instances. + +`ForceFlush` SHOULD complete or abort within a configurable timeout and +SHOULD provide a way for the caller to determine whether it succeeded, +failed, or timed out. + +## AuditLogger + +### Emit an AuditRecord + +When `emit` is called, the SDK MUST: + +1. Set `ObservedTimestamp` to the current time if the caller did not + provide it. +2. Validate that all required fields (`Timestamp`, `EventName`, + `Actor`, `ActorType`, `Action`, `Outcome`) are present and non-empty. + If any required field is missing, `emit` MUST surface a hard error + to the caller and MUST NOT silently drop the record. +3. Enqueue the `AuditRecord` in the [AuditRecord Queue](#auditrecord-queue). +4. Pass the record through all registered + `AuditRecordProcessor` instances via + [`OnEmit`](#onemit). +5. Block until the exporter returns a successful acknowledgement from + the audit sink. +6. Return the [`AuditReceipt`](./data-model.md#auditreceipt) provided + by the sink. + +If the synchronous `emit` cannot obtain an acknowledgement within the +configured timeout, it MUST surface a hard error. It MUST NOT return +successfully without a valid receipt. It may implement retries with back-off, +but this MUST be transparent to the caller and MUST NOT cause `emit` to block +indefinitely. + +## AuditRecord Queue + +### Durability guarantees + +The SDK MUST NOT use a bounded queue that silently drops records when +full. The SDK MUST use one of the following strategies: + +- **Disk-backed queue** (default): records are persisted to local + storage before acknowledgement by the exporter. This guarantees + delivery across SDK restarts and process crashes. +- **Unbounded in-memory queue** (opt-in): the queue grows without a + fixed limit. The SDK MUST expose a configurable high-water-mark + warning threshold. When the queue depth exceeds this threshold, the + SDK SHOULD emit a warning via the SDK's internal logging. The SDK SHOULD + provide meaningfull metrics (for example, `audit.queue.depth`) to allow + operators to monitor the queue depth and set up alerts. +- **Other lossless queuing strategy**: if the SDK implements a different + strategy that guarantees no record loss (for example, DB persistence queue + with back-pressure blocking), it MUST provide the same + durability guarantees and operational visibility as the above options. + The SDK MUST NOT use a strategy that can result in silent record loss + under any circumstances. + +### Back-pressure handling + +If the in-memory queue is exhausted (for example, because the exporter +is unavailable for an extended period): + +- The SDK MUST either block the calling thread (applying back-pressure + to the application) or persist the record to a local durable buffer. +- The SDK MUST NOT silently discard the record. + +The back-pressure strategy (blocking vs. disk-backed) MUST be +configurable. + +## AuditRecordProcessor + +`AuditRecordProcessor` is an interface that allows hooks into the +`AuditRecord` pipeline for enrichment and forwarding. + +### Processor restrictions + +The Audit Logging pipeline enforces strict restrictions on processors +to guarantee that no record is lost or altered: + +- Only **additive** operations are permitted: processors MAY enrich + records by adding `Attributes`, but MUST NOT remove + existing attributes, filter records, aggregate records, or alter the + mandatory fields (`Timestamp`, `EventName`, `Actor`, `ActorType`, + `Action`, `Outcome`). +- Processors that delete attributes, suppress records, or aggregate + records MUST be rejected at configuration time with an explanatory + error. +- Processors MUST NOT introduce sampling or conditional forwarding. + +### AuditRecordProcessor operations + +#### OnEmit + +`OnEmit` is called synchronously on the thread that called `emit`, +after the `AuditRecord` has been queued and before the response is +returned to the caller. + +**Parameters:** + +- `auditRecord` – a mutable `AuditRecord` for the emitted record. + Processors MAY enrich the record (add attributes) but + MUST NOT remove mandatory fields or filter the record. +- `context` – the resolved `Context` at the time of emission. + +**Returns:** `Void` + +`OnEmit` MUST NOT block indefinitely. It SHOULD complete quickly to +minimise the latency of synchronous `emit` calls. + +Mutations made by a processor to `auditRecord` MUST be visible to +subsequently registered processors. + +#### Shutdown + +Shuts down the processor. Called when the `AuditProvider` is shut down. + +`Shutdown` MUST be called at most once per `AuditRecordProcessor` +instance. + +`Shutdown` MUST include the effects of `ForceFlush`. + +`Shutdown` SHOULD complete or abort within a configurable timeout. + +#### ForceFlush + +Requests that the processor export all buffered records as soon as +possible, preferably before returning. + +`ForceFlush` SHOULD provide a way for the caller to determine whether +it succeeded, failed, or timed out. + +### Built-in processors + +#### Simple processor + +The simple processor passes each `AuditRecord` directly to the +configured `AuditRecordExporter` synchronously on the calling thread. +This guarantees that the exporter acknowledges the record before +`emit` returns. + +The processor MUST synchronize calls to the exporter's `Export` to +prevent concurrent invocations. + +**Configurable parameters:** + +- `exporter` – the `AuditRecordExporter` to which records are pushed. + +The SDK MUST provide the simple processor as a built-in. It is the +default processor for synchronous `emit`. + +#### Signing processor (Sig) + +The signing processor adds a digital signature to each `AuditRecord` to +guarantee integrity and non-repudiation. It MUST be implemented as a separate +processor that can be added to the pipeline in addition to the simple +processor, rather than as a modification to the simple processor, to allow +flexibility in the choice of signing algorithm and key management strategy. + +## AuditRecordExporter + +`AuditRecordExporter` is the interface that protocol-specific exporters +implement to transmit `AuditRecord`s to the audit sink. + +Each implementation MUST document its concurrency characteristics. + +### AuditRecordExporter operations + +#### Export + +Exports `AuditRecord`s to the audit sink. + +`Export` MUST NOT be called concurrently on the same exporter instance. + +`Export` MUST NOT block indefinitely. There MUST be a configurable +upper time limit after which the call times out with a `Failure` result. + +**Parameters:** + +- Optional `batch` – a collection of `AuditRecord`s to export. The concrete + type is language-specific. + +**Returns:** `ExportResult` + +`ExportResult` has values of either `Success` or `Failure`: + +- `Success` – all records in the batch have been acknowledged by the + audit sink. The exporter MUST include the `AuditReceipt` for each + record in the result. +- `Failure` – the export failed. The SDK MUST retain all records in + the batch and retry according to the configured retry policy. The + SDK MUST NOT drop records on a `Failure` result. + +#### ForceFlush + +Requests that any buffered `AuditRecord`s be exported immediately. + +`ForceFlush` SHOULD provide a way for the caller to determine whether +it succeeded, failed, or timed out. + +#### Shutdown + +Shuts down the exporter. Called when the `AuditProvider` is shut down. + +`Shutdown` MUST be called at most once per `AuditRecordExporter` +instance. After `Shutdown`, subsequent calls to `Export` MUST return +`Failure`. + +### OTLP transport requirements + +#### Dedicated endpoint + +Audit records MUST be exported to a dedicated OTLP endpoint, distinct +from the standard `/v1/logs` endpoint. The default path is `/v1/audit`. + +Using a dedicated endpoint enables: + +- Network-level isolation (firewall rules, separate TLS certificates + and credentials). +- Independent QoS policies with no back-pressure shedding from the + observability pipeline. +- Clear separation in the sink's data model. + + +The payload MUST use the standard `ExportLogsServiceRequest` proto, +with the `audit` flag set to `true` in the `ResourceLogs` message to +identify the batch as audit records. + +#### No partial success + +The OTLP receiver MUST NOT respond with a `partial_success` for an +audit log export request. + +If the receiver cannot process one or more records in a batch, it MUST +reject the **entire** batch and return an error status. The SDK exporter +MUST treat any `partial_success` response as a hard `Failure`, retain +**all** records in the batch (including those that would have been +accepted), and retry the full batch. + +This constraint ensures that no audit record can be silently lost by a +receiver that acknowledges only part of a batch. + +#### InstrumentationScope + +`InstrumentationScope` has no meaning for audit logging. Audit records +are emitted by application or system code acting on behalf of an actor, +not by instrumentation libraries. + +Exporters MUST leave the `InstrumentationScope` field empty, or MAY +populate it with the SDK name and version as a technical marker. +Receivers MUST NOT use `InstrumentationScope` for routing, filtering, +or processing audit records. + +## Failure handling + +If the exporter cannot reach the audit sink, the SDK MUST follow this +failure-handling sequence: + +1. **Buffer**: Retain the record in the SDK's queue (disk-backed or + in-memory, depending on configuration). The record MUST NOT be + discarded. +2. **Retry**: Attempt re-export with exponential back-off and a + configurable maximum number of retries. The initial retry interval, + back-off multiplier, and maximum interval SHOULD be configurable. +3. **Hard error**: If the retry budget is exhausted or the buffer is + full (for example, the disk is full), the SDK MUST surface a hard + error to the caller (exception, error code, or equivalent). It MUST + NOT silently drop the record. +4. **Metric**: The SDK MUST increment the `audit.records.dropped` + metric counter for every record that is lost due to an unrecoverable + error. See [Observability](#observability). + +## Clock synchronisation + +Accurate timestamps are a requirement of ISO 27001 Annex A (clock +synchronisation). The SDK SHOULD: + +- Warn at startup if the system clock NTP offset exceeds one second. +- Warn when `Timestamp` and `ObservedTimestamp` differ by more than a + configurable threshold (default: 5 seconds), which may indicate clock + skew between the event source and the SDK host. + +The SDK MUST set `ObservedTimestamp` to the wall-clock time at the +moment `emit` is called, using the SDK host's system clock. + +## Interaction with the Log signal + +Audit records MUST NOT be routed through the standard `LoggerProvider` +pipeline. The two signals are independent and MUST use separate +providers, queues, and exporters. + +An application or library MAY emit both an `AuditRecord` (for +compliance) and a regular `LogRecord` (for observability) for the same +event. The records travel through independent pipelines to independent +backends and MUST NOT interfere with each other. + +## Observability + +The SDK MUST expose the following internal metrics to allow operators +to monitor the health of the audit pipeline: + +| Metric name | Type | Description | +|--------------------------|-----------|---------------------------------------------| +| `audit.records.emitted` | Counter | Total records passed to the SDK pipeline. | +| `audit.records.exported` | Counter | Records successfully acknowledged by sink. | +| `audit.records.dropped` | Counter | Records lost due to unrecoverable errors. | +| `audit.queue.depth` | Gauge | Current number of records in the SDK queue. | +| `audit.export.duration` | Histogram | Round-trip time from `Export` to receipt. | + +A non-zero value for `audit.records.dropped` MUST be treated as a +critical operational alert. SDK authors SHOULD provide a mechanism (for +example, a log message or a callback) to notify operators immediately +when this counter is incremented. + +## Concurrency requirements + +For languages that support concurrent execution: + +- **AuditProvider** – logger creation, `ForceFlush`, and `Shutdown` + MUST be safe for concurrent use. +- **AuditLogger** – `emit` MUST be safe for concurrent use. +- **AuditRecordExporter** – `ForceFlush` and `Shutdown` MUST be safe + for concurrent use. `Export` MUST NOT be called concurrently on the + same instance. + +## References + +- [OTEP 0267 – Audit Logging Signal](../../oteps/0267-audit-logging.md) +- [OTEP 0150 – Logging Library SDK Prototype Specification](../../oteps/logs/0150-logging-library-sdk.md) From bc030391208ca6b9857b83e6b142285137ff6a94 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Thu, 23 Apr 2026 17:17:49 +0200 Subject: [PATCH 04/16] touch changelog Signed-off-by: Hilmar Falkenberg --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index d6cb1481647..9f239761a77 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,11 @@ release. ### Logs +### Audit + +- Introduce Audit-Logging signal and related specification sections. + ([#1](https://github.com/apeirora/opentelemetry-specification/pull/1)) + ### Baggage ### Profiles From 91cc415832138de9954b45ce3fdb41d8103d703d Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Thu, 23 Apr 2026 17:26:12 +0200 Subject: [PATCH 05/16] simpler shutdown handling Signed-off-by: Hilmar Falkenberg --- specification/audit/sdk.md | 42 ++++++++++++++++++++++++++++---------- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index b0b0e888784..0f2e74e9717 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -116,9 +116,7 @@ that reduces the number of `AuditRecord`s reaching the exporter. ### Shutdown -`Shutdown` MUST be called at most once per `AuditProvider` instance. - -Before returning, `Shutdown` MUST: +On the first call, `Shutdown` MUST: 1. Call `ForceFlush` to ensure all buffered records are exported. 2. Call `Shutdown` on every registered @@ -126,11 +124,16 @@ Before returning, `Shutdown` MUST: `Shutdown` SHOULD complete or abort within a configurable timeout. After `Shutdown` returns, subsequent calls to `GetAuditLogger` SHOULD -return an error or throw an exception. +return an error or throw an exception. Any call to `emit` on an +`AuditLogger` obtained from a shut-down `AuditProvider` MUST raise a +hard error or throw an exception and MUST NOT silently drop the record. `Shutdown` SHOULD provide a way for the caller to determine whether it succeeded, failed, or timed out. +Subsequent calls to `Shutdown` or `ForceFlush` after the first +`Shutdown` call MUST be no-ops and MUST NOT return an error. + ### ForceFlush `ForceFlush` requests that all buffered `AuditRecord`s be exported @@ -143,10 +146,17 @@ immediately. SHOULD provide a way for the caller to determine whether it succeeded, failed, or timed out. +If `Shutdown` has already been called, `ForceFlush` MUST be a no-op +and MUST NOT return an error. + ## AuditLogger ### Emit an AuditRecord +If `emit` is called after the owning `AuditProvider` has been shut +down, the SDK MUST raise a hard error or throw an exception. The SDK +MUST NOT silently drop the record. + When `emit` is called, the SDK MUST: 1. Set `ObservedTimestamp` to the current time if the caller did not @@ -248,17 +258,21 @@ minimise the latency of synchronous `emit` calls. Mutations made by a processor to `auditRecord` MUST be visible to subsequently registered processors. +If `OnEmit` is called after the processor has been shut down, the +processor MUST raise a hard error or throw an exception and MUST NOT +silently accept or forward the record. + #### Shutdown Shuts down the processor. Called when the `AuditProvider` is shut down. -`Shutdown` MUST be called at most once per `AuditRecordProcessor` -instance. - -`Shutdown` MUST include the effects of `ForceFlush`. +On the first call, `Shutdown` MUST include the effects of `ForceFlush`. `Shutdown` SHOULD complete or abort within a configurable timeout. +Subsequent calls to `Shutdown` or `ForceFlush` after the first +`Shutdown` call MUST be no-ops and MUST NOT return an error. + #### ForceFlush Requests that the processor export all buffered records as soon as @@ -267,6 +281,9 @@ possible, preferably before returning. `ForceFlush` SHOULD provide a way for the caller to determine whether it succeeded, failed, or timed out. +If `Shutdown` has already been called, `ForceFlush` MUST be a no-op +and MUST NOT return an error. + ### Built-in processors #### Simple processor @@ -339,9 +356,12 @@ it succeeded, failed, or timed out. Shuts down the exporter. Called when the `AuditProvider` is shut down. -`Shutdown` MUST be called at most once per `AuditRecordExporter` -instance. After `Shutdown`, subsequent calls to `Export` MUST return -`Failure`. +On the first call, `Shutdown` MUST flush any internally buffered records +and release all resources. After `Shutdown` completes, subsequent calls +to `Export` MUST return `Failure`. + +Subsequent calls to `Shutdown` or `ForceFlush` after the first +`Shutdown` call MUST be no-ops and MUST NOT return an error. ### OTLP transport requirements From c14d9e0601050d4ab69ffa7c75fc109f8907a518 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Fri, 24 Apr 2026 10:01:28 +0200 Subject: [PATCH 06/16] pacify textlint + markdown-link-check Signed-off-by: Hilmar Falkenberg --- specification/audit/README.md | 6 +++--- specification/audit/api.md | 8 ++++---- specification/audit/sdk.md | 4 ++-- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/specification/audit/README.md b/specification/audit/README.md index 24b12485dbc..a8353dbb5c5 100644 --- a/specification/audit/README.md +++ b/specification/audit/README.md @@ -81,7 +81,7 @@ compliance drivers that motivate them: | Operator | Configuration change | ISO 27001 A.8.15 | | Service account | Elevated-privilege action | ISO 27001 A.5.18 | | Service | Outbound call to external API | PCI-DSS Req. 10 | -| Runtime | Antivirus / IDS rule fired | ISO 27001 A.8.16 | +| Runtime | Antivirus / IDs rule fired | ISO 27001 A.8.16 | ## Signal Overview @@ -118,10 +118,10 @@ The principal data components are: - **AuditRecord** – the audit event payload containing mandatory fields for actor, action, outcome, timestamps, and optional integrity - metadata. See [Audit Record Data Model](./data-model.md#auditrecord). + metadata. See [Audit Record Data Model](./data-model.md#auditrecord-definition). - **AuditReceipt** – proof-of-delivery returned by the sink, containing a unique record identifier and a SHA-256 integrity hash. - See [Audit Record Data Model](./data-model.md#auditreceipt). + See [Audit Record Data Model](./data-model.md#auditreceipt-definition). ## Relationship to the Log Signal diff --git a/specification/audit/api.md b/specification/audit/api.md index b94af43ec16..257d8b12137 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -26,7 +26,7 @@ weight: 1 The Audit Logging API provides application code with a means to emit -[`AuditRecord`s](./data-model.md#auditrecord) that are guaranteed to +[`AuditRecord`s](./data-model.md#auditrecord-definition) that are guaranteed to reach the configured audit sink without loss or modification. The API consists of these main components: @@ -34,7 +34,7 @@ The API consists of these main components: - [**AuditProvider**](#auditprovider) – the entry point of the API. Provides access to `AuditLogger` instances. - [**AuditLogger**](#auditlogger) – emits `AuditRecord`s and returns - an [`AuditReceipt`](./data-model.md#auditreceipt). + an [`AuditReceipt`](./data-model.md#auditreceipt-definition). ```mermaid graph TD @@ -50,7 +50,7 @@ Unlike the [Logs API](../logs/api.md), the Audit Logging API: is prohibited. - Does **not** accept a sampling-related configuration: the pipeline MUST NOT sample or drop records. -- Has `emit` return an [`AuditReceipt`](./data-model.md#auditreceipt) +- Has `emit` return an [`AuditReceipt`](./data-model.md#auditreceipt-definition) as proof-of-delivery once the sink acknowledges the record. ## AuditProvider @@ -153,7 +153,7 @@ The API MUST accept the following optional parameters: public-key certificate for signature verification. **Return value**: The API MUST return an -[`AuditReceipt`](./data-model.md#auditreceipt) when the audit sink +[`AuditReceipt`](./data-model.md#auditreceipt-definition) when the audit sink acknowledges that the record has been persisted. The receipt contains a `RecordId`, an `IntegrityHash`, and a `SinkTimestamp`. diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index 0f2e74e9717..809388bdd00 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -171,7 +171,7 @@ When `emit` is called, the SDK MUST: [`OnEmit`](#onemit). 5. Block until the exporter returns a successful acknowledgement from the audit sink. -6. Return the [`AuditReceipt`](./data-model.md#auditreceipt) provided +6. Return the [`AuditReceipt`](./data-model.md#auditreceipt-definition) provided by the sink. If the synchronous `emit` cannot obtain an acknowledgement within the @@ -416,7 +416,7 @@ failure-handling sequence: 1. **Buffer**: Retain the record in the SDK's queue (disk-backed or in-memory, depending on configuration). The record MUST NOT be discarded. -2. **Retry**: Attempt re-export with exponential back-off and a +2. **Retry**: Attempt reexport with exponential back-off and a configurable maximum number of retries. The initial retry interval, back-off multiplier, and maximum interval SHOULD be configurable. 3. **Hard error**: If the retry budget is exhausted or the buffer is From 43452814751276f882675218ef5974c99b5279eb Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Fri, 24 Apr 2026 11:17:58 +0200 Subject: [PATCH 07/16] incorporated important differences from https://github.com/apeirora/opentelemetry-specification/pull/2 Signed-off-by: Hilmar Falkenberg --- specification/audit/README.md | 11 +- specification/audit/api.md | 21 ++- specification/audit/collector.md | 288 ++++++++++++++++++++++++++++++ specification/audit/data-model.md | 166 ++++++++++++++--- specification/audit/sdk.md | 113 +++++++++++- 5 files changed, 562 insertions(+), 37 deletions(-) create mode 100644 specification/audit/collector.md diff --git a/specification/audit/README.md b/specification/audit/README.md index a8353dbb5c5..e4889b55ce1 100644 --- a/specification/audit/README.md +++ b/specification/audit/README.md @@ -94,13 +94,17 @@ Application code │ ▼ AuditLogger.emit(AuditRecord) ──► returns AuditReceipt - │ + │ RecordId (caller-generated) ▼ AuditProvider (no sampling, no dropping) │ └── AuditRecordProcessor (enrich only – never redact or drop) + │ ├── Simple processor (sync, default) + │ ├── Batching processor (async, high-volume) + │ └── Signing processor (adds Signature or Hmac) ▼ - AuditExporter ──► Audit sink (OpenSearch, Splunk, S3 WORM, SIEM, …) - │ + AuditExporter ──► Tier-2 Collector (optional) ──► Audit sinks + or ──► Audit sink directly │ + │ (OpenSearch, Splunk, S3 WORM …) └── returns AuditReceipt (RecordId + IntegrityHash + SinkTimestamp) ``` @@ -149,6 +153,7 @@ payload carrier. The following OTLP envelope layers apply: - [Audit Logging API](./api.md) - [Audit Logging SDK](./sdk.md) - [Audit Record Data Model](./data-model.md) +- [Audit Logging Collector (Tier-2)](./collector.md) ## References diff --git a/specification/audit/api.md b/specification/audit/api.md index 257d8b12137..dd218a6751d 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -20,7 +20,6 @@ weight: 1 - [Optional and required parameters](#optional-and-required-parameters) - [Concurrency requirements](#concurrency-requirements) - [References](#references) - @@ -120,6 +119,10 @@ returning an `AuditReceipt`. The API MUST accept the following parameters: +- [`RecordId`](./data-model.md#field-recordid) (required): a + caller-generated unique identifier for this record. If the caller + does not provide one the SDK MUST generate a UUID v4. The value + MUST be stable across retries of the same event. - [`Timestamp`](./data-model.md#field-timestamp) (required): the time at which the auditable action occurred. - [`EventName`](./data-model.md#field-eventname) (required): the @@ -137,6 +140,8 @@ The API MUST accept the following optional parameters: - [`ObservedTimestamp`](./data-model.md#field-observedtimestamp) (optional): if not set, the SDK MUST set this to the current time. +- [`SchemaVersion`](./data-model.md#field-schemaversion) (optional): + the schema version of the audit payload. SHOULD be set. - [`TargetResource`](./data-model.md#field-targetresource) (optional): the object upon which the action was performed. - [`SourceIP`](./data-model.md#field-sourceip) (optional): the source @@ -151,11 +156,21 @@ The API MUST accept the following optional parameters: signature algorithm; MUST be set if `Signature` is set. - [`Certificate`](./data-model.md#field-certificate) (optional): the public-key certificate for signature verification. +- [`Hmac`](./data-model.md#field-hmac) (optional): symmetric HMAC over + the record (alternative to `Signature`). MUST NOT be set together + with `Signature`. +- [`HmacAlgorithm`](./data-model.md#field-hmacalgorithm) (optional): + the HMAC algorithm; MUST be set if `Hmac` is set. +- [`SequenceNo`](./data-model.md#field-sequenceno) (optional): + monotonic counter for hash-chain continuity. +- [`PrevHash`](./data-model.md#field-prevhash) (optional): SHA-256 of + the preceding record in the audit stream. **Return value**: The API MUST return an [`AuditReceipt`](./data-model.md#auditreceipt-definition) when the audit sink -acknowledges that the record has been persisted. The receipt contains -a `RecordId`, an `IntegrityHash`, and a `SinkTimestamp`. +acknowledges that the record has been persisted. The receipt echoes the +caller's `RecordId` and contains an `IntegrityHash` and a +`SinkTimestamp`. **Delivery semantics**: `emit` is synchronous by default. It MUST block the calling thread until the exporter receives a successful diff --git a/specification/audit/collector.md b/specification/audit/collector.md new file mode 100644 index 00000000000..46549aaab00 --- /dev/null +++ b/specification/audit/collector.md @@ -0,0 +1,288 @@ + + +# Audit Logging Collector (Tier-2) + +**Status**: [Development](../document-status.md) + +
+Table of Contents + + + +- [Audit Logging Collector (Tier-2)](#audit-logging-collector-tier-2) + - [Overview](#overview) + - [Endpoint Definition](#endpoint-definition) + - [Single-Record Ingest](#single-record-ingest) + - [Batch Ingest](#batch-ingest) + - [Request Model](#request-model) + - [Required Record Fields](#required-record-fields) + - [Optional Record Fields](#optional-record-fields) + - [Response Model](#response-model) + - [Single-Record Response Fields](#single-record-response-fields) + - [Batch Response Fields](#batch-response-fields) + - [Domain Status Values](#domain-status-values) + - [HTTP Status Codes](#http-status-codes) + - [Idempotency and Duplicate Handling](#idempotency-and-duplicate-handling) + - [Verification and Sink Delivery](#verification-and-sink-delivery) + - [Integrity Verification](#integrity-verification) + - [Hash-Chain Validation](#hash-chain-validation) + - [Per-Sink Delivery Status](#per-sink-delivery-status) + - [Record Lifecycle](#record-lifecycle) + - [Retry Guidance](#retry-guidance) + - [Security Requirements](#security-requirements) + - [References](#references) + + + +
+ +## Overview + +This document specifies the behavior of an OpenTelemetry Collector +acting as a Tier-2 intermediary in an audit logging pipeline. The +collector receives `AuditRecord`s from SDK exporters (Tier-1), +verifies their integrity, and distributes them to one or more required +audit sinks. + +The Tier-2 collector is optional. SDK exporters MAY deliver directly +to the audit sink when no collector is deployed. When a collector is +present it MUST satisfy the same no-drop and at-least-once delivery +guarantees as the SDK itself. + +``` +SDK Exporter ──POST /v1/audit──► Tier-2 Collector + │ + verify integrity + validate hash-chain + │ + ┌────────────┼────────────┐ + ▼ ▼ ▼ + Primary sink Secondary SIEM / archive + (OpenSearch) (S3 WORM) (Splunk) +``` + +## Endpoint Definition + +The Tier-2 collector MUST expose an HTTP endpoint that accepts audit +records from SDK exporters. + +### Single-Record Ingest + +- **Path**: `/v1/audit` (MUST; matches the SDK exporter default). +- **Method**: `POST`. +- **Body**: one serialized `AuditRecord` payload. + +### Batch Ingest + +- **Path**: `/v1/audit` (same endpoint; batch indicated by content). +- **Method**: `POST`. +- **Body**: batch payload containing multiple `AuditRecord`s. + +The collector MUST support both single-record and batch requests on +the same endpoint. The request content type MUST follow the OTLP +serialization convention (protobuf or JSON). + +## Request Model + +Request payloads MUST follow the +[Audit Record Data Model](./data-model.md). + +### Required Record Fields + +Each record MUST include: + +- `RecordId` +- `Timestamp` +- `EventName` +- `Actor` +- `ActorType` +- `Action` +- `Outcome` + +### Optional Record Fields + +Each record MAY include any field defined in the +[AuditRecord Definition](./data-model.md#auditrecord-definition), +including `SchemaVersion`, `Signature`, `Algorithm`, `Certificate`, +`Hmac`, `HmacAlgorithm`, `SequenceNo`, and `PrevHash`. + +## Response Model + +Tier-2 responses MUST include: + +- A transport-level HTTP status code. +- A domain-level processing status in the response body. + +### Single-Record Response Fields + +- `record_id` – echoes the `RecordId` from the request. +- `status` – domain status value (see [Domain Status Values](#domain-status-values)). +- `integrity_hash` – SHA-256 of the record as persisted by the + collector (same semantics as `AuditReceipt.IntegrityHash`). +- `sink_timestamp` – nanoseconds since UNIX epoch when the collector + persisted the record to all required sinks. +- `verify_status` – `passed`, `failed`, or `deferred`. +- `sink_status_map` – map of sink name → delivery status (see + [Per-Sink Delivery Status](#per-sink-delivery-status)). +- `reason` – human-readable explanation when `status` is not success. +- `retry_after` – seconds to wait before retrying, when applicable. + +### Batch Response Fields + +- `batch_id` – collector-assigned identifier for the batch. +- `status` – aggregate domain status. +- `record_status_map` – map of `RecordId` → single-record response. +- `sink_status_aggregate` – aggregate delivery status across all sinks. + +### Domain Status Values + +The collector SHOULD use the following domain status values: + +- `accepted_pending_verify` – received; integrity check in progress. +- `verified_queued` – integrity passed; queued for sink delivery. +- `verified_exporting` – actively delivering to required sinks. +- `delivered_all_sinks` – every required sink confirmed receipt. +- `delivered_partial` – at least one required sink failed; treated as + non-success (see [Verification and Sink Delivery](#verification-and-sink-delivery)). +- `rejected_verify_failed` – integrity check failed. +- `rejected_schema_invalid` – schema validation failed. +- `rejected_authz` – authentication or authorisation failure. +- `failed_internal` – unexpected internal collector error. + +## HTTP Status Codes + +The collector SHOULD use the following HTTP status codes: + +| Code | Meaning | +|---------------|----------------------------------------------------------------| +| `200` | Fully processed; all required sinks confirmed. | +| `202` | Accepted for async verify or export; final state pending. | +| `400` | Malformed payload, schema mismatch, or canonicalization error. | +| `401` / `403` | Authentication or authorisation failure. | +| `409` | Duplicate `RecordId` with differing payload hash. | +| `413` | Payload too large. | +| `429` | Intake throttled by policy. | +| `500` | Unexpected internal error. | +| `503` | Temporarily unavailable or no reliable intake capacity. | + +The collector MUST NOT return HTTP 207 (Multi-Status) to indicate +partial success. If any required sink fails the collector MUST reject +the entire batch (see +[No partial success](./sdk.md#no-partial-success)). + +## Idempotency and Duplicate Handling + +The collector MUST treat `RecordId` as the primary idempotency key. + +When a duplicate `RecordId` is received: + +- If the canonical payload hash is **identical**, the collector SHOULD + return a deterministic success response for the existing record + without creating a duplicate entry. +- If the canonical payload hash **differs**, the collector MUST return + HTTP 409. The SDK exporter MUST treat HTTP 409 as a non-retryable + `Failure`. + +## Verification and Sink Delivery + +### Integrity Verification + +When `Signature` or `Hmac` is present the collector SHOULD verify the +field against the declared algorithm and the public key or shared +secret configured in the collector's trust policy. + +The collector MAY defer verification for low-latency ingest (returning +`accepted_pending_verify`) but MUST complete verification before +acknowledging delivery to any required sink. + +If verification fails the collector MUST: + +1. Reject the record with domain status `rejected_verify_failed`. +2. NOT forward the record to any sink. +3. Return HTTP 400 with a descriptive `reason`. + +### Hash-Chain Validation + +When `SequenceNo` and `PrevHash` are present the collector SHOULD +validate chain continuity: + +- `SequenceNo` MUST be strictly greater than the previous record's + `SequenceNo` in the same audit stream. +- `PrevHash` MUST equal the `IntegrityHash` of the preceding record. + +A broken chain SHOULD be surfaced as a warning metric and logged as a +security event. The collector MAY still accept and forward a broken +chain record but MUST annotate the delivery status with a chain +validation warning. + +### Per-Sink Delivery Status + +For each required sink the collector SHOULD report: + +- `exporter_name` – configured name of the sink exporter. +- `sink_type` – type of sink (e.g. `opensearch`, `s3`, `splunk`). +- `delivery_status` – `success`, `failed`, `timeout`, `retrying`, or + `unknown`. +- `attempt_count` – number of delivery attempts made. +- `last_attempt_at` – timestamp of the most recent attempt. +- `sink_ack_id` – acknowledgement identifier returned by the sink, if + available. +- `error_code` – sink-specific error code on failure. +- `error_message` – human-readable error description on failure. + +A record outcome is successful only when every required sink reports +`delivery_status: success`. + +### Record Lifecycle + +Recommended state transitions: + +``` +received + → accepted_pending_verify + → verified_queued + → verified_exporting + → delivered_all_sinks (terminal: success) +``` + +Alternative terminal states: + +- `rejected_schema_invalid` +- `rejected_verify_failed` +- `rejected_authz` +- `delivered_partial` (treated as non-success) +- `failed_internal` + +## Retry Guidance + +SDK exporters SHOULD use the `retry_after` field when present. + +Retry-ability by status code: + +| Status | Behaviour | +|-----------------------------------|--------------------------------------| +| `202`, `429`, `503` | Retryable with exponential back-off. | +| `400`, `401`, `403`, `409`, `413` | Non-retryable. | +| `500` | Implementation and policy dependent. | + +## Security Requirements + +The collector ingress MUST require secure transport and SHOULD support: + +- TLS with server authentication (minimum TLS 1.2). +- mTLS for stronger client identity where required. +- Token-based authentication (e.g. Bearer token) where mTLS is not + used. + +The collector SHOULD validate `Signature` or `Hmac` fields according +to a configured verification policy before forwarding records to sinks. + +## References + +- [Audit Logging API](./api.md) +- [Audit Logging SDK](./sdk.md) +- [Audit Record Data Model](./data-model.md) +- [OTEP 0267 – Audit Logging Signal](../../oteps/0267-audit-logging.md) diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md index c9edf46bc54..c4af15e85bf 100644 --- a/specification/audit/data-model.md +++ b/specification/audit/data-model.md @@ -18,8 +18,10 @@ weight: 2 - [Relationship to LogRecord](#relationship-to-logrecord) - [OTLP Envelope Layers](#otlp-envelope-layers) - [AuditRecord Definition](#auditrecord-definition) + - [Field: `RecordId`](#field-recordid) - [Field: `Timestamp`](#field-timestamp) - [Field: `ObservedTimestamp`](#field-observedtimestamp) + - [Field: `SchemaVersion`](#field-schemaversion) - [Field: `EventName`](#field-eventname) - [Actor Fields](#actor-fields) - [Field: `Actor`](#field-actor) @@ -34,6 +36,11 @@ weight: 2 - [Field: `Signature`](#field-signature) - [Field: `Algorithm`](#field-algorithm) - [Field: `Certificate`](#field-certificate) + - [Field: `Hmac`](#field-hmac) + - [Field: `HmacAlgorithm`](#field-hmacalgorithm) + - [Optional Ordering Fields](#optional-ordering-fields) + - [Field: `SequenceNo`](#field-sequenceno) + - [Field: `PrevHash`](#field-prevhash) - [AuditReceipt Definition](#auditreceipt-definition) - [Field: `RecordId`](#field-recordid) - [Field: `IntegrityHash`](#field-integrityhash) @@ -101,22 +108,47 @@ application or system code on behalf of an actor. The following table provides a summary of all fields. Detailed descriptions follow. -| Field | Type | Req. | Description | -|---------------------|-------------------------|--------|------------------------------------------------| -| `Timestamp` | `fixed64` | MUST | Event time, ns since UNIX epoch (UTC). | -| `ObservedTimestamp` | `fixed64` | MUST | SDK observation time, ns since UNIX epoch. | -| `EventName` | `string` | MUST | Semantic name of the audit event. | -| `Actor` | `AnyValue` | MUST | Identity that performed the action. | -| `ActorType` | `enum` | MUST | `USER`, `SERVICE`, or `SYSTEM`. | -| `Action` | `string` | MUST | Verb describing what was done. | -| `Outcome` | `enum` | MUST | `SUCCESS`, `FAILURE`, or `UNKNOWN`. | -| `TargetResource` | `AnyValue` | SHOULD | The object acted upon. | -| `SourceIP` | `string` | MAY | Source network address. | -| `Body` | `AnyValue` | MAY | Free-form additional event details. | -| `Attributes` | `map` | MAY | Arbitrary key-value context. | -| `Signature` | `bytes` | MAY | Digital signature of the record. | -| `Algorithm` | `string` | MAY | Signature algorithm (e.g. `ES256`). | -| `Certificate` | `bytes` | MAY | Public certificate for signature verification. | +| Field | Type | Req. | Description | +|---------------------|-------------------------|--------|----------------------------------------------------| +| `RecordId` | `string` | MUST | Caller-generated unique identifier. | +| `Timestamp` | `fixed64` | MUST | Event time, ns since UNIX epoch (UTC). | +| `ObservedTimestamp` | `fixed64` | MUST | SDK observation time, ns since UNIX epoch. | +| `SchemaVersion` | `string` | SHOULD | Schema version of the audit payload. | +| `EventName` | `string` | MUST | Semantic name of the audit event. | +| `Actor` | `AnyValue` | MUST | Identity that performed the action. | +| `ActorType` | `enum` | MUST | `USER`, `SERVICE`, or `SYSTEM`. | +| `Action` | `string` | MUST | Verb describing what was done. | +| `Outcome` | `enum` | MUST | `SUCCESS`, `FAILURE`, or `UNKNOWN`. | +| `TargetResource` | `AnyValue` | SHOULD | The object acted upon. | +| `SourceIP` | `string` | MAY | Source network address. | +| `Body` | `AnyValue` | MAY | Free-form additional event details. | +| `Attributes` | `map` | MAY | Arbitrary key-value context. | +| `Signature` | `bytes` | MAY | Asymmetric digital signature of the record. | +| `Algorithm` | `string` | MAY | Signature algorithm (e.g. `ES256`). | +| `Certificate` | `bytes` | MAY | Public certificate for signature verification. | +| `Hmac` | `bytes` | MAY | Symmetric HMAC of the record (alt. to Signature). | +| `HmacAlgorithm` | `string` | MAY | HMAC algorithm (e.g. `HMAC-SHA256`). | +| `SequenceNo` | `uint64` | MAY | Monotonic counter for hash-chain continuity. | +| `PrevHash` | `string` | MAY | SHA-256 of the previous record in the chain. | + +### Field: `RecordId` + +| Property | Value | +|----------|--------------------------------| +| Type | `string` | +| Required | MUST be set; MUST NOT be empty | + +A caller-generated unique identifier for this `AuditRecord`. The value +MUST be immutable once set and MUST remain stable across retries so +that receivers can deduplicate records delivered more than once. + +The caller SHOULD use a UUID v4 or an equivalent unpredictable unique +identifier. The SDK MUST generate a UUID v4 if the caller does not +provide one. + +`RecordId` is echoed back unchanged in the +[`AuditReceipt`](#auditreceipt-definition), allowing the emitting +application to correlate receipts with the original `emit` call. ### Field: `Timestamp` @@ -157,6 +189,20 @@ sink. A significant difference between `Timestamp` and `ObservedTimestamp` SHOULD be treated as a clock synchronisation warning. +### Field: `SchemaVersion` + +| Property | Value | +|----------|------------------| +| Type | `string` | +| Required | SHOULD be set | + +The version of the audit payload schema used to produce this record. +`SchemaVersion` allows receivers and long-term archives to validate +and interpret records correctly even after the schema evolves. + +The value SHOULD follow semantic versioning (e.g. `1.0.0`). If not +set, the receiver MUST treat the schema version as unknown. + ### Field: `EventName` | Property | Value | @@ -386,6 +432,76 @@ configuration on the relying party side to resolve the key reference. If `Certificate` is omitted, the relying party MUST obtain the public key through a separately configured trust anchor. +#### Field: `Hmac` + +| Property | Value | +|----------|-------------------------------------------| +| Type | `bytes` | +| Required | MAY be set | +| Mutual | MUST NOT be set together with `Signature` | + +A symmetric HMAC over the canonical serialization of the +`AuditRecord`. `Hmac` is an alternative to `Signature` for deployments +where a full asymmetric PKI is not available. The HMAC MUST cover all +mandatory fields plus any `Attributes` and `Body` present at emission +time. + +If `Hmac` is set, `HmacAlgorithm` MUST also be set. +`Hmac` and `Signature` MUST NOT both be set on the same record. + +#### Field: `HmacAlgorithm` + +| Property | Value | +|----------|---------------------------------| +| Type | `string` | +| Required | MUST be set if `Hmac` is set | + +The algorithm used to compute `Hmac`. The value SHOULD be a registered +IANA MAC algorithm identifier, for example `HMAC-SHA256` or +`HMAC-SHA512`. + +### Optional Ordering Fields + +The optional ordering fields enable hash-chain validation across a +sequence of `AuditRecord`s. When populated, receivers can detect +whether records have been deleted, inserted, or reordered by verifying +that the sequence numbers are monotonically increasing and that each +`PrevHash` matches the `IntegrityHash` returned in the preceding +record's `AuditReceipt`. + +Implementations that require strong tamper-evidence for ordered +sequences SHOULD populate both `SequenceNo` and `PrevHash`. + +#### Field: `SequenceNo` + +| Property | Value | +|----------|------------| +| Type | `uint64` | +| Required | MAY be set | + +A monotonically increasing counter, scoped to a single +`AuditLogger` instance or a named audit stream. The first record in a +stream SHOULD have `SequenceNo` equal to `1`. + +A gap between two consecutive `SequenceNo` values indicates that one +or more records were lost or deleted and SHOULD trigger an alert. + +#### Field: `PrevHash` + +| Property | Value | +|----------|------------| +| Type | `string` | +| Required | MAY be set | + +The `IntegrityHash` of the immediately preceding record in the same +audit stream (as returned in the preceding `AuditReceipt`). The first +record in a stream SHOULD set `PrevHash` to the SHA-256 hash of the +empty string (`e3b0c44298fc1c149afb…`). + +A `PrevHash` that does not match the stored `IntegrityHash` of the +previous record indicates tampering and MUST be treated as a critical +integrity violation. + ## AuditReceipt Definition An `AuditReceipt` is returned by `AuditLogger.emit` once the audit @@ -394,18 +510,20 @@ and enables integrity verification by the emitting application. | Field | Type | Required | Description | |-----------------|-----------|-------------|--------------------------------------------------------------| -| `RecordId` | `string` | MUST be set | Unique identifier assigned by the sink. | +| `RecordId` | `string` | MUST be set | Echoes the caller's `AuditRecord.RecordId`. | | `IntegrityHash` | `string` | MUST be set | SHA-256 of the record as persisted by the sink. | | `SinkTimestamp` | `fixed64` | MUST be set | Nanoseconds since UNIX epoch when the sink wrote the record. | ### Field: `RecordId` -A unique, stable identifier for the persisted record, assigned by the -audit sink. The format is sink-specific (for example, a UUID, a -sequential integer, or a content-addressed hash). +The `RecordId` from the corresponding `AuditRecord` as provided by the +caller. The sink MUST echo this value unchanged. Callers use it to +correlate the receipt with the original `emit` call, and to confirm +that the correct record was persisted. -The emitting application MAY store `RecordId` for later retrieval or -cross-referencing. +If the sink generates its own internal identifier it MUST still return +the caller's `RecordId` here. The internal identifier MAY be conveyed +as a separate field in a protocol extension. ### Field: `IntegrityHash` @@ -436,8 +554,10 @@ abnormally long delivery times. ```json { + "RecordId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "Timestamp": 1714041600000000000, "ObservedTimestamp": 1714041600001000000, + "SchemaVersion": "1.0.0", "EventName": "user.login.success", "Actor": "u8472", "ActorType": "USER", @@ -456,8 +576,10 @@ abnormally long delivery times. ```json { + "RecordId": "f7e8d9c0-b1a2-3456-cdef-0987654321ab", "Timestamp": 1714041700000000000, "ObservedTimestamp": 1714041700002000000, + "SchemaVersion": "1.0.0", "EventName": "config.change", "Actor": "svc-deployer", "ActorType": "SERVICE", diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index 809388bdd00..7bdb49f1a13 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -33,6 +33,7 @@ weight: 3 - [ForceFlush](#forceflush-1) - [Built-in processors](#built-in-processors) - [Simple processor](#simple-processor) + - [Batching processor](#batching-processor) - [Signing processor (Sig)](#signing-processor-sig) - [AuditRecordExporter](#auditrecordexporter) - [AuditRecordExporter operations](#auditrecordexporter-operations) @@ -42,10 +43,12 @@ weight: 3 - [OTLP transport requirements](#otlp-transport-requirements) - [Dedicated endpoint](#dedicated-endpoint) - [No partial success](#no-partial-success) + - [Idempotency](#idempotency) - [InstrumentationScope](#instrumentationscope) - [Failure handling](#failure-handling) - [Clock synchronisation](#clock-synchronisation) - [Interaction with the Log signal](#interaction-with-the-log-signal) + - [Collector pipeline (Tier-2)](#collector-pipeline-tier-2) - [Observability](#observability) - [Concurrency requirements](#concurrency-requirements) - [References](#references) @@ -159,19 +162,21 @@ MUST NOT silently drop the record. When `emit` is called, the SDK MUST: -1. Set `ObservedTimestamp` to the current time if the caller did not +1. If the caller did not provide a `RecordId`, generate a UUID v4 and + set it on the record before any further processing. +2. Set `ObservedTimestamp` to the current time if the caller did not provide it. -2. Validate that all required fields (`Timestamp`, `EventName`, - `Actor`, `ActorType`, `Action`, `Outcome`) are present and non-empty. - If any required field is missing, `emit` MUST surface a hard error - to the caller and MUST NOT silently drop the record. -3. Enqueue the `AuditRecord` in the [AuditRecord Queue](#auditrecord-queue). -4. Pass the record through all registered +3. Validate that all required fields (`RecordId`, `Timestamp`, + `EventName`, `Actor`, `ActorType`, `Action`, `Outcome`) are present + and non-empty. If any required field is missing, `emit` MUST surface + a hard error to the caller and MUST NOT silently drop the record. +4. Enqueue the `AuditRecord` in the [AuditRecord Queue](#auditrecord-queue). +5. Pass the record through all registered `AuditRecordProcessor` instances via [`OnEmit`](#onemit). -5. Block until the exporter returns a successful acknowledgement from +6. Block until the exporter returns a successful acknowledgement from the audit sink. -6. Return the [`AuditReceipt`](./data-model.md#auditreceipt-definition) provided +7. Return the [`AuditReceipt`](./data-model.md#auditreceipt-definition) provided by the sink. If the synchronous `emit` cannot obtain an acknowledgement within the @@ -303,6 +308,36 @@ prevent concurrent invocations. The SDK MUST provide the simple processor as a built-in. It is the default processor for synchronous `emit`. +#### Batching processor + +The batching processor accumulates `AuditRecord`s in an internal queue +and exports them in batches asynchronously. It reduces per-record +round-trip overhead while retaining the no-drop guarantee: records +MUST NOT be discarded when the batch queue is full; the processor MUST +apply back-pressure or switch to disk-backed storage instead. + +The batching processor MUST still satisfy the at-least-once delivery +requirement: if the process exits before a batch is exported, buffered +records MUST be recoverable (e.g. via a disk-backed queue). + +**Configurable parameters:** + +- `exporter` – the `AuditRecordExporter` to which batches are pushed. +- `maxQueueSize` – maximum number of records held in memory before + back-pressure is applied. Default: 2048. +- `scheduledDelayMillis` – how long the processor waits before + exporting an incomplete batch. Default: 5000 ms. +- `exportTimeoutMillis` – maximum time allowed for a single `Export` + call. Default: 30 000 ms. +- `maxExportBatchSize` – maximum number of records per exported batch. + Default: 512. +- `maxRetryCount` – maximum export retry attempts before surfacing a + hard error. Default: 5. +- `initialBackoffMillis` – initial wait between retries; doubled on + each attempt (exponential back-off). Default: 1000 ms. + +The SDK SHOULD provide the batching processor as a built-in. + #### Signing processor (Sig) The signing processor adds a digital signature to each `AuditRecord` to @@ -397,6 +432,23 @@ accepted), and retry the full batch. This constraint ensures that no audit record can be silently lost by a receiver that acknowledges only part of a batch. +#### Idempotency + +The OTLP receiver SHOULD treat `RecordId` as an idempotency key. + +When a record with an already-known `RecordId` arrives: + +- If the canonical payload hash is **identical**, the receiver SHOULD + return a deterministic success response for the existing record + without creating a duplicate entry. +- If the canonical payload hash **differs**, the receiver MUST return + a conflict error (HTTP 409 or equivalent). The SDK exporter MUST + treat a conflict error as a non-retryable `Failure`. + +This ensures that retries after transient failures do not produce +duplicate audit entries, provided the caller uses a stable `RecordId` +across retry attempts. + #### InstrumentationScope `InstrumentationScope` has no meaning for audit logging. Audit records @@ -451,6 +503,49 @@ compliance) and a regular `LogRecord` (for observability) for the same event. The records travel through independent pipelines to independent backends and MUST NOT interfere with each other. +## Collector pipeline (Tier-2) + +In enterprise deployments an OpenTelemetry Collector typically sits +between the SDK exporter and the final audit sink. This collector acts +as a **Tier-2** intermediary that can verify record integrity, +distribute records to multiple sinks, and provide delivery tracking. + +``` +Application + │ + ▼ /v1/audit (SDK exporter) +OTel Collector ──► verify hash/signature + │ └── forward to required sinks + ├──► Primary sink (OpenSearch, Splunk, …) + ├──► Secondary sink (S3 WORM, cold archive) + └──► SIEM + +Collector returns delivery status per sink to the SDK exporter. +``` + +The SDK exporter interacts with the Tier-2 collector via the same +`/v1/audit` endpoint; no API changes are required at the SDK level. + +The Tier-2 collector SHOULD: + +- Verify the `IntegrityHash`, `Signature`, or `Hmac` of each received + record against the declared algorithm and key. +- Validate `SequenceNo` continuity and `PrevHash` chain integrity when + these optional fields are present. +- Deliver each record to every configured required sink before + returning a success response to the SDK exporter. +- Return a conflict error (HTTP 409) when a duplicate `RecordId` with + a differing payload hash is received. +- Expose per-sink delivery status in its response so that operators can + detect partial delivery failures. + +The Tier-2 collector MUST NOT respond with partial success (see +[No partial success](#no-partial-success)). If any required sink +rejects the record the collector MUST reject the entire request. + +For the full Tier-2 collector specification see +[Audit Logging Collector](./collector.md). + ## Observability The SDK MUST expose the following internal metrics to allow operators From d048bbd63076b3c6d15aebc8870226a26299ebc8 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Tue, 28 Apr 2026 10:57:50 +0200 Subject: [PATCH 08/16] https://github.com/apeirora/opentelemetry-specification/pull/6 replaces https://github.com/apeirora/opentelemetry-specification/pull/1 Signed-off-by: Hilmar Falkenberg --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f239761a77..27a285a2462 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,7 @@ release. - Introduce Audit-Logging signal and related specification sections. ([#1](https://github.com/apeirora/opentelemetry-specification/pull/1)) + ([#6](https://github.com/apeirora/opentelemetry-specification/pull/6)) ### Baggage From bf04d7f4d1b281e003b2707c5cdb3d771e197a04 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Wed, 29 Apr 2026 11:37:49 +0200 Subject: [PATCH 09/16] Signature/HMAC merged into IntegrityValue Signed-off-by: Hilmar Falkenberg --- oteps/0267-audit-logging.md | 34 ++++---- specification/audit/README.md | 4 +- specification/audit/api.md | 17 ++-- specification/audit/collector.md | 18 ++-- specification/audit/data-model.md | 133 +++++++++++++++--------------- specification/audit/sdk.md | 15 ++-- 6 files changed, 115 insertions(+), 106 deletions(-) diff --git a/oteps/0267-audit-logging.md b/oteps/0267-audit-logging.md index 788af778cfe..5effd584aba 100644 --- a/oteps/0267-audit-logging.md +++ b/oteps/0267-audit-logging.md @@ -124,9 +124,7 @@ failure. | `SourceIP` | `string` | Source network address, if applicable. | | `Body` | `AnyValue` | Free-form text or protobuf with additional details about the event. | | `Attributes` | `map` | Arbitrary key-value pairs for additional context. | -| `Signature` | `bytes` | Digital signature of the audit record for integrity verification. | -| `Algorithm` | `string` | Algorithm used for the digital signature (e.g., `RS256`, `ES256`). | -| `Certificate` | `bytes` | Digital public certificate used for the signature verification. | +| `IntegrityValue` | `bytes` | Cryptographic integrity proof (signature or HMAC); algorithm in `audit.integrity.algorithm`, key in `audit.integrity.certificate`. | ### AuditReceipt @@ -178,11 +176,19 @@ records. The three OTLP envelope layers that **do** apply to audit logging are: -| OTLP layer | Role in audit logging | -|--------------------|----------------------------------------------------------------| -| `Resource` | Identifies the emitting service/host – reused without change. | -| `LogRecord` (body) | Carries the `AuditRecord` payload. | -| `Attributes` | Key-value context (user ID, IP, outcome, …) – reused as-is. | +| OTLP layer | Role in audit logging | +|--------------------|-------------------------------------------------------------| +| `Resource` | Emitting service / host; carries integrity attrs. | +| `LogRecord` (body) | Carries the `AuditRecord` payload. | +| `Attributes` | Key-value context (user ID, IP, outcome, …) – reused as-is. | + +The `Resource` MUST carry the following attributes whenever any +record in the batch includes an `IntegrityValue`: + +| Attribute | Type | Description | +|-------------------------------|----------|-------------------------------------------------------------------------------------------------------------| +| `audit.integrity.algorithm` | `string` | Algorithm used to compute `IntegrityValue` (JWA identifier for signatures, IANA MAC identifier for HMACs). | +| `audit.integrity.certificate` | `string` | Key reference: full DER cert (base64), fingerprint, Key ID, SKI, or Issuer+Serial. Not for HMAC algorithms. | #### Durability at the sink is out of scope @@ -240,12 +246,12 @@ If the exporter cannot reach the sink: ## Trade-offs and mitigations -| Trade-off | Mitigation | -|------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------| -| Synchronous `emit` adds latency to the calling thread | An async variant with a durable local queue mitigates this; the queue size and flush interval are configurable. | -| A separate pipeline increases SDK complexity | The audit pipeline deliberately reuses `LogRecord` as its wire format, so OTLP exporters can be reused with minimal changes. | -| SHA-256 receipt requires a round-trip to the sink | The receipt is optional for callers that do not need proof-of-delivery; a fire-and-forget async mode MAY omit it. | -| Disk-backed queue introduces a dependency on localStorage | This is opt-in; the default is an in-memory queue with blocking back-pressure. | +| Trade-off | Mitigation | +|-----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------| +| Synchronous `emit` adds latency to the calling thread | An async variant with a durable local queue mitigates this; the queue size and flush interval are configurable. | +| A separate pipeline increases SDK complexity | The audit pipeline deliberately reuses `LogRecord` as its wire format, so OTLP exporters can be reused with minimal changes. | +| SHA-256 receipt requires a round-trip to the sink | The receipt is optional for callers that do not need proof-of-delivery; a fire-and-forget async mode MAY omit it. | +| Disk-backed queue introduces a dependency on localStorage | This is opt-in; the default is an in-memory queue with blocking back-pressure. | ## Prior art and alternatives diff --git a/specification/audit/README.md b/specification/audit/README.md index e4889b55ce1..95130644f05 100644 --- a/specification/audit/README.md +++ b/specification/audit/README.md @@ -100,7 +100,7 @@ Application code │ └── AuditRecordProcessor (enrich only – never redact or drop) │ ├── Simple processor (sync, default) │ ├── Batching processor (async, high-volume) - │ └── Signing processor (adds Signature or Hmac) + │ └── Signing processor (adds IntegrityValue) ▼ AuditExporter ──► Tier-2 Collector (optional) ──► Audit sinks or ──► Audit sink directly │ @@ -143,7 +143,7 @@ payload carrier. The following OTLP envelope layers apply: | OTLP layer | Role in audit logging | |------------------------|--------------------------------------------------------| -| `Resource` | Identifies the emitting service / host – reused as-is. | +| `Resource` | Emitting service / host; integrity attrs on Resource. | | `LogRecord` (body) | Carries the `AuditRecord` payload. | | `Attributes` | Key-value context (actor, outcome, …) – reused as-is. | | `InstrumentationScope` | **Not applicable.** MUST be left empty by exporters. | diff --git a/specification/audit/api.md b/specification/audit/api.md index dd218a6751d..c8511c2d3fa 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -150,17 +150,12 @@ The API MUST accept the following optional parameters: event details. - [`Attributes`](./data-model.md#field-attributes) (optional): arbitrary key-value context pairs. -- [`Signature`](./data-model.md#field-signature) (optional): digital - signature over the record. -- [`Algorithm`](./data-model.md#field-algorithm) (optional): the - signature algorithm; MUST be set if `Signature` is set. -- [`Certificate`](./data-model.md#field-certificate) (optional): the - public-key certificate for signature verification. -- [`Hmac`](./data-model.md#field-hmac) (optional): symmetric HMAC over - the record (alternative to `Signature`). MUST NOT be set together - with `Signature`. -- [`HmacAlgorithm`](./data-model.md#field-hmacalgorithm) (optional): - the HMAC algorithm; MUST be set if `Hmac` is set. +- [`IntegrityValue`](./data-model.md#field-integrityvalue) (optional): + cryptographic integrity proof over the record – either an asymmetric + digital signature or a symmetric HMAC. The signing algorithm is + configured via the `audit.integrity.algorithm` `Resource` attribute + on the `AuditProvider`; the key reference via + `audit.integrity.certificate`. - [`SequenceNo`](./data-model.md#field-sequenceno) (optional): monotonic counter for hash-chain continuity. - [`PrevHash`](./data-model.md#field-prevhash) (optional): SHA-256 of diff --git a/specification/audit/collector.md b/specification/audit/collector.md index 46549aaab00..37c12f47342 100644 --- a/specification/audit/collector.md +++ b/specification/audit/collector.md @@ -106,8 +106,10 @@ Each record MUST include: Each record MAY include any field defined in the [AuditRecord Definition](./data-model.md#auditrecord-definition), -including `SchemaVersion`, `Signature`, `Algorithm`, `Certificate`, -`Hmac`, `HmacAlgorithm`, `SequenceNo`, and `PrevHash`. +including `SchemaVersion`, `IntegrityValue`, `SequenceNo`, and +`PrevHash`. The signing algorithm and key reference are carried as +`Resource` attributes `audit.integrity.algorithm` and +`audit.integrity.certificate`. ## Response Model @@ -190,9 +192,10 @@ When a duplicate `RecordId` is received: ### Integrity Verification -When `Signature` or `Hmac` is present the collector SHOULD verify the -field against the declared algorithm and the public key or shared -secret configured in the collector's trust policy. +When `IntegrityValue` is present the collector SHOULD verify it +against the algorithm declared in the `Resource` attribute +`audit.integrity.algorithm` and the key material referenced by +`audit.integrity.certificate` in the collector's trust policy. The collector MAY defer verification for low-latency ingest (returning `accepted_pending_verify`) but MUST complete verification before @@ -277,8 +280,9 @@ The collector ingress MUST require secure transport and SHOULD support: - Token-based authentication (e.g. Bearer token) where mTLS is not used. -The collector SHOULD validate `Signature` or `Hmac` fields according -to a configured verification policy before forwarding records to sinks. +The collector SHOULD validate `IntegrityValue` according to the +algorithm in `audit.integrity.algorithm` and the configured +verification policy before forwarding records to sinks. ## References diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md index c4af15e85bf..46f7ae60f7e 100644 --- a/specification/audit/data-model.md +++ b/specification/audit/data-model.md @@ -33,11 +33,8 @@ weight: 2 - [Field: `Body`](#field-body) - [Field: `Attributes`](#field-attributes) - [Integrity Fields](#integrity-fields) - - [Field: `Signature`](#field-signature) - - [Field: `Algorithm`](#field-algorithm) - - [Field: `Certificate`](#field-certificate) - - [Field: `Hmac`](#field-hmac) - - [Field: `HmacAlgorithm`](#field-hmacalgorithm) + - [Field: `IntegrityValue`](#field-integrityvalue) + - [Integrity Resource Attributes](#integrity-resource-attributes) - [Optional Ordering Fields](#optional-ordering-fields) - [Field: `SequenceNo`](#field-sequenceno) - [Field: `PrevHash`](#field-prevhash) @@ -69,8 +66,9 @@ requirements: - The data model MUST include timestamps that allow clock-skew detection across distributed services (ISO 27001 – clock synchronisation). -- The data model MUST support optional digital signatures so that - records can be verified for integrity after delivery. +- The data model MUST support optional cryptographic integrity proofs + (asymmetric digital signatures or symmetric HMACs) so that records + can be verified for integrity after delivery. - The data model MUST be efficiently representable in OTLP by reusing the existing `LogRecord` proto message as a transport container. @@ -95,11 +93,15 @@ The following OTLP envelope layers apply to audit records: | OTLP layer | Role in audit logging | |------------------------|--------------------------------------------------------| -| `Resource` | Identifies the emitting service / host – reused as-is. | +| `Resource` | Emitting service / host; integrity attrs (see below). | | `LogRecord` | Carries the `AuditRecord` payload (see mapping below). | | `Attributes` | Key-value context – reused as-is. | | `InstrumentationScope` | **Not applicable.** MUST be left empty. | +The `Resource` also carries `audit.integrity.algorithm` and +`audit.integrity.certificate` — see +[Integrity Resource Attributes](#integrity-resource-attributes). + ## AuditRecord Definition An `AuditRecord` is a single security-relevant event emitted by @@ -123,11 +125,7 @@ descriptions follow. | `SourceIP` | `string` | MAY | Source network address. | | `Body` | `AnyValue` | MAY | Free-form additional event details. | | `Attributes` | `map` | MAY | Arbitrary key-value context. | -| `Signature` | `bytes` | MAY | Asymmetric digital signature of the record. | -| `Algorithm` | `string` | MAY | Signature algorithm (e.g. `ES256`). | -| `Certificate` | `bytes` | MAY | Public certificate for signature verification. | -| `Hmac` | `bytes` | MAY | Symmetric HMAC of the record (alt. to Signature). | -| `HmacAlgorithm` | `string` | MAY | HMAC algorithm (e.g. `HMAC-SHA256`). | +| `IntegrityValue` | `bytes` | MAY | Cryptographic integrity proof (signature or HMAC). | | `SequenceNo` | `uint64` | MAY | Monotonic counter for hash-chain continuity. | | `PrevHash` | `string` | MAY | SHA-256 of the previous record in the chain. | @@ -389,76 +387,79 @@ The integrity fields enable a relying party to verify that an SHOULD be set in environments where tamper evidence is required by the applicable compliance framework. -#### Field: `Signature` +#### Field: `IntegrityValue` | Property | Value | |----------|------------| | Type | `bytes` | | Required | MAY be set | -A digital signature over the canonical serialization of the -`AuditRecord`. The signature MUST cover all mandatory fields plus any +A cryptographic integrity proof over the canonical serialization of the +`AuditRecord`. `IntegrityValue` MUST cover all mandatory fields plus any `Attributes` and `Body` that are present at emission time. -If `Signature` is set, `Algorithm` MUST also be set. - -#### Field: `Algorithm` - -| Property | Value | -|----------|-----------------------------------| -| Type | `string` | -| Required | MUST be set if `Signature` is set | - -The algorithm used to compute `Signature`. The value SHOULD be a -registered JWA algorithm identifier, for example `RS256`, `ES256`, or -`EdDSA`. - -#### Field: `Certificate` +The proof is either an asymmetric digital signature or a symmetric +HMAC, as indicated by `audit.integrity.algorithm`. If +`IntegrityValue` is set, `audit.integrity.algorithm` MUST be set as +a `Resource` attribute (see +[Integrity Resource Attributes](#integrity-resource-attributes)). -| Property | Value | -|----------|------------| -| Type | `bytes` | -| Required | MAY be set | +### Integrity Resource Attributes -The public-key certificate (DER-encoded X.509) corresponding to the -signing key, included so that relying parties can verify `Signature` -without an out-of-band key lookup. +The signing algorithm and key reference are identical for every +`AuditRecord` emitted by a given service instance. They are therefore +carried once as OTel `Resource` attributes instead of being repeated +in every record. -Alternatively, Key ID, Subject Key Identifier (SKI), Issuer + SerialNumber or -other key reference MAY be included as an attribute instead of the full -certificate, but this is less self-contained and requires additional -configuration on the relying party side to resolve the key reference. +#### Attribute: `audit.integrity.algorithm` -If `Certificate` is omitted, the relying party MUST obtain the public -key through a separately configured trust anchor. +| Property | Value | +|----------|----------------------------------------| +| Type | `string` | +| Required | MUST be set if `IntegrityValue` is set | -#### Field: `Hmac` +The algorithm used to compute `IntegrityValue` for all records +produced by this resource. -| Property | Value | -|----------|-------------------------------------------| -| Type | `bytes` | -| Required | MAY be set | -| Mutual | MUST NOT be set together with `Signature` | +- For asymmetric signatures the value SHOULD be a registered JWA + algorithm identifier, for example `RS256`, `ES256`, or `EdDSA`. +- For symmetric HMACs the value SHOULD be a registered IANA MAC + algorithm identifier, for example `HMAC-SHA256` or `HMAC-SHA512`. -A symmetric HMAC over the canonical serialization of the -`AuditRecord`. `Hmac` is an alternative to `Signature` for deployments -where a full asymmetric PKI is not available. The HMAC MUST cover all -mandatory fields plus any `Attributes` and `Body` present at emission -time. +#### Attribute: `audit.integrity.certificate` -If `Hmac` is set, `HmacAlgorithm` MUST also be set. -`Hmac` and `Signature` MUST NOT both be set on the same record. - -#### Field: `HmacAlgorithm` - -| Property | Value | -|----------|---------------------------------| -| Type | `string` | -| Required | MUST be set if `Hmac` is set | +| Property | Value | +|----------|------------| +| Type | `string` | +| Required | MAY be set | -The algorithm used to compute `Hmac`. The value SHOULD be a registered -IANA MAC algorithm identifier, for example `HMAC-SHA256` or -`HMAC-SHA512`. +A reference to the signing key used to produce `IntegrityValue`. +MUST NOT be set when `audit.integrity.algorithm` denotes a symmetric +MAC algorithm. + +The value MUST be one of the following forms, listed from most to +least self-contained: + +- **Full certificate** – base64 (standard encoding, no line wrapping) + of the DER-encoded X.509 public-key certificate. Relying parties + can verify `IntegrityValue` without any out-of-band lookup. +- **Fingerprint** – A hash string in the form `sha256:` or + `sha1:`, where `` is the colon-separated hex encoding of + the DER certificate hash (e.g. `sha256:4A:6C:9F:…`). Requires + matching against a locally trusted certificate. +- **Key ID** – An opaque string identifier (e.g. a JWK `kid` claim + such as `key-2024-01`) agreed between emitter and relying party. + Requires key retrieval via JWK Set URI or equivalent. +- **SKI** – Subject Key Identifier in colon-separated hex + (e.g. `3C:F0:…`). Requires a PKI lookup. +- **Issuer + Serial** – Issuer Distinguished Name and serial number + separated by a slash (e.g. `CN=MyCA,O=Acme Corp/12345`). Requires + a PKI lookup. + +If `audit.integrity.certificate` is omitted and +`audit.integrity.algorithm` denotes an asymmetric algorithm, the +relying party MUST obtain the public key through a separately +configured trust anchor. ### Optional Ordering Fields diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index 7bdb49f1a13..7dc99571884 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -340,10 +340,12 @@ The SDK SHOULD provide the batching processor as a built-in. #### Signing processor (Sig) -The signing processor adds a digital signature to each `AuditRecord` to -guarantee integrity and non-repudiation. It MUST be implemented as a separate -processor that can be added to the pipeline in addition to the simple -processor, rather than as a modification to the simple processor, to allow +The signing processor computes an `IntegrityValue` for each `AuditRecord` +to provide tamper-evidence. The value is either an asymmetric digital +signature or a symmetric HMAC, as determined by the +`audit.integrity.algorithm` `Resource` attribute configured on the +`AuditProvider`. It MUST be implemented as a separate processor that can +be added to the pipeline in addition to the simple processor, to allow flexibility in the choice of signing algorithm and key management strategy. ## AuditRecordExporter @@ -528,8 +530,9 @@ The SDK exporter interacts with the Tier-2 collector via the same The Tier-2 collector SHOULD: -- Verify the `IntegrityHash`, `Signature`, or `Hmac` of each received - record against the declared algorithm and key. +- Verify `IntegrityValue` of each received record against the algorithm + declared in the `Resource` attribute `audit.integrity.algorithm` and + the key material referenced by `audit.integrity.certificate`. - Validate `SequenceNo` continuity and `PrevHash` chain integrity when these optional fields are present. - Deliver each record to every configured required sink before From cf85f350d79ec4280bf7430294f3703c4b1cc306 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Wed, 29 Apr 2026 14:47:16 +0200 Subject: [PATCH 10/16] `AuditRecord` is a `LogRecord` ... ... that uses the standard dedicated `LogRecord` fields for universal concepts and carries all audit-specific data as `Attributes` following the `audit.*` semantic convention namespace Signed-off-by: Hilmar Falkenberg --- oteps/0267-audit-logging.md | 88 +++-- specification/audit/README.md | 4 +- specification/audit/api.md | 75 ++-- specification/audit/collector.md | 47 +-- specification/audit/data-model.md | 595 ++++++++++++++---------------- specification/audit/sdk.md | 55 +-- 6 files changed, 433 insertions(+), 431 deletions(-) diff --git a/oteps/0267-audit-logging.md b/oteps/0267-audit-logging.md index 5effd584aba..7d1ed2c1074 100644 --- a/oteps/0267-audit-logging.md +++ b/oteps/0267-audit-logging.md @@ -109,22 +109,56 @@ failure. ### AuditRecord data model -`AuditRecord` is quite similar to `LogRecord` with the following mandatory fields: - -| Field | Type | Description | -|---------------------|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------| -| `Timestamp` | `fixed64` | Nanoseconds since UNIX epoch (UTC). MUST be set. `<= ObservedTimestamp <= now(UTC)` | -| `ObservedTimestamp` | `fixed64` | Nanoseconds since UNIX epoch when the SDK observed the event. Used for clock skew detection. MUST be set. `>= Timestamp <= now(UTC)` | -| `EventName` | `string` | Semantic name of the audit event (e.g. `user.login.success`). MUST be set. | -| `Actor` | `AnyValue` | Identity that performed the action (user ID, service account, …). MUST be set. | -| `ActorType` | `enum` | `USER`, `SERVICE`, `SYSTEM`. MUST be set. | -| `Outcome` | `enum` | `SUCCESS`, `FAILURE`, `UNKNOWN`. MUST be set. | -| `Resource` | `AnyValue` | The target resource (file, endpoint, table, …). SHOULD be set. | -| `Action` | `string` | Verb describing what was done (e.g. `READ`, `WRITE`, `DELETE`, `LOGIN`). MUST be set. | -| `SourceIP` | `string` | Source network address, if applicable. | -| `Body` | `AnyValue` | Free-form text or protobuf with additional details about the event. | -| `Attributes` | `map` | Arbitrary key-value pairs for additional context. | -| `IntegrityValue` | `bytes` | Cryptographic integrity proof (signature or HMAC); algorithm in `audit.integrity.algorithm`, key in `audit.integrity.certificate`. | +`AuditRecord` is a `LogRecord` that uses the standard dedicated +`LogRecord` fields for universal concepts and carries all +audit-specific data as `Attributes` following the `audit.*` semantic +convention namespace. + +#### LogRecord fields used + +| Field | Type | Description | +|---------------------|-------------------------|-----------------------------------------------------------------------------------| +| `Timestamp` | `fixed64` | Time the auditable action occurred. MUST be set. | +| `ObservedTimestamp` | `fixed64` | Time the SDK observed the event. MUST be set. | +| `EventName` | `string` | Semantic name of the audit event. MUST be set. | +| `Body` | `AnyValue` | Free-form or structured event details. MAY be set. | +| `Attributes` | `map` | Carries audit semantic attributes (see below). MUST set all mandatory attributes. | +| `Resource` | – | Emitting service / host; carries integrity attributes. | + +`SeverityNumber`, `SeverityText`, and `InstrumentationScope` SHOULD NOT +be set / SHOULD be left empty for audit records. + +#### Audit semantic attributes + +Audit-specific data is carried in `Attributes` using the `audit.*` +semantic convention namespace, or by reusing existing OTel semantic +convention attributes where a direct mapping exists. + +**Mandatory attributes:** + +| Attribute name | Type | Description | +|--------------------|----------|-------------------------------------| +| `audit.record.id` | `string` | Unique stable identifier (UUID v4). | +| `audit.actor.id` | `string` | Identity that performed the action. | +| `audit.actor.type` | `string` | `user`, `service`, or `system`. | +| `audit.action` | `string` | Verb: `LOGIN`, `READ`, `DELETE`, … | +| `audit.outcome` | `string` | `success`, `failure`, or `unknown`. | + +**Optional / recommended attributes:** + +| Attribute name | Type | Description | +|-------------------------|----------|-------------------------------------------------------| +| `audit.target.id` | `string` | Identifier of the resource acted upon. | +| `audit.target.type` | `string` | Type of the target resource. | +| `audit.source.id` | `string` | Network address or identifier of the source. | +| `audit.source.type` | `string` | Type of the source (e.g. `ipv4`, `ipv6`, `hostname`). | +| `audit.integrity.value` | `string` | Base64-encoded signature or HMAC. | +| `audit.sequence.number` | `int` | Monotonic counter for hash-chain continuity. | +| `audit.prev.hash` | `string` | SHA-256 of the preceding record. | +| `audit.schema.version` | `string` | Schema version (e.g. `1.0.0`). | + +The `Resource` carries `audit.integrity.algorithm` and +`audit.integrity.certificate` (unchanged per-service metadata). ### AuditReceipt @@ -133,7 +167,7 @@ the exporter/sink: | Field | Type | Description | |-----------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------| -| `RecordId` | `string` | Unique, stable identifier assigned by the sink. | +| `RecordId` | `string` | Echoes the `audit.record.id` attribute of the persisted record. | | `IntegrityHash` | `string` | SHA-256 of the canonical serialization of the `AuditRecord`, computed by the sink after successful write. Used for integrity verification. | | `SinkTimestamp` | `fixed64` | Nanoseconds since UNIX epoch when the sink persisted the record. | @@ -176,20 +210,24 @@ records. The three OTLP envelope layers that **do** apply to audit logging are: -| OTLP layer | Role in audit logging | -|--------------------|-------------------------------------------------------------| -| `Resource` | Emitting service / host; carries integrity attrs. | -| `LogRecord` (body) | Carries the `AuditRecord` payload. | -| `Attributes` | Key-value context (user ID, IP, outcome, …) – reused as-is. | +| OTLP layer | Role in audit logging | +|--------------|--------------------------------------------------------------------| +| `Resource` | Emitting service / host; carries integrity attrs. | +| `LogRecord` | Carries `AuditRecord` via dedicated fields and `Attributes`. | +| `Attributes` | Mandatory and optional `audit.*` attributes; see data model above. | The `Resource` MUST carry the following attributes whenever any -record in the batch includes an `IntegrityValue`: +record in the batch includes an `audit.integrity.value`: -| Attribute | Type | Description | -|-------------------------------|----------|-------------------------------------------------------------------------------------------------------------| +| Attribute | Type | Description | +|-------------------------------|----------|---------------------------------------| | `audit.integrity.algorithm` | `string` | Algorithm used to compute `IntegrityValue` (JWA identifier for signatures, IANA MAC identifier for HMACs). | | `audit.integrity.certificate` | `string` | Key reference: full DER cert (base64), fingerprint, Key ID, SKI, or Issuer+Serial. Not for HMAC algorithms. | +See +[Integrity Resource Attributes](../specification/audit/data-model.md#integrity-resource-attributes) +for the full field descriptions. + #### Durability at the sink is out of scope This OTEP does not prescribe how the audit sink (e.g. OpenSearch, Splunk, diff --git a/specification/audit/README.md b/specification/audit/README.md index 95130644f05..23f973cae35 100644 --- a/specification/audit/README.md +++ b/specification/audit/README.md @@ -144,8 +144,8 @@ payload carrier. The following OTLP envelope layers apply: | OTLP layer | Role in audit logging | |------------------------|--------------------------------------------------------| | `Resource` | Emitting service / host; integrity attrs on Resource. | -| `LogRecord` (body) | Carries the `AuditRecord` payload. | -| `Attributes` | Key-value context (actor, outcome, …) – reused as-is. | +| `LogRecord` | Carries AuditRecord via dedicated fields + Attributes. | +| `Attributes` | Mandatory and optional `audit.*` semantic attributes. | | `InstrumentationScope` | **Not applicable.** MUST be left empty by exporters. | ## Specifications diff --git a/specification/audit/api.md b/specification/audit/api.md index c8511c2d3fa..033e6837ef3 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -119,53 +119,46 @@ returning an `AuditReceipt`. The API MUST accept the following parameters: -- [`RecordId`](./data-model.md#field-recordid) (required): a - caller-generated unique identifier for this record. If the caller - does not provide one the SDK MUST generate a UUID v4. The value - MUST be stable across retries of the same event. -- [`Timestamp`](./data-model.md#field-timestamp) (required): the time - at which the auditable action occurred. -- [`EventName`](./data-model.md#field-eventname) (required): the +- [`Timestamp`](./data-model.md#logrecord-field-usage) (required): the + time at which the auditable action occurred. +- [`EventName`](./data-model.md#logrecord-field-usage) (required): the semantic name of the audit event. -- [`Actor`](./data-model.md#field-actor) (required): the identity that - performed the action. -- [`ActorType`](./data-model.md#field-actortype) (required): the type - of the actor (`USER`, `SERVICE`, or `SYSTEM`). -- [`Action`](./data-model.md#field-action) (required): the verb - describing what was done. -- [`Outcome`](./data-model.md#field-outcome) (required): the result of - the action (`SUCCESS`, `FAILURE`, or `UNKNOWN`). +- [`Attributes`](./data-model.md#audit-semantic-attributes) (required): + a map of key-value pairs that carries all audit-specific data. + The following attribute keys MUST be present and non-empty: + + | Attribute key | Type | Description | + |---------------------|----------|--------------------------------------| + | `audit.actor.id` | `string` | Identity that performed the action. | + | `audit.actor.type` | `string` | `user`, `service`, or `system`. | + | `audit.action` | `string` | Verb describing what was done. | + | `audit.outcome` | `string` | `success`, `failure`, or `unknown`. | + + The attribute key `audit.record.id` is also required in every + persisted record. If the caller omits it, the SDK MUST generate a + UUID v4 and inject it into `Attributes` before any further + processing. The value MUST be stable across retries of the same + event. The API MUST accept the following optional parameters: -- [`ObservedTimestamp`](./data-model.md#field-observedtimestamp) - (optional): if not set, the SDK MUST set this to the current time. -- [`SchemaVersion`](./data-model.md#field-schemaversion) (optional): - the schema version of the audit payload. SHOULD be set. -- [`TargetResource`](./data-model.md#field-targetresource) (optional): - the object upon which the action was performed. -- [`SourceIP`](./data-model.md#field-sourceip) (optional): the source - network address. -- [`Body`](./data-model.md#field-body) (optional): free-form additional - event details. -- [`Attributes`](./data-model.md#field-attributes) (optional): - arbitrary key-value context pairs. -- [`IntegrityValue`](./data-model.md#field-integrityvalue) (optional): - cryptographic integrity proof over the record – either an asymmetric - digital signature or a symmetric HMAC. The signing algorithm is - configured via the `audit.integrity.algorithm` `Resource` attribute - on the `AuditProvider`; the key reference via - `audit.integrity.certificate`. -- [`SequenceNo`](./data-model.md#field-sequenceno) (optional): - monotonic counter for hash-chain continuity. -- [`PrevHash`](./data-model.md#field-prevhash) (optional): SHA-256 of - the preceding record in the audit stream. +- [`ObservedTimestamp`](./data-model.md#logrecord-field-usage) + (optional): if not provided, the SDK MUST set it to the current time. +- [`Body`](./data-model.md#logrecord-field-usage) (optional): free-form + or structured event details. + +Additional optional attributes MAY be provided in `Attributes`. See +the [Audit Record Data Model](./data-model.md#audit-semantic-attributes) +for the full list, which includes `audit.target.id`, `audit.target.type`, +`audit.source.id`, `audit.source.type`, `audit.integrity.value`, +`audit.sequence.number`, +`audit.prev.hash`, and `audit.schema.version`. **Return value**: The API MUST return an -[`AuditReceipt`](./data-model.md#auditreceipt-definition) when the audit sink -acknowledges that the record has been persisted. The receipt echoes the -caller's `RecordId` and contains an `IntegrityHash` and a -`SinkTimestamp`. +[`AuditReceipt`](./data-model.md#auditreceipt-definition) when the +audit sink acknowledges that the record has been persisted. The receipt +echoes the caller's `audit.record.id` and contains an `IntegrityHash` +and a `SinkTimestamp`. **Delivery semantics**: `emit` is synchronous by default. It MUST block the calling thread until the exporter receives a successful diff --git a/specification/audit/collector.md b/specification/audit/collector.md index 37c12f47342..3e1f12b85ff 100644 --- a/specification/audit/collector.md +++ b/specification/audit/collector.md @@ -92,24 +92,27 @@ Request payloads MUST follow the ### Required Record Fields -Each record MUST include: +Each record MUST include the following LogRecord fields: -- `RecordId` - `Timestamp` - `EventName` -- `Actor` -- `ActorType` -- `Action` -- `Outcome` + +And the following mandatory attributes in the `Attributes` map: + +- `audit.record.id` +- `audit.actor.id` +- `audit.actor.type` +- `audit.action` +- `audit.outcome` ### Optional Record Fields -Each record MAY include any field defined in the -[AuditRecord Definition](./data-model.md#auditrecord-definition), -including `SchemaVersion`, `IntegrityValue`, `SequenceNo`, and -`PrevHash`. The signing algorithm and key reference are carried as -`Resource` attributes `audit.integrity.algorithm` and -`audit.integrity.certificate`. +Each record MAY include any attribute defined in the +[Audit Semantic Attributes](./data-model.md#audit-semantic-attributes) +section, including `audit.schema.version`, `audit.integrity.value`, +`audit.sequence.number`, and `audit.prev.hash`. The signing algorithm +and key reference are carried as `Resource` attributes +`audit.integrity.algorithm` and `audit.integrity.certificate`. ## Response Model @@ -177,9 +180,10 @@ the entire batch (see ## Idempotency and Duplicate Handling -The collector MUST treat `RecordId` as the primary idempotency key. +The collector MUST treat `audit.record.id` as the primary idempotency +key. -When a duplicate `RecordId` is received: +When a duplicate `audit.record.id` is received: - If the canonical payload hash is **identical**, the collector SHOULD return a deterministic success response for the existing record @@ -192,8 +196,8 @@ When a duplicate `RecordId` is received: ### Integrity Verification -When `IntegrityValue` is present the collector SHOULD verify it -against the algorithm declared in the `Resource` attribute +When `audit.integrity.value` is present the collector SHOULD verify +it against the algorithm declared in the `Resource` attribute `audit.integrity.algorithm` and the key material referenced by `audit.integrity.certificate` in the collector's trust policy. @@ -209,12 +213,13 @@ If verification fails the collector MUST: ### Hash-Chain Validation -When `SequenceNo` and `PrevHash` are present the collector SHOULD -validate chain continuity: +When `audit.sequence.number` and `audit.prev.hash` are present the +collector SHOULD validate chain continuity: -- `SequenceNo` MUST be strictly greater than the previous record's - `SequenceNo` in the same audit stream. -- `PrevHash` MUST equal the `IntegrityHash` of the preceding record. +- `audit.sequence.number` MUST be strictly greater than the previous + record's `audit.sequence.number` in the same audit stream. +- `audit.prev.hash` MUST equal the `IntegrityHash` of the preceding + record. A broken chain SHOULD be surfaced as a warning metric and logged as a security event. The collector MAY still accept and forward a broken diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md index 46f7ae60f7e..47e32c2050a 100644 --- a/specification/audit/data-model.md +++ b/specification/audit/data-model.md @@ -18,30 +18,27 @@ weight: 2 - [Relationship to LogRecord](#relationship-to-logrecord) - [OTLP Envelope Layers](#otlp-envelope-layers) - [AuditRecord Definition](#auditrecord-definition) - - [Field: `RecordId`](#field-recordid) - - [Field: `Timestamp`](#field-timestamp) - - [Field: `ObservedTimestamp`](#field-observedtimestamp) - - [Field: `SchemaVersion`](#field-schemaversion) - - [Field: `EventName`](#field-eventname) - - [Actor Fields](#actor-fields) - - [Field: `Actor`](#field-actor) - - [Field: `ActorType`](#field-actortype) - - [Field: `Action`](#field-action) - - [Field: `Outcome`](#field-outcome) - - [Field: `TargetResource`](#field-targetresource) - - [Field: `SourceIP`](#field-sourceip) - - [Field: `Body`](#field-body) - - [Field: `Attributes`](#field-attributes) - - [Integrity Fields](#integrity-fields) - - [Field: `IntegrityValue`](#field-integrityvalue) - - [Integrity Resource Attributes](#integrity-resource-attributes) - - [Optional Ordering Fields](#optional-ordering-fields) - - [Field: `SequenceNo`](#field-sequenceno) - - [Field: `PrevHash`](#field-prevhash) + - [LogRecord Field Usage](#logrecord-field-usage) + - [Field: `Timestamp`](#field-timestamp) + - [Field: `ObservedTimestamp`](#field-observedtimestamp) + - [Field: `EventName`](#field-eventname) + - [Field: `Body`](#field-body) + - [Field: `Attributes`](#field-attributes) + - [Audit Semantic Attributes](#audit-semantic-attributes) + - [Mandatory Attributes](#mandatory-attributes) + - [Attribute: `audit.record.id`](#attribute-auditrecordid) + - [Actor Attributes](#actor-attributes) + - [Attribute: `audit.action`](#attribute-auditaction) + - [Attribute: `audit.outcome`](#attribute-auditoutcome) + - [Optional Attributes](#optional-attributes) + - [Target Attributes](#target-attributes) + - [Source Attributes](#source-attributes) + - [Integrity Attributes](#integrity-attributes) + - [Ordering Attributes](#ordering-attributes) + - [Integrity Resource Attributes](#integrity-resource-attributes) + - [Attribute: `audit.integrity.algorithm`](#attribute-auditintegrityalgorithm) + - [Attribute: `audit.integrity.certificate`](#attribute-auditintegritycertificate) - [AuditReceipt Definition](#auditreceipt-definition) - - [Field: `RecordId`](#field-recordid) - - [Field: `IntegrityHash`](#field-integrityhash) - - [Field: `SinkTimestamp`](#field-sinktimestamp) - [Example AuditRecords](#example-auditrecords) - [Successful user login](#successful-user-login) - [Failed privileged configuration change](#failed-privileged-configuration-change) @@ -78,25 +75,31 @@ requirements: ### Relationship to LogRecord -`AuditRecord` is transported as an OTLP `LogRecord`. The mandatory -audit-specific fields are stored in dedicated `LogRecord` fields where -a natural mapping exists, and in the `Attributes` map otherwise. The -`Body` field carries the free-form `AuditRecord.Body` payload. +An `AuditRecord` is a `LogRecord` that uses the standard dedicated +`LogRecord` fields for the universal concepts they represent, and +carries all audit-specific information as `Attributes` following the +`audit.*` semantic convention namespace. + +This follows the established OTel convention: universal concepts +(timestamps, event name, body, context) receive dedicated top-level +`LogRecord` fields; signal- and domain-specific information is +expressed as semantic-convention attributes. Audit records are +therefore processable by any OTel tooling that understands `LogRecord`s. The `SeverityNumber` and `SeverityText` fields of the underlying -`LogRecord` MUST NOT be used for audit records. Severity is not a -meaningful concept for audit events; `Outcome` serves a similar purpose. +`LogRecord` SHOULD NOT be set for audit records. Severity is not a +meaningful concept for audit events; the `audit.outcome` attribute +serves a structurally similar purpose. ### OTLP Envelope Layers The following OTLP envelope layers apply to audit records: -| OTLP layer | Role in audit logging | -|------------------------|--------------------------------------------------------| -| `Resource` | Emitting service / host; integrity attrs (see below). | -| `LogRecord` | Carries the `AuditRecord` payload (see mapping below). | -| `Attributes` | Key-value context – reused as-is. | -| `InstrumentationScope` | **Not applicable.** MUST be left empty. | +| OTLP layer | Role in audit logging | +|------------------------|---------------------------------------------------------| +| `Resource` | Emitting service / host; integrity attrs (see below). | +| `LogRecord` | Carries the `AuditRecord` via dedicated fields and `Attributes`. | +| `InstrumentationScope` | **Not applicable.** MUST be left empty by exporters. | The `Resource` also carries `audit.integrity.algorithm` and `audit.integrity.certificate` — see @@ -105,50 +108,28 @@ The `Resource` also carries `audit.integrity.algorithm` and ## AuditRecord Definition An `AuditRecord` is a single security-relevant event emitted by -application or system code on behalf of an actor. - -The following table provides a summary of all fields. Detailed -descriptions follow. - -| Field | Type | Req. | Description | -|---------------------|-------------------------|--------|----------------------------------------------------| -| `RecordId` | `string` | MUST | Caller-generated unique identifier. | -| `Timestamp` | `fixed64` | MUST | Event time, ns since UNIX epoch (UTC). | -| `ObservedTimestamp` | `fixed64` | MUST | SDK observation time, ns since UNIX epoch. | -| `SchemaVersion` | `string` | SHOULD | Schema version of the audit payload. | -| `EventName` | `string` | MUST | Semantic name of the audit event. | -| `Actor` | `AnyValue` | MUST | Identity that performed the action. | -| `ActorType` | `enum` | MUST | `USER`, `SERVICE`, or `SYSTEM`. | -| `Action` | `string` | MUST | Verb describing what was done. | -| `Outcome` | `enum` | MUST | `SUCCESS`, `FAILURE`, or `UNKNOWN`. | -| `TargetResource` | `AnyValue` | SHOULD | The object acted upon. | -| `SourceIP` | `string` | MAY | Source network address. | -| `Body` | `AnyValue` | MAY | Free-form additional event details. | -| `Attributes` | `map` | MAY | Arbitrary key-value context. | -| `IntegrityValue` | `bytes` | MAY | Cryptographic integrity proof (signature or HMAC). | -| `SequenceNo` | `uint64` | MAY | Monotonic counter for hash-chain continuity. | -| `PrevHash` | `string` | MAY | SHA-256 of the previous record in the chain. | - -### Field: `RecordId` - -| Property | Value | -|----------|--------------------------------| -| Type | `string` | -| Required | MUST be set; MUST NOT be empty | - -A caller-generated unique identifier for this `AuditRecord`. The value -MUST be immutable once set and MUST remain stable across retries so -that receivers can deduplicate records delivered more than once. - -The caller SHOULD use a UUID v4 or an equivalent unpredictable unique -identifier. The SDK MUST generate a UUID v4 if the caller does not -provide one. - -`RecordId` is echoed back unchanged in the -[`AuditReceipt`](#auditreceipt-definition), allowing the emitting -application to correlate receipts with the original `emit` call. - -### Field: `Timestamp` +application or system code on behalf of an actor. It is transported +as a `LogRecord`; the following two tables summarise how `LogRecord` +fields are used and which semantic attributes carry audit-specific +data. + +### LogRecord Field Usage + +| LogRecord Field | Audit Usage | Required | +|------------------------|-----------------------------------------------|--------------------------------| +| `Timestamp` | Time the auditable action occurred | MUST be set | +| `ObservedTimestamp` | Time the SDK observed the event | MUST be set | +| `EventName` | Semantic name of the audit event type | MUST be set; MUST NOT be empty | +| `Body` | Free-form or structured event details | MAY be set | +| `Attributes` | Audit semantic attributes (see below) | MUST contain mandatory attrs | +| `Resource` | Emitting service / host metadata | MUST be set | +| `TraceId` | Correlation to an active trace, if any | MAY be set | +| `SpanId` | Correlation to an active span, if any | MAY be set | +| `SeverityNumber` | Not applicable for audit records | SHOULD NOT be set | +| `SeverityText` | Not applicable for audit records | SHOULD NOT be set | +| `InstrumentationScope` | Not applicable for audit records | MUST be left empty | + +#### Field: `Timestamp` | Property | Value | |------------|----------------------------------| @@ -161,14 +142,14 @@ since the UNIX epoch (UTC). The application MUST set this field to the time the auditable action actually took place. If the application cannot determine the precise event time, it SHOULD -use the current time at the moment of the `emit` call and document this -in the `Body` or `Attributes`. +use the current time at the moment of the `emit` call and document +this in the `Body` or `Attributes`. The clock used for `Timestamp` MUST be a wall-clock source synchronised via NTP or an equivalent protocol. The SDK SHOULD warn at startup if the system clock offset exceeds one second. -### Field: `ObservedTimestamp` +#### Field: `ObservedTimestamp` | Property | Value | |------------|----------------------------------------------------| @@ -182,26 +163,11 @@ to the current time when `emit` is called if the application does not provide it. `ObservedTimestamp` enables the audit sink to detect clock skew between -the emitting service and the SDK host, and between the SDK host and the -sink. A significant difference between `Timestamp` and -`ObservedTimestamp` SHOULD be treated as a clock synchronisation -warning. - -### Field: `SchemaVersion` - -| Property | Value | -|----------|------------------| -| Type | `string` | -| Required | SHOULD be set | - -The version of the audit payload schema used to produce this record. -`SchemaVersion` allows receivers and long-term archives to validate -and interpret records correctly even after the schema evolves. - -The value SHOULD follow semantic versioning (e.g. `1.0.0`). If not -set, the receiver MUST treat the schema version as unknown. +the emitting service and the SDK host. A significant difference between +`Timestamp` and `ObservedTimestamp` SHOULD be treated as a clock +synchronisation warning. -### Field: `EventName` +#### Field: `EventName` | Property | Value | |----------|--------------------------------| @@ -219,67 +185,113 @@ follow a hierarchical naming convention, for example: - `privilege.escalation` Semantic conventions for common `EventName` values SHOULD be defined -in the [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/). +in the +[OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/). + +#### Field: `Body` + +| Property | Value | +|----------|------------| +| Type | `AnyValue` | +| Required | MAY be set | + +Free-form additional information about the audit event. The value MAY +be a string, a byte buffer, or a structured object. Applications SHOULD +store information that does not fit naturally into the named mandatory +attributes here. + +The `Body` MUST NOT duplicate information already present in the +mandatory `Attributes`. + +#### Field: `Attributes` + +| Property | Value | +|----------|-----------------------------------------| +| Type | `map` | +| Required | MUST contain mandatory audit attributes | + +Key-value pairs that carry audit-specific context. The mandatory and +optional attribute names are defined in +[Audit Semantic Attributes](#audit-semantic-attributes) below. +Applications MAY add additional attributes beyond those listed here +using standard OTel +[attribute naming conventions](../common/attribute-naming.md). + +Attribute keys MUST be unique within a record. If the same key is set +multiple times, the last value MUST be used. -### Actor Fields +### Audit Semantic Attributes -Actor fields identify the entity that performed the auditable action. -Together, `Actor` and `ActorType` provide the minimum identity context -required by ISO 27001 Annex A item 1 (user IDs). +All audit-specific data that does not map to a standard `LogRecord` +dedicated field is expressed as `Attributes` following the `audit.*` +semantic convention namespace, or by reusing existing semantic +convention attributes where a direct mapping exists. -#### Field: `Actor` +#### Mandatory Attributes -| Property | Value | -|----------|-------------| -| Type | `AnyValue` | -| Required | MUST be set | +| Attribute name | Type | Required | Description | +|----------------------|----------|----------|------------------------------------------------| +| `audit.record.id` | `string` | MUST | Unique stable identifier; UUID v4 recommended. | +| `audit.actor.id` | `string` | MUST | Identity that performed the action. | +| `audit.actor.type` | `string` | MUST | `user`, `service`, or `system`. | +| `audit.action` | `string` | MUST | Verb describing what was done. | +| `audit.outcome` | `string` | MUST | `success`, `failure`, or `unknown`. | -The identity of the entity that performed the action. This MAY be: +#### Attribute: `audit.record.id` -- A user ID (string or integer) -- A service account name -- A fully qualified system identifier -- A structured object (e.g. `{ "id": "u123", "email": "a@example.com" }`) +A caller-generated unique identifier for this `AuditRecord`. The value +MUST be immutable once set and MUST remain stable across retries so +that receivers can deduplicate records delivered more than once. -If the actor identity cannot be determined (for example, an -unauthenticated request), `Actor` MUST still be set to a value -indicating the unknown state (e.g. `"anonymous"` or `"unknown"`). +The caller SHOULD use a UUID v4 or an equivalent unpredictable unique +identifier. The SDK MUST generate a UUID v4 if the caller does not +provide one. -#### Field: `ActorType` +`audit.record.id` is echoed back unchanged in the +[`AuditReceipt`](#auditreceipt-definition), allowing the emitting +application to correlate receipts with the original `emit` call. -| Property | Value | -|----------|-----------------------------| -| Type | `enum` | -| Required | MUST be set | -| Values | `USER`, `SERVICE`, `SYSTEM` | +#### Actor Attributes + +Actor attributes identify the entity that performed the auditable +action. Together, `audit.actor.id` and `audit.actor.type` provide the +minimum identity context required by ISO 27001 Annex A item 1 (user +IDs). + +**`audit.actor.id`** + +The identity of the entity that performed the action. This MAY be a +user ID (string or integer cast to string), a service account name, +or a fully qualified system identifier. If the actor identity cannot +be determined (for example, an unauthenticated request), +`audit.actor.id` MUST still be set to a value indicating the unknown +state (e.g. `"anonymous"` or `"unknown"`). + +Applications where the actor is an authenticated end user MAY set +`enduser.id` in addition to `audit.actor.id` when the values differ +(for example, when `audit.actor.id` is a stable internal identifier +and `enduser.id` is the authentication subject claim). + +**`audit.actor.type`** Classifies the kind of entity that performed the action: | Value | Meaning | |-----------|--------------------------------------------------------| -| `USER` | A human user, identified by a user account. | -| `SERVICE` | An automated service, daemon, or service account. | -| `SYSTEM` | The operating system or a privileged system component. | +| `user` | A human user, identified by a user account. | +| `service` | An automated service, daemon, or service account. | +| `system` | The operating system or a privileged system component. | -In times of AI agents and autonomous systems, the distinction between `USER` -and `SERVICE` may become blurred. The `ActorType` values are intended to be -broad categories rather than rigid classifications, and the SDK MUST NOT -enforce any particular semantics beyond what is described in the table above. -Applications SHOULD choose the most appropriate `ActorType` based on the -context of the action and the identity of the actor. Use `SERVICE` for -non-human actors that operate autonomously, and `USER` for human-initiated -actions, even if they are performed by an AI agent on behalf of a user. +Applications SHOULD choose the most appropriate value based on context. +Use `service` for non-human actors that operate autonomously, and +`user` for human-initiated actions, even if performed by an AI agent +on behalf of a user. -### Field: `Action` - -| Property | Value | -|----------|--------------------------------| -| Type | `string` | -| Required | MUST be set; MUST NOT be empty | +#### Attribute: `audit.action` A short verb or verb phrase that describes what the actor did. -Applications SHOULD use uppercase verbs to distinguish actions from -event names. Common values include: +Applications SHOULD use uppercase verbs to distinguish action values +from event names. Common values include: `LOGIN`, `LOGOUT`, `READ`, `WRITE`, `CREATE`, `UPDATE`, `DELETE`, `EXECUTE`, `APPROVE`, `REJECT`, `EXPORT`, `IMPORT`. @@ -287,122 +299,107 @@ event names. Common values include: Applications MAY define domain-specific action verbs. Action names SHOULD be stable across releases. -### Field: `Outcome` - -| Property | Value | -|----------|---------------------------------| -| Type | `enum` | -| Required | MUST be set | -| Values | `SUCCESS`, `FAILURE`, `UNKNOWN` | +#### Attribute: `audit.outcome` The result of the auditable action: | Value | Meaning | |-----------|--------------------------------------------------------------| -| `SUCCESS` | The action completed successfully. | -| `FAILURE` | The action was attempted but did not complete successfully. | -| `UNKNOWN` | The outcome could not be determined at the time of emission. | +| `success` | The action completed successfully. | +| `failure` | The action was attempted but did not complete successfully. | +| `unknown` | The outcome could not be determined at the time of emission. | ISO 27001 Annex A items 5 and 6 require logging both successful and -unsuccessful access attempts. Using `UNKNOWN` SHOULD be avoided except +unsuccessful access attempts. Using `unknown` SHOULD be avoided except for fire-and-forget actions where acknowledgement is not possible. -### Field: `TargetResource` +#### Optional Attributes -| Property | Value | -|----------|---------------| -| Type | `AnyValue` | -| Required | SHOULD be set | +| Attribute name | Type | Required | Description | +|--------------------------|----------|----------|-------------------------------------------------------| +| `audit.target.id` | `string` | SHOULD | Identifier of the resource acted upon. | +| `audit.target.type` | `string` | SHOULD | Type of the target resource. | +| `audit.source.id` | `string` | MAY | Network address or identifier of the source. | +| `audit.source.type` | `string` | MAY | Type of the source (e.g. `ipv4`, `ipv6`, `hostname`). | +| `audit.integrity.value` | `string` | MAY | Base64-encoded cryptographic integrity proof. | +| `audit.sequence.number` | `int` | MAY | Monotonic counter for hash-chain continuity. | +| `audit.prev.hash` | `string` | MAY | SHA-256 of the previous record in the stream. | +| `audit.schema.version` | `string` | SHOULD | Schema version of the audit payload. | -The object upon which the action was performed. This MAY be: +#### Target Attributes -- A file path (string) -- A database table or row identifier -- A REST endpoint path -- A structured object with type and identifier fields +**`audit.target.id`** SHOULD be set to an identifier for the object +upon which the action was performed. Examples: -If the action was not directed at a specific resource (for example, a -system startup event), this field MAY be omitted. +- A file path (`/etc/passwd`) +- A database table name (`users`) +- A REST endpoint path (`/api/v1/orders/42`) +- A cloud resource ARN or ID -Note: this field identifies the *target* of the auditable action and is -distinct from the OTLP `Resource` envelope, which identifies the -*emitting service or host*. +**`audit.target.type`** SHOULD classify the type of resource, for +example `file`, `database.table`, `http.endpoint`, `k8s.configmap`. +When applicable, values SHOULD align with existing OTel semantic +convention resource types. -### Field: `SourceIP` - -| Property | Value | -|----------|------------| -| Type | `string` | -| Required | MAY be set | - -The network address of the source of the auditable action. This field -SHOULD be set for network-initiated actions (for example, a login -attempt over HTTPS) and MAY be omitted for local or intra-process -actions. - -The value SHOULD conform to the IPv4 dotted-decimal notation -(`203.0.113.42`) or the IPv6 full notation (`2001:db8::1`). Port -numbers MAY be appended using the `host:port` convention. - -### Field: `Body` +If the action was not directed at a specific resource (for example, a +system startup event), both attributes MAY be omitted. -| Property | Value | -|----------|------------| -| Type | `AnyValue` | -| Required | MAY be set | +#### Source Attributes -Free-form additional information about the audit event. The value MAY -be a string, a byte buffer, or a structured object. Applications SHOULD -store information that does not fit naturally into the named fields here. +**`audit.source.id`** SHOULD be set for network-initiated actions (for +example, a login attempt over HTTPS). The value SHOULD be an IPv4 +address in dotted-decimal notation (`203.0.113.42`) or a full IPv6 +address (`2001:db8::1`), or a hostname. -The `Body` MUST NOT be used to store fields that duplicate the named -mandatory fields (`Actor`, `Action`, `Outcome`, etc.). +**`audit.source.type`** SHOULD classify the address format. Recommended +values are `ipv4`, `ipv6`, and `hostname`. When the source cannot be +determined, both attributes MAY be omitted. -### Field: `Attributes` +#### Integrity Attributes -| Property | Value | -|----------|-------------------------| -| Type | `map` | -| Required | MAY be set | +**`audit.integrity.value`** -Arbitrary key-value pairs that provide additional context about the -audit event. Applications SHOULD use the OpenTelemetry -[attribute naming conventions](../common/attribute-naming.md). +A base64-encoded cryptographic integrity proof over the canonical +serialisation of the `AuditRecord`. The proof is either an asymmetric +digital signature or a symmetric HMAC, as indicated by the +`audit.integrity.algorithm` `Resource` attribute (see +[Integrity Resource Attributes](#integrity-resource-attributes)). -Examples of useful attributes: +When `audit.integrity.value` is set, `audit.integrity.algorithm` MUST +be set as a `Resource` attribute. -- `session.id` – the authenticated session identifier -- `request.id` – a correlation identifier for the originating request -- `tenant.id` – the tenant in a multi-tenant system -- `geolocation.country` – country of origin for network requests -- `tls.version` – the TLS protocol version used +#### Ordering Attributes -Attribute keys MUST be unique within an `AuditRecord`. If the same key -is set multiple times, the last value MUST be used. +The optional ordering attributes enable hash-chain validation across a +sequence of `AuditRecord`s. When populated, receivers can detect +whether records have been deleted, inserted, or reordered by verifying +that the sequence numbers are monotonically increasing and that each +`audit.prev.hash` matches the `IntegrityHash` returned in the preceding +record's `AuditReceipt`. -### Integrity Fields +Implementations that require strong tamper-evidence for ordered +sequences SHOULD populate both `audit.sequence.number` and +`audit.prev.hash`. -The integrity fields enable a relying party to verify that an -`AuditRecord` was not altered after emission. They are OPTIONAL but -SHOULD be set in environments where tamper evidence is required by the -applicable compliance framework. +**`audit.sequence.number`** -#### Field: `IntegrityValue` +A monotonically increasing integer counter, scoped to a single +`AuditLogger` instance or a named audit stream. The first record in a +stream SHOULD have `audit.sequence.number` equal to `1`. A gap between +two consecutive values indicates that one or more records were lost or +deleted and SHOULD trigger an alert. -| Property | Value | -|----------|------------| -| Type | `bytes` | -| Required | MAY be set | +**`audit.prev.hash`** -A cryptographic integrity proof over the canonical serialization of the -`AuditRecord`. `IntegrityValue` MUST cover all mandatory fields plus any -`Attributes` and `Body` that are present at emission time. +The `IntegrityHash` of the immediately preceding record in the same +audit stream (as returned in the preceding `AuditReceipt`). The first +record in a stream SHOULD set `audit.prev.hash` to the SHA-256 hash of +the empty string +(`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`). -The proof is either an asymmetric digital signature or a symmetric -HMAC, as indicated by `audit.integrity.algorithm`. If -`IntegrityValue` is set, `audit.integrity.algorithm` MUST be set as -a `Resource` attribute (see -[Integrity Resource Attributes](#integrity-resource-attributes)). +A `audit.prev.hash` that does not match the stored `IntegrityHash` of +the previous record indicates tampering and MUST be treated as a +critical integrity violation. ### Integrity Resource Attributes @@ -413,12 +410,12 @@ in every record. #### Attribute: `audit.integrity.algorithm` -| Property | Value | -|----------|----------------------------------------| -| Type | `string` | -| Required | MUST be set if `IntegrityValue` is set | +| Property | Value | +|----------|----------------------------------------------------------| +| Type | `string` | +| Required | MUST be set if `audit.integrity.value` is set on any record in the batch | -The algorithm used to compute `IntegrityValue` for all records +The algorithm used to compute `audit.integrity.value` for all records produced by this resource. - For asymmetric signatures the value SHOULD be a registered JWA @@ -433,7 +430,7 @@ produced by this resource. | Type | `string` | | Required | MAY be set | -A reference to the signing key used to produce `IntegrityValue`. +A reference to the signing key used to produce `audit.integrity.value`. MUST NOT be set when `audit.integrity.algorithm` denotes a symmetric MAC algorithm. @@ -442,12 +439,12 @@ least self-contained: - **Full certificate** – base64 (standard encoding, no line wrapping) of the DER-encoded X.509 public-key certificate. Relying parties - can verify `IntegrityValue` without any out-of-band lookup. -- **Fingerprint** – A hash string in the form `sha256:` or + can verify `audit.integrity.value` without any out-of-band lookup. +- **Fingerprint** – a hash string in the form `sha256:` or `sha1:`, where `` is the colon-separated hex encoding of the DER certificate hash (e.g. `sha256:4A:6C:9F:…`). Requires matching against a locally trusted certificate. -- **Key ID** – An opaque string identifier (e.g. a JWK `kid` claim +- **Key ID** – an opaque string identifier (e.g. a JWK `kid` claim such as `key-2024-01`) agreed between emitter and relying party. Requires key retrieval via JWK Set URI or equivalent. - **SKI** – Subject Key Identifier in colon-separated hex @@ -461,48 +458,6 @@ If `audit.integrity.certificate` is omitted and relying party MUST obtain the public key through a separately configured trust anchor. -### Optional Ordering Fields - -The optional ordering fields enable hash-chain validation across a -sequence of `AuditRecord`s. When populated, receivers can detect -whether records have been deleted, inserted, or reordered by verifying -that the sequence numbers are monotonically increasing and that each -`PrevHash` matches the `IntegrityHash` returned in the preceding -record's `AuditReceipt`. - -Implementations that require strong tamper-evidence for ordered -sequences SHOULD populate both `SequenceNo` and `PrevHash`. - -#### Field: `SequenceNo` - -| Property | Value | -|----------|------------| -| Type | `uint64` | -| Required | MAY be set | - -A monotonically increasing counter, scoped to a single -`AuditLogger` instance or a named audit stream. The first record in a -stream SHOULD have `SequenceNo` equal to `1`. - -A gap between two consecutive `SequenceNo` values indicates that one -or more records were lost or deleted and SHOULD trigger an alert. - -#### Field: `PrevHash` - -| Property | Value | -|----------|------------| -| Type | `string` | -| Required | MAY be set | - -The `IntegrityHash` of the immediately preceding record in the same -audit stream (as returned in the preceding `AuditReceipt`). The first -record in a stream SHOULD set `PrevHash` to the SHA-256 hash of the -empty string (`e3b0c44298fc1c149afb…`). - -A `PrevHash` that does not match the stored `IntegrityHash` of the -previous record indicates tampering and MUST be treated as a critical -integrity violation. - ## AuditReceipt Definition An `AuditReceipt` is returned by `AuditLogger.emit` once the audit @@ -511,36 +466,28 @@ and enables integrity verification by the emitting application. | Field | Type | Required | Description | |-----------------|-----------|-------------|--------------------------------------------------------------| -| `RecordId` | `string` | MUST be set | Echoes the caller's `AuditRecord.RecordId`. | +| `RecordId` | `string` | MUST be set | Echoes the caller's `audit.record.id` attribute. | | `IntegrityHash` | `string` | MUST be set | SHA-256 of the record as persisted by the sink. | | `SinkTimestamp` | `fixed64` | MUST be set | Nanoseconds since UNIX epoch when the sink wrote the record. | ### Field: `RecordId` -The `RecordId` from the corresponding `AuditRecord` as provided by the -caller. The sink MUST echo this value unchanged. Callers use it to -correlate the receipt with the original `emit` call, and to confirm -that the correct record was persisted. - -If the sink generates its own internal identifier it MUST still return -the caller's `RecordId` here. The internal identifier MAY be conveyed -as a separate field in a protocol extension. +The value of the `audit.record.id` attribute from the corresponding +`AuditRecord` as provided by the caller. The sink MUST echo this value +unchanged. Callers use it to correlate the receipt with the original +`emit` call and to confirm that the correct record was persisted. ### Field: `IntegrityHash` -The SHA-256 hash of the canonical serialization of the `AuditRecord` -as it was written to persistent storage, computed by the sink. The hash -is returned to the emitting application so that it can verify that the -record was not altered between emission and persistence. - -The `IntegrityHash` is computed by the **sink**, not by the SDK, so it -reflects the record as actually stored – not merely as transmitted. +The SHA-256 hash of the canonical serialisation of the `AuditRecord` +as it was written to persistent storage, computed by the sink. Returned +to the emitting application so that it can verify that the record was +not altered between emission and persistence. The emitting application SHOULD compute the same hash locally immediately after calling `emit` and compare it to the received `IntegrityHash`. A mismatch indicates that the record was altered in -transit or by the sink and SHOULD trigger an alert (for example, log an -error or raise an incident). +transit or by the sink and SHOULD trigger an alert. ### Field: `SinkTimestamp` @@ -551,48 +498,60 @@ abnormally long delivery times. ## Example AuditRecords +The examples below show the `LogRecord` representation of an +`AuditRecord`. Resource attributes are shown separately. + ### Successful user login ```json { - "RecordId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "Timestamp": 1714041600000000000, "ObservedTimestamp": 1714041600001000000, - "SchemaVersion": "1.0.0", "EventName": "user.login.success", - "Actor": "u8472", - "ActorType": "USER", - "Action": "LOGIN", - "Outcome": "SUCCESS", - "TargetResource": "/api/auth/session", - "SourceIP": "203.0.113.42", + "Body": null, "Attributes": { + "audit.record.id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", + "audit.actor.id": "u8472", + "audit.actor.type": "user", + "audit.action": "LOGIN", + "audit.outcome": "success", + "audit.target.id": "/api/auth/session", + "audit.target.type": "http.endpoint", + "audit.source.id": "203.0.113.42", + "audit.source.type": "ipv4", + "audit.schema.version": "1.0.0", "session.id": "sess-abc123", "tls.version": "TLSv1.3" } } ``` +Resource attributes: + +```json +{ + "service.name": "auth-service", + "service.version": "2.3.1" +} +``` + ### Failed privileged configuration change ```json { - "RecordId": "f7e8d9c0-b1a2-3456-cdef-0987654321ab", "Timestamp": 1714041700000000000, "ObservedTimestamp": 1714041700002000000, - "SchemaVersion": "1.0.0", "EventName": "config.change", - "Actor": "svc-deployer", - "ActorType": "SERVICE", - "Action": "UPDATE", - "Outcome": "FAILURE", - "TargetResource": { - "type": "kubernetes/configmap", - "namespace": "production", - "name": "app-secrets" - }, "Body": "Authorization denied: missing role 'config-writer'", "Attributes": { + "audit.record.id": "f7e8d9c0-b1a2-3456-cdef-0987654321ab", + "audit.actor.id": "svc-deployer", + "audit.actor.type": "service", + "audit.action": "UPDATE", + "audit.outcome": "failure", + "audit.target.id": "production/app-secrets", + "audit.target.type": "k8s.configmap", + "audit.schema.version": "1.0.0", "request.id": "req-xyz789", "k8s.namespace": "production" } diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index 7dc99571884..914338beca3 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -162,14 +162,17 @@ MUST NOT silently drop the record. When `emit` is called, the SDK MUST: -1. If the caller did not provide a `RecordId`, generate a UUID v4 and - set it on the record before any further processing. +1. If `Attributes` does not contain `audit.record.id`, generate a + UUID v4 and inject it into `Attributes` before any further + processing. 2. Set `ObservedTimestamp` to the current time if the caller did not provide it. -3. Validate that all required fields (`RecordId`, `Timestamp`, - `EventName`, `Actor`, `ActorType`, `Action`, `Outcome`) are present - and non-empty. If any required field is missing, `emit` MUST surface - a hard error to the caller and MUST NOT silently drop the record. +3. Validate that the required LogRecord fields (`Timestamp`, + `EventName`) and the mandatory attributes (`audit.actor.id`, + `audit.actor.type`, `audit.action`, `audit.outcome`) are present + and non-empty. If any required field or attribute is missing, + `emit` MUST surface a hard error to the caller and MUST NOT + silently drop the record. 4. Enqueue the `AuditRecord` in the [AuditRecord Queue](#auditrecord-queue). 5. Pass the record through all registered `AuditRecordProcessor` instances via @@ -231,10 +234,11 @@ The Audit Logging pipeline enforces strict restrictions on processors to guarantee that no record is lost or altered: - Only **additive** operations are permitted: processors MAY enrich - records by adding `Attributes`, but MUST NOT remove - existing attributes, filter records, aggregate records, or alter the - mandatory fields (`Timestamp`, `EventName`, `Actor`, `ActorType`, - `Action`, `Outcome`). + records by adding `Attributes`, but MUST NOT remove existing + attributes, filter records, aggregate records, or alter the mandatory + LogRecord fields (`Timestamp`, `EventName`) or the mandatory + attributes (`audit.actor.id`, `audit.actor.type`, `audit.action`, + `audit.outcome`). - Processors that delete attributes, suppress records, or aggregate records MUST be rejected at configuration time with an explanatory error. @@ -340,13 +344,14 @@ The SDK SHOULD provide the batching processor as a built-in. #### Signing processor (Sig) -The signing processor computes an `IntegrityValue` for each `AuditRecord` -to provide tamper-evidence. The value is either an asymmetric digital -signature or a symmetric HMAC, as determined by the -`audit.integrity.algorithm` `Resource` attribute configured on the -`AuditProvider`. It MUST be implemented as a separate processor that can -be added to the pipeline in addition to the simple processor, to allow -flexibility in the choice of signing algorithm and key management strategy. +The signing processor computes an `audit.integrity.value` attribute +for each `AuditRecord` to provide tamper-evidence. The value is a +base64-encoded asymmetric digital signature or symmetric HMAC, as +determined by the `audit.integrity.algorithm` `Resource` attribute +configured on the `AuditProvider`. It MUST be implemented as a +separate processor that can be added to the pipeline in addition to +the simple processor, to allow flexibility in the choice of signing +algorithm and key management strategy. ## AuditRecordExporter @@ -436,9 +441,10 @@ receiver that acknowledges only part of a batch. #### Idempotency -The OTLP receiver SHOULD treat `RecordId` as an idempotency key. +The OTLP receiver SHOULD treat `audit.record.id` as an idempotency +key. -When a record with an already-known `RecordId` arrives: +When a record with an already-known `audit.record.id` arrives: - If the canonical payload hash is **identical**, the receiver SHOULD return a deterministic success response for the existing record @@ -530,11 +536,12 @@ The SDK exporter interacts with the Tier-2 collector via the same The Tier-2 collector SHOULD: -- Verify `IntegrityValue` of each received record against the algorithm - declared in the `Resource` attribute `audit.integrity.algorithm` and - the key material referenced by `audit.integrity.certificate`. -- Validate `SequenceNo` continuity and `PrevHash` chain integrity when - these optional fields are present. +- Verify `audit.integrity.value` of each received record against the + algorithm declared in the `Resource` attribute + `audit.integrity.algorithm` and the key material referenced by + `audit.integrity.certificate`. +- Validate `audit.sequence.number` continuity and `audit.prev.hash` + chain integrity when these optional attributes are present. - Deliver each record to every configured required sink before returning a success response to the SDK exporter. - Return a conflict error (HTTP 409) when a duplicate `RecordId` with From 82452b1fc53fec27844cda97ff6735d110eb1f74 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Wed, 29 Apr 2026 15:04:05 +0200 Subject: [PATCH 11/16] prettify MD tables, typo: serialization Signed-off-by: Hilmar Falkenberg --- oteps/0267-audit-logging.md | 4 +- specification/audit/api.md | 29 ++++++----- specification/audit/data-model.md | 85 ++++++++++++++++--------------- 3 files changed, 61 insertions(+), 57 deletions(-) diff --git a/oteps/0267-audit-logging.md b/oteps/0267-audit-logging.md index 7d1ed2c1074..fe17ec57f89 100644 --- a/oteps/0267-audit-logging.md +++ b/oteps/0267-audit-logging.md @@ -219,8 +219,8 @@ The three OTLP envelope layers that **do** apply to audit logging are: The `Resource` MUST carry the following attributes whenever any record in the batch includes an `audit.integrity.value`: -| Attribute | Type | Description | -|-------------------------------|----------|---------------------------------------| +| Attribute | Type | Description | +|-------------------------------|----------|-------------------------------------------------------------------------------------------------------------| | `audit.integrity.algorithm` | `string` | Algorithm used to compute `IntegrityValue` (JWA identifier for signatures, IANA MAC identifier for HMACs). | | `audit.integrity.certificate` | `string` | Key reference: full DER cert (base64), fingerprint, Key ID, SKI, or Issuer+Serial. Not for HMAC algorithms. | diff --git a/specification/audit/api.md b/specification/audit/api.md index 033e6837ef3..82c4e83f340 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -12,14 +12,15 @@ weight: 1 -- [AuditProvider](#auditprovider) - * [AuditProvider operations](#auditprovider-operations) - + [Get an AuditLogger](#get-an-auditlogger) -- [AuditLogger](#auditlogger) - * [Emit an AuditRecord](#emit-an-auditrecord) -- [Optional and required parameters](#optional-and-required-parameters) -- [Concurrency requirements](#concurrency-requirements) -- [References](#references) +- [Audit Logging API](#audit-logging-api) + - [AuditProvider](#auditprovider) + - [AuditProvider operations](#auditprovider-operations) + - [Get an AuditLogger](#get-an-auditlogger) + - [AuditLogger](#auditlogger) + - [Emit an AuditRecord](#emit-an-auditrecord) + - [Optional and required parameters](#optional-and-required-parameters) + - [Concurrency requirements](#concurrency-requirements) + - [References](#references) @@ -127,12 +128,12 @@ The API MUST accept the following parameters: a map of key-value pairs that carries all audit-specific data. The following attribute keys MUST be present and non-empty: - | Attribute key | Type | Description | - |---------------------|----------|--------------------------------------| - | `audit.actor.id` | `string` | Identity that performed the action. | - | `audit.actor.type` | `string` | `user`, `service`, or `system`. | - | `audit.action` | `string` | Verb describing what was done. | - | `audit.outcome` | `string` | `success`, `failure`, or `unknown`. | + | Attribute key | Type | Description | + |--------------------|----------|-------------------------------------| + | `audit.actor.id` | `string` | Identity that performed the action. | + | `audit.actor.type` | `string` | `user`, `service`, or `system`. | + | `audit.action` | `string` | Verb describing what was done. | + | `audit.outcome` | `string` | `success`, `failure`, or `unknown`. | The attribute key `audit.record.id` is also required in every persisted record. If the caller omits it, the SDK MUST generate a diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md index 47e32c2050a..c3b546e174c 100644 --- a/specification/audit/data-model.md +++ b/specification/audit/data-model.md @@ -39,6 +39,9 @@ weight: 2 - [Attribute: `audit.integrity.algorithm`](#attribute-auditintegrityalgorithm) - [Attribute: `audit.integrity.certificate`](#attribute-auditintegritycertificate) - [AuditReceipt Definition](#auditreceipt-definition) + - [Field: `RecordId`](#field-recordid) + - [Field: `IntegrityHash`](#field-integrityhash) + - [Field: `SinkTimestamp`](#field-sinktimestamp) - [Example AuditRecords](#example-auditrecords) - [Successful user login](#successful-user-login) - [Failed privileged configuration change](#failed-privileged-configuration-change) @@ -95,11 +98,11 @@ serves a structurally similar purpose. The following OTLP envelope layers apply to audit records: -| OTLP layer | Role in audit logging | -|------------------------|---------------------------------------------------------| -| `Resource` | Emitting service / host; integrity attrs (see below). | -| `LogRecord` | Carries the `AuditRecord` via dedicated fields and `Attributes`. | -| `InstrumentationScope` | **Not applicable.** MUST be left empty by exporters. | +| OTLP layer | Role in audit logging | +|------------------------|------------------------------------------------------------------| +| `Resource` | Emitting service / host; integrity attrs (see below). | +| `LogRecord` | Carries the `AuditRecord` via dedicated fields and `Attributes`. | +| `InstrumentationScope` | **Not applicable.** MUST be left empty by exporters. | The `Resource` also carries `audit.integrity.algorithm` and `audit.integrity.certificate` — see @@ -115,19 +118,19 @@ data. ### LogRecord Field Usage -| LogRecord Field | Audit Usage | Required | -|------------------------|-----------------------------------------------|--------------------------------| -| `Timestamp` | Time the auditable action occurred | MUST be set | -| `ObservedTimestamp` | Time the SDK observed the event | MUST be set | -| `EventName` | Semantic name of the audit event type | MUST be set; MUST NOT be empty | -| `Body` | Free-form or structured event details | MAY be set | -| `Attributes` | Audit semantic attributes (see below) | MUST contain mandatory attrs | -| `Resource` | Emitting service / host metadata | MUST be set | -| `TraceId` | Correlation to an active trace, if any | MAY be set | -| `SpanId` | Correlation to an active span, if any | MAY be set | -| `SeverityNumber` | Not applicable for audit records | SHOULD NOT be set | -| `SeverityText` | Not applicable for audit records | SHOULD NOT be set | -| `InstrumentationScope` | Not applicable for audit records | MUST be left empty | +| LogRecord Field | Audit Usage | Required | +|------------------------|----------------------------------------|--------------------------------| +| `Timestamp` | Time the auditable action occurred | MUST be set | +| `ObservedTimestamp` | Time the SDK observed the event | MUST be set | +| `EventName` | Semantic name of the audit event type | MUST be set; MUST NOT be empty | +| `Body` | Free-form or structured event details | MAY be set | +| `Attributes` | Audit semantic attributes (see below) | MUST contain mandatory attrs | +| `Resource` | Emitting service / host metadata | MUST be set | +| `TraceId` | Correlation to an active trace, if any | MAY be set | +| `SpanId` | Correlation to an active span, if any | MAY be set | +| `SeverityNumber` | Not applicable for audit records | SHOULD NOT be set | +| `SeverityText` | Not applicable for audit records | SHOULD NOT be set | +| `InstrumentationScope` | Not applicable for audit records | MUST be left empty | #### Field: `Timestamp` @@ -229,13 +232,13 @@ convention attributes where a direct mapping exists. #### Mandatory Attributes -| Attribute name | Type | Required | Description | -|----------------------|----------|----------|------------------------------------------------| -| `audit.record.id` | `string` | MUST | Unique stable identifier; UUID v4 recommended. | -| `audit.actor.id` | `string` | MUST | Identity that performed the action. | -| `audit.actor.type` | `string` | MUST | `user`, `service`, or `system`. | -| `audit.action` | `string` | MUST | Verb describing what was done. | -| `audit.outcome` | `string` | MUST | `success`, `failure`, or `unknown`. | +| Attribute name | Type | Required | Description | +|--------------------|----------|----------|------------------------------------------------| +| `audit.record.id` | `string` | MUST | Unique stable identifier; UUID v4 recommended. | +| `audit.actor.id` | `string` | MUST | Identity that performed the action. | +| `audit.actor.type` | `string` | MUST | `user`, `service`, or `system`. | +| `audit.action` | `string` | MUST | Verb describing what was done. | +| `audit.outcome` | `string` | MUST | `success`, `failure`, or `unknown`. | #### Attribute: `audit.record.id` @@ -315,16 +318,16 @@ for fire-and-forget actions where acknowledgement is not possible. #### Optional Attributes -| Attribute name | Type | Required | Description | -|--------------------------|----------|----------|-------------------------------------------------------| -| `audit.target.id` | `string` | SHOULD | Identifier of the resource acted upon. | -| `audit.target.type` | `string` | SHOULD | Type of the target resource. | -| `audit.source.id` | `string` | MAY | Network address or identifier of the source. | -| `audit.source.type` | `string` | MAY | Type of the source (e.g. `ipv4`, `ipv6`, `hostname`). | -| `audit.integrity.value` | `string` | MAY | Base64-encoded cryptographic integrity proof. | -| `audit.sequence.number` | `int` | MAY | Monotonic counter for hash-chain continuity. | -| `audit.prev.hash` | `string` | MAY | SHA-256 of the previous record in the stream. | -| `audit.schema.version` | `string` | SHOULD | Schema version of the audit payload. | +| Attribute name | Type | Required | Description | +|-------------------------|----------|----------|-------------------------------------------------------| +| `audit.target.id` | `string` | SHOULD | Identifier of the resource acted upon. | +| `audit.target.type` | `string` | SHOULD | Type of the target resource. | +| `audit.source.id` | `string` | MAY | Network address or identifier of the source. | +| `audit.source.type` | `string` | MAY | Type of the source (e.g. `ipv4`, `ipv6`, `hostname`). | +| `audit.integrity.value` | `string` | MAY | Base64-encoded cryptographic integrity proof. | +| `audit.sequence.number` | `int` | MAY | Monotonic counter for hash-chain continuity. | +| `audit.prev.hash` | `string` | MAY | SHA-256 of the previous record in the stream. | +| `audit.schema.version` | `string` | SHOULD | Schema version of the audit payload. | #### Target Attributes @@ -360,7 +363,7 @@ determined, both attributes MAY be omitted. **`audit.integrity.value`** A base64-encoded cryptographic integrity proof over the canonical -serialisation of the `AuditRecord`. The proof is either an asymmetric +serialization of the `AuditRecord`. The proof is either an asymmetric digital signature or a symmetric HMAC, as indicated by the `audit.integrity.algorithm` `Resource` attribute (see [Integrity Resource Attributes](#integrity-resource-attributes)). @@ -410,10 +413,10 @@ in every record. #### Attribute: `audit.integrity.algorithm` -| Property | Value | -|----------|----------------------------------------------------------| -| Type | `string` | -| Required | MUST be set if `audit.integrity.value` is set on any record in the batch | +| Property | Value | +|----------|--------------------------------------------------------------------------| +| Type | `string` | +| Required | MUST be set if `audit.integrity.value` is set on any record in the batch | The algorithm used to compute `audit.integrity.value` for all records produced by this resource. @@ -479,7 +482,7 @@ unchanged. Callers use it to correlate the receipt with the original ### Field: `IntegrityHash` -The SHA-256 hash of the canonical serialisation of the `AuditRecord` +The SHA-256 hash of the canonical serialization of the `AuditRecord` as it was written to persistent storage, computed by the sink. Returned to the emitting application so that it can verify that the record was not altered between emission and persistence. From 496fdef3003c1f96902b3aa7bc7b33b6206edadd Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Fri, 29 May 2026 14:29:33 +0200 Subject: [PATCH 12/16] remove No-op provider/logger Signed-off-by: Hilmar Falkenberg --- specification/audit/api.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/specification/audit/api.md b/specification/audit/api.md index 82c4e83f340..d456aa0026d 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -61,11 +61,6 @@ The `AuditProvider` is expected to be accessed from a central place. The API SHOULD provide a way to set and access a global default `AuditProvider`. -The API MUST provide a No-op `AuditProvider` implementation that is -used when no SDK is installed. The No-op provider MUST return a No-op -`AuditLogger` whose `emit` method completes without error and returns a -No-op `AuditReceipt`. - ### AuditProvider operations The `AuditProvider` MUST provide the following function: From 768ddaee238b3f3a34586ca2d3cb65f4c90e6459 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Mon, 1 Jun 2026 13:05:29 +0200 Subject: [PATCH 13/16] keep in sync with main Signed-off-by: Hilmar Falkenberg --- .github/scripts/triage-helper/Pipfile.lock | 136 +++++++++++---------- .github/workflows/ossf-scorecard.yml | 2 +- .github/workflows/stale-pr.yaml | 2 +- 3 files changed, 71 insertions(+), 69 deletions(-) diff --git a/.github/scripts/triage-helper/Pipfile.lock b/.github/scripts/triage-helper/Pipfile.lock index 42b0a008fac..f8c0b8e7cfc 100644 --- a/.github/scripts/triage-helper/Pipfile.lock +++ b/.github/scripts/triage-helper/Pipfile.lock @@ -18,11 +18,11 @@ "default": { "certifi": { "hashes": [ - "sha256:3c52e209ba0a4ad7aebe60436a4ab349c39e1e602e8c134221e546902ad25897", - "sha256:69dea482ab64caa7b9f6aba1c6bf48bb6a5448d1c0f1b17ab42ad8c763a5344d" + "sha256:027692e4402ad994f1c42e52a4997a9763c646b73e4096e4d5d6db8af1d6f0fa", + "sha256:e887ab5cee78ea814d3472169153c2d12cd43b14bd03329a39a9c6e2e80bfba7" ], "markers": "python_version >= '3.7'", - "version": "==2026.5.20" + "version": "==2026.2.25" }, "cffi": { "hashes": [ @@ -251,66 +251,67 @@ }, "cryptography": { "hashes": [ - "sha256:0890f502ddf7d9c6426129c3f49f5c0a39278ed7cd6322c8755ffca6ee675a13", - "sha256:0c558d2cdffd8f4bbb30fc7134c74d2ca9a476f830bb053074498fbc86f41ed6", - "sha256:16cd65b9330583e4619939b3a3843eec1e6e789744bb01e7c7e2e62e33c239c8", - "sha256:18349bbc56f4743c8b12dc32e2bccb2cf83ee8b69a3bba74ef8ae857e26b3d25", - "sha256:1e2d54c8be6152856a36f0882ab231e70f8ec7f14e93cf87db8a2ed056bf160c", - "sha256:22a5cb272895dce158b2cacdfdc3debd299019659f42947dbdac6f32d68fe832", - "sha256:27241b1dc9962e056062a8eef1991d02c3a24569c95975bd2322a8a52c6e5e12", - "sha256:2b4d59804e8408e2fea7d1fbaf218e5ec984325221db76e6a241a9abd6cdd95c", - "sha256:2eb992bbd4661238c5a397594c83f5b4dc2bc5b848c365c8f991b6780efcc5c7", - "sha256:369a6348999f94bbd53435c894377b20ab95f25a9065c283570e70150d8abc3c", - "sha256:3cb07a3ed6431663cd321ea8a000a1314c74211f823e4177fefa2255e057d1ec", - "sha256:40ba1f85eaa6959837b1d51c9767e230e14612eea4ef110ee8854ada22da1bf5", - "sha256:4defde8685ae324a9eb9d818717e93b4638ef67070ac9bc15b8ca85f63048355", - "sha256:55b7718303bf06a5753dcdccf2f3945cf18ad7bffde41b61226e4db31ab89a9c", - "sha256:561215ea3879cb1cbbf272867e2efda62476f240fb58c64de6b393ae19246741", - "sha256:58d00498e8933e4a194f3076aee1b4a97dfec1a6da444535755822fe5d8b0b86", - "sha256:59baa2cb386c4f0b9905bd6eb4c2a79a69a128408fd31d32ca4d7102d4156321", - "sha256:5a5ed8fde7a1d09376ca0b40e68cd59c69fe23b1f9768bd5824f54681626032a", - "sha256:5b012212e08b8dd5edc78ef54da83dd9892fd9105323b3993eff6bea65dc21d7", - "sha256:5c3932f4436d1cccb036cb0eaef46e6e2db91035166f1ad6505c3c9d5a635920", - "sha256:614d0949f4790582d2cc25553abd09dd723025f0c0e7c67376a1d77196743d6e", - "sha256:76341972e1eff8b4bea859f09c0d3e64b96ce931b084f9b9b7db8ef364c30eff", - "sha256:77a2ccbbe917f6710e05ba9adaa25fb5075620bf3ea6fb751997875aff4ae4bd", - "sha256:7995ef305d7165c3f11ae07f2517e5a4f1d5c18da1376a0a9ed496336b69e5f3", - "sha256:7ce4bfae76319a532a2dc68f82cc32f5676ee792a983187dac07183690e5c66f", - "sha256:7e8eac43dfca5c4cccc6dad9a80504436fca53bb9bc3100a2386d730fbe6b602", - "sha256:84cf79f0dc8b36ac5da873481716e87aef31fcfa0444f9e1d8b4b2cece142855", - "sha256:8c7378637d7d88016fa6791c159f698b3d3eed28ebf844ac36b9dc04a14dae18", - "sha256:8cd666227ef7af430aa5914a9910e0ddd703e75f039cef0825cd0da71b6b711a", - "sha256:906cbf0670286c6e0044156bc7d4af9cbb0ef6db9f73e52c3ec56ba6bdde5336", - "sha256:9071196d81abc88b3516ac8cdfad32e2b66dd4a5393a8e68a961e9161ddc6239", - "sha256:9249e3cd978541d665967ac2cb2787fd6a62bddf1e75b3e347a594d7dacf4f74", - "sha256:984a20b0f62a26f48a3396c72e4bc34c66e356d356bf370053066b3b6d54634a", - "sha256:9be5aafa5736574f8f15f262adc81b2a9869e2cfe9014d52a44633905b40d52c", - "sha256:9c459db21422be75e2809370b829a87eb37f74cd785fc4aa9ea1e5f43b47cda4", - "sha256:9ccdac7d40688ecb5a3b4a604b8a88c8002e3442d6c60aead1db2a89a041560c", - "sha256:a0e692c683f4df67815a2d258b324e66f4738bd7a96a218c826dce4f4bd05d8f", - "sha256:a5da777e32ffed6f85a7b2b3f7c5cbc88c146bfcd0a1d7baf5fcc6c52ee35dd4", - "sha256:a64697c641c7b1b2178e573cbc31c7c6684cd56883a478d75143dbb7118036db", - "sha256:ad64688338ed4bc1a6618076ba75fd7194a5f1797ac60b47afe926285adb3166", - "sha256:bd72e68b06bb1e96913f97dd4901119bc17f39d4586a5adf2d3e47bc2b9d58b5", - "sha256:c17dfe85494deaeddc5ce251aebd1d60bbe6afc8b62071bb0b469431a000124f", - "sha256:c18684a7f0cc9a3cb60328f496b8e3372def7c5d2df39ac267878b05565aaaae", - "sha256:cc90c0b39b2e3c65ef52c804b72e3c58f8a04ab2a1871272798e5f9572c17d20", - "sha256:db63bf618e5dea46c07de12e900fe1cdd2541e6dc9dbae772a70b7d4d4765f6a", - "sha256:ea8990436d914540a40ab24b6a77c0969695ed52f4a4874c5137ccf7045a7057", - "sha256:ecde28a596bead48b0cfd2a1b4416c3d43074c2d785e3a398d7ec1fc4d0f7fbb", - "sha256:f5333311663ea94f75dd408665686aaf426563556bb5283554a3539177e03b8c", - "sha256:fdfef35d751d510fcef5252703621574364fec16418c4a1e5e1055248401054b" + "sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65", + "sha256:128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832", + "sha256:1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067", + "sha256:24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de", + "sha256:258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4", + "sha256:35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0", + "sha256:397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b", + "sha256:3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968", + "sha256:420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef", + "sha256:42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b", + "sha256:462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4", + "sha256:506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3", + "sha256:5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308", + "sha256:5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e", + "sha256:5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163", + "sha256:60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f", + "sha256:65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee", + "sha256:73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77", + "sha256:7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85", + "sha256:80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99", + "sha256:84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7", + "sha256:8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83", + "sha256:91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85", + "sha256:935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006", + "sha256:9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb", + "sha256:a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e", + "sha256:abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba", + "sha256:b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325", + "sha256:b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d", + "sha256:c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1", + "sha256:cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1", + "sha256:cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2", + "sha256:cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0", + "sha256:d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455", + "sha256:d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842", + "sha256:d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457", + "sha256:d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15", + "sha256:d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2", + "sha256:db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c", + "sha256:e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb", + "sha256:e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5", + "sha256:ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4", + "sha256:ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902", + "sha256:f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246", + "sha256:fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022", + "sha256:fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f", + "sha256:fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e", + "sha256:fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298", + "sha256:ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce" ], - "markers": "python_version >= '3.9' and python_full_version not in '3.9.0, 3.9.1'", - "version": "==48.0.0" + "markers": "python_version >= '3.8' and python_full_version not in '3.9.0, 3.9.1'", + "version": "==46.0.7" }, "idna": { "hashes": [ - "sha256:cc246e3a3f89580c3a951b5ad298ca4638078b2cdd4f115654332b5c26daded5", - "sha256:d7a6da03db833450fca25d2358ac9ff06cd624577a4aea3a596d5c0f77b8e03d" + "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", + "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc" ], - "markers": "python_version >= '3.9'", - "version": "==3.16" + "index": "pypi", + "markers": "python_version >= '3.8'", + "version": "==3.15" }, "pycparser": { "hashes": [ @@ -334,11 +335,11 @@ "crypto" ], "hashes": [ - "sha256:41571c89ca91598c79e8ef18a2d07367d4810fbbd6f637794879baf1b7703423", - "sha256:66adcc2aff09b3f1bbd95fc1e1577df8ac8723c978552fd43304c8a290ac5728" + "sha256:28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c", + "sha256:c74a7a2adf861c04d002db713dd85f84beb242228e671280bf709d765b03672b" ], "markers": "python_version >= '3.9'", - "version": "==2.13.0" + "version": "==2.12.1" }, "pynacl": { "hashes": [ @@ -373,19 +374,19 @@ }, "pytz": { "hashes": [ - "sha256:04156e608bee23d3792fd45c94ae47fae1036688e75032eea2e3bf0323d1f126", - "sha256:0e60b47b29f21574376f218fe21abc009894a2321ea16c6754f3cad6eb7cdd6a" + "sha256:3378dde6a0c3d26719182142c56e60c7f9af7e968076f31aae569d72a0358ee1", + "sha256:f2fd16142fda348286a75e1a524be810bb05d444e5a081f37f7affc635035f7a" ], "index": "pypi", - "version": "==2026.2" + "version": "==2026.1.post1" }, "requests": { "hashes": [ - "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", - "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed" + "sha256:18817f8c57c6263968bc123d237e3b8b08ac046f5456bd1e307ee8f4250d3517", + "sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a" ], "markers": "python_version >= '3.10'", - "version": "==2.34.2" + "version": "==2.33.1" }, "typing-extensions": { "hashes": [ @@ -400,6 +401,7 @@ "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897" ], + "index": "pypi", "markers": "python_version >= '3.10'", "version": "==2.7.0" } diff --git a/.github/workflows/ossf-scorecard.yml b/.github/workflows/ossf-scorecard.yml index 70cf64cf8f0..cb68672852d 100644 --- a/.github/workflows/ossf-scorecard.yml +++ b/.github/workflows/ossf-scorecard.yml @@ -43,6 +43,6 @@ jobs: # Upload the results to GitHub's code scanning dashboard (optional). # Commenting out will disable upload of results to your repo's Code Scanning dashboard - name: "Upload to code-scanning" - uses: github/codeql-action/upload-sarif@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4.36.0 + uses: github/codeql-action/upload-sarif@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4.35.5 with: sarif_file: results.sarif diff --git a/.github/workflows/stale-pr.yaml b/.github/workflows/stale-pr.yaml index f305fa6a840..b7dd9cb4336 100644 --- a/.github/workflows/stale-pr.yaml +++ b/.github/workflows/stale-pr.yaml @@ -13,7 +13,7 @@ jobs: pull-requests: write # required for marking and closing stale PRs runs-on: ubuntu-latest steps: - - uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0 + - uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0 with: repo-token: ${{ secrets.GITHUB_TOKEN }} stale-pr-message: 'This PR was marked stale. It will be closed in 14 days without additional activity.' From 0f9e76568b5a526b3556f013bace2f908168890e Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Mon, 1 Jun 2026 13:12:09 +0200 Subject: [PATCH 14/16] keep in sync with main Signed-off-by: Hilmar Falkenberg --- .github/scripts/triage-helper/Pipfile.lock | 136 ++++++++++----------- .github/workflows/ossf-scorecard.yml | 2 +- .github/workflows/stale-pr.yaml | 2 +- 3 files changed, 69 insertions(+), 71 deletions(-) diff --git a/.github/scripts/triage-helper/Pipfile.lock b/.github/scripts/triage-helper/Pipfile.lock index f8c0b8e7cfc..42b0a008fac 100644 --- a/.github/scripts/triage-helper/Pipfile.lock +++ b/.github/scripts/triage-helper/Pipfile.lock @@ -18,11 +18,11 @@ "default": { "certifi": { "hashes": [ - "sha256:027692e4402ad994f1c42e52a4997a9763c646b73e4096e4d5d6db8af1d6f0fa", - "sha256:e887ab5cee78ea814d3472169153c2d12cd43b14bd03329a39a9c6e2e80bfba7" + "sha256:3c52e209ba0a4ad7aebe60436a4ab349c39e1e602e8c134221e546902ad25897", + "sha256:69dea482ab64caa7b9f6aba1c6bf48bb6a5448d1c0f1b17ab42ad8c763a5344d" ], "markers": "python_version >= '3.7'", - "version": "==2026.2.25" + "version": "==2026.5.20" }, "cffi": { "hashes": [ @@ -251,67 +251,66 @@ }, "cryptography": { "hashes": [ - "sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65", - "sha256:128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832", - "sha256:1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067", - "sha256:24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de", - "sha256:258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4", - "sha256:35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0", - "sha256:397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b", - "sha256:3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968", - "sha256:420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef", - "sha256:42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b", - "sha256:462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4", - "sha256:506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3", - "sha256:5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308", - "sha256:5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e", - "sha256:5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163", - "sha256:60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f", - "sha256:65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee", - "sha256:73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77", - "sha256:7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85", - "sha256:80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99", - "sha256:84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7", - "sha256:8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83", - "sha256:91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85", - "sha256:935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006", - "sha256:9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb", - "sha256:a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e", - "sha256:abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba", - "sha256:b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325", - "sha256:b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d", - "sha256:c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1", - "sha256:cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1", - "sha256:cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2", - "sha256:cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0", - "sha256:d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455", - "sha256:d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842", - "sha256:d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457", - "sha256:d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15", - "sha256:d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2", - "sha256:db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c", - "sha256:e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb", - "sha256:e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5", - "sha256:ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4", - "sha256:ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902", - "sha256:f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246", - "sha256:fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022", - "sha256:fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f", - "sha256:fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e", - "sha256:fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298", - "sha256:ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce" + "sha256:0890f502ddf7d9c6426129c3f49f5c0a39278ed7cd6322c8755ffca6ee675a13", + "sha256:0c558d2cdffd8f4bbb30fc7134c74d2ca9a476f830bb053074498fbc86f41ed6", + "sha256:16cd65b9330583e4619939b3a3843eec1e6e789744bb01e7c7e2e62e33c239c8", + "sha256:18349bbc56f4743c8b12dc32e2bccb2cf83ee8b69a3bba74ef8ae857e26b3d25", + "sha256:1e2d54c8be6152856a36f0882ab231e70f8ec7f14e93cf87db8a2ed056bf160c", + "sha256:22a5cb272895dce158b2cacdfdc3debd299019659f42947dbdac6f32d68fe832", + "sha256:27241b1dc9962e056062a8eef1991d02c3a24569c95975bd2322a8a52c6e5e12", + "sha256:2b4d59804e8408e2fea7d1fbaf218e5ec984325221db76e6a241a9abd6cdd95c", + "sha256:2eb992bbd4661238c5a397594c83f5b4dc2bc5b848c365c8f991b6780efcc5c7", + "sha256:369a6348999f94bbd53435c894377b20ab95f25a9065c283570e70150d8abc3c", + "sha256:3cb07a3ed6431663cd321ea8a000a1314c74211f823e4177fefa2255e057d1ec", + "sha256:40ba1f85eaa6959837b1d51c9767e230e14612eea4ef110ee8854ada22da1bf5", + "sha256:4defde8685ae324a9eb9d818717e93b4638ef67070ac9bc15b8ca85f63048355", + "sha256:55b7718303bf06a5753dcdccf2f3945cf18ad7bffde41b61226e4db31ab89a9c", + "sha256:561215ea3879cb1cbbf272867e2efda62476f240fb58c64de6b393ae19246741", + "sha256:58d00498e8933e4a194f3076aee1b4a97dfec1a6da444535755822fe5d8b0b86", + "sha256:59baa2cb386c4f0b9905bd6eb4c2a79a69a128408fd31d32ca4d7102d4156321", + "sha256:5a5ed8fde7a1d09376ca0b40e68cd59c69fe23b1f9768bd5824f54681626032a", + "sha256:5b012212e08b8dd5edc78ef54da83dd9892fd9105323b3993eff6bea65dc21d7", + "sha256:5c3932f4436d1cccb036cb0eaef46e6e2db91035166f1ad6505c3c9d5a635920", + "sha256:614d0949f4790582d2cc25553abd09dd723025f0c0e7c67376a1d77196743d6e", + "sha256:76341972e1eff8b4bea859f09c0d3e64b96ce931b084f9b9b7db8ef364c30eff", + "sha256:77a2ccbbe917f6710e05ba9adaa25fb5075620bf3ea6fb751997875aff4ae4bd", + "sha256:7995ef305d7165c3f11ae07f2517e5a4f1d5c18da1376a0a9ed496336b69e5f3", + "sha256:7ce4bfae76319a532a2dc68f82cc32f5676ee792a983187dac07183690e5c66f", + "sha256:7e8eac43dfca5c4cccc6dad9a80504436fca53bb9bc3100a2386d730fbe6b602", + "sha256:84cf79f0dc8b36ac5da873481716e87aef31fcfa0444f9e1d8b4b2cece142855", + "sha256:8c7378637d7d88016fa6791c159f698b3d3eed28ebf844ac36b9dc04a14dae18", + "sha256:8cd666227ef7af430aa5914a9910e0ddd703e75f039cef0825cd0da71b6b711a", + "sha256:906cbf0670286c6e0044156bc7d4af9cbb0ef6db9f73e52c3ec56ba6bdde5336", + "sha256:9071196d81abc88b3516ac8cdfad32e2b66dd4a5393a8e68a961e9161ddc6239", + "sha256:9249e3cd978541d665967ac2cb2787fd6a62bddf1e75b3e347a594d7dacf4f74", + "sha256:984a20b0f62a26f48a3396c72e4bc34c66e356d356bf370053066b3b6d54634a", + "sha256:9be5aafa5736574f8f15f262adc81b2a9869e2cfe9014d52a44633905b40d52c", + "sha256:9c459db21422be75e2809370b829a87eb37f74cd785fc4aa9ea1e5f43b47cda4", + "sha256:9ccdac7d40688ecb5a3b4a604b8a88c8002e3442d6c60aead1db2a89a041560c", + "sha256:a0e692c683f4df67815a2d258b324e66f4738bd7a96a218c826dce4f4bd05d8f", + "sha256:a5da777e32ffed6f85a7b2b3f7c5cbc88c146bfcd0a1d7baf5fcc6c52ee35dd4", + "sha256:a64697c641c7b1b2178e573cbc31c7c6684cd56883a478d75143dbb7118036db", + "sha256:ad64688338ed4bc1a6618076ba75fd7194a5f1797ac60b47afe926285adb3166", + "sha256:bd72e68b06bb1e96913f97dd4901119bc17f39d4586a5adf2d3e47bc2b9d58b5", + "sha256:c17dfe85494deaeddc5ce251aebd1d60bbe6afc8b62071bb0b469431a000124f", + "sha256:c18684a7f0cc9a3cb60328f496b8e3372def7c5d2df39ac267878b05565aaaae", + "sha256:cc90c0b39b2e3c65ef52c804b72e3c58f8a04ab2a1871272798e5f9572c17d20", + "sha256:db63bf618e5dea46c07de12e900fe1cdd2541e6dc9dbae772a70b7d4d4765f6a", + "sha256:ea8990436d914540a40ab24b6a77c0969695ed52f4a4874c5137ccf7045a7057", + "sha256:ecde28a596bead48b0cfd2a1b4416c3d43074c2d785e3a398d7ec1fc4d0f7fbb", + "sha256:f5333311663ea94f75dd408665686aaf426563556bb5283554a3539177e03b8c", + "sha256:fdfef35d751d510fcef5252703621574364fec16418c4a1e5e1055248401054b" ], - "markers": "python_version >= '3.8' and python_full_version not in '3.9.0, 3.9.1'", - "version": "==46.0.7" + "markers": "python_version >= '3.9' and python_full_version not in '3.9.0, 3.9.1'", + "version": "==48.0.0" }, "idna": { "hashes": [ - "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", - "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc" + "sha256:cc246e3a3f89580c3a951b5ad298ca4638078b2cdd4f115654332b5c26daded5", + "sha256:d7a6da03db833450fca25d2358ac9ff06cd624577a4aea3a596d5c0f77b8e03d" ], - "index": "pypi", - "markers": "python_version >= '3.8'", - "version": "==3.15" + "markers": "python_version >= '3.9'", + "version": "==3.16" }, "pycparser": { "hashes": [ @@ -335,11 +334,11 @@ "crypto" ], "hashes": [ - "sha256:28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c", - "sha256:c74a7a2adf861c04d002db713dd85f84beb242228e671280bf709d765b03672b" + "sha256:41571c89ca91598c79e8ef18a2d07367d4810fbbd6f637794879baf1b7703423", + "sha256:66adcc2aff09b3f1bbd95fc1e1577df8ac8723c978552fd43304c8a290ac5728" ], "markers": "python_version >= '3.9'", - "version": "==2.12.1" + "version": "==2.13.0" }, "pynacl": { "hashes": [ @@ -374,19 +373,19 @@ }, "pytz": { "hashes": [ - "sha256:3378dde6a0c3d26719182142c56e60c7f9af7e968076f31aae569d72a0358ee1", - "sha256:f2fd16142fda348286a75e1a524be810bb05d444e5a081f37f7affc635035f7a" + "sha256:04156e608bee23d3792fd45c94ae47fae1036688e75032eea2e3bf0323d1f126", + "sha256:0e60b47b29f21574376f218fe21abc009894a2321ea16c6754f3cad6eb7cdd6a" ], "index": "pypi", - "version": "==2026.1.post1" + "version": "==2026.2" }, "requests": { "hashes": [ - "sha256:18817f8c57c6263968bc123d237e3b8b08ac046f5456bd1e307ee8f4250d3517", - "sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a" + "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", + "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed" ], "markers": "python_version >= '3.10'", - "version": "==2.33.1" + "version": "==2.34.2" }, "typing-extensions": { "hashes": [ @@ -401,7 +400,6 @@ "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897" ], - "index": "pypi", "markers": "python_version >= '3.10'", "version": "==2.7.0" } diff --git a/.github/workflows/ossf-scorecard.yml b/.github/workflows/ossf-scorecard.yml index cb68672852d..70cf64cf8f0 100644 --- a/.github/workflows/ossf-scorecard.yml +++ b/.github/workflows/ossf-scorecard.yml @@ -43,6 +43,6 @@ jobs: # Upload the results to GitHub's code scanning dashboard (optional). # Commenting out will disable upload of results to your repo's Code Scanning dashboard - name: "Upload to code-scanning" - uses: github/codeql-action/upload-sarif@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v4.35.5 + uses: github/codeql-action/upload-sarif@7211b7c8077ea37d8641b6271f6a365a22a5fbfa # v4.36.0 with: sarif_file: results.sarif diff --git a/.github/workflows/stale-pr.yaml b/.github/workflows/stale-pr.yaml index b7dd9cb4336..f305fa6a840 100644 --- a/.github/workflows/stale-pr.yaml +++ b/.github/workflows/stale-pr.yaml @@ -13,7 +13,7 @@ jobs: pull-requests: write # required for marking and closing stale PRs runs-on: ubuntu-latest steps: - - uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0 + - uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0 with: repo-token: ${{ secrets.GITHUB_TOKEN }} stale-pr-message: 'This PR was marked stale. It will be closed in 14 days without additional activity.' From c3933f13a6ebbd2fee874d9fbf35bf8301087a1f Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Mon, 1 Jun 2026 16:31:53 +0200 Subject: [PATCH 15/16] fingerprint notation + JCS (RFC 8785) Signed-off-by: Hilmar Falkenberg --- specification/audit/collector.md | 9 +++++++++ specification/audit/data-model.md | 30 +++++++++++++++++++++++------- specification/audit/sdk.md | 10 +++++++++- 3 files changed, 41 insertions(+), 8 deletions(-) diff --git a/specification/audit/collector.md b/specification/audit/collector.md index 3e1f12b85ff..3025884c695 100644 --- a/specification/audit/collector.md +++ b/specification/audit/collector.md @@ -200,6 +200,15 @@ When `audit.integrity.value` is present the collector SHOULD verify it against the algorithm declared in the `Resource` attribute `audit.integrity.algorithm` and the key material referenced by `audit.integrity.certificate` in the collector's trust policy. +Before verifying, the collector MUST serialize the `AuditRecord` to +JSON and canonicalize it using +[RFC 8785 – JSON Canonicalization Scheme (JCS)][rfc8785]; the +`audit.integrity.*` attributes MUST be excluded from the canonical +form — they carry the proof itself and MUST NOT be part of the +verified payload. The canonical byte sequence of the remaining +record is the input to the verification operation. + +[rfc8785]: https://www.rfc-editor.org/rfc/rfc8785 The collector MAY defer verification for low-latency ingest (returning `accepted_pending_verify`) but MUST complete verification before diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md index c3b546e174c..f2fb72e3b46 100644 --- a/specification/audit/data-model.md +++ b/specification/audit/data-model.md @@ -368,6 +368,18 @@ digital signature or a symmetric HMAC, as indicated by the `audit.integrity.algorithm` `Resource` attribute (see [Integrity Resource Attributes](#integrity-resource-attributes)). +Before signing or verifying, the `AuditRecord` MUST be serialized to +JSON and then canonicalized using +[RFC 8785 – JSON Canonicalization Scheme (JCS)][rfc8785]. The +`audit.integrity.*` attributes MUST be excluded from the canonical +form before signing — they carry the proof itself and MUST NOT be +part of the signed payload. The canonical byte sequence of the +remaining record is the input to the signing or HMAC operation. +Implementations MUST NOT use any other serialization or +canonicalization method for this purpose. + +[rfc8785]: https://www.rfc-editor.org/rfc/rfc8785 + When `audit.integrity.value` is set, `audit.integrity.algorithm` MUST be set as a `Resource` attribute. @@ -444,14 +456,16 @@ least self-contained: of the DER-encoded X.509 public-key certificate. Relying parties can verify `audit.integrity.value` without any out-of-band lookup. - **Fingerprint** – a hash string in the form `sha256:` or - `sha1:`, where `` is the colon-separated hex encoding of - the DER certificate hash (e.g. `sha256:4A:6C:9F:…`). Requires + `sha1:`, where `` is the lowercase hex encoding of + the DER certificate hash **without** any separators + (e.g. `sha256:3a5f9c2e117b4d8f…`). Colons or other separator + characters within the hex portion are NOT permitted. Requires matching against a locally trusted certificate. - **Key ID** – an opaque string identifier (e.g. a JWK `kid` claim such as `key-2024-01`) agreed between emitter and relying party. Requires key retrieval via JWK Set URI or equivalent. -- **SKI** – Subject Key Identifier in colon-separated hex - (e.g. `3C:F0:…`). Requires a PKI lookup. +- **SKI** – Subject Key Identifier as lowercase hex without separators + (e.g. `3cf012ab…`). Requires a PKI lookup. - **Issuer + Serial** – Issuer Distinguished Name and serial number separated by a slash (e.g. `CN=MyCA,O=Acme Corp/12345`). Requires a PKI lookup. @@ -483,9 +497,11 @@ unchanged. Callers use it to correlate the receipt with the original ### Field: `IntegrityHash` The SHA-256 hash of the canonical serialization of the `AuditRecord` -as it was written to persistent storage, computed by the sink. Returned -to the emitting application so that it can verify that the record was -not altered between emission and persistence. +as it was written to persistent storage, computed by the sink. The +record MUST be serialized to JSON and canonicalized using +[RFC 8785 – JSON Canonicalization Scheme (JCS)][rfc8785] before +hashing. Returned to the emitting application so that it can verify +that the record was not altered between emission and persistence. The emitting application SHOULD compute the same hash locally immediately after calling `emit` and compare it to the received diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index 914338beca3..af935c4a460 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -348,11 +348,19 @@ The signing processor computes an `audit.integrity.value` attribute for each `AuditRecord` to provide tamper-evidence. The value is a base64-encoded asymmetric digital signature or symmetric HMAC, as determined by the `audit.integrity.algorithm` `Resource` attribute -configured on the `AuditProvider`. It MUST be implemented as a +configured on the `AuditProvider`. Before signing, the processor +MUST serialize the `AuditRecord` to JSON and canonicalize it using +[RFC 8785 – JSON Canonicalization Scheme (JCS)][rfc8785]; the +`audit.integrity.*` attributes MUST be excluded from the canonical +form — they carry the proof itself and MUST NOT be part of the +signed payload. The canonical byte sequence of the remaining record +is the input to the signing or HMAC operation. It MUST be implemented as a separate processor that can be added to the pipeline in addition to the simple processor, to allow flexibility in the choice of signing algorithm and key management strategy. +[rfc8785]: https://www.rfc-editor.org/rfc/rfc8785 + ## AuditRecordExporter `AuditRecordExporter` is the interface that protocol-specific exporters From a48a63180b145b385c7cc8d5b54692bb5e771b42 Mon Sep 17 00:00:00 2001 From: Hilmar Falkenberg Date: Wed, 3 Jun 2026 11:51:35 +0200 Subject: [PATCH 16/16] rename audit.prev.hash -> audit.sequence.prev_hash Signed-off-by: Hilmar Falkenberg --- oteps/0267-audit-logging.md | 2 +- specification/audit/api.md | 2 +- specification/audit/collector.md | 6 +++--- specification/audit/data-model.md | 12 ++++++------ specification/audit/sdk.md | 2 +- 5 files changed, 12 insertions(+), 12 deletions(-) diff --git a/oteps/0267-audit-logging.md b/oteps/0267-audit-logging.md index fe17ec57f89..e97b57dbf66 100644 --- a/oteps/0267-audit-logging.md +++ b/oteps/0267-audit-logging.md @@ -154,7 +154,7 @@ convention attributes where a direct mapping exists. | `audit.source.type` | `string` | Type of the source (e.g. `ipv4`, `ipv6`, `hostname`). | | `audit.integrity.value` | `string` | Base64-encoded signature or HMAC. | | `audit.sequence.number` | `int` | Monotonic counter for hash-chain continuity. | -| `audit.prev.hash` | `string` | SHA-256 of the preceding record. | +| `audit.sequence.prev_hash` | `string` | SHA-256 of the preceding record. | | `audit.schema.version` | `string` | Schema version (e.g. `1.0.0`). | The `Resource` carries `audit.integrity.algorithm` and diff --git a/specification/audit/api.md b/specification/audit/api.md index d456aa0026d..7fa6331ed31 100644 --- a/specification/audit/api.md +++ b/specification/audit/api.md @@ -148,7 +148,7 @@ the [Audit Record Data Model](./data-model.md#audit-semantic-attributes) for the full list, which includes `audit.target.id`, `audit.target.type`, `audit.source.id`, `audit.source.type`, `audit.integrity.value`, `audit.sequence.number`, -`audit.prev.hash`, and `audit.schema.version`. +`audit.sequence.prev_hash`, and `audit.schema.version`. **Return value**: The API MUST return an [`AuditReceipt`](./data-model.md#auditreceipt-definition) when the diff --git a/specification/audit/collector.md b/specification/audit/collector.md index 3025884c695..959199a55fc 100644 --- a/specification/audit/collector.md +++ b/specification/audit/collector.md @@ -110,7 +110,7 @@ And the following mandatory attributes in the `Attributes` map: Each record MAY include any attribute defined in the [Audit Semantic Attributes](./data-model.md#audit-semantic-attributes) section, including `audit.schema.version`, `audit.integrity.value`, -`audit.sequence.number`, and `audit.prev.hash`. The signing algorithm +`audit.sequence.number`, and `audit.sequence.prev_hash`. The signing algorithm and key reference are carried as `Resource` attributes `audit.integrity.algorithm` and `audit.integrity.certificate`. @@ -222,12 +222,12 @@ If verification fails the collector MUST: ### Hash-Chain Validation -When `audit.sequence.number` and `audit.prev.hash` are present the +When `audit.sequence.number` and `audit.sequence.prev_hash` are present the collector SHOULD validate chain continuity: - `audit.sequence.number` MUST be strictly greater than the previous record's `audit.sequence.number` in the same audit stream. -- `audit.prev.hash` MUST equal the `IntegrityHash` of the preceding +- `audit.sequence.prev_hash` MUST equal the `IntegrityHash` of the preceding record. A broken chain SHOULD be surfaced as a warning metric and logged as a diff --git a/specification/audit/data-model.md b/specification/audit/data-model.md index f2fb72e3b46..d672f931e9e 100644 --- a/specification/audit/data-model.md +++ b/specification/audit/data-model.md @@ -326,7 +326,7 @@ for fire-and-forget actions where acknowledgement is not possible. | `audit.source.type` | `string` | MAY | Type of the source (e.g. `ipv4`, `ipv6`, `hostname`). | | `audit.integrity.value` | `string` | MAY | Base64-encoded cryptographic integrity proof. | | `audit.sequence.number` | `int` | MAY | Monotonic counter for hash-chain continuity. | -| `audit.prev.hash` | `string` | MAY | SHA-256 of the previous record in the stream. | +| `audit.sequence.prev_hash` | `string` | MAY | SHA-256 of the previous record in the stream. | | `audit.schema.version` | `string` | SHOULD | Schema version of the audit payload. | #### Target Attributes @@ -389,12 +389,12 @@ The optional ordering attributes enable hash-chain validation across a sequence of `AuditRecord`s. When populated, receivers can detect whether records have been deleted, inserted, or reordered by verifying that the sequence numbers are monotonically increasing and that each -`audit.prev.hash` matches the `IntegrityHash` returned in the preceding +`audit.sequence.prev_hash` matches the `IntegrityHash` returned in the preceding record's `AuditReceipt`. Implementations that require strong tamper-evidence for ordered sequences SHOULD populate both `audit.sequence.number` and -`audit.prev.hash`. +`audit.sequence.prev_hash`. **`audit.sequence.number`** @@ -404,15 +404,15 @@ stream SHOULD have `audit.sequence.number` equal to `1`. A gap between two consecutive values indicates that one or more records were lost or deleted and SHOULD trigger an alert. -**`audit.prev.hash`** +**`audit.sequence.prev_hash`** The `IntegrityHash` of the immediately preceding record in the same audit stream (as returned in the preceding `AuditReceipt`). The first -record in a stream SHOULD set `audit.prev.hash` to the SHA-256 hash of +record in a stream SHOULD set `audit.sequence.prev_hash` to the SHA-256 hash of the empty string (`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`). -A `audit.prev.hash` that does not match the stored `IntegrityHash` of +A `audit.sequence.prev_hash` that does not match the stored `IntegrityHash` of the previous record indicates tampering and MUST be treated as a critical integrity violation. diff --git a/specification/audit/sdk.md b/specification/audit/sdk.md index af935c4a460..81fc3f28dfc 100644 --- a/specification/audit/sdk.md +++ b/specification/audit/sdk.md @@ -548,7 +548,7 @@ The Tier-2 collector SHOULD: algorithm declared in the `Resource` attribute `audit.integrity.algorithm` and the key material referenced by `audit.integrity.certificate`. -- Validate `audit.sequence.number` continuity and `audit.prev.hash` +- Validate `audit.sequence.number` continuity and `audit.sequence.prev_hash` chain integrity when these optional attributes are present. - Deliver each record to every configured required sink before returning a success response to the SDK exporter.