diff --git a/doc/logging/README.md b/doc/logging/README.md new file mode 100644 index 0000000000..6d9c486ea8 --- /dev/null +++ b/doc/logging/README.md @@ -0,0 +1,202 @@ +# Ecstasy logging — start here + +`lib_logging` is the XDK's injectable logging library. Application code asks the +runtime for a logger and writes events; the host container chooses where the +output goes. + +```ecstasy +@Inject Logger logger; +logger.info("processed {} records in {}ms", [count, elapsed]); +``` + +That is the whole hello-world. There is no factory call, no static initializer, +no configuration file. The default `ConsoleLogSink` writes the line; the host +can swap that for JSON, async, fanout, hierarchical levels, or a test capture +sink without changing caller code. + +> **New here?** Read [`usage/injected-logger-example.md`](usage/injected-logger-example.md) next — it walks +> from the smallest possible app up to a realistic per-class logger setup. Come +> back to this README for the design context. + +## Reading paths + +| If you want to… | Read in this order | +|---|---| +| **Write code against `lib_logging`** | [`usage/injected-logger-example.md`](usage/injected-logger-example.md) → [`usage/structured-logging.md`](usage/structured-logging.md) → [`usage/lazy-logging.md`](usage/lazy-logging.md) | +| **Decide whether to merge this PR** | [`roadmap.md`](roadmap.md) → [`lib-logging-vs-lib-slogging.md`](lib-logging-vs-lib-slogging.md) → [`open-questions.md`](open-questions.md) | +| **Understand the API choice** | [`lib-logging-vs-lib-slogging.md`](lib-logging-vs-lib-slogging.md) → [`design/why-slf4j-and-injection.md`](design/why-slf4j-and-injection.md) → [`cloud-integration.md`](cloud-integration.md) | +| **Migrate from SLF4J** | [`usage/slf4j-parity.md`](usage/slf4j-parity.md) → [`usage/ecstasy-vs-java-examples.md`](usage/ecstasy-vs-java-examples.md) | +| **Migrate from Go `slog`** | [`usage/slog-parity.md`](usage/slog-parity.md) → [`api-cross-reference.md`](api-cross-reference.md) | +| **Write a custom backend** | [`usage/custom-sinks.md`](usage/custom-sinks.md) → [`design/design.md`](design/design.md) | +| **Read the architecture** | [`design/design.md`](design/design.md) → [`design/xdk-alignment.md`](design/xdk-alignment.md) | + +Each doc has a "Next →" footer that follows its column above. You can also use +the [Full doc index](#full-doc-index) at the bottom. + +## Goals + +Why this library exists. These are the directional aims, not contractual specs: + +- **Familiar.** A JVM or Go engineer should recognize the call shape on first + read, with no Ecstasy-specific learning curve for the common case. +- **Container-aware.** The host application — not the calling library — decides + where logs go. Per-container override is a one-line change, not a classpath + ceremony. +- **Cloud-ready.** Structured fields (level, message, request context, exception) + flow into JSON output and from there into Cloud Logging / CloudWatch / App + Insights through small, well-known adapters. See [`cloud-integration.md`](cloud-integration.md). +- **Pay-for-what-you-use.** Disabled log calls cost a level check. Expensive + message and value construction can be deferred behind that check. +- **Testable.** Capturing log output in a test should be one line, with no + global state to reset. + +## Requirements + +The narrower, verifiable list — what the accepted XDK logging API **must** do. +Status reflects what the POC in this branch ships: + +| # | Requirement | Status | Evidence | +|---|---|---|---| +| R1 | One canonical injectable logger acquired by `@Inject Logger logger;`. | ✓ done | [`Logger.x`](../../lib_logging/src/main/x/logging/Logger.x), [`NativeContainer.java`](../../javatools/src/main/java/org/xvm/runtime/NativeContainer.java) `registerLoggingResources` | +| R2 | Per-name child loggers via API, not via wildcard injection. | ✓ done | `Logger.named(String)` in [`Logger.x`](../../lib_logging/src/main/x/logging/Logger.x); [`NamedLoggerTest`](../../lib_logging/src/test/x/LoggingTest/NamedLoggerTest.x) | +| R3 | Events carry first-class structured key/value fields, exception, MDC, markers. | ✓ done | [`LogEvent.x`](../../lib_logging/src/main/x/logging/LogEvent.x); [`StructuredLoggingTest`](../../lib_logging/src/test/x/LoggingTest/StructuredLoggingTest.x) | +| R4 | Disabled log calls are cheap; expensive args/values can be supplied lazily. | ✓ done | `MessageSupplier`, `addLazyArgument` in [`LoggingEventBuilder.x`](../../lib_logging/src/main/x/logging/LoggingEventBuilder.x); [`LazyLoggingTest`](../../lib_logging/src/test/x/LoggingTest/LazyLoggingTest.x) | +| R5 | Backends are swappable, composable, async-wrappable, and test-capturable. | ✓ done | [`CompositeLogSink.x`](../../lib_logging/src/main/x/logging/CompositeLogSink.x), [`AsyncLogSink.x`](../../lib_logging/src/main/x/logging/AsyncLogSink.x), [`MemoryLogSink.x`](../../lib_logging/src/main/x/logging/MemoryLogSink.x) | +| R6 | Per-logger-name level routing (Logback longest-prefix). | ✓ done | [`HierarchicalLogSink.x`](../../lib_logging/src/main/x/logging/HierarchicalLogSink.x); [`HierarchicalLogSinkTest`](../../lib_logging/src/test/x/LoggingTest/HierarchicalLogSinkTest.x) | +| R7 | JSON output with redaction. | ✓ done | [`JsonLogSink.x`](../../lib_logging/src/main/x/logging/JsonLogSink.x), [`JsonLogSinkOptions.x`](../../lib_logging/src/main/x/logging/JsonLogSinkOptions.x); [`JsonLogSinkTest`](../../lib_logging/src/test/x/LoggingTest/JsonLogSinkTest.x) | +| R8 | Backend failures must not break callers. | ✓ contract | Documented in [`LogSink.x`](../../lib_logging/src/main/x/logging/LogSink.x); enforcement still up to each sink. | +| R9 | In-memory capture sink for tests. | ✓ done | [`MemoryLogSink.x`](../../lib_logging/src/main/x/logging/MemoryLogSink.x); used throughout `lib_logging/src/test/` | +| R10 | Default logger name and source-location capture have a stable lowering target the compiler can fill in later. | ~ partial | `Logger.logAt(...)` populates source explicitly; runtime fallback derives logger name from caller namespace. Compiler-synthesized source/name still TBD — see [`roadmap.md`](roadmap.md). | +| R11 | Configuration-file driven backend (Logback-equivalent ops). | → v1+ | Sketched in [`future/logback-integration.md`](future/logback-integration.md); not in this branch. | + +## Non-goals (this branch) + +These are explicitly out of scope for the POC. Listing them prevents reviewer +churn over things that were never the target: + +- **A configuration file format.** `JsonLogSinkOptions` covers the JSON sink's + knobs; XML/JSON/YAML config and hot reload belong to a future configured + backend. +- **Distributed tracing context propagation.** MDC carries strings; tracing + IDs can ride on it. Trace/span propagation itself is a separate library. +- **Automatic compiler call-site capture.** `Logger.logAt(sourceFile, sourceLine)` + is the lowering target; the compiler does not yet synthesize those args for + ordinary `logger.info(...)` calls. +- **Rolling/network/cloud destinations.** They are pure-XTC sink modules built + on the SPI; out of scope for the base library. + +## Two POCs in this branch + +This branch carries two parallel libraries so reviewers can compare API shapes +in real Ecstasy code: + +| Library | Prior art | Tests | +|---|---|---| +| [`lib_logging`](../../lib_logging/) | SLF4J 2.x + Logback | 70 XTC test methods, [`TestLogging.x`](../../manualTests/src/main/x/TestLogging.x) demo | +| [`lib_slogging`](../../lib_slogging/) | Go `log/slog` | 41 XTC test methods, [`TestSLogging.x`](../../manualTests/src/main/x/TestSLogging.x) demo | + +The branch **recommends `lib_logging`** as the canonical XDK shape on +familiarity and ecosystem grounds — the [API-choice doc](lib-logging-vs-lib-slogging.md) +explains why. Whichever shape wins should ship as `lib_logging`; `lib_slogging` +is a POC name, not a proposed permanent sibling. Both modules are deliberately +optional in the runtime: a build that only ships one of them must still start. + +## Status & roadmap + +The POC implements every requirement above except R11 (configuration-file +backend) and the compiler-side half of R10. The merge-blocking decisions, the +v1 scope, and the explicit-future items are tracked in +[`roadmap.md`](roadmap.md). Outstanding language/runtime questions for the XTC +designers are in [`open-questions.md`](open-questions.md) (Q-D1…Q-D7). + +## Full doc index + +| Doc | One-line description | +|---|---| +| **Top-level** | | +| [`README.md`](README.md) | This file. The "start here" entry point. | +| [`roadmap.md`](roadmap.md) | What blocks merge, what lands in v1, what is explicit future. | +| [`lib-logging-vs-lib-slogging.md`](lib-logging-vs-lib-slogging.md) | API-choice summary first, then depth-first rationale on levels, context, markers, formatting, and backend shape. | +| [`api-cross-reference.md`](api-cross-reference.md) | Official SLF4J / Go slog API links mapped to local Ecstasy types and difference notes. | +| [`cloud-integration.md`](cloud-integration.md) | Why the API choice is the entry point to cloud observability ecosystems. | +| [`open-questions.md`](open-questions.md) | Decision tracker: resolved calls, designer questions (Q-D1..Q-D7), W-item parity list, and remaining follow-up. | +| **Concepts** | | +| [`concepts/markers.md`](concepts/markers.md) | What a log marker is, who uses them, when they earn their API surface. | +| **Design (`lib_logging` side)** | | +| [`design/design.md`](design/design.md) | `lib_logging` architecture: types, API↔impl boundary, sink-type rule, MDC mechanism, per-container override. | +| [`design/why-slf4j-and-injection.md`](design/why-slf4j-and-injection.md) | The original rationale for the SLF4J shape and injection-first acquisition. | +| [`design/xdk-alignment.md`](design/xdk-alignment.md) | Alignment with the conventions of other XDK libraries (`lib_ecstasy`, `lib_cli`, etc.). | +| **Usage / examples** | | +| [`usage/injected-logger-example.md`](usage/injected-logger-example.md) | End-to-end working example of `@Inject Logger`. **Suggested first stop for new users.** | +| [`usage/ecstasy-vs-java-examples.md`](usage/ecstasy-vs-java-examples.md) | Per-feature SLF4J Java vs `lib_logging` Ecstasy. | +| [`usage/slf4j-parity.md`](usage/slf4j-parity.md) | Exhaustive type/method mapping reference. | +| [`usage/slog-parity.md`](usage/slog-parity.md) | Go `log/slog` vs `lib_slogging` mapping reference. | +| [`usage/platform-and-examples-adaptation.md`](usage/platform-and-examples-adaptation.md) | Migration survey for existing Ecstasy code bases. | +| **Extension** | | +| [`usage/custom-sinks.md`](usage/custom-sinks.md) | How to write a custom `LogSink`, with worked examples. | +| [`usage/custom-handlers.md`](usage/custom-handlers.md) | How to write a custom slog `Handler`, with worked examples. | +| [`usage/structured-logging.md`](usage/structured-logging.md) | Structured logging dive: key/value pairs, JSON output, fluent builder. | +| [`usage/lazy-logging.md`](usage/lazy-logging.md) | Lazy message/value construction: Kotlin blocks, Java suppliers, and the Ecstasy API. | +| [`usage/configuration.md`](usage/configuration.md) | Logback XML vs proposed Ecstasy JSON configuration, dynamic sink/handler reload, and equivalence mapping. | +| **Follow-up backend/compiler work** | | +| [`future/logback-integration.md`](future/logback-integration.md) | Configuration-driven Logback-style backend on top of the shipped primitives. | + +## Where the actual code lives + +``` +lib_logging/ SLF4J-shaped library +├── build.gradle.kts +├── README.md module-level intro +└── src/ + ├── main/x/ + │ ├── logging.x module declaration + │ └── logging/ + │ ├── Level.x severity enum + │ ├── Marker.x marker interface (org.slf4j.Marker) + │ ├── BasicMarker.x default marker impl + │ ├── MarkerFactory.x stateful marker factory + │ ├── MDC.x per-fiber mapped diagnostic context (const) + │ ├── LogEvent.x immutable event const (Marker[] markers) + │ ├── LogSink.x SPI — the API/impl boundary + │ ├── Logger.x user-facing facade interface + │ ├── LoggingEventBuilder.x SLF4J 2.x fluent builder + │ ├── LoggerFactory.x non-injection acquisition path + │ ├── LoggerRegistry.x identity-stable interning of named children + │ ├── MessageFormatter.x {}-substitution + throwable promotion + │ ├── BasicLogger.x canonical Logger impl (const) + │ ├── BasicEventBuilder.x canonical builder impl + │ ├── ConsoleLogSink.x default sink (const), forwards to @Inject Console + │ ├── JsonLogSink.x JSON-Lines sink (const), rendered by lib_json + │ ├── JsonLogSinkOptions.x root level, redaction, inclusion, field names + │ ├── CompositeLogSink.x multi-destination fanout (const) + │ ├── HierarchicalLogSink.x longest-prefix per-logger thresholds (service) + │ ├── AsyncLogSink.x bounded async wrapper (service) + │ ├── NoopLogSink.x drops every event (const) + │ └── MemoryLogSink.x test-helper, captures events (service) + └── test/x/LoggingTest/ 70 focused XTC test methods + +lib_slogging/ slog-shaped sibling library +├── build.gradle.kts +└── src/ + ├── main/x/ + │ ├── slogging.x module declaration + │ └── slogging/ + │ ├── Level.x open Int severity (Level(severity, label)) + │ ├── Attr.x the single structured-data carrier + │ ├── Record.x immutable event const (the LogEvent equivalent) + │ ├── Handler.x SPI (the LogSink equivalent) + │ ├── BoundHandler.x default with/withGroup derivation wrapper + │ ├── Logger.x concrete user-facing const + │ ├── LoggerContext.x optional SharedContext helper + │ ├── TextHandler.x default human-readable handler (const) + │ ├── JSONHandler.x JSON-Lines handler (const, lib_json renderer) + │ ├── HandlerOptions.x threshold, redaction, source, field-name knobs + │ ├── AsyncHandler.x bounded async wrapper (service) + │ ├── NopHandler.x drops every record (const) + │ └── MemoryHandler.x test-helper (service) + └── test/x/SLoggingTest/ 41 focused XTC test methods +``` + +--- + +Next: [`usage/injected-logger-example.md`](usage/injected-logger-example.md) → diff --git a/doc/logging/api-cross-reference.md b/doc/logging/api-cross-reference.md new file mode 100644 index 0000000000..80f19a86bf --- /dev/null +++ b/doc/logging/api-cross-reference.md @@ -0,0 +1,122 @@ +# Logging API cross-reference + +> **Audience:** reviewers and porters who want a single page mapping every Ecstasy type to its SLF4J or Go slog counterpart. + +This page maps the Ecstasy POC types to the well-known APIs they intentionally +mirror. Use it when reading the source: the point is not "we copied Java" or "we +copied Go", but "we reused the smallest familiar shape, then adapted acquisition, +services, consts, and fiber context to Ecstasy." + +Primary references: + +- SLF4J API docs: [`org.slf4j.Logger`](https://www.slf4j.org/api/org/slf4j/Logger.html), + [`LoggerFactory`](https://www.slf4j.org/api/org/slf4j/LoggerFactory.html), + [`Marker`](https://www.slf4j.org/api/org/slf4j/Marker.html), + [`MarkerFactory`](https://www.slf4j.org/api/org/slf4j/MarkerFactory.html), + [`MDC`](https://www.slf4j.org/api/org/slf4j/MDC.html), + [`LoggingEventBuilder`](https://www.slf4j.org/apidocs/org/slf4j/spi/LoggingEventBuilder.html), + [`MessageFormatter`](https://www.slf4j.org/api/org/slf4j/helpers/MessageFormatter.html), + [`Level`](https://www.slf4j.org/api/org/slf4j/event/Level.html), + [`LoggingEvent`](https://www.slf4j.org/api/org/slf4j/event/LoggingEvent.html). +- Go `log/slog` package docs: [`log/slog`](https://pkg.go.dev/log/slog). +- Related Java structured/fluent APIs: + [`Log4j 2 fluent API`](https://logging.apache.org/log4j/2.x/manual/logbuilder.html), + [`Google Flogger`](https://google.github.io/flogger/), and + [`logstash-logback-encoder`](https://github.com/logfellow/logstash-logback-encoder). + +## SLF4J-shaped `lib_logging` + +| Ecstasy implementation | Prior-art API | Same idea | Ecstasy difference | +|---|---|---|---| +| [`Logger`](../../lib_logging/src/main/x/logging/Logger.x) | SLF4J [`Logger`](https://www.slf4j.org/api/org/slf4j/Logger.html) | Per-level methods, level checks, parameterized `{}` messages, markers, fluent `atInfo()` family. | Ecstasy collapses SLF4J's many overloads into default arguments. Acquisition is `@Inject Logger logger;` plus `logger.named("full.name")`, not a static classpath binding. | +| [`BasicLogger`](../../lib_logging/src/main/x/logging/BasicLogger.x) | SLF4J binding logger implementations such as `SimpleLogger` / Logback logger | One canonical implementation funnels all call shapes through a single emission path. | It is a `const` over a `LogSink`, so calls run on the caller's fiber and can see `MDC` `SharedContext` state. | +| [`LoggerFactory`](../../lib_logging/src/main/x/logging/LoggerFactory.x), [`LoggerRegistry`](../../lib_logging/src/main/x/logging/LoggerRegistry.x) | SLF4J [`LoggerFactory`](https://www.slf4j.org/api/org/slf4j/LoggerFactory.html) / `ILoggerFactory` | Name-keyed logger acquisition and identity-stable lookup. | The factory is an injected service escape hatch, not the primary acquisition path. The registry is scoped to one sink/container instead of JVM-global classpath state. | +| [`Level`](../../lib_logging/src/main/x/logging/Level.x) | SLF4J [`org.slf4j.event.Level`](https://www.slf4j.org/api/org/slf4j/event/Level.html) | `Trace`, `Debug`, `Info`, `Warn`, `Error`. | Adds sink-side `Off` and exposes integer `severity` for cheap threshold checks. | +| [`Marker`](../../lib_logging/src/main/x/logging/Marker.x), [`BasicMarker`](../../lib_logging/src/main/x/logging/BasicMarker.x), [`MarkerFactory`](../../lib_logging/src/main/x/logging/MarkerFactory.x) | SLF4J [`Marker`](https://www.slf4j.org/api/org/slf4j/Marker.html), [`MarkerFactory`](https://www.slf4j.org/api/org/slf4j/MarkerFactory.html) | Named event tags, child-marker containment, detached vs interned markers, multiple markers through the fluent API. | `Marker` also extends XDK `Freezable`, `Stringable`, and `Hashable` so markers can ride on `const LogEvent`s and cross service boundaries. | +| [`MDC`](../../lib_logging/src/main/x/logging/MDC.x) | SLF4J [`MDC`](https://www.slf4j.org/api/org/slf4j/MDC.html) | Request-scoped string map captured on each event. | Acquired by `@Inject MDC mdc;` and backed by `SharedContext` for per-fiber semantics instead of Java static thread-local access. | +| [`MessageFormatter`](../../lib_logging/src/main/x/logging/MessageFormatter.x) | SLF4J [`MessageFormatter`](https://www.slf4j.org/api/org/slf4j/helpers/MessageFormatter.html) | `{}` substitution, escape handling, and trailing-exception promotion. | Returns `(String, Exception?)`; an explicit `cause=` on the logger call wins over a promoted trailing exception. | +| [`LoggingEventBuilder`](../../lib_logging/src/main/x/logging/LoggingEventBuilder.x), [`BasicEventBuilder`](../../lib_logging/src/main/x/logging/BasicEventBuilder.x) | SLF4J [`LoggingEventBuilder`](https://www.slf4j.org/apidocs/org/slf4j/spi/LoggingEventBuilder.html) | Fluent `setMessage`, `addArgument`, `addMarker`, `setCause`, `addKeyValue`, `log`, plus lazy message/value suppliers. | Ecstasy keeps lazy value methods explicit (`addLazyArgument`, `addLazyKeyValue`) because `function Object()` is also an `Object`, unlike Java's distinct `Supplier` overload. The level check runs before suppliers are invoked. | +| [`LogEvent`](../../lib_logging/src/main/x/logging/LogEvent.x) | SLF4J [`LoggingEvent`](https://www.slf4j.org/api/org/slf4j/event/LoggingEvent.html), Logback `ILoggingEvent` | Immutable event payload carrying logger name, level, message, markers, cause, arguments, MDC, source metadata, and key/value pairs. | Ecstasy uses a `const`, with `Marker[] markers` plus `marker` convenience accessor and a `Map` for structured KV pairs. `sourceFile` / `sourceLine` are explicit today and ready for compiler call-site capture later. | +| [`LogSink`](../../lib_logging/src/main/x/logging/LogSink.x) | Logback appender concept | Backend SPI: cheap enabled check plus event emission. | One interface allows both `const` forwarders and `service` stateful sinks; the `const`/`service` choice is documented in [`design/design.md`](design/design.md). | +| [`ConsoleLogSink`](../../lib_logging/src/main/x/logging/ConsoleLogSink.x), [`NoopLogSink`](../../lib_logging/src/main/x/logging/NoopLogSink.x), [`MemoryLogSink`](../../lib_logging/src/main/x/logging/MemoryLogSink.x) | `slf4j-simple`, `slf4j-nop`, Logback `ListAppender`-style test capture | Minimal default output, silent output, and test capture. | Shipped as tiny Ecstasy sinks so the base library is useful without a Logback-style backend. | +| [`JsonLogSink`](../../lib_logging/src/main/x/logging/JsonLogSink.x), [`JsonLogSinkOptions`](../../lib_logging/src/main/x/logging/JsonLogSinkOptions.x) | Logback JSON encoders such as LogstashEncoder / cloud appenders | One JSON object per event, rendered with `lib_json`, preserving MDC, markers, key/value pairs, exception structure, and source metadata. | Options are programmatic rather than XML: root threshold, redacted keys, redaction token, field names, and inclusion flags. | +| [`CompositeLogSink`](../../lib_logging/src/main/x/logging/CompositeLogSink.x) | Multiple Logback appenders attached to one logger | Fans out an event to every delegate sink that enables it. | Composition is an ordinary `LogSink`, so callers and `BasicLogger` stay unchanged. | +| [`HierarchicalLogSink`](../../lib_logging/src/main/x/logging/HierarchicalLogSink.x) | Logback `LoggerContext` / per-logger level configuration | Longest-prefix logger-name threshold routing before delegating. | Programmatic API (`setLevel`, `clearLevel`, `effectiveLevel`) instead of `logback.xml`; config loaders can target this later. | +| [`AsyncLogSink`](../../lib_logging/src/main/x/logging/AsyncLogSink.x) | Logback `AsyncAppender` | Bounded queue wrapper around any sink, with `flush`, `close`, dropped-count, and pending-count probes. | Implemented as an Ecstasy `service`; events already captured MDC/source metadata before enqueue. | + +## slog-shaped `lib_slogging` + +| Ecstasy implementation | Prior-art API | Same idea | Ecstasy difference | +|---|---|---|---| +| [`Logger`](../../lib_slogging/src/main/x/slogging/Logger.x) | Go [`slog.Logger`](https://pkg.go.dev/log/slog#Logger) | A logger holds a handler; per-level methods create records and forward them. `with` / `withGroup` derive handlers. | Ecstasy `Logger` is a `const` and is injectable via `@Inject slogging.Logger logger;`. It omits Go's `context.Context` parameter from per-call APIs and adds lazy message suppliers. | +| [`LoggerContext`](../../lib_slogging/src/main/x/slogging/LoggerContext.x) | Go [`context.Context`](https://pkg.go.dev/context) as used by `log/slog` | Optional request-scoped propagation for the active logger. | Uses Ecstasy `SharedContext` instead of threading `context.Context` through every logging call. Explicit logger passing remains the default recommendation. | +| [`Handler`](../../lib_slogging/src/main/x/slogging/Handler.x), [`BoundHandler`](../../lib_slogging/src/main/x/slogging/BoundHandler.x) | Go [`slog.Handler`](https://pkg.go.dev/log/slog#Handler) | `enabled`, `handle`, `withAttrs`, `withGroup`; derivation lets handlers pre-resolve context. | Ecstasy drops the `context.Context` parameter and `error` return for now; `BoundHandler` supplies the default derivation semantics for simple handlers. | +| [`Record`](../../lib_slogging/src/main/x/slogging/Record.x) | Go [`slog.Record`](https://pkg.go.dev/log/slog#Record) | One immutable record carries time, level, message, source, and attrs. | Adds an explicit `Exception?` slot because Ecstasy exceptions are first-class and tests can assert on them directly. | +| [`Attr`](../../lib_slogging/src/main/x/slogging/Attr.x), [`LazyValue`](../../lib_slogging/src/main/x/slogging/LazyValue.x) | Go [`slog.Attr`](https://pkg.go.dev/log/slog#Attr), [`slog.Group`](https://pkg.go.dev/log/slog#Group), `LogValuer` | All structured data is key/value attrs; groups are nested attrs; lazy values resolve after enablement. | Uses `Object` values instead of Go's `slog.Value` kind union. `Attr.lazy("k", () -> v)` is the POC's compact `LogValuer` equivalent. | +| [`Level`](../../lib_slogging/src/main/x/slogging/Level.x) | Go [`slog.Level`](https://pkg.go.dev/log/slog#Level) | Open integer severity line with canonical `Debug`, `Info`, `Warn`, `Error`. | Stores a display `label` alongside severity for simple rendering. | +| [`HandlerOptions`](../../lib_slogging/src/main/x/slogging/HandlerOptions.x) | Go [`HandlerOptions`](https://pkg.go.dev/log/slog#HandlerOptions) | Root level, source inclusion, field names, and replacement policy live beside handlers. | This POC models the production knobs it needs now: threshold, redacted keys, redaction token, source inclusion, and JSON/text field names. | +| [`TextHandler`](../../lib_slogging/src/main/x/slogging/TextHandler.x) | Go [`TextHandler`](https://pkg.go.dev/log/slog#TextHandler) | Human-readable `key=value` output and group prefixing. | Writes through `@Inject Console` and honors `HandlerOptions` threshold, source inclusion, and redaction. | +| [`JSONHandler`](../../lib_slogging/src/main/x/slogging/JSONHandler.x) | Go [`JSONHandler`](https://pkg.go.dev/log/slog#JSONHandler) | One line-delimited JSON object per record, nested groups, structured exceptions. | Renders through XDK `lib_json`, writes through injected `Console`, and honors `HandlerOptions` field names, source inclusion, and redaction. | +| [`AsyncHandler`](../../lib_slogging/src/main/x/slogging/AsyncHandler.x) | Go async handler wrappers are usually third-party; same role as Logback `AsyncAppender` | Bounded queue wrapper around any handler, preserving already-constructed records. | Implemented as an Ecstasy `service` and keeps `withAttrs` / `withGroup` derivation by wrapping derived delegates. | +| [`MemoryHandler`](../../lib_slogging/src/main/x/slogging/MemoryHandler.x), [`NopHandler`](../../lib_slogging/src/main/x/slogging/NopHandler.x) | Go test/discard handler patterns | Capture records for assertions or drop records entirely. | Small Ecstasy convenience handlers; not direct Go stdlib types. | + +## No slog counterpart for logger categories or markers + +Go `slog.Logger.WithGroup` is grouped attribute namespacing, not a named logger +hierarchy. It lets a handler qualify output keys (for example `payments.amount` in text +or `"payments": {"amount": ...}` in JSON), but it does not create a logger named +`com.acme.payments` that a backend can route with Logback's longest-prefix rule. + +Go `slog` also has no `Marker` / `MarkerFactory` equivalent. The slog equivalent is an +attribute such as `slog.Bool("audit", true)`. That is simpler and often preferable for +JSON output, but it does not carry SLF4J marker identity, marker containment, or +marker-aware enabled checks. + +## No slog counterpart for `MessageFormatter` + +`lib_logging.MessageFormatter` intentionally has no `lib_slogging` equivalent. SLF4J +uses a templated message plus positional arguments: + +```ecstasy +logger.info("processed {} records", [count]); +``` + +Go `log/slog` uses a completed message plus structured attrs: + +```ecstasy +logger.info("processed records", [Attr.of("count", count)]); +``` + +Reusing `MessageFormatter` inside `lib_slogging` would be architecturally wrong for +the core API because it would add a second structured-data channel (positional +arguments) to the design whose main advantage is having exactly one channel (`Attr`). +If migration sugar is useful later, it should live in an adapter/helper module that +makes the compromise explicit, for example `lib_slogging_slf4j_adapter`. + +## Call-shape comparison + +| Intent | Java / Go prior art | Ecstasy POC | +|---|---|---| +| Named logger | `LoggerFactory.getLogger(PaymentService.class)` | `@Inject Logger root; Logger log = root.named("PaymentService");` | +| SLF4J-style formatted message | `log.info("processed {}", count)` | `log.info("processed {}", [count])` | +| SLF4J-style structured event | `log.atInfo().addKeyValue("id", id).log("processed")` | `log.atInfo().addKeyValue("id", id).log("processed")` | +| Request context with MDC | `MDC.put("requestId", id)` | `@Inject MDC mdc; mdc.put("requestId", id);` | +| slog-style derived logger | `logger.With("requestId", id)` | `Logger reqLog = logger.with([Attr.of("requestId", id)]);` | +| slog-style context logger | `logger.LogAttrs(ctx, level, msg, attrs...)` | `using (loggerContext.bind(reqLog)) { ... }` plus `loggerContext.currentOr(root)` | +| slog-style structured event | `logger.Info("processed", "count", count)` | `logger.info("processed", [Attr.of("count", count)]);` | + +## What to keep in mind while reviewing + +- `lib_logging` optimizes for **JVM familiarity and Logback-style operational + extension**. It keeps separate concepts for message args, markers, MDC, and + structured key/value pairs because SLF4J users already know those concepts. +- `lib_slogging` optimizes for **conceptual economy**. Everything structured is an + `Attr`; categories and request context are not special. +- Both designs put real deployment behavior behind one extension point: + `LogSink` for the SLF4J shape, `Handler` for the slog shape. +- The biggest Ecstasy-specific change in both designs is acquisition: the long-term + shape should be container-controlled injection, not global static configuration. + +--- + +Reference doc — dipped into rather than read straight through. Up: [`README.md`](README.md) diff --git a/doc/logging/cloud-integration.md b/doc/logging/cloud-integration.md new file mode 100644 index 0000000000..552b97266b --- /dev/null +++ b/doc/logging/cloud-integration.md @@ -0,0 +1,238 @@ +# Cloud-native logging — why API familiarity matters + +This doc explains *why* the choice between SLF4J-shaped and slog-shaped logging +APIs (`lib_logging` vs `lib_slogging`, see `lib-logging-vs-lib-slogging.md`) is +not just an aesthetic call. It is the entry point to the major cloud +observability ecosystems — and the wrong API choice silently rules Ecstasy out +of common deployment patterns even when the code itself is fine. + +The doc is written assuming the reader is an expert systems engineer — runtimes, +compilers, JIT — but has not personally deployed a service to GCP / AWS / Azure +in production. That is a common, perfectly reasonable shape; the cloud +ecosystems are sprawling and most of the day-to-day knowledge lives in +operations teams, not language teams. So we start by laying out what +"deploying an application" looks like in 2026, then explain how logging fits +into that, then come back to "and that is why this API choice matters." + +## A brief geography of how applications are deployed today + +For a long time — through about 2010 in mainstream practice — the unit of +software delivery was an **installer that produced a process on a machine**: an +MSI on Windows, a `.deb` on Linux, an installed shell command. Users started +the process, the process wrote log lines to a file under `/var/log/` or +`%PROGRAMDATA%\\logs`, and ops people logged into the machine and tailed +the file when something went wrong. The logging API's job was to format text +into that file. + +That world still exists for desktop applications and CLI tools (the XDK itself +is delivered this way, and rightly so). But for **server-side applications and +services** — anything that handles requests, reads from a database, calls a +peer service — the deployment model has been replaced by a fundamentally +different shape: + +1. The application is built into a **container image** (most often via Docker / + OCI). The image is a self-contained filesystem with the runtime, the app, + and its dependencies — conceptually "an executable, but for a Linux + sandbox." +2. The image is run not on one machine but on a **cluster orchestrator**: GKE + (GCP), EKS (AWS), AKS (Azure), or a managed equivalent (Cloud Run, App + Runner, Container Apps). The orchestrator decides which physical node runs + the container, can run dozens or thousands of replicas, and replaces them + automatically when nodes fail. +3. There is **no persistent local filesystem** the application can rely on for + logs. Containers come and go; their disks are wiped between restarts. A + log file inside a container is gone the moment that container dies — which + for a healthy fleet might be every 12 hours. +4. So, by convention, **the application writes structured log records to its + own stdout/stderr** as one record per line. The orchestrator captures + those streams and forwards them, fully automatically, to a managed + logging product (Cloud Logging, CloudWatch Logs, Azure Monitor) — which + indexes them, makes them queryable through a web UI, sends alerts when + patterns match, and retains them per a configured policy. + +Two consequences flow from (4) and shape every modern logging API: + +- **Logs are queried, not tailed.** Operations engineers don't `ssh` into the + machine and `tail -f`; they open a web console, type a structured query + ("show me all `ERROR`-level events in the last hour with `requestId=r_42`"), + and the cloud product returns results across thousands of containers as one + search. +- **Each log line is consumed as a record, not as text.** The cloud product + parses the line as a JSON object (or an equivalent structured format) and + indexes every field. A string `"INFO user u_3 logged in"` is a single + searchable string. A JSON object `{"level":"INFO","user":"u_3","action":"login"}` + has every field independently queryable, filterable, groupable, and + alert-able. + +The shift from "logs are text in a file" to "logs are queryable JSON streams" +is the single largest operational change in server software since the year +2000. Every popular logging API since 2014 (slf4j-2.x, log4j2, slog, +zap, tracing, Serilog) was retrofitted or freshly designed around it. + +## What "the cloud product on the other side" actually does + +Pick GCP Cloud Logging as the canonical example. When a container's stdout +emits a JSON line, the GKE node's logging agent (`fluent-bit` by default) +ships it to the Cloud Logging API. The product then: + +1. **Parses** the JSON. Each top-level key becomes an indexed field. +2. **Maps** specific keys to its own first-class concepts: + - `severity` (or sometimes `level`) — must be one of + `DEBUG / INFO / WARNING / ERROR / CRITICAL`. Drives the colour of the row + and feeds alert policies. + - `message` (or `msg`) — the human-readable text shown in the UI's main + column. + - `trace` and `spanId` — if present, link the log entry to a distributed + trace (Cloud Trace, Jaeger). Click the row, see the entire request flow. + - `labels.*` — a string-only key/value map exposed as filterable tags in + the UI (the same role as Logback's `Marker` plays in older configs). + - `jsonPayload.*` — anything else, fully indexed and queryable. +3. **Persists** the record according to retention policy (30 days default + for the basic tier, up to 10 years on the audit tier). +4. **Indexes** the fields for search, alerting, and metric extraction. + +CloudWatch Logs (AWS) and Azure Monitor (Application Insights) work the same +way — different product names, near-identical conventions. JSON in, structured +search out. + +The key takeaway: the cloud-side product **reads the structured fields the +logging API put on the wire**. If the API never emits a `requestId` field, +you can't filter on it. If the API emits everything as one big string, all +querying degrades to substring search. The expressiveness of the logging API +sets a hard ceiling on the operability of the deployed system. + +## The standard adapters — where API familiarity lives + +The logging APIs we are comparing exist not in isolation but in a +*pre-existing graph of adapters* that translate them into the schemas the +cloud products expect. + +### Google Cloud Logging + +- **SLF4J / Logback**: Google ships [`com.google.cloud.logging.logback.LoggingAppender`](https://cloud.google.com/logging/docs/setup/java). + Drop it into your `logback.xml`, and every Logback `LoggingEvent` becomes a + Cloud Logging entry with `severity` mapped from level, `message` mapped from + the formatted message, MDC keys mapped to `jsonPayload`, **markers mapped to + `labels`**, and trace IDs mapped from the SLF4J `MDC` keys + `traceId`/`spanId`/`traceSampled`. Existing Java code that already used SLF4J + needs *no source changes* to send logs to Cloud Logging — it is a config + file change. +- **Go `log/slog`**: [`cloud.google.com/go/logging.HandlerOptions`](https://pkg.go.dev/cloud.google.com/go/logging#HandlerOptions) + produces a `slog.Handler` that writes records directly to the Cloud Logging + API. slog `Attr`s become `jsonPayload` fields; the slog level numbers map + cleanly to GCP's severity ladder. + +### AWS CloudWatch Logs + +- **SLF4J / Logback**: the community-maintained [`cloudwatch-logback-appender`](https://github.com/j256/cloudwatchlogbackappender) + or the more common pattern of `logstash-logback-encoder` writing JSON to + stdout, which the Lambda runtime / ECS Firelens picks up automatically. MDC + keys become top-level JSON fields; markers go into a `markers` array. AWS + X-Ray trace IDs propagate through the same MDC channel. +- **Go `log/slog`**: stdout JSON is the default; AWS Lambda's runtime captures + it directly. Alternatively, the [`slog-cloudwatch`](https://pkg.go.dev/github.com/jutkko/slog-cloudwatch) + community handler writes to CloudWatch Logs directly. + +### Azure Monitor / Application Insights + +- **SLF4J / Logback**: [`applicationinsights-logging-logback`](https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-arguments) + ships every `LoggingEvent` as a `TraceTelemetry`. MDC becomes "custom + properties"; **Logback markers become custom properties** keyed + `markerName -> "true"`. App Insights' Kusto query language indexes them all. +- **Go `log/slog`**: any slog handler that writes JSON to stdout works + through Azure Container Apps' built-in log capture; community packages also + exist. + +The pattern is identical across the three providers: the API's concepts +(level, MDC keys, markers, structured fields) are pre-mapped to the cloud +product's concepts (severity, custom properties, labels, indexed JSON +payload) **by adapters that already exist and are battle-tested**. + +## Why this matters for the `lib_logging` vs `lib_slogging` decision + +If Ecstasy chooses an API shape that is *isomorphic* to one of these +two ecosystems, the path to "drop in an Ecstasy server, view its logs in +Cloud Logging / CloudWatch / App Insights" is "write a thin adapter that +translates `LogEvent` / `Record` to the provider's wire format" — typically a +weekend project, often community-maintained. + +If Ecstasy chooses a *novel* shape — well-designed, perhaps cleaner than +either — every prospective adopter has to write a custom adapter from scratch +for every provider they care about, no community will share that work, and +every team's adapter is a private liability they have to maintain. The +"cleaner" API ends up gated on the willingness to maintain a parallel +ecosystem of adapters. + +Concrete read on the two libraries' positions: + +- **`lib_logging`** (SLF4J shape) gets the largest ready-made adapter graph. + Every JVM team's mental model maps onto it: `Logger`, `MDC`, `Marker`, + `Level`, fluent builder. The cloud-side schema mappings (level → severity, + MDC → jsonPayload, marker → labels) carry over essentially untouched. A + prospective Ecstasy user who has been writing SLF4J in Java for fifteen + years can deploy an Ecstasy service to GCP and have its logs render + correctly in the Cloud Logging UI with `requestId` filterable, without + having to learn a new mental model. + +- **`lib_slogging`** (Go slog shape) gets the second-largest adapter graph. + Cloud Logging, CloudWatch, and App Insights all have native slog handlers. + The audience is narrower — cloud-native Go engineers — but it is the + fastest-growing audience in production observability, and the slog + attribute model maps to JSON-payload-shaped logging *more* directly than + SLF4J does (no markers / MDC reconciliation step; everything is already an + attribute). + +A library that picks neither shape — say, a clean-slate Ecstasy-native API +based on tagged unions and `SharedContext` — *can* be built, and may even be +simpler to use in pure Ecstasy, but it ships with no adapter graph at all. +The first user who tries to deploy to Cloud Logging finds out the hard way +that they're writing the integration themselves. + +## What "do the right thing for cloud" means in API terms + +Independent of the SLF4J-vs-slog decision, here are the API features that +every modern cloud product relies on, ordered by how badly things break if +the API doesn't expose them: + +1. **Per-event structured fields.** Every emission must be able to carry + key/value pairs that survive verbatim into the wire format. SLF4J 2.x's + `KeyValuePair` list and slog's `Attr` both satisfy this. **Both + `lib_logging` and `lib_slogging` already do.** +2. **A field for the trace/span ID.** Distributed tracing is now table + stakes; the logging product correlates trace IDs to render the request + flow. SLF4J does this through MDC keys (`traceId`, `spanId`) by + convention; slog through dedicated attrs. **Both libraries can carry + these; the integration code chooses which keys to use.** +3. **A canonical level ladder that maps to provider severity.** GCP wants + `DEBUG / INFO / WARNING / ERROR / CRITICAL`. SLF4J's five levels map + directly. slog's open `Int` levels map via comparison. **Both fine.** +4. **A way to attach context without threading parameters.** SLF4J's MDC, + slog's `Logger.With(...)` chains. Without this, request-scoped fields + (like `requestId`) need to be passed through every function, which is + a non-starter for real applications. **Both libraries provide this in + different ways — see the comparison doc § 3.2.** +5. **An async / batched sink.** When a service emits 10k log events per + second, blocking the request thread on disk I/O is unacceptable. Both + libraries' `AsyncLogSink` / `AsyncHandler` wrappers cover this. + +Note that *all five* of these features exist for the cloud-deployment story, +not for the local-tail-the-log-file story. They are exactly the features that +were retrofitted onto SLF4J between 2018 and 2022, and the features that +slog was designed to have from day one. + +## TL;DR + +Choosing between `lib_logging` (SLF4J) and `lib_slogging` (slog) is **not** a +question about which API reads better in isolation. It is a question about +which existing cloud-side adapter ecosystem an Ecstasy deployment plugs into +on day one. SLF4J's adapter graph is larger and more mature; slog's is +younger, growing fastest, and has a slightly cleaner mapping into structured +JSON output. Either choice is defensible; a third "cleaner" choice means +shipping into an empty ecosystem and writing the integrations yourself. + +This document exists so that decision is made with that context, not without +it. + +--- + +Previous: [`concepts/markers.md`](concepts/markers.md) | Next: [`design/design.md`](design/design.md) → | Up: [`README.md`](README.md) diff --git a/doc/logging/concepts/markers.md b/doc/logging/concepts/markers.md new file mode 100644 index 0000000000..3f22e65fdb --- /dev/null +++ b/doc/logging/concepts/markers.md @@ -0,0 +1,299 @@ +# Log markers + +> **Audience:** anyone evaluating `lib_logging` who has not used SLF4J/Logback +> markers in production. Read this if you see `Marker` in the API and wonder +> what it earns. +> +> **Length-conscious read:** the [bottom line](#bottom-line) tells you the +> outcome in two paragraphs. The rest is supporting detail. + +## What a marker is + +A **log marker** is a *named tag* attached to a log record that gives sinks a +first-class "category" to filter on, separate from the message text and the +severity level. SLF4J introduced the concept in 2005; the same idea exists in +Log4j 2 (`Marker`) and is conceptually similar to `MDC` keys in Logback config +or "properties" in log4net — but markers travel *with the event* whereas MDC +travels *with the thread*. + +Concrete use cases — all from production deployments using SLF4J/Logback: + +- **`AUDIT`** — route every event tagged with this marker to an *append-only* + audit log that satisfies SOX / PCI-DSS / HIPAA retention rules. The same + logger writes ordinary events to a normal file; the audit log only sees + marker-tagged events. +- **`SECURITY`** / **`SECURITY.BREACH`** — forward security-relevant events to + a SIEM (Splunk, Elastic, Datadog) on a separate channel, optionally with + stricter encryption / redaction filters in front. +- **`SLOW_QUERY`** / **`PERFORMANCE`** — emit perf-suspect events to an APM + pipeline (New Relic, AppDynamics) without the noise of regular debug logs. +- **`BILLING`** / **`CONFIDENTIAL`** — route financial-impact events to a + system with longer retention and tighter access controls. +- **`PUBLIC_API_REQUEST`** — tee API-boundary events to a separate access log + used for traffic analysis and rate-limit tuning. + +Markers also support a *parent/child DAG*: in SLF4J, +`SECURITY.contains(BREACH)` returns true if `BREACH` was added as a child of +`SECURITY`. A filter configured on `SECURITY` therefore catches any descendant +marker without needing to know their names. This is the marker model's only +real edge over a plain `is_security: true` attribute — and almost no +production code uses it. + +## Who actually uses markers + +Inside the JVM ecosystem, marker support is widespread: + +- **Logback** (the dominant SLF4J binding): markers are first-class. + `TurboFilter` and `EvaluatorFilter` can route events by marker. Most JSON + encoders emit the marker name as a structured field. Used in **almost every + Java enterprise app** built between roughly 2010 and 2020. +- **Log4j 2**: `MarkerManager` interns markers; `MarkerFilter` accepts/denies + by marker name. Used heavily in Apache projects (Kafka, Cassandra, Solr) and + in legacy enterprise codebases. +- **SLF4J 2.x** itself: `Logger.atInfo().addMarker(...)` is the canonical + fluent builder usage, and `LoggingEvent.getMarkers(): List` is the + v2 wire format. +- **Apache Camel, Spring Boot, JBoss / WildFly, ActiveMQ, Hazelcast** — all + surface marker-based filtering in their default Logback / Log4j configs. + +Outside the JVM, markers are deliberately absent: + +- **Go's `log/slog`** rejects markers. Categorisation is just + `slog.Bool("audit", true)` or `slog.String("category", "security")`. +- **Python's `logging`** has no markers. Categories live in the logger name + hierarchy (`security.breach`) and in `extra={...}` dict attrs. +- **.NET's `Microsoft.Extensions.Logging`** has no markers. Categories are + logger names; structured fields go in the `state` object. +- **Rust's `tracing`** has no markers. The `target` field on each event + approximates the same idea; otherwise `field::display(...)` attributes. + +The pattern: **markers are a JVM-ecosystem feature**, born of SLF4J, used +widely *inside* the JVM, and deliberately excluded from every well-known +modern logger designed since 2015. + +## What is the actual payoff + +Three things, in decreasing order of how often they justify the API surface: + +1. **Backend routing.** A sink looks at the marker and decides where the event + goes (audit log vs. main log vs. SIEM). This is how 90% of production + marker usage works in practice. It is the *only* real pull for keeping + markers. +2. **Marker DAG queries.** `SECURITY.contains(BREACH)` and `marker.iterator` + walking parent references. Mostly unused outside of niche audit frameworks. +3. **Marker as a typed-tag** — a way to say "this is the audit log line" + without spelling out a `"category"` string. Same as a string-keyed boolean + attr; the "type-ness" is a thin gain. + +Backend routing is real, but it is the same operation as filtering on +`attrs["category"] == "AUDIT"`. The marker concept is *one type to learn* and +gives you a slightly cheaper filter (interned reference identity vs string +equality). Whether that's worth a separate type in the API depends on whether +the audience reflexively reaches for `MarkerFactory.getMarker("AUDIT")` (JVM +veterans: yes; everyone else: no). + +## Marker patterns side-by-side + +The cases below cover what marker users actually do in production. Each shows +the SLF4J idiom (`lib_logging`) and the slog equivalent (`lib_slogging`). + +### Single category tag — "this is an audit event" + +```ecstasy +// lib_logging +logger.atInfo() + .addMarker(MarkerFactory.getMarker("AUDIT")) + .log("user signed in"); +``` + +```ecstasy +// lib_slogging +logger.info("user signed in", [Attr.of("audit", True)]); +// or, if you prefer string-tag style: +logger.info("user signed in", [Attr.of("category", "AUDIT")]); +``` + +Cloud-side rendering (GCP Cloud Logging, JSON): + +```jsonc +// from lib_logging via the Logback Cloud Logging appender: +{ "severity": "INFO", "message": "user signed in", "labels": { "AUDIT": "true" } } + +// from lib_slogging via slog's JSONHandler: +{ "level": "INFO", "msg": "user signed in", "audit": true } +``` + +The SLF4J adapter shoves markers into `labels` (a string-only flat map). The +slog version emits the boolean directly into `jsonPayload`. **Both are +queryable in Cloud Logging.** A query for "all audit events" is +`labels.AUDIT="true"` in the SLF4J case and `jsonPayload.audit=true` in the +slog case — same operational outcome, different cell in the JSON. + +### Multiple markers on one event + +```ecstasy +// lib_logging — fluent builder accumulates +Marker AUDIT = MarkerFactory.getMarker("AUDIT"); +Marker SECURITY = MarkerFactory.getMarker("SECURITY"); + +logger.atInfo() + .addMarker(AUDIT) + .addMarker(SECURITY) + .log("login attempt"); +``` + +```ecstasy +// lib_slogging — just two attrs +logger.info("login attempt", [ + Attr.of("audit", True), + Attr.of("security", True), +]); +``` + +Multi-marker is what motivated W-1 in [`../open-questions.md`](../open-questions.md). +In slog the question doesn't arise — repeated attrs are the only shape. + +### Hierarchical relationships — "BREACH is a kind of SECURITY" + +This is the one case where markers genuinely add something attributes don't. +SLF4J markers can have child references; a filter on `SECURITY` catches every +descendant marker without enumerating them. + +```ecstasy +// lib_logging — DAG via Marker.add +Marker SECURITY = MarkerFactory.getMarker("SECURITY"); +Marker BREACH = MarkerFactory.getMarker("BREACH"); +SECURITY.add(BREACH); + +logger.atError().addMarker(BREACH).log("token leak detected"); + +// any filter configured to match SECURITY *also* matches BREACH: +assert SECURITY.contains(BREACH); +``` + +```ecstasy +// lib_slogging — emit both names; the filter does prefix matching +logger.error("token leak detected", [ + Attr.of("category", "SECURITY.BREACH"), + Attr.of("parentCategory", "SECURITY"), // optional, helps simple filters +]); +``` + +**This is the strongest argument for keeping markers in `lib_logging`** — the +DAG model is genuinely more expressive than the slog equivalent. **It is also +the marker feature production code rarely uses.** Most operational filters are +`category == "SECURITY.*"` glob matches at the cloud-product level, not +transitive containment queries inside the application. + +### Sink/handler-side routing — "send audit events to a separate file" + +```ecstasy +// lib_logging — a sink that filters on event.markers +const AuditFilteringLogSink(LogSink delegate, String requiredMarkerName) + implements LogSink { + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) = + marker?.containsName(requiredMarkerName) ?: False; + @Override + void log(LogEvent event) { + for (Marker m : event.markers) { + if (m.containsName(requiredMarkerName)) { + delegate.log(event); + return; + } + } + } +} + +new AuditFilteringLogSink(new FileLogSink("/var/log/audit.log"), "AUDIT"); +``` + +```ecstasy +// lib_slogging — a handler that filters on attrs +const AuditFilteringHandler(Handler delegate, String requiredKey) + implements Handler { + @Override + Boolean enabled(Level level) = delegate.enabled(level); + @Override + void handle(Record record) { + for (Attr a : record.attrs) { + if (a.key == requiredKey && a.value == True) { + delegate.handle(record); + return; + } + } + } + @Override Handler withAttrs(Attr[] attrs) = this; + @Override Handler withGroup(String name) = this; +} + +new AuditFilteringHandler(new FileHandler("/var/log/audit.log"), "audit"); +``` + +Both implementations are about the same length. The slog one is slightly +simpler (no nested `containsName` walk on the marker DAG). The SLF4J one is +more powerful (matches transitively). + +### Programmatic categorisation from a dynamic source + +When the category isn't known at compile time (e.g. it's read from a request +header): + +```ecstasy +// lib_logging +String catName = request.header("x-log-category") ?: "DEFAULT"; +Marker cat = MarkerFactory.getMarker(catName); // interned by name +logger.atInfo().addMarker(cat).log("event"); +``` + +```ecstasy +// lib_slogging +String catName = request.header("x-log-category") ?: "DEFAULT"; +logger.info("event", [Attr.of("category", catName)]); +``` + +The marker version interns by name, so repeated requests for `"AUDIT"` get +the same `Marker` instance — slightly cheaper to compare against in a filter. +The attribute version allocates a new `String` value each time. For most +production workloads this difference is too small to measure. + +## Summary table + +| Concern | `lib_logging` (markers) | `lib_slogging` (attrs) | +|---|---|---| +| Single category | `addMarker(M)` | `Attr.of("k", v)` | +| Multiple categories | `addMarker(M1); addMarker(M2)` | `[Attr.of(...), Attr.of(...)]` | +| Hierarchical (DAG) categories | `parent.add(child); marker.contains(other)` | dotted naming + prefix filter | +| Sink-side routing | `marker?.containsName(X)` | `attr.key == X && attr.value == True` | +| Dynamic/runtime categorisation | `MarkerFactory.getMarker(name)` (interned) | `Attr.of("category", name)` | +| Identity comparison cost | reference equality on interned Marker | string equality on `key`/`value` | +| Cloud-side rendering | usually emitted as `labels.{name}` | usually emitted as a top-level JSON field | +| Familiar to Java/JVM teams | yes | no | +| Familiar to Go / cloud-native teams | no | yes | +| Forces a separate API concept | yes | no | + +## Bottom line + +For an **SLF4J-shaped library**, markers are part of the contract. Production +Logback configs filter on markers; existing code uses `MarkerFactory`; the +fluent builder's `.addMarker(...)` is the standard idiom. Dropping it forces +JVM users to rewrite their mental model, which is exactly what an SLF4J-shaped +library is trying to avoid. **`lib_logging` keeps multi-marker support.** + +For a **slog-shaped library**, importing markers would be the wrong move. +slog's single-concept attribute model is the half of its design that *is* +worth copying — markers are a separate type for what is otherwise just a +string-keyed flag, and the Go ecosystem deliberately voted against them. +**`lib_slogging` ships without markers.** Categorisation is +`Attr.of("category", "AUDIT")` and DAG queries are simulated by either dotted +naming + prefix-match filters in the handler, or by emitting both `category` +and `parentCategory` attrs. + +If the comparison surfaces "we want markers in the slog-shaped library," that +would be a strong signal the slog design is *not* the right fit and the SLF4J +shape should win — because at that point the simplification motivating slog +has been undone. + +--- + +Previous: [`../lib-logging-vs-lib-slogging.md`](../lib-logging-vs-lib-slogging.md) | Up: [`../README.md`](../README.md) diff --git a/doc/logging/design/design.md b/doc/logging/design/design.md new file mode 100644 index 0000000000..d711428f25 --- /dev/null +++ b/doc/logging/design/design.md @@ -0,0 +1,282 @@ +# lib_logging — Design + +> **Audience:** anyone reading `lib_logging` source. Architecture and the API↔impl boundary. + +## High-level architecture + +``` + ┌──────────────────────────────────┐ + user code → │ Logger (interface) │ ← public API + │ Level / Marker / MDC / LogEvent │ + │ LoggingEventBuilder │ + │ LoggerFactory │ + ├──────────────────────────────────┤ + │ BasicLogger / BasicEventBuilder │ ← canonical impl of the API + │ MessageFormatter │ + ├──────────────────────────────────┤ + │ LogSink (interface) │ ← API ↔ impl boundary + ├──────────────────────────────────┤ + │ ConsoleLogSink │ ← shipped impls + │ JsonLogSink │ + │ CompositeLogSink │ + │ HierarchicalLogSink │ + │ AsyncLogSink │ + │ NoopLogSink │ + │ MemoryLogSink │ + │ (future) configured/destination │ + │ sinks │ + └──────────────────────────────────┘ +``` + +User code holds a `Logger`. The `Logger` is a `BasicLogger` that holds a `LogSink`. The +sink is whatever the runtime injected. Everything above the `LogSink` line is sink-agnostic +and stable; everything below is swappable. + +This mirrors the layering that makes SLF4J/Logback durable in Java: the public facade +is small, the backend boundary is narrow, and concrete output policy lives behind that +boundary. The Ecstasy version keeps that shape but makes injection the acquisition +mechanism and keeps `LogSink` as the XDK-native extension point. + +## Why injection (and why per-name loggers come from `Logger.named(String)`) + +Ecstasy already routes `Console` through `@Inject Console console;`; the runtime +controls which `Console` implementation gets wired up. Loggers fit the same shape: the +runtime decides where output goes (in v0, `ConsoleLogSink`; later, possibly a +configured sink graph), and user code is sink-agnostic. + +`@Inject Logger logger;` resolves to a single fixed-name root logger — exactly one +supplier (`("logger", loggerType)`) is registered with the injector. Per-name loggers +are *derived* via `Logger.named(String)` rather than acquired through additional +injection sites: + +```ecstasy +@Inject Logger logger; +static Logger PaymentLogger = logger.named("payments"); +``` + +This mirrors SLF4J's `LoggerFactory.getLogger(MyClass.class)` idiom one-to-one, and +keeps the injector's resource table as the single registry of allowed injections: +there is one fixed `"logger"` resource, and per-name loggers are derived in XTC rather +than acquired through wildcard injection. + +## API surface + +### `Logger` + +The user-facing facade. Five level methods (`trace`, `debug`, `info`, `warn`, `error`), +each accepting `(message, arguments, cause, marker)` with sensible defaults. Five level +checks (`infoEnabled`, etc.). A polymorphic `log(level, ...)`. A fluent `atInfo()` / +`atLevel(...)` family that returns a `LoggingEventBuilder`. + +The signatures intentionally collapse SLF4J's many overloads into one canonical signature +per level using Ecstasy's default-argument support. The fluent builder covers the cases +where the parameter list isn't known statically. + +### `Level` + +`Trace, Debug, Info, Warn, Error, Off`. `Off` is sink-side only; never emitted. +`severity` field for cheap threshold comparisons. + +### `Marker` / `MarkerFactory` / `BasicMarker` + +`MarkerFactory.getMarker(name)` returns interned markers; `getDetachedMarker(name)` +returns fresh ones. Markers can have child references (DAG), and `contains` does the +transitive check. The default `BasicMarker` is straight from the SLF4J playbook — +intentionally minimal, the rich filtering is the sink's job. + +### `MDC` + +Service exposing `put / get / remove / clear / copyOfContextMap`. Sinks that care about +MDC capture `copyOfContextMap` when emitting; the snapshot is included in `LogEvent`. + +### `LogEvent` + +Immutable `const` record carrying loggerName, level, message (already substituted), +marker, exception, arguments, mdc snapshot, thread/fiber name, timestamp. + +### `LogSink` + +Two methods: `isEnabled(loggerName, level, marker)` and `log(event)`. This is the +boundary; everything below is replaceable. + +### `LoggingEventBuilder` + +SLF4J 2.x fluent builder: `setMessage / addArgument / addMarker / setCause / +addKeyValue / log`, plus lazy `setMessage`, `log`, `addLazyArgument`, and +`addLazyKeyValue` forms. The builder checks the sink at final `log(...)` time after +markers are attached. If disabled, it does not format, snapshot MDC, allocate a +`LogEvent`, or invoke lazy suppliers. Eager argument expressions still run before the +method call, exactly as in Java; use the explicit lazy methods for expensive values. + +## Default implementations + +### `BasicLogger` + +Holds `(name, sink)`. Each emission method: +1. Calls `sink.isEnabled(...)`. If False, return immediately. +2. Calls `MessageFormatter.format(message, arguments)` to substitute placeholders and + detect the SLF4J throwable-promotion case. +3. Snapshots MDC. +4. Constructs a `LogEvent` and hands it to the sink. + +### `ConsoleLogSink` + +`const` (not service) holding `@Inject Console console;` and a construct-time `rootLevel` +threshold (default `Info`). Emits one line per event in a fixed format: timestamp, thread, +padded level, logger name, message, optional `[marker=NAME]` suffix; appends the +exception on a following line. + +### `NoopLogSink` + +`const`. Drops everything. `isEnabled` returns False so callers that respect the check +skip all formatting work. Useful for libraries that want quiet defaults. + +### `MemoryLogSink` + +`service` (mutable `events[]` shared across fibers). Captures events in an array. +Test-only; not registered as a default. + +## Sink type: `const` vs `service` + +`BasicLogger` is itself a `const`. Its `sink: LogSink` field therefore requires +implementations to be `Passable`, i.e. either `immutable` (a `const`) or a `service`. +The choice between the two is not arbitrary — it follows a clean rule that mirrors what +both the SLF4J/Logback ecosystem and the surrounding XDK codebases already do: + +| Property | use `const` | use `service` | +|---|---|---| +| Has its own mutable state shared across fibers? | no | yes | +| Aggregates events for later inspection? | no | yes | +| Owns external resources (writers, sockets, queues)? | no | yes | +| Pure forwarder over an injected service? | yes | — | +| Configuration fixed at construction? | yes | (typically yes either way) | + +Concretely, in this library: + +- `NoopLogSink` — `const` (stateless). +- `ConsoleLogSink` — `const` (forwarder over `@Inject Console`, threshold fixed at + construction). Structurally identical to + `lib_ecstasy/src/main/x/ecstasy/io/ConsoleAppender.x` and `ConsoleLog.x`, both `class`. +- `MemoryLogSink`, `ListLogSink` — `service` (collect `events[]`). +- `AsyncLogSink` — `service` (owns a bounded queue and drain state). +- `HierarchicalLogSink` — `service` (owns mutable per-prefix level configuration). +- A future `FileLogSink` owning a `Writer` — `service`. + +This matches the pattern used elsewhere in the XDK / platform ecosystem: + +- `service ErrorLog extends SimpleLog` — `platform/common/src/main/x/common.x:23`, + collects log entries from many callers. +- `service ConsoleExecutionListener implements ExecutionListener` — + `xvmrepo/lib_xunit_engine/.../console/ConsoleExecutionListener.x:4`, captures test + events into a private `Map`. **Structurally identical to `MemoryLogSink`.** +- `service HostManager` with `Map> activityHistogram` — + `platform/host/.../HostManager.x:40`. +- `class ConsoleAppender(Console console, ...)` — `lib_ecstasy/.../ConsoleAppender.x:4`, + no own mutable state. + +Mapping back to Logback: Logback collapses both shapes into one `Appender` interface +(everything is a Java class with a `synchronized log()` method). In Ecstasy the +service/const distinction makes the intent explicit at the type level — and is enforced +by the compiler whenever a sink is held by a `const` such as `BasicLogger`. + +Open question: is one-interface-with-two-allowed-impl-shapes the right call, or should +`LogSink` split into two interfaces? See `../open-questions.md` (item 6). + +## Future: `lib_logging_logback` + +A separate module providing a configuration-driven sink. Reads a config tree +(programmatic, possibly XML or JSON) describing: + +- per-logger thresholds (`com.example.foo` at `Debug`, root at `Info`); +- appenders (Console, File, Rolling, Network, JSON); +- layouts (PatternLayout, JSONLayout); +- filters (level, marker, MDC, regex on message); +- async wrappers. + +The whole thing is just a different `LogSink` implementation. User code that already +worked against `lib_logging` keeps working. Detailed sketch in `../future/logback-integration.md`. + +## Per-container sink override + +Each Ecstasy container has its own injector. Host code that wants a nested +container to use a *different* sink than the host's default does so by +configuring the child container's injector explicitly — there is no +`lib_logging`-side helper for this, by design (see `../open-questions.md` item 8). + +The pattern matches how every other injectable resource in the XDK is +overridden per child container: + +```ecstasy +import logging.Logger; +import logging.LogSink; +import logging.MemoryLogSink; + +// Inside host code that's about to spawn a child container: +LogSink childSink = new MemoryLogSink(); // or any other LogSink + +// When constructing the child container, pass an injector that resolves +// `("logger", Logger)` against `childSink` instead of the host's default. +// The exact host API depends on which container-construction helper the host +// uses; the principle is the same everywhere — replace the resource supplier +// for the `Logger` injection key. +``` + +In practice, host runtimes typically expose a `withResource(...)` / +`addResourceSupplier(...)` style API on the container's injector; the host +adds an entry `("logger", Logger) → new BasicLogger(name, childSink)` before +the child container starts running. The child container's `@Inject Logger` +resolves to that supplier; the host's loggers are unaffected. + +We deliberately do not ship a `ContainerLoggingConfig.set(child, sink)` +convenience because: + +1. The container/injector API is the right place for *all* per-container + resource overrides; every library inventing its own helper would create N + parallel ways to do the same thing. +2. Real-world usage of this is rare — most programs run with a single sink + per process. Adding a helper now would lock in an API for a use case we + haven't seen demand for yet. + +If a future user needs this often enough that a helper is worth it, the +helper lives in the host runtime's injector library, not here. + +## Non-goals (v0) + +Items deliberately *not* in scope for the base POC of `lib_logging`. + +- **Distributed tracing context propagation.** `MDC` carries strings; that's + it. A future tracing library can write its trace ID into MDC and let the + logging library carry it, but `lib_logging` itself does not implement + trace/span propagation. +- **Configuration file format.** The default sink has one knob (`rootLevel`) + set at construction, and `JsonLogSinkOptions` covers JSON field names, + inclusion, threshold, and redaction. A real config story (XML / programmatic / + hot reload) belongs to the future Logback-style backend; + `../future/logback-integration.md` sketches it. +- **Automatic call-site source capture.** `Logger.logAt(...)` gives the runtime/compiler + a stable lowering target and sinks can render `sourceFile` / `sourceLine`, but the + compiler does not yet synthesize those arguments for ordinary `logger.info(...)` + calls. + +## What isn't here yet + +- **Compiler-side exact default names** — the interpreter runtime now falls back from + the field name `"logger"` to the caller namespace for canonical `logging.Logger` + injections. A compiler pass can still improve this by emitting the exact + module/class logger name as the resource name. +- **External Logback-style configuration loader** — the base backend + primitives (`AsyncLogSink`, `CompositeLogSink`, `HierarchicalLogSink`, `JsonLogSink`) + are in this module. XML/JSON config loading, rolling files, network destinations, and + filters remain explicit follow-up modules; see `../future/logback-integration.md`. +- **Per-container override convenience** — open question 8. + +The temporary interpreter-side injection wiring lives in +`javatools/src/main/java/org/xvm/runtime/NativeContainer.java` (`ensureLogger`, +`ensureConst`); `BasicLogger` is the `const` returned for `@Inject Logger` so MDC +fiber-locals survive injection. The real `MessageFormatter` is implemented (12 tests in +`MessageFormatterTest`). Tests live in `lib_logging/src/test/x/LoggingTest/` (66 +passing as of this commit). + +--- + +Previous: [`../cloud-integration.md`](../cloud-integration.md) | Next: [`why-slf4j-and-injection.md`](why-slf4j-and-injection.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/design/why-slf4j-and-injection.md b/doc/logging/design/why-slf4j-and-injection.md new file mode 100644 index 0000000000..2f4dc902c5 --- /dev/null +++ b/doc/logging/design/why-slf4j-and-injection.md @@ -0,0 +1,333 @@ +# Why model `lib_logging` on SLF4J and on Ecstasy injection + +> **Audience:** reviewers asking "why SLF4J? why injection?" — the rationale doc. + +This is the rationale doc. It exists to make the case that the two big shape-defining +choices in `lib_logging` — _model the API on SLF4J_, _acquire loggers via `@Inject`_ — +are not arbitrary preferences but the only choices that combine into a logging library +worth shipping. Everything else has been tried, in many languages, and falls short in +specific, predictable ways. + +## TL;DR + +- **Modeling on SLF4J means we inherit thirty years of hard-won design.** SLF4J is the + one logging API in the Java ecosystem that survived the wars (log4j 1.x → log4j 2 → + Logback → JUL → JBoss Logging → ...). It survived because it got the API↔impl boundary + right. Copying that boundary copies the wins. +- **Modeling injection on `@Inject` means user code never depends on a backend.** This is + the thing static `LoggerFactory.getLogger` *almost* gives you in Java but never quite + does — there is always a static binding choice somewhere on the classpath. In Ecstasy + the runtime container provides the binding. There is no classpath race. +- **The combination is greater than the sum of the parts.** Ecstasy gives us + per-container injection; SLF4J gives us a stable user-facing API. Together: a logging + library where the host application controls every embedded module's logging without + any embedded module having to know. + +The rest of this document expands each of those. + +--- + +## Part 1 — Why SLF4J is the right model + +### 1.1 The API/impl boundary is the only thing that has ever held up + +Logging in Java has churned through at least seven serious facades and frameworks: +`System.err.println`, log4j 1.x, JUL (`java.util.logging`), Apache Commons Logging, +JBoss Logging, log4j 2, SLF4J + Logback. Each replacement had a moment of "this is the +new way". Most of them are now legacy. + +The thing that has stuck is the **SLF4J API**. Logback replaced log4j as the canonical +backend. log4j 2 came along. Then `slf4j-jul-impl`, `log4j-slf4j-impl`, etc. — bridges +in every direction. None of this churn touched user code. The reason is that SLF4J got +exactly one thing right that Commons Logging got wrong and JUL got wrong: **the +caller-facing API is one stable shape, and the binding is a separate jar selected at +deploy time.** Caller code does not import the binding; it cannot, by construction. + +This is the decision we must inherit, because the decision pays for itself for decades. +A new language that ships a logging library that mixes API and binding (the way JUL +does, the way Python's stdlib `logging` does) has just signed up for the same churn. + +### 1.2 SLF4J's specific design choices, why they're each load-bearing + +The signature shapes look small but they encode hard-won lessons. Walking through them: + +- **`{}` placeholder formatting**, *not* `String.format`. `String.format` is expensive + even when the level is disabled. The whole reason SLF4J exists is that log4j 1.x users + paid the cost of `"some msg " + obj.toString()` even when DEBUG was off. Deferred + formatting via `{}` placeholders + arrays of arguments lets the implementation skip + the work entirely when `isDebugEnabled()` is false. This is the original SLF4J + motivation; it is still load-bearing. + +- **Throwable promotion**. The trailing `Throwable` is *not* a substitution argument; it + is the cause. This rule looks small but it eliminates an entire class of footgun + ("why is my stack trace just `java.io.IOException@1c2d4f` in the message?"). SLF4J + users learn this rule once. + +- **Level checks as first-class methods.** `if (log.isInfoEnabled())` is the standard + Java idiom for "this log call has an expensive argument I want to elide". Frameworks + that lack this method force callers to either pay the formatting cost or write + unreadable wrapper code. + +- **Markers as a structured tag.** Markers are how you say "this event belongs to the + AUDIT category" without resorting to either a separate logger name or substring + matching on messages. Backend filters can route on markers without grep. + +- **MDC for context propagation.** MDC is _the_ idiom for request-scoped context. + Every observable Java service stack uses it. If you want a log line tagged with + `requestId` without your business code having to thread it through, you set it once + in the request handler and the encoder picks it up. + +- **The fluent event builder (SLF4J 2.x).** Eight years after SLF4J 1.x shipped, the + community decided the explosion of overload signatures (`info(String)`, `info(String, + Object)`, `info(String, Object, Object)`, `info(Marker, ...)`, ...) was a mistake. The + fluent builder is the answer: one builder type with `addArgument`, `addMarker`, + `setCause`, `addKeyValue`, then `log()`. SLF4J kept the legacy overloads for + backwards compatibility, but new code is moving to the builder. We get to skip the + legacy and ship the builder as a first-class option from day one. + +Every one of these is in `lib_logging`. None of them is novel; all of them are +load-bearing. Throw any one out and we have a worse logging library. + +### 1.3 The alternatives, why none of them is good enough + +| Model | Why we don't copy it | +|---|---| +| **`java.util.logging` (JUL).** The "ship something with the language" approach. | Levels are weird (`FINE`/`FINER`/`FINEST` instead of `DEBUG`/`TRACE`); MDC support was added late and is awkward; configuration is a flat properties file with no programmatic equivalent; performance is poor on hot paths. JUL is the cautionary tale — it's what you get when "we'll just put a logger in the language" goes uncritically. | +| **log4j 1.x.** Once the dominant model. | Eager string concat in caller code (no `{}` deferred formatting), `Logger.getLogger` static factory races, no MDC at the API level. SLF4J was created specifically because of these. | +| **log4j 2.** Strong API, strong impl, but bundled. | The API and the impl are tightly coupled. You cannot replace the impl without giving up the API. We need the impl-as-plugin property; log4j 2 doesn't preserve it cleanly. | +| **Apache Commons Logging.** Tried to be a facade. | Famously broken classloader behaviour ("the JCL nightmare"). The dynamic discovery model didn't survive the move to OSGi/modular classloaders. SLF4J's static binding fixed exactly this. | +| **Python `logging`.** Stdlib. | Hierarchical logger configuration is good; the API is verbose; no fluent builder; no markers; bound to a global root by default. Decent for Python culture; doesn't fit Ecstasy's injection-first culture. | +| **Go `log/slog` (1.21+).** Modern. | Reasonable shape. `slog` is _newer_ than SLF4J 2.x and arrived at a similar place: structured KV, levels, handlers. We could equally well copy `slog`. We don't, because (a) far fewer engineers know `slog` than know SLF4J, and (b) `slog`'s ergonomics around groups and context don't add anything an MDC + KV combo doesn't already give us. | +| **Roll our own.** Greenfield. | The space is well-trodden. Every novel decision is a decision we'd have to defend forever. The only good reason to depart from SLF4J is to fit Ecstasy's injection model — which is what we're doing for acquisition, but not for the API surface itself. | + +The strongest competing argument is "Go `slog` is a cleaner design and Ecstasy is a new +language; why anchor to a Java idiom?" The answer: developer onboarding. We will pay the +cost of every developer learning a new logging API forever. We don't need to. SLF4J's +shape is in the muscle memory of probably the largest population of working backend +engineers in the world. Free familiarity is not nothing. + +### 1.4 What the boundary buys us _specifically_ + +Concretely, here's what splitting `Logger` from `LogSink` and copying the SLF4J API +shape gets us: + +- **A library author writes `@Inject Logger logger; logger.info(...)` and is done.** + They do not pick a backend, they do not have a config file, they cannot break + downstream consumers by their choice. A library that does logging is a library that + does logging *politely*. +- **An application author can swap `ConsoleLogSink` for `JsonLogSink`, + `CompositeLogSink`, `HierarchicalLogSink`, or a future configured backend without + recompiling any library.** Behind a stable injection point. +- **Test authors get `MemoryLogSink` for free.** They wire it up once and assert on + `events`. No global state, no `LogManager.getLogManager().reset()`, no flaky tests. +- **Future structured-logging consumers (log aggregators, dashboards) get a stable event + shape.** We add a `keyValues` map to `LogEvent` once; every sink that wants to render + it does, every sink that doesn't ignores it. No format wars. + +These are not theoretical. Every Java shop that runs Logback + LogstashEncoder for +production and SimpleLogger for tests is benefiting from exactly this layering today. +We get the same property out of the box. + +--- + +## Part 2 — Why injection (`@Inject Logger`) is the right acquisition mechanism + +### 2.1 What `LoggerFactory.getLogger` actually is in Java + +The Java idiom is: + +```java +private static final Logger log = LoggerFactory.getLogger(MyClass.class); +``` + +This is a static factory call resolved against whatever SLF4J binding is on the +classpath at startup. In a single-application JVM with one classpath, it works. In +anything more complex it gets ugly: + +- **Two bindings on the classpath.** Random one wins. SLF4J prints a warning at + startup, but nothing else. +- **Library shading.** Two libraries each shade their own SLF4J binding into their jars. + Classloader lottery decides which wins. +- **Multi-tenant containers.** Different tenants want different log routing. Java has no + natural way to express this; you end up plumbing tenant-scoped loggers manually or + using `LogContext`-style hacks. +- **Tests that mutate global state.** Setting up `ListAppender` in a test means + registering it on a logger that is global to the JVM, with cleanup discipline. + +These are paper cuts in Java. They are accepted because the language has no better +option. **Ecstasy does have a better option.** The injection model is the better option. + +### 2.2 What `@Inject Logger` gives us that static factories cannot + +Ecstasy's `@Inject` is resolved per-container by the runtime, with a `(Type, name)` +key. Concretely, this means: + +- **The host application's container chooses the sink.** A test container injects + `MemoryLogSink`. A production container injects a configuration-driven sink. The + embedded library has no opinion. Static `LoggerFactory.getLogger` cannot do this + because the choice is made at JVM startup by classpath discovery. + +- **Per-container override is a one-line change.** Want module X to log to a different + destination than module Y? Provide a different Injector for module X's container. + No bridges, no `MarkerFilter` hacks, no per-tenant `LogContext`. + +- **No classpath surprises.** There is no "two bindings, who wins?" question because + there is no classpath-level binding. The binding is a function in the suppliers map. + +- **Testing is trivial.** A test sets up a child container with a `MemoryLogSink`, + runs the system under test, asserts on captured events. No global mutation, no + per-test `BeforeEach` cleanup of root logger appenders, no race when tests run in + parallel. + +- **Library authors don't depend on a configuration model.** A Java library that wants + to log _to a specific destination_ in some scenarios usually adds a `setLogger(Logger)` + method. With injection that's no longer needed; the *consumer* of the library is the + one who decides what the library's `Logger` resolves to. + +### 2.3 The `Console` precedent + +Ecstasy already does this for `Console`. `@Inject Console console` is a stable contract; +the runtime swaps `TerminalConsole`, headless test consoles, captured-output consoles, +embedded-host-controlled consoles, all transparently. Logging is exactly the same +problem with the same shape: it's an output channel the platform should control on the +caller's behalf. Treating loggers like consoles is *the obvious choice once you see it*. + +The alternative — "loggers are special, they need a static factory because that's what +SLF4J does" — gives up the property that motivated `Console` injection in the first +place. We would not accept that for `Console`; we should not accept it for `Logger`. + +### 2.4 What about the `LoggerFactory` static escape hatch? + +`lib_logging` ships `LoggerFactory` for code that genuinely cannot use injection (e.g., +static initializers, anonymous closures with no injection scope). The escape hatch is +itself a service that consults an injected default sink — the same plumbing as +`@Inject Logger`, just exposed via a classmethod-like accessor. + +This means even the escape hatch respects per-container overrides. Static is not the +same as global. + +--- + +## Part 3 — Why the combination is greater than the sum of the parts + +Now put the two together. + +### 3.1 The library author's experience + +Writes: +```ecstasy +@Inject Logger logger; +logger.info("processed {} records", count); +``` + +Reads like SLF4J. Costs nothing to learn for an SLF4J veteran. Imports nothing +backend-specific. Cannot accidentally pin a binding. Cannot leak a binding choice into +downstream consumers. Cannot affect behaviour in unrelated modules. Will keep working if +the runtime swaps the sink, the binding, the config file, the encoder, anything. + +This is what every facade has aspired to, and almost-but-not-quite achieved. + +### 3.2 The host application's experience + +Wants production-grade logging? Provide a composed `LogSink` such as +`new AsyncLogSink(new CompositeLogSink([...]))`, with `JsonLogSink` or a future +destination-specific backend inside it. Provide it *to the container* and every +library inside the container picks it up. No "add slf4j-logback to the classpath" +ceremony, no version-skew warnings, no shading games. The host owns the injector; the +host owns the routing. + +Wants logs disabled for some embedded module? Override the injection for that module's +sub-container. There is no Java equivalent that doesn't involve `MarkerFilter` hacks. + +Wants a custom JSON wire format? Configure `JsonLogSink` or write another `LogSink`, +register it. Forty-plus embedded libraries in the application start emitting JSON +without one of them having done anything. + +### 3.3 The ops engineer's experience + +Same story as the host: the routing decision is exactly one place — the resource +provider in the injector. There is no second place. There is no `logback.xml` somewhere +in a transitive dependency overriding the choice. + +### 3.4 The test author's experience + +Wires `MemoryLogSink` into the test container, runs the system under test, asserts on +`sink.events`. Test isolation is automatic because each test container has its own +suppliers map. No global root logger to reset. + +This last property is, frankly, hard to overstate. Anyone who has chased a flaky CI test +caused by a `ListAppender` not being detached at the end of a previous test knows what +the alternative looks like. + +--- + +## Part 4 — Anticipating objections + +**"SLF4J is showing its age. Should we copy something newer?"** + +The shape of the API is exactly what is _not_ aging. The features added in SLF4J 2.x +(KV pairs, fluent builder) are themselves modern. There is no working logging library in +mainstream use whose user-facing surface is meaningfully different *and* better. `slog` +in Go is closest, and the user-facing differences (groups, attribute encoding) are not +improvements so much as different trade-offs. None of them justify giving up SLF4J's +familiarity. + +**"Why not let users pick their style — slog-like *or* SLF4J-like?"** + +Two APIs is worse than one. Ecstasy is a new language; the cost of forcing one idiom is +small, and the cost of supporting two forever is large. + +**"Why not lean on `@Inject Console console` and skip the whole thing?"** + +`println` doesn't have levels. It doesn't have markers. It doesn't have MDC. It doesn't +have parameterized messages with deferred formatting. It doesn't have the API/impl +boundary. Building any of those on top of `Console` ad hoc, in every library that needs +them, is exactly the world before SLF4J — and we know how that turns out. + +**"What about the runtime cost of the indirection?"** + +The level check (`sink.isEnabled(name, level, marker)`) is the fast path. When the +level is disabled, no formatting work happens, no MDC snapshot is taken, no event is +constructed. This is the same fast-path SLF4J has and it is well-trodden. + +**"Could we change our minds later?"** + +Most of the API surface is in the `Logger` interface and a few small types +(`Level`, `Marker`, `MDC`, `LogEvent`, `LoggingEventBuilder`). Anything below `LogSink` +is replaceable today; anything above is replaceable only by breaking caller code. +Choosing the SLF4J shape now is the choice we want locked in. It's the choice with the +shortest list of failure modes. + +--- + +## Summary + +We're not picking SLF4J because we like Java. We're picking SLF4J because: + +1. The API/impl split it pioneered is the only logging architecture that has survived + thirty years of churn intact. +2. The specific signatures it standardized (`{}` deferred formatting, throwable + promotion, level checks, markers, MDC, fluent KV) each solve a real problem nobody + wants to re-solve. +3. The audience we are onboarding to Ecstasy already speaks this dialect. + +We're not picking `@Inject Logger` because we like dependency injection. We're picking +it because: + +1. Static `LoggerFactory.getLogger` has known failure modes (classpath races, shading + conflicts, global state in tests) that Ecstasy's container model just doesn't + exhibit. +2. The same property already pays for itself with `@Inject Console`. Logging is + strictly the same shape as console output and benefits from exactly the same + mechanism. +3. The combination — SLF4J's API surface plus per-container injection — is a logging + library that no Java framework can quite ship, because Java doesn't have + per-container injection at the language level. Ecstasy does. We should use it. + +The result is still recognizable to SLF4J users, but acquisition and backend +selection follow Ecstasy's container model instead of Java's process-global binding +model. + +--- + +Previous: [`design.md`](design.md) | Next: [`xdk-alignment.md`](xdk-alignment.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/design/xdk-alignment.md b/doc/logging/design/xdk-alignment.md new file mode 100644 index 0000000000..50f298e59d --- /dev/null +++ b/doc/logging/design/xdk-alignment.md @@ -0,0 +1,217 @@ +# XDK lib alignment — making `lib_logging` a first-class citizen + +> **Audience:** XDK maintainers checking `lib_logging` against the conventions of other shipped libs. + +This document audits `lib_logging` against the conventions used by the other shipped XDK +libs (`lib_ecstasy`, `lib_cli`, `lib_crypto`, `lib_json`, `lib_jsondb`, `lib_net`, +`lib_oodb`, `lib_sec`, `lib_web`, `lib_webauth`, `lib_xenia`, `lib_xunit`) and against +how they are actually used in `~/src/platform` and `~/src/examples`. Every deviation has +to be a deliberate choice. + +## The XDK conventions, observed + +### 1. Injectables are interface types acquired by `@Inject Type ref;` + +Across the XDK and downstream code, every injected resource is an interface that the +runtime resolves to an implementation: + +| Resource type | Where defined | Used like | +|---|---|---| +| `Console` | `lib_ecstasy/src/main/x/ecstasy/io/Console.x` | `@Inject Console console;` | +| `Clock` | `lib_ecstasy/src/main/x/ecstasy/temporal/Clock.x` | `@Inject Clock clock;` | +| `Random` | `lib_ecstasy/src/main/x/ecstasy/numbers/Random.x` | `@Inject Random random;` | +| `Directory` | `lib_ecstasy/src/main/x/ecstasy/fs/Directory.x` | `@Inject Directory curDir;` | +| `FileSystem` | `lib_ecstasy/src/main/x/ecstasy/fs/FileSystem.x` | `@Inject FileSystem fs;` | +| `ModuleRepository`| `lib_ecstasy/.../mgmt/ModuleRepository.x` | `@Inject ModuleRepository repository;` | +| `Compiler` | `lib_ecstasy/.../mgmt/Compiler.x` | `@Inject Compiler compiler;` | +| `Algorithms` | `lib_crypto/.../crypto/Algorithms.x` | `@Inject Algorithms algorithms;` | +| `CertificateManager` | `lib_crypto/.../crypto/CertificateManager.x` | `@Inject CertificateManager manager;` | +| `Authenticator?` | `lib_webauth/...` | `@Inject Authenticator? authenticator;` | + +The pattern is so consistent it is the language's idiom. **`Logger` slots into exactly +this list.** No code in the XDK uses a static factory to acquire a runtime resource; +that's the SLF4J/Java pattern that `lib_logging` deliberately turns into injection. + +### 2. Named injection where multiple instances are reasonable + +Where there is more than one of something, the resource name disambiguates: + +```ecstasy +@Inject Directory curDir; +@Inject Directory testOutput; +@Inject Directory testOutputRoot; +@Inject Directory tmpDir; +@Inject("homeDir") Directory home; // from BasicResourceProvider +``` + +`lib_logging` deliberately does **not** use named injection for logger names. It uses +`@Inject Logger logger;` for the root logger, then derives full-name loggers by API: +`Logger payments = logger.named("com.example.payments");`. This keeps the injector's +resource table exact-match and avoids a one-off wildcard injection rule for logging. + +### 3. Defaults via the `Inject` annotation's `resourceName` + +`lib_ecstasy/src/main/x/ecstasy/annotations/Inject.x` defines the annotation as +`Inject(String? resourceName = Null, Options opts = Null)`. The `resourceName` +is the dispatch key; null means "default for this type". Every XDK lib follows this. + +`lib_logging` follows the default-resource part (`@Inject Logger logger;`) but not +the named-resource part for logger names. Directory names identify distinct host +resources; logger names are application categories. Those categories belong in the +logging API (`logger.named(...)`), not in the runtime injector. + +### 4. The four canonical interfaces for shareable values + +Other XDK libs that ship value types reach for one or more of these mixins consistently: + +- **`Freezable`** (`lib_ecstasy/src/main/x/ecstasy/Freezable.x`) — for anything that may + cross a service boundary. Sinks are typically services; anything they receive must be + freezable. Examples: `Path`, `URI`, `Tuple`, every `const`. +- **`Stringable`** (`lib_ecstasy/src/main/x/ecstasy/text/Stringable.x`) — for anything + routinely formatted into log lines, JSON, or CLI output. Lets the renderer pre-size + buffers via `estimateStringLength()`. +- **`Hashable`** (`lib_ecstasy/src/main/x/ecstasy/collections/Hashable.x`) — for anything + that may end up as a `Map` or `Set` key. +- **`Orderable`** (`lib_ecstasy/src/main/x/ecstasy/Orderable.x`) — for anything that has + a natural ordering. + +`lib_ecstasy/.../Const.x:149-152` defines `Const` as `extends Hashable, Orderable, +Freezable, Stringable` — all four. So every `const` gets them automatically. + +### 5. Const for value types, service for stateful actors, class otherwise + +- **`const`** for immutable value records: `Path`, `URI`, `Time`, `Duration`, `Range`. +- **`service`** for stateful, thread-safe shared resources: `TerminalConsole`, + `Clock`, `Random`, sinks that hold writers / queues / counters. +- **`class`** for everything else: small mutable helpers, builders. + +### 6. Module declaration shape + +Every lib_* declares one module file at `src/main/x/.x` with a brief module +doc-comment, naming the package as `.xtclang.org`. Subordinate types live under +`src/main/x//.x`. Tests live under `src/test/x/.x` plus +`src/test/x//.x`. + +### 7. Build script shape + +Every lib_* has a tiny `build.gradle.kts`: + +```kotlin +plugins { + alias(libs.plugins.xtc) +} + +dependencies { + xdkJavaTools(libs.javatools) + xtcModule(libs.xdk.ecstasy) + // optional test deps: + xtcModuleTest(libs.javatools.bridge) + xtcModuleTest(libs.xdk.xunit.engine) +} +``` + +`lib_logging`'s build script is identical. + +## How `~/src/platform` and `~/src/examples` use these conventions + +### `~/src/platform` patterns + +- `kernel.x` injects `Console`, `Clock`, `ModuleRepository`, `CertificateManager`, + `Configuration` — all type-keyed, no resource names except where explicitly needed. +- `host/HostManager.x` injects `Directory testOutput;` — a named directory. +- `auth/OAuthEndpoint.x` and `auth/OAuthProvider.x` use `@Inject Console console;` + followed by ad-hoc `console.print($"Info : ...")` / `console.print($"Error: ...")` + patterns — exactly the surface `lib_logging` is meant to absorb. + +### `~/src/examples` patterns + +- `welcome.x` uses `@Inject Console console;` for genuine user-facing output. +- `CardGame/cardGame.x` and friends inject `Console` for game state messages — these + remain `Console` after lib_logging adoption (they are user output, not log events). +- `chess-game/...` similarly mixes user output and would benefit from a clean split. + +The clean split between "user output" (`Console`) and "diagnostic event" (`Logger`) +is exactly the property `lib_logging` introduces. Both repos already inject `Console` +the right way; they just lack a `Logger` to round out the injection surface. + +## Where `lib_logging` already follows the conventions + +| Convention | `lib_logging` | +|---|---| +| Injectable interface type | `Logger` and `MDC` are wired by the runtime in this branch. `LogSink`, `MarkerFactory`, and `LoggerFactory` are designed as injectable extension points but not yet registered by the interpreter's native container. | +| Named logger acquisition | `@Inject Logger logger; Logger named = logger.named("com.example");` | +| `Const` for immutable records | `LogEvent` | +| `const` for stateless/passable values | `BasicLogger`, `ConsoleLogSink`, `NoopLogSink`, `MDC`, `LogEvent` | +| `service` for stateful actors | `LoggerFactory`, `LoggerRegistry`, `MemoryLogSink` | +| `class` for mutable helpers/builders | `BasicEventBuilder`, `BasicMarker`, `MarkerFactory` | + +The slog-shaped sibling follows the same injection convention: `slogging.Logger` is +also registered under the resource name `logger`, and the native injector disambiguates +by requested type. +| Module file layout | `lib_logging/src/main/x/logging.x` declaring `module logging.xtclang.org` | +| Subordinate types under same dir | `lib_logging/src/main/x/logging/.x` | +| Build script shape | matches `lib_cli` / `lib_json` exactly | +| Test layout | `lib_logging/src/test/x/LoggingTest.x` + `LoggingTest/.x` | +| `Stringable` / `Hashable` / `Freezable` on shareable types | `Marker` extends all three | + +## Where `lib_logging` deliberately *does* deviate, and why + +- **`Marker` is an interface, not a const.** SLF4J's contract requires markers to be + mutable (`add(child)` / `remove(child)`). A `const` cannot be mutable. We keep + `Marker` as an interface, implement `Freezable` for boundary crossing, and let + `BasicMarker` enforce the SLF4J contract on the unfrozen path. + +- **`LogSink` does not extend `Service`.** Most `LogSink` impls *are* services, but the + `LogSink` interface itself doesn't require it. Reason: trivial sinks (`NoopLogSink`) + don't need to be services and shouldn't pay for the per-fiber isolation. This is + consistent with `Stringable` and `Hashable` — XDK interfaces leave the implementation + policy to the implementer rather than forcing one shape. + +- **`Logger` does not extend `Stringable` or `Hashable`.** Loggers are not values; they + are facades. You wouldn't put a `Logger` in a `Map` or render it into a log line. + Compare `Console` — also a façade interface, also not Stringable/Hashable. + +- **`Level` is an enum, not a const.** Enums in Ecstasy already get `Hashable + + Orderable + Freezable + Stringable + Comparable` automatically (via the same `Const` + base they inherit), so there is no extra work to do — but the design choice is + noted because it's the same shape as `lib_ecstasy/.../io/SeekDirection.x` and other + XDK enums. + +## Concrete improvements made to `lib_logging` for XDK alignment + +Pre-alignment, `Marker` was a plain interface and `LogEvent` carried a `String? +markerName` because the unfreezable Marker couldn't ride on a `const`. Post-alignment: + +- `Marker extends Freezable, Stringable, Hashable`. +- `BasicMarker` implements `freeze` (deep-freezes children, makes self immutable) and + acquires `Stringable.appendTo` / `estimateStringLength` for free from `Marker`'s + default methods. +- `LogEvent.marker` carries the full `Marker?` reference again. Sinks doing + `containsName` filtering or `marker.references` traversal work end-to-end without + giving up the const safety. +- The mutable `Object[] arguments` from the fluent builder is converted to + `Constant`-mode (frozen) before being handed to the sink — same boundary-crossing + story Ecstasy applies to every other shareable value type. + +Together: every value that may end up on a `LogEvent` honours the XDK's `Freezable` +contract, every type that wants to be a map key honours `Hashable`, every type that +wants to be efficiently rendered honours `Stringable`. There is no part of +`lib_logging` that an experienced XDK engineer would point at and say "that's not how +we do things." + +## Summary + +`lib_logging` is now structurally indistinguishable from the other shipped XDK libs: + +- Same module / file / build conventions. +- Same injection idiom. +- Same value-type interfaces (`Freezable`, `Stringable`, `Hashable`) where appropriate. +- Same const / service / class split. + +The only thing missing for full first-class status is the runtime-side resource +registration (`@Inject Logger` actually resolving), which is the next chunk of work +described in `../open-questions.md` items 1, 2, and 9. + +--- + +Previous: [`why-slf4j-and-injection.md`](why-slf4j-and-injection.md) | Next: [`../roadmap.md`](../roadmap.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/future/logback-integration.md b/doc/logging/future/logback-integration.md new file mode 100644 index 0000000000..866f71b521 --- /dev/null +++ b/doc/logging/future/logback-integration.md @@ -0,0 +1,300 @@ +# Logback-style integration + +> **Audience:** future-work readers — what a configuration-driven backend module would look like. + +`lib_logging` now ships the base Logback-style primitives directly: +`CompositeLogSink`, `HierarchicalLogSink`, `AsyncLogSink`, `JsonLogSink`, and +`JsonLogSinkOptions`. + +This document is still forward-looking, but its scope is narrower now: it sketches the +optional configuration-driven backend layer that could sit on top of those primitives +and add config files, richer appenders, filters, hot reload, rolling files, and +destination-specific policies. + +In other words: the branch already has programmatic Logback-style building blocks in +XTC. This document is about the missing production configuration layer around them, +not about whether Ecstasy can express Logback-like backend behavior at all. + +## What "Logback-style" means + +For SLF4J users, "Logback" means a specific bundle of features beyond the SLF4J facade: + +| Feature | Description | +|---|---| +| **Per-logger thresholds** | `com.example.foo` at `DEBUG`, root at `INFO`. The threshold lookup walks the logger-name hierarchy. | +| **Multi-appender** | The same event can fan out to console, file, network, syslog, etc. | +| **Layouts** | Pluggable output formatters: `PatternLayout`, `JSONLayout`, `LogstashEncoder`, etc. | +| **Filters** | Per-appender pre-emit decisions based on level, marker, MDC, regex, etc. | +| **Configuration files** | `logback.xml` (or programmatic). Reload on change. | +| **Async appenders** | A queue + worker thread so producer threads aren't blocked by slow I/O. | +| **Context-aware MDC rendering** | MDC values appear in formatted output without the message having to mention them. | +| **Rolling file appenders** | Time-based, size-based, or composite rotation. | + +The base library covers the first three in programmatic form and includes async/JSON +building blocks. The goal of a future `lib_logging_logback` module is to provide the +configuration and destination ecosystem around them. + +## The architectural fit + +`lib_logging` already separates: + +``` +Logger ──→ LogSink ──→ (whatever) +``` + +A logback-style backend slots into the `LogSink` slot. From the caller's perspective +nothing changes — `@Inject Logger logger;` still produces a `Logger`, the methods are +the same, the events are the same. The backend just happens to be much smarter about +what it does with them. + +Conceptually the backend itself decomposes into a smaller hierarchy: + +``` +ConfiguredLogSink (the LogSink the runtime injects) + │ + ├── LoggerContext (the per-logger threshold tree, "INFO at root, DEBUG at com.example.foo") + │ + └── Appender[] (each appender = its own "where does the event go" target) + │ + ├── Filter[] (pre-emit predicates: level, marker, MDC, regex) + │ + ├── Layout (LogEvent → bytes, e.g. PatternLayout, JSONLayout) + │ + └── Output (Console, File, RollingFile, Network, ...) +``` + +This isn't novel; it's Logback's mental model expressed as XTC code. The advantage of +writing it as an Ecstasy module is that it uses Ecstasy's own primitives: services for +thread-safety, fibers for async work, and the file abstraction from `lib_ecstasy`. + +## Sketch — the public API of `lib_logging_logback` + +```ecstasy +module logback.xtclang.org { + package log import logging.xtclang.org; + + /** + * The configurable sink that user code wires up via `@Inject log.LogSink`. + */ + service ConfiguredLogSink + implements log.LogSink { + // populated by Configurator + } + + /** + * Programmatic configuration entry point. Equivalent to Logback's JoranConfigurator. + */ + service Configurator { + Configurator setRootLevel(log.Level level); + Configurator setLevel(String loggerPrefix, log.Level level); + Configurator addAppender(Appender appender); + Configurator clear(); + } + + interface Appender { + @RO String name; + Boolean isEnabled(log.LogEvent event); + void append(log.LogEvent event); + } + + interface Layout { + String render(log.LogEvent event); + } + + interface Filter { + Decision decide(log.LogEvent event); + enum Decision { Accept, Neutral, Deny } + } + + // shipped appenders + service ConsoleAppender (String name, Layout layout, Filter[] filters = []) implements Appender; + service FileAppender (String name, Path file, Layout layout, Filter[] filters = []) implements Appender; + service RollingFileAppender(String name, Path file, RollingPolicy policy, Layout layout, Filter[] filters = []) implements Appender; + service AsyncAppender (String name, Appender delegate, Int queueSize = 1024) implements Appender; + + // shipped layouts + service PatternLayout(String pattern) implements Layout; + service JsonLayout implements Layout; + + // shipped filters + service LevelFilter (log.Level threshold) implements Filter; + service MarkerFilter(log.Marker required) implements Filter; + service MDCFilter (String key, String valuePrefix) implements Filter; +} +``` + +## Sketch — programmatic configuration + +```ecstasy +module MyApp { + package log import logging.xtclang.org; + package logback import logback.xtclang.org; + + void run() { + @Inject logback.Configurator config; + + logback.PatternLayout text = new logback.PatternLayout( + "%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"); + + logback.JsonLayout json = new logback.JsonLayout(); + + config + .setRootLevel(log.Info) + .setLevel("com.example.payments", log.Debug) + .addAppender(new logback.ConsoleAppender("STDOUT", text)) + .addAppender(new logback.RollingFileAppender( + name = "FILE", + file = Path:/var/log/app.log, + policy = new logback.SizeBasedRollingPolicy(maxSize = 100M, maxBackups = 10), + layout = json, + filters = [new logback.MarkerFilter(MarkerFactory.getMarker("AUDIT"))] + )); + } +} +``` + +## Sketch — file-based configuration + +For parity with Logback's `logback.xml`, the module could ship an XML or JSON loader. +A possible JSON shape (more idiomatic for Ecstasy than XML): + +```json +{ + "rootLevel": "INFO", + "loggers": { + "com.example.payments": "DEBUG", + "com.example.silly": "OFF" + }, + "appenders": [ + { + "name": "STDOUT", + "type": "ConsoleAppender", + "layout": { "type": "PatternLayout", "pattern": "%d [%thread] %-5level %logger - %msg%n" } + }, + { + "name": "FILE", + "type": "RollingFileAppender", + "file": "/var/log/app.log", + "policy": { "type": "SizeBased", "maxSize": "100M", "maxBackups": 10 }, + "layout": { "type": "JsonLayout" }, + "filters": [ + { "type": "MarkerFilter", "marker": "AUDIT" } + ] + } + ] +} +``` + +Loading this via `lib_json`: + +```ecstasy +@Inject FileSystem fs; +@Inject json.Schema schema; // (sketch — actual API depends on lib_json) +@Inject logback.Configurator config; + +File configFile = fs.fileFor(Path:/etc/myapp/logging.json); +String text = configFile.contents.toString(); +LoggingConfig cfg = schema.deserialize(LoggingConfig, text); +config.applyConfig(cfg); +``` + +## Hot reload + +Logback's auto-reload watches the config file and rebuilds the logger context on change. +The Ecstasy equivalent would use the file-system change-notification API to do the same. +The atomic swap is on the `Configurator` side; once it commits a new config, every +existing `Logger` reference (since it goes through the injected sink) sees the new +behaviour on the next call. + +## Per-logger lookup + +The longest-prefix-match lookup illustrated in `../usage/custom-sinks.md` is the engine: + +```ecstasy +private Level effectiveLevel(String loggerName) { + String name = loggerName; + while (True) { + if (Level lvl := perLogger.get(name)) { + return lvl; + } + Int dot = name.lastIndexOf('.'); + if (dot <= 0) { + return rootLevel; + } + name = name[0 ..< dot]; + } +} +``` + +This is what `LoggerContext.getLogger(name)` does in Logback, just in fewer lines +because we don't have to maintain a tree object. + +## Async appenders + +```ecstasy +service AsyncAppender(String name, Appender delegate, Int queueSize) + implements Appender { + + private Queue queue = new BlockingQueue(queueSize); + + construct(String name, Appender delegate, Int queueSize) { + // start a worker fiber that drains the queue into the delegate + worker.runAsync(drainLoop); + } + + @Override + void append(log.LogEvent event) { + if (!queue.offer(event)) { + // queue full: drop or block — configurable via policy + } + } + + private void drainLoop() { + while (running) { + log.LogEvent event = queue.take(); + delegate.append(event); + } + } +} +``` + +This is the Ecstasy-native equivalent of Logback's `AsyncAppender`. The base library +already ships `AsyncLogSink`; a full backend can replace or compose it when it needs +drop/block policies, batching, or shutdown hooks. + +## Migration story for SLF4J + Logback users + +A team running an SLF4J + Logback service today and porting to Ecstasy + `lib_logging` + +`lib_logging_logback`: + +| In Java + Logback | In Ecstasy + lib_logging_logback | +|---|---| +| `org.slf4j.LoggerFactory.getLogger(MyClass.class)` | `@Inject Logger logger;` (or `LoggerFactory.getLogger(MyClass)`) | +| `logback.xml` in classpath root | `logging.json` (or programmatic) loaded by `Configurator` | +| `` | `config.setRootLevel(Info)` | +| `` | `config.setLevel("com.example.foo", Debug)` | +| `` | `new ConsoleAppender("STDOUT", layout)` | +| `` | `new PatternLayout(pattern)` | +| `` | `new LevelFilter(threshold)` | +| `MDC.put("requestId", id)` | `mdc.put("requestId", id)` | +| `org.slf4j.MarkerFactory.getMarker("AUDIT")` | `markers.getMarker("AUDIT")` | + +Caller code is unchanged. Configuration is structurally similar but expressed in +Ecstasy idiom. + +## Out of scope for `lib_logging_logback` v1 + +These are explicitly later: + +- SMTP appender (email on errors). +- DBAppender (write to RDBMS). +- Sift appender (separate appender per MDC value). +- TurboFilters (filters that run *before* the level check; meaningful only for very + high-volume systems). + +These exist in Java Logback. They are infrequently used and can be added incrementally +without API changes. + +--- + +Future-work sketch. Up: [`../roadmap.md`](../roadmap.md), [`../README.md`](../README.md) diff --git a/doc/logging/lib-logging-vs-lib-slogging.md b/doc/logging/lib-logging-vs-lib-slogging.md new file mode 100644 index 0000000000..13f7ae31eb --- /dev/null +++ b/doc/logging/lib-logging-vs-lib-slogging.md @@ -0,0 +1,644 @@ +# `lib_logging` vs `lib_slogging` — design comparison + +> **Audience:** reviewers picking the canonical XDK logging API shape. + +Two parallel XDK logging libraries model the same end-to-end capability but follow +different prior-art traditions. + +They coexist only so this branch can compare the designs. User code should choose one +shape, not mix both as permanent XDK APIs. After review, the chosen injectable logging +shape should graduate as `lib_logging`; `lib_slogging` is a POC name for the Go +`log/slog`-shaped candidate, not a proposal to keep a second default logging facade. + +The branch does not introduce a clean XDK build switch between "logging" and +"slogging". During the POC, the meaningful validation is to compile both libraries and +run the two manual injection tests: `TestLogging` for the SLF4J-shaped API and +`TestSLogging` for the slog-shaped API. Runners that only carry one POC module should +still start; the native injector should simply expose the resources for whichever +logging module is present. That tolerance is a temporary comparison aid, not a +statement that both APIs should remain in the XDK. + +| Library | Prior art | Core shape | +|---|---|---| +| `lib_logging` | SLF4J 2.x + Logback | Named loggers, `{}` message formatting, MDC, markers, fluent event builders, and a `LogSink` backend boundary. | +| `lib_slogging` | Go 1.21 `log/slog` | Attribute-first events, derived loggers via `with(...)`, open integer levels, grouped attrs, and a `Handler` backend boundary. | + +Both libraries target the same set of features end-to-end so a reviewer can pick +either, drop it into an Ecstasy program, and get the equivalent operational +capability. The point of carrying two implementations is not to ship both forever — +it's to let the XTC language designers and prospective users see the two designs +side-by-side, in real Ecstasy, and tell us which one fits the language better. + +The sections below show the same scenario in both APIs, compare the designs across +the axes that matter for XDK code, and end with the reviewer questions that should +decide which shape survives. Direct links from each Ecstasy type to its SLF4J or +Go `log/slog` counterpart are in [`api-cross-reference.md`](api-cross-reference.md). + +> Status note: both libraries are working comparison POCs. `lib_logging` is the +> recommended canonical facade and has the fuller SLF4J surface (70 focused XTC test +> methods plus the dedicated injected manual demo), including async/composite/hierarchical/JSON +> backend building blocks. `lib_slogging` has the more compact slog surface (41 focused +> XTC test methods plus the dedicated injected manual demo), with runtime injection, async +> handler support, `lib_json` JSON rendering, redaction options, source metadata, +> context binding, and handler derivation semantics implemented. + +--- + +## Fast conclusion + +Use this section for the first pass; the later sections are the depth-first rationale. + +Choose **`lib_logging`** as the canonical XDK facade unless review explicitly decides +Ecstasy should prefer the Go `slog` mental model. The deciding point is operational +familiarity: `lib_logging` gives users named loggers, `{}` formatting, markers, MDC, +fluent event builders, and a Logback-style `LogSink` boundary, while still supporting +modern structured JSON/cloud output through `LogEvent.keyValues`. + +`lib_slogging` proves the alternative is viable and clean. It is smaller because all +structured data is `Attr`, and backend extension is through `Handler`. The cost is that +hierarchical logger names, markers, and message templates are not first-class; they must +be modeled as attrs or helper adapters. + +The comparison can be read in two passes: + +- **Sections 1-3:** enough to understand the API choice and what exists or does not + exist in each model. +- **Sections 4-8:** deeper rationale, marker examples, Ecstasy `const`/`service` + implications, reviewer questions, and exact branch contents. + +--- + +## 1. Same scenario in both APIs + +A nontrivial example: an HTTP request handler logs request entry, attaches a +correlation id and user id to a context that propagates into a child fiber, and +records a structured event with an exception cause. + +### SLF4J-shaped (`lib_logging`) + +```ecstasy +import logging.Logger; +import logging.MDC; +import logging.MarkerFactory; + +@Inject Logger logger; +@Inject MDC mdc; + +void handle(Request req) { + mdc.put("requestId", req.id); + mdc.put("user", req.userId); + try { + logger.info("processing {} {}", [req.method, req.path]); + process(req); + } catch (Exception e) { + logger.atError() + .setMessage("request failed") + .addMarker(MarkerFactory.getMarker("AUDIT")) + .addKeyValue("status", 500) + .setCause(e) + .log(); + } finally { + mdc.clear(); + } +} +``` + +### slog-shaped (`lib_slogging`) + +```ecstasy +import slogging.Logger; +import slogging.Attr; + +@Inject Logger logger; + +void handle(Request req) { + Logger reqLog = logger.with([ + Attr.of("requestId", req.id), + Attr.of("user", req.userId), + ]); + try { + reqLog.info("processing", [ + Attr.of("method", req.method), + Attr.of("path", req.path), + ]); + process(req); + } catch (Exception e) { + reqLog.error("request failed", [ + Attr.of("audit", True), + Attr.of("status", 500), + Attr.of("err", e), + ]); + } +} +``` + +Both produce equivalent structured output. The shapes diverge in how +*correlation* (per-request id) and *categorisation* (audit channel) are expressed. + +--- + +## 2. Per-axis comparison + +| Axis | `lib_logging` (SLF4J 2.x) | `lib_slogging` (Go slog) | +|---|---|---| +| **Levels** | Five named: `Trace`, `Debug`, `Info`, `Warn`, `Error` (+ `Off`). Closed enum. | Integer `Level(severity)` with named constants `Debug=-4`, `Info=0`, `Warn=4`, `Error=8`. Open: callers can define `Level(2, "NOTICE")`. | +| **Context propagation** | Per-fiber `MDC` (`SharedContext`-backed map). Implicit — every emission snapshots it. | Explicit derived loggers via `Logger.with(attrs)` by default. Optional `LoggerContext` uses `SharedContext` when framework/request code wants implicit propagation. | +| **Structured data** | `Marker` for category, `addKeyValue("k", v)` for KV pairs, `arguments` for `{}` substitution slots. Three concepts. | One concept: `Attr(key, value)`. Attributes are the first-class carrier. | +| **Message** | Templated: `info("user {} did {}", [name, action])`. SLF4J-style `{}` placeholders. | Free-form string + attrs separately. No interpolation. | +| **Sinks / handlers** | `LogSink.isEnabled(name, level, marker)`, `LogSink.log(event)`. Two methods. | `Handler.enabled(level)`, `Handler.handle(record)`, `Handler.withAttrs(attrs)`, `Handler.withGroup(name)`. Four methods — handler can pre-resolve attrs at derivation time. | +| **Logger naming** | Hierarchical names: `logger.named("payments.stripe")`. The caller supplies the full category name, like `LoggerFactory.getLogger("...")`. | Loggers are uniform; categorisation lives in attrs (`"component" -> "payments.stripe"`). | +| **Source location** | `Logger.logAt(...)` explicitly populates `LogEvent.sourceFile` / `sourceLine`; automatic call-site capture is future runtime/compiler work. | `Logger.logAt(...)` explicitly populates `Record.sourceFile` / `sourceLine`; automatic call-site capture is future runtime/compiler work. | +| **Async / batching** | `AsyncLogSink` wraps any sink with a bounded queue. | `AsyncHandler` wraps any handler with the same bounded-queue shape. | +| **Familiarity bench** | Java/Kotlin/Scala. Anyone who has done structured logging on the JVM. | Go (1.21+). Increasingly the modern go-to in cloud-native code. | + +--- + +## 3. Exists / Missing Matrix + +This is the blunt version of the comparison. "Missing" does not mean impossible; it +means the concept is not first-class in that API and must be modeled with a more general +mechanism such as attributes or a custom backend. + +| Capability | SLF4J / Logback | Go `log/slog` | `lib_logging` | `lib_slogging` | Why it matters | +|---|---|---|---|---|---| +| Hierarchical logger names | **Exists.** `LoggerFactory.getLogger(PaymentService.class)`, Logback ``. | **No first-class logger name hierarchy.** `Logger.With` and `WithGroup` attach attrs / qualify attr keys; they do not define a logger category string with longest-prefix routing. | **Exists.** `logger.named("com.acme.payments")` plus `HierarchicalLogSink.setLevel(...)`. | **Modeled as attrs/groups.** Use `logger.with([Attr.of("component", "...")])` or `withGroup("payments")`; custom handlers can filter attrs, but there is no built-in prefix tree. | Per-package/per-module level control is an operational staple in Logback configs. | +| Markers | **Exists.** `Marker`, `MarkerFactory`, marker-aware level checks, Logback marker filters. | **No Marker type.** Use attrs such as `"audit"=true` or `"category"="audit"`. | **Exists.** `Marker`, marker DAG, multi-marker fluent events, marker-aware sinks. | **Modeled as attrs.** `Attr.of("audit", True)` is simpler but has no marker containment semantics. | Markers are useful for audit/security routing that should not depend on message text. | +| Message templates | **Exists.** `"processed {}"` with argument arrays and trailing throwable promotion. | **Not the model.** Message is a stable string; variable data is attrs. | **Exists.** `MessageFormatter` implements SLF4J-style `{}` formatting. | **Intentionally absent.** Use `logger.info("processed", [Attr.of("count", count)])`. | SLF4J favors migration familiarity; slog favors structured data over interpolation. | +| Mapped context | **Exists.** `MDC.put(...)`; Logback layouts can render it. | **Different shape.** Context-aware methods accept `context.Context`; persistent fields usually come from `Logger.With(...)`. | **Exists.** `@Inject MDC` backed by `SharedContext`. | **Different shape.** `Logger.with(...)` is explicit; `LoggerContext` is an optional `SharedContext` helper. | Request IDs and tenant/user IDs should not be threaded through every logging call by hand. | +| Open custom levels | **No.** SLF4J has the standard fixed ladder. | **Exists.** `slog.Level` is an integer; `Level(2)` is valid. | **No.** Closed enum plus `Off`. | **Exists.** `new Level(2, "NOTICE")`. | Useful for environments with `NOTICE`, `CRITICAL`, or domain-specific levels; less canonical for generic tooling. | +| Handler derivation hooks | **No direct facade equivalent.** Backends configure appenders/loggers; event builders carry per-event data. | **Exists.** `Handler.WithAttrs` and `WithGroup` let handlers pre-resolve context. | **No direct equivalent.** `LogSink` stays two-method and relies on logger names/MDC/builders. | **Exists.** `Handler.withAttrs` / `withGroup`, with `BoundHandler` default semantics. | Helps high-volume structured handlers avoid recomputing prefixes per record. | +| Source location | **Backend-dependent.** Logback can calculate caller data, but SLF4J does not make source a normal facade argument. | **Exists in the record model.** `Record.PC`, `Record.Source()`, `HandlerOptions.AddSource`. | **Explicit API.** `Logger.logAt(...)` populates `LogEvent.sourceFile` / `sourceLine`; compiler capture is future work. | **Explicit API.** `Logger.logAt(...)` populates `Record.sourceFile` / `sourceLine`; compiler capture is future work. | Useful for debugging, but automatic capture has runtime/compiler cost. | +| Attribute replacement / redaction | **Backend concern.** Logback filters/encoders decide. | **Exists in `HandlerOptions.ReplaceAttr`.** | **Exists in shipped JSON sink options.** `JsonLogSinkOptions.redactedKeys`. | **Exists in POC options.** `HandlerOptions.redactedKeys`. | Production JSON output needs redaction without caller code remembering every sink policy. | + +### Examples for the two contentious claims + +**Hierarchical category routing exists in SLF4J / Logback and `lib_logging`:** + +```java +// Java + SLF4J +private static final Logger LOG = + LoggerFactory.getLogger("com.acme.payments.stripe"); +LOG.debug("charged {}", paymentId); +``` + +```xml + + + +``` + +```ecstasy +// Ecstasy + lib_logging +HierarchicalLogSink sink = new HierarchicalLogSink(new ConsoleLogSink(), Info); +sink.setLevel("com.acme.payments", Debug); + +Logger log = new BasicLogger("root", sink) + .named("com.acme.payments.stripe"); +log.debug("charged {}", [paymentId]); +``` + +Go `slog` can carry a category, but it is data, not a built-in logger hierarchy: + +```go +// Go + slog +log := slog.New(handler).With("component", "com.acme.payments.stripe") +log.Debug("charged", "paymentId", paymentID) +``` + +```ecstasy +// Ecstasy + lib_slogging +SLogger log = logger.with([ + Attr.of("component", "com.acme.payments.stripe"), +]); +log.debug("charged", [Attr.of("paymentId", paymentId)]); +``` + +That is perfectly workable for structured output. What it does not give by itself is +Logback's standard longest-prefix rule: "turn on DEBUG for `com.acme.payments` and all +children." A slog handler can implement attr-based filtering, but it is not the same +standard category mechanism. + +**Markers exist in SLF4J / Logback and `lib_logging`; slog uses attrs instead:** + +```java +// Java + SLF4J +Marker audit = MarkerFactory.getMarker("AUDIT"); +LOG.info(audit, "user signed in"); +``` + +```xml + + + AUDIT + ACCEPT + DENY + +``` + +```ecstasy +// Ecstasy + lib_logging +Marker audit = MarkerFactory.getMarker("AUDIT"); +logger.info("user signed in", marker=audit); +``` + +```go +// Go + slog: model the same signal as an attribute +logger.Info("user signed in", "audit", true) +``` + +```ecstasy +// Ecstasy + lib_slogging +logger.info("user signed in", [Attr.of("audit", True)]); +``` + +The attr version is simpler and often enough. The marker version has marker identity, +marker containment (`SECURITY` can contain `AUDIT`), marker-aware enabled checks, and +direct compatibility with existing Logback marker filters. + +### Are Java APIs moving toward slog? + +They are moving toward the same structured/fluent direction, but they are not the same +API shape. + +| Java API / framework | Looks slog-like because | Still differs from Go slog | +|---|---|---| +| **SLF4J 2.x fluent API** | `logger.atInfo().addKeyValue("k", v).log("msg")` gives a message plus structured fields, close to slog's `Info("msg", "k", v)`. | Keeps SLF4J concepts: named loggers, `{}` message templates, markers, MDC, and backend binding through Logback/other providers. | +| **Log4j 2 API** | Has `LOGGER.atInfo().withMarker(...).withThrowable(...).withLocation().log(...)`, custom levels, Thread Context, JSON layouts, and rich message types. | It is a broader Java logging API, not an attr-only slog model. Markers, logger names, message factories, and config/plugins remain first-class. | +| **Google Flogger** | Uses a fluent API: `logger.atInfo().withCause(e).log("value: %s", value)` and supports lazy/rate-limited logging. | It is optimized for self-documenting Java call sites and performance, not Go's `Handler` / `Attr` / `Record` architecture. | +| **Logback + logstash-logback-encoder / structured arguments** | Many teams keep SLF4J and add structured fields at the call site plus a JSON encoder at the backend. | The facade remains SLF4J/Logback-shaped; structured data is added to that ecosystem rather than replacing it with slog semantics. | + +So the designs are not completely different things. They are two answers to the same +industry pressure: log events need structured fields. Go made attrs the central model. +Modern Java mostly retrofitted structured/fluent APIs onto the existing logger-name, +marker, MDC, appender, and configuration ecosystem. + +That is why `lib_logging` can be both SLF4J-shaped and modern: it keeps the Java +operational model but includes SLF4J 2-style `addKeyValue`, JSON sinks, redaction, and +async/composite/hierarchical backend primitives. + +--- + +## 4. Per-axis analysis + +### 4.1 Levels — closed enum vs open integer + +SLF4J's five levels are an industry consensus, easy to pattern-match on, and remove a +class of "what's the right level for this?" questions because the choice is small. +The cost: introducing `Notice`, `Critical`, or `Audit` requires either reinterpreting +an existing level or adding a new one to the enum (which is a breaking change). + +slog's integer levels accept extension without library changes. Custom levels +compose with comparison; tooling that filters on `>= Warn` keeps working when a +user adds `Notice = 2`. The cost: there is no canonical `Notice`; two libraries can +both define `Notice` at different integer values and silently disagree. + +**Ecstasy fit.** Both designs are expressible. The slog model is closer to how +Ecstasy already treats severity in many places (raw `Int`-shaped severity in +`Console`, `xunit_engine` levels), and integrates naturally with the +`Comparable`/`Orderable` machinery. The SLF4J model is a `const` enum with explicit +methods — also clean, but rigidly closed. + +### 4.2 Context propagation — `MDC` vs `With(attrs)` + +This is the single biggest design fork. + +**`MDC`.** Implicit. Every log call captures the current `MDC` snapshot regardless of +whether the calling code knows about it. This is enormously convenient for +cross-cutting concerns (request ids, tracing) — the framework injects ids once and +every downstream log line gets them for free. The cost is hidden state: code looks +pure but isn't, you can't tell from reading a log call whether it will pick up an MDC +key. + +In `lib_logging` the implementation is `SharedContext`-backed for per-fiber scope, so +unlike Java MDC it doesn't leak into sibling fibers. That removes the worst class of +real-world MDC bugs. But the *implicitness* remains. + +**`With(attrs)`.** Explicit. Code reads "derive a handler that always includes these +attributes." When a function wants to log with a request id, it normally accepts the +request-aware logger — either as a parameter or as a field set up at construction. +`LoggerContext` exists for framework/request code that needs implicit propagation. + +In Go, slog reaches for `context.Context` to pass loggers through the call graph +without adding parameters everywhere. The Ecstasy equivalent in this POC is +`LoggerContext`, a small `SharedContext` wrapper. + +**Ecstasy fit.** The MDC story is *already implemented and works*: `const MDC` over an +immutable-map `SharedContext` is small and per-fiber. The slog story is similarly +small: explicit `Logger.with(...)` for normal code, optional `LoggerContext` for +framework code that wants the "library code still sees the request logger" trick. We +want reviewer thoughts on which side of this tradeoff Ecstasy programs should be on. + +### 4.3 Structured data — three concepts vs one + +#### Markers — what they are and which library keeps them + +A **log marker** is a named, event-scoped tag SLF4J/Logback users reach for +when categorising events for backend routing (audit/security/billing channels). +The full explainer — what they are, who uses them, the patterns they cover, and +the cloud-side rendering — lives in [`concepts/markers.md`](concepts/markers.md). +The summary, for the comparison axis here: + +- **`lib_logging` keeps markers.** SLF4J users expect them; Logback configs + filter on them; the marker-aware Cloud Logging adapter exists. +- **`lib_slogging` drops markers.** Go `slog` deliberately omits them; every + use case maps to attributes, with one corner the attribute model loses (the + DAG containment query). Importing markers into a slog-shaped library would + undo the simplification that motivates copying slog in the first place. + +If the comparison surfaces "we want markers in the slog-shaped library", that +is a strong signal the slog design is not the right fit and the SLF4J shape +should win. + + +#### The three concepts in `lib_logging` + +SLF4J carries: + +1. `arguments` — positional substitution into `{}` slots in the message template. +2. `marker` — categorisation (audit, security, billing). +3. `keyValues` — explicit structured pairs (added in SLF4J 2.x). + +These were *bolted on over time* and the API shows it: the position of an `Object[]` +argument in the call signature is not the same as the position of a builder +`addKeyValue`. Some sinks (SimpleLogger) only use the message; some (Logback JSON +encoders) only use `keyValues` and ignore `arguments`. + +slog has *one* concept: `Attr`. Everything is an attribute. The message is a free +string; everything structured is an attribute; categories are attributes; the cause +of an error is an attribute. There is no "is this rendered into the message text or +emitted as a structured field" question — the handler decides per attr key. + +**Ecstasy fit.** slog wins on conceptual economy. SLF4J wins on familiarity and on +"you can have nicely-formatted free-text logs without writing handler code." A real +Ecstasy program will probably want both — which the SLF4J model gives explicitly and +the slog model gives via "the Text handler renders the message verbatim and appends +attrs." + +### 4.4 Message — `{}` template vs no interpolation + +SLF4J: `info("user {} did {}", [name, action])`. Lazy substitution: if the level is +disabled the `format` step is skipped. Familiar and concise. + +slog: `Info("user did action", "name", name, "action", action)` — no interpolation. +The handler decides how to render. Output is typically `msg="user did action" +name=alice action=login` for the text handler. + +**Ecstasy fit.** Both are easy. SLF4J's `{}` is more compact for human-readable +output; slog's structure-first is easier for machine consumption. The Ecstasy default +arguments and varargs make either ergonomic. + +**Formatter boundary.** `lib_logging.MessageFormatter` belongs only to the SLF4J-shaped +side. Go `log/slog` has no equivalent formatter: callers pass a finished message and +separate attrs. Reusing the SLF4J formatter in `lib_slogging` would erase the cleanest +part of slog's design by adding positional arguments next to attrs. If we want a +migration helper later, it should be an adapter outside the core slog API. + +### 4.5 Sink/handler shape — minimal vs richer + +`LogSink` has two methods. Everything (level filter, attr resolution, MDC capture, +formatting) happens above the sink. A new sink author writes `isEnabled` and `log` +and is done. + +slog `Handler` has four — the extras are `withAttrs(attrs)` and `withGroup(name)`, +which let the *handler* pre-resolve attached attributes when the user calls +`logger.With(...)`. This means a JSON handler can fold `requestId` into a +namespace prefix once, not every emission. It's a real performance win for +high-volume structured logging. + +**Ecstasy fit.** The slog handler shape is more work to implement but removes +allocation on the hot path for derived loggers. Whether that matters in Ecstasy +depends on workload. `lib_slogging` now ships +[`BoundHandler`](../../lib_slogging/src/main/x/slogging/BoundHandler.x) as the default +derivation wrapper; a production backend can still override those hooks to cache a +serialized prefix or backend-native context object. + +### 4.6 Logger naming — hierarchical vs attribute-based + +SLF4J's `logger.named("payments")` gives you a `Logger` whose name is `". +payments"`. Hierarchical names are how Logback configuration ("set +`com.example.payments` to `Debug`") works. It's a category system that doesn't need +attrs because the *name* IS the category. + +slog has no notion of named loggers. You'd express "this is the payments component" +as an attribute that travels via `With`. Filtering happens by attr equality, not +name-prefix matching. + +**Ecstasy fit.** SLF4J names map cleanly to Ecstasy module / package names, which is +a feature: a `@Inject Logger` resolved with the enclosing module's qualified name is +how SLF4J users intuitively expect it to work. slog forfeits this for uniformity. + +### 4.7 Source location + +Both libraries now expose an explicit source-aware call as the lowering target for +future compiler/runtime help: `logging.Logger.logAt(...)` populates +`LogEvent.sourceFile` / `sourceLine`; `slogging.Logger.logAt(...)` populates +`Record.sourceFile` / `sourceLine`. + +Automatic call-site capture remains compiler/runtime polish. The library-level +decision is made: source metadata belongs on the immutable event/record, not in a +backend-specific side channel. + +### 4.8 Async / batching + +Same shape in both: a wrapper handler/sink owns a bounded queue and drains on a +worker fiber. The base libraries now ship these as `AsyncLogSink` and +`AsyncHandler` so slow output can be isolated without waiting for a full +configuration backend. + +### 4.9 Familiarity + +A subjective dimension that nonetheless matters for adoption. SLF4J is the dominant +JVM logging API; Java/Kotlin engineers reach for it without thinking. slog is the +modern Go default; cloud-native engineers reach for it. Ecstasy targets a developer +audience that will overlap heavily with both. + +--- + +## 5. Ecstasy idiom fit — where the languages bite + +### 5.1 `const` vs `service` for the building blocks + +In `lib_logging`: + +- `BasicLogger` is a `const` (so `@Inject Logger` lives on the caller's fiber, MDC + works). +- `MDC` is a `const` over `SharedContext`, copy-on-write. +- Sinks split: stateless ones (`NoopLogSink`, `ConsoleLogSink`) are `const`; + stateful ones (`MemoryLogSink`, `ListLogSink`) are `service`. + +In `lib_slogging`: + +- `Logger` is a `const` carrying `Handler handler`. Derivation via `with(...)` + returns a new `const Logger` with a derived handler — naturally immutable. +- `LoggerContext` is the optional `SharedContext` helper for framework/request + propagation; it is not required for normal logging calls. +- Handlers split the same way: stateless adapters (`TextHandler`, `JSONHandler`, + `NopHandler`) are `const`; stateful collectors (`MemoryHandler`) are `service`. + +The slog model is *substantially* simpler on the const side because the core API does +not snapshot an MDC map on every event. The optional interaction — "I want a logger +that propagates through fibers without threading a parameter" — lives in +`LoggerContext`, not in the hot path. + +### 5.2 Fluent builder vs varargs + +SLF4J's fluent `.atInfo().setMessage(...).addKeyValue(...).log()` requires a +`LoggingEventBuilder` interface and a `BasicEventBuilder` implementation, plus a +no-op variant for short-circuiting disabled levels. That's three types of machinery +for one API surface. + +slog's `LogAttrs(level, msg, ...attrs)` doesn't need a builder at all — varargs of +typed `Attr` values cover the same ground. Fewer types, fewer allocations on the +disabled-level path (because the caller usually doesn't construct attrs unless the +level is enabled, via `if logger.Enabled(...) { ... }`). + +**Ecstasy fit.** The fluent builder feels Java-ish. The slog varargs feel closer to +how Ecstasy method signatures already work. But Ecstasy's default-argument support +already collapses many of SLF4J's fluent-builder use cases into one method call, so +the fluent surface is smaller in `lib_logging` than the SLF4J Java original. + +### 5.3 Marker subgraph vs attribute uniformity + +`Marker` in SLF4J/`lib_logging` is its own type with `add` (parent/child references) +and `contains` (transitive query). It also has identity (`MarkerFactory.getMarker` +interns). That's a small DAG type. + +slog has no marker concept. The same use cases (filter audit-only events) are +handled by `Attr.of("audit", True)` and a handler that filters on that attr. + +**Ecstasy fit.** Markers add a separate type; attributes don't. The marker model is +slightly more efficient (one interned reference vs a string equality check), but the +attribute model needs less surface area. + +### 5.4 Compiler-side default-name injection + +SLF4J expects `LoggerFactory.getLogger(MyClass.class)` to resolve to a class- or +module-named logger. `lib_logging` currently handles the simple interpreter case with +a runtime fallback from the injected field name to the caller namespace. Exact +compiler-synthesized names remain future polish. Without them, `@Inject Logger` gets a +fixed-name root logger and you call `.named("...")` to derive children. + +slog has no such machinery — there is one process-wide default Logger and you derive +from it. No naming hierarchy, no compiler change. + +**Ecstasy fit.** slog removes a "do we make the compiler smarter?" question +entirely. SLF4J keeps the door open for the nicer ergonomics if the compiler change +lands. + +--- + +## 6. What we want reviewer feedback on + +1. **`MDC` worth keeping?** SLF4J's per-fiber MDC is the most opinionated piece of + this comparison. It costs ~50 lines of `SharedContext` machinery + a small + compiler ask. The slog approach (explicit `with(attrs)`) removes the implicit + context entirely. Which side of the implicit/explicit fork should an Ecstasy + library live on? + +2. **One sink interface or two?** Both designs end up with sinks/handlers split into + "stateless `const`" and "stateful `service`" implementations. We currently allow + both shapes under one interface. Should the language convention be one interface + per shape (forcing implementers to declare intent at the type level)? + +3. **Fluent builder pull-back.** SLF4J's `.atInfo()...` family is a third API + surface that pays for cases the per-level methods can't easily express. With + Ecstasy default arguments and varargs, the fluent builder may be unnecessary. + Should `lib_logging`'s fluent surface go away? + +4. **Hierarchical names.** Are class- / module-qualified hierarchical logger names + worth a compiler change to make `@Inject Logger` give you a class-named logger + automatically? Or is the slog model — one logger, attributes for everything else — + what we want for new Ecstasy code? + +5. **Markers vs attrs.** Do reviewers see real value in the marker DAG model (audit + marker, security marker, etc.) over a string-keyed attribute? `lib_logging` + carries the marker type today; `lib_slogging` does not. + +6. **Levels — closed enum vs extensible Int.** Five named levels are familiar but + inflexible; integer levels are open but lose canonicality. Which idiom should + Ecstasy converge on across logging, telemetry, and tracing libraries? + +7. **`@Inject` ergonomics.** Both libraries currently funnel through `@Inject Logger + logger;`. Do reviewers want this as the entry point, or should one of the two + libraries demonstrate a non-injection idiom (a top-level `Logger.default` const, + say) for comparison? + +8. **Source location.** Both POCs expose explicit `logAt(...)` APIs. Do reviewers want + compiler/runtime sugar that automatically populates those fields on ordinary + `logger.info(...)` calls? + +9. **Performance baseline.** Neither library has been profiled. If reviewers want + benchmarks before deciding, name the workload (sustained 100k events/sec to a + no-op sink? burst with disabled levels? 95th-percentile latency at the call + site?) and we'll add it to the comparison. + +10. **Configuration home.** Both libraries now have async wrappers. Should the XDK + also ship a standard JSON config loader/reload service, or leave config-file + loading to applications/backends? + +11. **Should Ecstasy ship two logging libraries permanently?** The Java ecosystem has + two large facades (SLF4J, Log4j 2 API) and is by no means ergonomic for it. + Carrying both `lib_logging` and `lib_slogging` long-term has the same risk. The + intent of this experiment is to *pick one* and move on. We want reviewer input + on which. + +--- + +## 7. Recommendation + +Choose **`lib_logging` as the canonical Ecstasy logging library**. + +The decisive reason is not that the SLF4J shape is prettier. It is that the XDK +needs one logging facade that looks industrially boring to the largest likely +audience while still leaving room for structured output and backend innovation. +`lib_logging` now does that: + +- JVM users recognize the call surface immediately: `Logger`, named loggers, `{}` message + formatting, `Exception` cause handling, markers, MDC, and SLF4J 2.x fluent builders. +- Kotlin/SLF4J 2.x users get lazy message/value construction for hot paths: + suppliers are invoked only after the level/marker check accepts the event. +- Backend authors get one narrow extension point, `LogSink`, with working examples for + console, memory capture, JSON-Lines, async forwarding, multi-destination fanout, and + hierarchical per-logger level routing. +- Cloud and JSON users are not forced to parse message text. Structured key/value pairs + travel on `LogEvent.keyValues`; `JsonLogSink` renders them directly and applies + redaction policy before output. +- Runtime injection already works for the interpreter, including a default-name fallback + for canonical `logging.Logger` injections. + +`lib_slogging` remains valuable. It is smaller, cleaner, and closer to modern +attribute-first logging. It also documents a real alternative for teams that prefer +Go's `slog` mental model. The right long-term shape, however, is not to ship two +competing XDK facades. Keep `lib_slogging` as review material and possible adapter +surface; make `lib_logging` the default API users learn first. + +A hybrid SLF4J-facade with slog-shaped event internals is worth revisiting after v0, +but it should not be the starting point. Mixing `Marker`, `MDC`, `Object[] arguments`, +and `Attr[]` in one public contract would make the facade harder to teach than either +pure design. + +The remaining compiler/runtime work is polish around the chosen facade, not a reason +to reopen the API choice: automatic source-location capture, better generated logger +names, and optional backend configuration loaders can all target the existing +`Logger` -> `LogSink` boundary. + +--- + +## 8. What lives in this PR + +This branch (`lagergren/logging`) contains two comparable implementations and the +review material needed to choose between them. + +| Area | Contents | +|---|---| +| `lib_logging/` | Recommended canonical SLF4J-shaped library with 70 focused XTC test methods, runtime injection, lazy suppliers, source metadata API, JSON/redaction sink, async wrapper, composite fanout, and hierarchical per-logger thresholds. | +| `lib_slogging/` | slog-shaped sibling library with 41 focused XTC test methods, runtime injection, lazy message/attr suppliers, `lib_json` JSON rendering, handler options/redaction, async wrapper, explicit source metadata, `LoggerContext`, handler derivation, and handler contract tests. | +| `doc/logging/` | Design comparison, API cross-reference, language/runtime questions, usage guides, and backend follow-up sketches. | + +The branch is now opinionated: Q-D6 is answered as `lib_logging`. Review can still +challenge that decision, but the implementation and docs no longer leave the reader +with two equally recommended facades. + +--- + +Previous: [`usage/custom-handlers.md`](usage/custom-handlers.md) | Next: [`concepts/markers.md`](concepts/markers.md) → | Up: [`README.md`](README.md) diff --git a/doc/logging/open-questions.md b/doc/logging/open-questions.md new file mode 100644 index 0000000000..c11b6561b5 --- /dev/null +++ b/doc/logging/open-questions.md @@ -0,0 +1,271 @@ +# Logging decision tracker and open questions + +> **Audience:** XTC language/runtime designers and reviewers tracking what is decided vs still open. + +This is the running list of design decisions and remaining implementation questions, +kept deliberately short so it stays readable as the project moves. + +It is split into resolved design notes, implementation trackers, and the few items +that still need compiler/tooling/backend follow-up. + +--- + +## Runtime decisions + +| # | Question | Resolution | +|---|---|---| +| 1 | **Wildcard-name injection** — should `@Inject("any.name") Logger` acquire arbitrary named loggers directly? | **Rejected.** There is one fixed-name `("logger", loggerType)` supplier; per-name loggers come from `Logger.named(String)` instead. | +| 2 | **Default logger name when no `resourceName` supplied** — should `@Inject Logger logger;` (no name) get `"default"`, the enclosing module's name, or the class name? | Runtime fallback implemented for canonical `logging.Logger`: if the injected field name is `"logger"`, `NativeContainer` derives the caller namespace. Exact compiler-synthesized module/class names remain future polish. | +| 3 | **MDC scope: per-fiber, per-service, or per-call?** | Per-fiber semantics implemented in Ecstasy with `SharedContext`. The logger is injected as a `BasicLogger` const so calls stay on the caller fiber and can see the current MDC. | +| 5 | **Throwable promotion: where does it happen, and who wins when both are supplied?** | `MessageFormatter.format` does the promotion; explicit `cause=` always wins over a promoted-from-args throwable. | +| 9 | **Where does the runtime live?** | Interpreter-side wiring lives in `javatools/src/main/java/org/xvm/runtime/NativeContainer.java`. The earlier native-resource class plan is superseded for this branch. | +| 10 | **Bootstrap: do we need a tiny native fallback for early-runtime logging before `Console` is registered?** | Not for this branch. The default `ConsoleLogSink` writes through `@Inject Console`; a bootstrap-native fallback can be added later only if runtime startup logging needs it. | + +--- + +## Design notes and remaining follow-up + +Numbered to preserve traceability to earlier discussion. + +### 4. Multiple markers per event — *resolved* + +**Resolution.** Implemented option A. `LogEvent.markers: Marker[]` is now the canonical +field (see `LogEvent.x`); a `LogEvent.marker: Marker?` convenience accessor returns +`markers[0]` for sinks that only surface a single category line. `BasicEventBuilder` +accumulates per `addMarker(...)` call and freezes the array before crossing the sink +boundary. Per-level methods (`info`/`warn`/...) still take a single optional `Marker` +and wrap it; multi-marker is reachable through the fluent builder. SPI-level +`LogSink.isEnabled(...)` keeps its single-marker signature (matches SLF4J 1.x's +`Logger.isEnabledFor(Marker)`); the "primary" marker is `markers[0]`. Tests: +`MarkerTest.shouldPropagateMultipleMarkersThroughFluentBuilder`, +`MarkerTest.shouldEmitNoMarkersWhenNoneAttached`. + +### 6. `const` vs `service` for sinks — *resolved* + +**Resolution.** `LogSink` is an interface; implementations choose between `const` and +`service` according to the rule documented in `design/design.md` ("Sink type: `const` vs +`service`"): + +- stateless forwarders / pure adapters → `const` +- stateful collectors, resource owners, async workers → `service` + +`BasicLogger` is a `const`, so every concrete sink must be `Passable` (either +`immutable` or a `service`). Forcing `LogSink extends Service` in the interface +signature was rejected because it would prohibit the legitimate stateless cases +(`NoopLogSink`, `ConsoleLogSink`) and contradicts the `class ConsoleAppender` / +`class ConsoleLog` pattern already used in `lib_ecstasy/src/main/x/ecstasy/io/`. + +This rule is checked for parity against patterns in `platform/common`, +`platform/host`, and `lib_xunit_engine` — see design/design.md for the citations. + +### 7. Async / batched sinks — *resolved* + +**Resolution.** Adopted option A. `AsyncLogSink` ships in `lib_logging` as a small +bounded-queue wrapper around any `LogSink`; `AsyncHandler` mirrors the same shape for +`lib_slogging`. Slow output can now be isolated without waiting for a full +configuration backend. A future Logback-style module can still choose to compose or +replace the wrapper for richer batching policies. + +### 8. Per-container override convenience — *resolved* + +**Resolution.** Adopted option A — document the pattern, add no API. A host that +wants a nested container to use a different sink configures the child container's +injector explicitly to resolve the `("logger", Logger)` key against the alternate +sink. Pattern documented in `design/design.md` § "Per-container sink override". No +`lib_logging`-side helper, deliberately: per-container resource overrides are the +host runtime / injector library's job, not the logging library's. + +### 11. Defensive copy of caller-supplied `Object[] arguments` — *resolved* + +**Resolution.** Adopted option B for v0: callers contractually must not mutate the +`arguments` array between the return of `info(...)` and any sink consuming the +resulting `LogEvent`. Matches SLF4J's posture; documented as a one-paragraph policy +comment in `BasicLogger.emitWith` (the only entry point that receives caller-owned +args). `BasicEventBuilder` is already safe by construction because it freezes its +internally-accumulated `args` to `Constant`-mode before crossing the sink boundary. +Reconsider if/when async sinks become a default — at that point the cost of the +defensive copy is justified by the larger window during which the caller can +mutate. + +### 12. Compiler/tooling support for log statements + +SLF4J has linters that flag `info("count: " + n)` (eagerly formatted instead of using +`{}`) and missing exception arguments. Should Ecstasy? + +**Recommendation:** Out of scope for `lib_logging`. If we want it, it's an XTC linter +feature, not a library feature. + +--- + +## Questions for the XTC language / runtime designers + +These are points where we made a call but want explicit confirmation from the people +who own the language semantics, before locking the API. + +### Q-D1. Is "`const` if stateless / `service` if stateful with shared mutable state" the right pattern for an Ecstasy SPI boundary? + +We landed on a rule (see design/design.md, "Sink type") that lets `LogSink` accept both shapes, +mirroring the way `lib_ecstasy/src/main/x/ecstasy/io/ConsoleAppender.x` is a `class` +while `service ErrorLog` and `service ConsoleExecutionListener` exist elsewhere. The +alternative is forcing every implementation to be one shape (e.g. `LogSink extends +Service`) and accepting the loss of `const` stateless sinks. + +**Question:** is permitting both shapes idiomatic for an XDK SPI, or does the language +team prefer SPI interfaces that constrain to one shape for clarity? + +### Q-D2. `@Inject` inside a `const` — is this fully supported? + +`BasicLogger` is a `const` and declares `@Inject Clock clock; @Inject MDC mdc;`. +`ConsoleLogSink` is a `const` and declares `@Inject Console console;`. This works in +practice today, but the docs are thin on whether `@Inject` resolution semantics are +identical inside a `const` versus a `service` (timing, fiber affinity, caching). + +**Question:** is `@Inject` inside a `const` first-class, or is there a subtlety we +should know about (e.g. resolved at construction vs at first access; affinity to the +constructing fiber vs the calling fiber; freezing behaviour after construction)? + +### Q-D3. Calling a service from a `const` field — boundary semantics + +`const BasicLogger` holds a `LogSink sink` field. When that sink is a `service`, every +`sink.log(event)` is an inter-service hop with implicit freeze of the arguments. We +adopted "freeze the marker before the first sink hop" as a workaround +(`BasicLogger.emitWith`) because the diagnostic wording ("Property not freezable" / +"mutable object cannot be used as an argument to a service call") is hard to map back +to "the value crossed an implicit service boundary." + +**Question:** is there a more idiomatic way to express "this `const` adapts a service +under the hood, and the service boundary cost is paid at this method call"? Should +the compiler give us a clearer diagnostic when a value would need to be `Passable` to +cross such a boundary? + +### Q-D4. Per-fiber `MDC` via `SharedContext` — are we using it as intended? + +`MDC` is implemented as `const MDC` storing `immutable Map` in a +single static `ecstasy.SharedContext`, with copy-on-write derivation on every mutation +(`put / remove / clear`). This relies on (a) `const` methods running on the caller's +fiber so `mapContext.hasValue()` resolves against that fiber's tokens; and (b) the +runtime's clone-on-write fiber-token map for sync calls so child fibers see parent +state without being able to mutate it back. + +**Question:** does this match the intended use of `SharedContext` for fiber-local +context propagation, or should we be using a different primitive (a built-in +`MDC`-equivalent, a service-injected ledger, fiber-local slots) for SLF4J-grade MDC +semantics? + +### Q-D6. SLF4J vs slog as the API shape — *resolved for this branch* + +We're maintaining two parallel libraries (`lib_logging` modeled after SLF4J 2.x and +`lib_slogging` modeled after Go's `slog`) so the design tradeoffs can be evaluated +side-by-side. See `lib-logging-vs-lib-slogging.md` for the comparison and the explicit +list of things we want reviewer feedback on. + +**Resolution.** Recommend `lib_logging` as the canonical XDK facade. The deciding +factors are ecosystem familiarity, hierarchical logger routing, markers/MDC/fluent +builder parity for SLF4J users, and the now-working `LogSink` backend path. Keep +`lib_slogging` as comparison material and possible adapter surface, not as a second +permanent default facade. + +### Q-D7. Cross-module default-argument resolution on `const` constructors + +Calling `new ConsoleLogSink()` from the `TestLogging` module fails with +`Unresolvable function "void construct()"` even though `ConsoleLogSink` was +declared as `const ConsoleLogSink(Level rootLevel = Info)`. Same shape failed +for `new BasicLogger(name, sink)` against +`const BasicLogger(String name, LogSink sink, LoggerRegistry? registry = Null)`. +Adding explicit no-arg / shorter-arg `construct(...)` forms that delegate to the +primary constructor fixes it; same-module callers (the convenience constructor +inside `BasicLogger.x` itself, the unit tests inside `LoggingTest`) work fine +without the explicit forms. + +We worked around it by writing every "shorter form" explicitly — see +`BasicLogger.x`, `ConsoleLogSink.x`, `Logger.x`, `TextHandler.x`, `JSONHandler.x`. +Each has a `Why:` comment naming the workaround. + +**Question:** is the synthesis of "callable" forms from default-argument +constructors only available within the declaring module, or is this a bug? If +intended, the `construct(Level rootLevel = Info)` syntax is misleading because +it reads as "this defaults to Info" but practically requires explicit +delegating constructors for cross-module ergonomics. If a bug, fixing it would +remove the explicit-delegation clutter from every const that wants a no-arg +form. + +--- + +## SLF4J-style library (`lib_logging`) — implementation tracker + +Tracking list so the SLF4J-shaped library stays comparable to `lib_slogging` in +feature scope. Each item is a concrete deliverable with a one-line summary. + +| # | Item | Notes | +|---|---|---| +| W-1 | Multiple markers per event | **Done.** `LogEvent.markers: Marker[]`, `marker` convenience accessor, `BasicEventBuilder.addMarker(...)` accumulation, and `ConsoleLogSink` multi-marker rendering are implemented and tested. | +| W-2 | Logger interning in `Logger.named` | **Done.** Implemented `service LoggerRegistry(LogSink sink)` keyed by logger name; `BasicLogger.registry: LoggerRegistry?` (optional, defaulted `Null` so unattached loggers stay allocation-light); `LoggerFactory` lazily constructs a registry over its `defaultSink` and routes both `getLogger(...)` and `named(...)` through it. Tests: `NamedLoggerTest.shouldInternChildrenWhenRegistryAttached`, `NamedLoggerTest.shouldNotInternWhenRegistryAbsent`. | +| W-3 | Real `MessageFormatter` | **Done.** Full SLF4J ParameterFormatter parity: `{}` substitution, `\{}` literal escape, `\\{}` double-escape, trailing-throwable promotion (with explicit-cause precedence enforced in `BasicLogger.emit`), defensive `safeToString`. 12 tests in `MessageFormatterTest`. The "stub" claim in earlier docs was stale. | +| W-4 | Default injected logger naming | **Partially done.** `NativeContainer` now falls back from the field name `"logger"` to the caller namespace for canonical `logging.Logger` injections. Exact module/class names still need compiler lowering if we want deterministic names outside the runtime fallback. | +| W-5 | Async / batched sink (`AsyncLogSink`) | **Done.** Bounded queue + worker-fiber drain wrapper ships in the base library and is tested. | +| W-6 | Logback-shaped backend primitives | **Done at the base-programmatic level.** `CompositeLogSink`, `HierarchicalLogSink`, `JsonLogSink`, and `JsonLogSinkOptions` cover multi-appender fanout, per-logger threshold routing, JSON rendering, and redaction/field-name knobs. External config-file loading, rolling files, hot reload, network sinks, and filters remain follow-up backend modules. | +| W-8 | Per-container override convenience | **Resolved: no library helper.** Child containers should override the injectable resource through the host injector API. | +| W-9 | Defensive copy of caller-supplied `Object[] arguments` | **Resolved: document, no copy.** Listed here so it can be revisited if async sinks become default-on. | +| W-10 | Compiler/tooling lints for log statements | Out of scope for the library; would be an XTC linter feature. Open question 12. | +| W-11 | Doc cleanup | **Done for active docs.** Start-here docs, parity docs, examples, runtime-injection notes, and source comments now describe the current `BasicLogger`/`MDC`/multi-marker/formatter shape. Historical future-plan docs remain as historical context. | + +The `lib_slogging` section mirrors the same tracking shape so the API decision is +made between two implementations at the same proof level. + +### Implementation tiers + +The W-items above are grouped into three tiers to show what landed in this branch +and what remains deliberate follow-up work. + +| Tier | Scope | Items | Approx. cost | +|---|---|---|---| +| **1** | Close visible loose ends. No new design choices. | W-1 (multi-marker), W-11 (doc cleanup) | ~1 hour | +| **2** | Make `lib_logging` fully comparable as a shippable library. | Tier 1 + W-2 (logger interning, needs a `LoggerRegistry` service since `BasicLogger` is `const`) + W-3 (real `MessageFormatter` with `{}` substitution + trailing-throwable promotion) | ~half a day | +| **3** | Backend/runtime polish. | W-4 runtime fallback naming, W-5 async wrapper, W-6 base Logback-style primitives, W-8 per-container override decision, W-9 defensive-copy decision, W-10 linter deferral | mostly landed; W-4 compiler lowering and W-10 linting remain separate work | + +**Decision (revised):** Tier 1, Tier 2, and the backend-primitives part of Tier 3 +land in this PR for both libraries. The only remaining Tier 3 implementation work is +outside the base libraries: compiler-synthesized call-site metadata, linter support, +and full config-file loading/destinations. + +The `lib_slogging` parity translation: + +| `lib_logging` (W) | `lib_slogging` analogue | +|---|---| +| W-1 multi-marker on event | n/a — slog uses repeated attrs, already supported by `Attr[]` | +| W-2 logger interning in `Logger.named` | n/a — slog has no hierarchical names; ergonomic `Logger.with(...)` already returns derived loggers | +| W-3 real `MessageFormatter` with `{}` substitution | n/a — slog has no message templating; `TextHandler` / `JSONHandler` formatting completeness instead | +| W-5 async sink | `AsyncHandler` | +| W-6 backend options / JSON / redaction | `HandlerOptions`, `JSONHandler`, `TextHandler` | +| W-11 doc cleanup | mirrored: keep `lib_slogging` doc references in sync | +| (test suite parity) | mirror `LoggingTest` → `SLoggingTest`: per-level routing, fluent-equivalent (`logger.with`), level thresholds, exception carriage, structured attrs, group nesting, async wrapper, JSON/redaction | + +## slog-style library (`lib_slogging`) — parity work addressed + +`lib_slogging` is now a working comparison POC at the same proof level as the +SLF4J-shaped library for the base API. The concrete parity gaps called out during +review have been closed as follows: + +| # | Item | Notes | +|---|---|---| +| S-1 | Runtime injection for `@Inject slogging.Logger logger;` | **Done.** `NativeContainer` now resolves resources by `(name, requested type)`, so `logging.Logger logger` and `slogging.Logger logger` can both use the default name. `manualTests/TestLogging.x` and `manualTests/TestSLogging.x` exercise the two injected APIs separately. | +| S-2 | Production JSON handler | **Done for the base POC.** `JSONHandler` renders through `lib_json`, escapes strings through `Printer`, preserves nested groups, emits source metadata, renders exceptions structurally, and honors `HandlerOptions` redaction/field-name/source knobs. Output destinations beyond `Console` remain a future handler family. | +| S-3 | Source-location capture | **Done via explicit API.** `Logger.logAt(level, message, sourceFile, sourceLine, attrs, cause)` populates `Record.sourceFile` / `sourceLine`. Automatic compiler/runtime call-site capture remains a future enhancement. | +| S-4 | Context story | **Done.** `LoggerContext` wraps `SharedContext` for framework/request propagation. Explicit logger passing remains the recommended default. | +| S-5 | Handler pre-resolution example | **Done.** `BoundHandler` implements the default `withAttrs` / `withGroup` semantics and shipped handlers use it; production backends can replace it with cached prefix implementations. | +| S-6 | Handler test kit | **Done.** `SLoggingTest.HandlerContract` provides a small `testing/slogtest`-style helper for third-party handler conformance. | + +## Decision ordering + +| Decision | Why it comes first | +|---|---| +| MDC scope (#3) | Runtime wiring needs a settled per-fiber propagation model. | +| Multiple markers per event (#4) | `LogEvent` and `BasicEventBuilder` should be stable before more sinks depend on them. | +| Defensive copy (#11) | The current decision is "document, no copy"; revisit if async sinks become default-on. | + +The remaining genuinely open items are compiler/tooling work, destination-specific +production backends, and any optional Java bridge. + +--- + +Previous: [`roadmap.md`](roadmap.md) | Next: [`README.md`](README.md) → | Up: [`README.md`](README.md) diff --git a/doc/logging/roadmap.md b/doc/logging/roadmap.md new file mode 100644 index 0000000000..efe8288cf1 --- /dev/null +++ b/doc/logging/roadmap.md @@ -0,0 +1,73 @@ +# Logging roadmap + +> **Audience:** reviewers deciding whether to merge this branch and what should +> follow. Read this once before [`open-questions.md`](open-questions.md). + +This page is the single answer to *"what blocks merge, what lands in v1, what +is explicit future?"*. The detailed history of each item — alternatives +considered, rejected variants, who pushed back — lives in +[`open-questions.md`](open-questions.md). Treat that as the supporting evidence, +this as the index. + +## Status legend + +| Mark | Meaning | +|---|---| +| ✓ | Done in this branch — code, tests, docs all present | +| ~ | Partial — runtime works, compiler/tooling polish missing | +| → | Planned for v1 (post-merge) | +| + | Future, no commitment | +| ? | Open — needs a decision before merge | + +## Merge-blocking + +Items the reviewers must resolve before this branch can graduate from POC. + +| Item | State | Where | +|---|---|---| +| Pick `lib_logging` (SLF4J shape) vs `lib_slogging` (slog shape) as the canonical XDK API. | ? | [`lib-logging-vs-lib-slogging.md`](lib-logging-vs-lib-slogging.md), open question Q-D6 | +| Confirm the `const`/`service` rule for SPI implementations is idiomatic XDK. | ? | open question Q-D1 | +| Confirm `@Inject` inside a `const` is fully supported (semantics, fiber affinity, freeze). | ? | open question Q-D2 | +| Confirm per-fiber `MDC` via `SharedContext` matches intended use. | ? | open question Q-D4 | +| Decide whether cross-module default-arg synthesis on `const` constructors is a bug or expected. | ? | open question Q-D7 | + +## v1 (post-merge, before users depend on it) + +Items the chosen library must land before downstream code starts to migrate. + +| Item | State | Notes | +|---|---|---| +| Compiler-synthesized default logger names. | → | Runtime fallback derives from caller namespace today; the compiler should emit the exact module/class name as the resource name. R10 in the README. | +| Compiler-synthesized source location. | → | `Logger.logAt(...)` is the lowering target. `logger.info(...)` calls do not yet get sourceFile/sourceLine populated automatically. R10 in the README. | +| `AsyncLogSink.drain` — `try/finally` to release `draining` on delegate exception. | → | Defensive: `LogSink.log` is contractually non-throwing, but the async wrapper should recover from a misbehaving delegate. Code review item. | +| `AsyncLogSink` queue dequeue performance — replace `Array.delete(0)` (O(n)) with a circular buffer. | → | Currently O(n²) per drain; only matters at scale. | +| Strip the asymmetric `xdk-logging` dep from `javatools_bridge/build.gradle.kts` (or add `xdk-slogging` for parity). | → | Bridge `_native.x` imports neither. Code review item. | +| Migrate `~/src/platform` ad-hoc `console.print($"… Info :")` lines to the chosen `Logger`. | → | Survey done in [`usage/platform-and-examples-adaptation.md`](usage/platform-and-examples-adaptation.md). | +| Confirm `addResourceSupplier` lookup performance is fine (O(n) loop over ~20 entries vs the previous HashMap by name). | → | Likely a non-issue, but worth checking once the resource table grows. | + +## Future, no commitment + +Items deliberately out of scope for the base library. They are pure-XTC modules +built on top of the SPI; nothing about them belongs in `lib_logging` itself. + +| Item | State | Where | +|---|---|---| +| Configuration-file driven backend (`lib_logging_logback`). | + | Sketched in [`future/logback-integration.md`](future/logback-integration.md) | +| Rolling/network/file destinations. | + | Live behind `LogSink`; out of scope for the base lib. | +| Provider-specific cloud sinks (GCP Cloud Logging, CloudWatch, App Insights). | + | Each is a thin adapter over `LogEvent`. See [`cloud-integration.md`](cloud-integration.md). | +| Distributed tracing context propagation (trace/span IDs as a first-class concern). | + | MDC carries strings; tracing is a separate library. | +| XTC linter for log-statement footguns (`info("count: " + n)`, missing exception arg). | + | Linter feature, not a library feature. Open question 12. | +| Adapter modules (`lib_logging_slf4j_adapter`, `lib_logging_slogging_adapter`) bridging the losing API onto the canonical one. | + | Only if the losing POC has users worth migrating. | +| `service MarkerRegistry` for fiber-safe global marker interning + DAG mutation. | + | The current `class MarkerFactory` covers the per-component pattern; a service-shaped registry would let multiple fibers share one interned marker space. See [`MarkerFactory.x`](../../lib_logging/src/main/x/logging/MarkerFactory.x) "Why a class, not a service" for the design constraint that keeps these as separate types. | + +## Reading order + +Once you have read this page: + +- For the API-choice debate: [`lib-logging-vs-lib-slogging.md`](lib-logging-vs-lib-slogging.md). +- For the open language/runtime questions: [`open-questions.md`](open-questions.md). +- For the cloud-side context behind the merge decision: [`cloud-integration.md`](cloud-integration.md). + +--- + +Previous: [`README.md`](README.md) | Next: [`lib-logging-vs-lib-slogging.md`](lib-logging-vs-lib-slogging.md) → diff --git a/doc/logging/usage/configuration.md b/doc/logging/usage/configuration.md new file mode 100644 index 0000000000..cf752ddf77 --- /dev/null +++ b/doc/logging/usage/configuration.md @@ -0,0 +1,955 @@ +# Logging configuration and dynamic backends + +> **Audience:** ops/host engineers who own backend wiring. Library authors should not need to read this. + +Logging configuration belongs below the facade. Application/library code should keep +using `@Inject Logger logger;`; the host/container decides which sinks or handlers are +active, where output goes, which fields are redacted, and which logger categories are +debug-enabled. + +This is exactly the SLF4J/Logback split: SLF4J is the facade, Logback is the configured +backend. The Ecstasy shape should preserve that split. + +## Fast path + +The proposed XDK configuration model is: + +1. **JSON for the full backend graph.** XML is familiar from Logback, but JSON maps more + directly to typed Ecstasy config objects and existing XDK JSON tooling. +2. **Small overrides for operations.** Properties, environment variables, and command-line + args should patch common values such as root level, per-component level, file path, + cloud provider, and redacted keys. +3. **Stable injected loggers.** Runtime injection hands application code a logger whose + sink/handler is stable; a host-owned config service rebuilds the backend graph and + swaps the delegate on reload. +4. **Core vs follow-up modules.** The POC already implements the facade, SPI, JSON + console output, async wrappers, fanout, hierarchical routing, and test capture. File + destinations, rolling files, provider-specific cloud clients, JSON config parsing, and + hot reload belong in first-class follow-up XDK modules such as `lib_logging_config`, + `lib_logging_file`, and `lib_logging_cloud`. + +If you only need the shape, jump to +[`What is implemented vs proposed`](#what-is-implemented-vs-proposed), +[`Self-contained lib_logging dynamic backend sketch`](#self-contained-lib_logging-dynamic-backend-sketch), +and +[`Self-contained lib_slogging dynamic handler sketch`](#self-contained-lib_slogging-dynamic-handler-sketch). +The rest of this document is the detailed prior-art mapping for Logback, Log4j 2, +Spring Boot, and Go `slog`. + +## Logback-style pieces already implemented in XTC + +Do not read every Logback-style term in this guide as future work. The base +`lib_logging` POC already implements several backend primitives that Java teams +normally associate with Logback rather than SLF4J itself: + +| Logback concept | Implemented XTC type | Status | +|---|---|---| +| Console appender with a fixed human-readable layout | `ConsoleLogSink` | Implemented; default injected sink. | +| Multiple appender refs on one logger | `CompositeLogSink` | Implemented; fans out to multiple delegate sinks. | +| Logger-name hierarchy and longest-prefix level lookup | `HierarchicalLogSink` | Implemented; mutable per-prefix thresholds. | +| Async appender wrapper | `AsyncLogSink` | Implemented; bounded queue, flush/close, drop counter. | +| JSON encoder / logstash-style structured output | `JsonLogSink`, `JsonLogSinkOptions` | Implemented with `lib_json`, source/MDC/marker/exception fields, and redaction. | +| Test capture appender | `MemoryLogSink` | Implemented; equivalent to Logback's `ListAppender`. | + +What is **not** implemented in the base library is the full Logback ecosystem: +configuration-file parsing, filters, pattern layouts, file and rolling-file outputs, +network/cloud destinations, and hot reload. Those belong in follow-up modules layered +on top of the implemented `LogSink` primitives. + +Java Logback: + +```xml + + + + + + + + + 8192 + + +``` + +Same implemented idea in Ecstasy code today: + +```ecstasy +LogSink json = new JsonLogSink(new JsonLogSinkOptions(Info, + ["authorization", "password", "token"])); +LogSink fanout = new CompositeLogSink([ + new ConsoleLogSink(), + new AsyncLogSink(json, capacity=8192) + ]); + +HierarchicalLogSink configured = new HierarchicalLogSink(fanout, Info); +configured.setLevel("com.acme.payments", Debug); + +Logger logger = new BasicLogger("com.acme.payments.Checkout", configured); +logger.debug("charged {}", [amount]); +``` + +## Configuration prior art to account for + +Logback is the best-known SLF4J backend, but it is not the only operational model that +reviewers will recognize. The XDK design should be understandable to teams coming from +these systems too: + +| System | Why it matters | What the XDK design should preserve | +|---|---|---| +| [Logback](https://logback.qos.ch/manual/configuration.html) | The default backend in many SLF4J deployments and the mental model most Java teams mean by "configured logging". | Named loggers, root level, appenders, encoders, async appenders, reload, MDC, markers. | +| [Apache Log4j 2](https://logging.apache.org/log4j/2.x/manual/configuration.html) | The other major JVM logging framework. It has XML, JSON, YAML, and properties configuration, appenders, layouts, filters, async logging, and custom plugins. | The same backend graph should be expressible without requiring Logback XML: loggers, appenders/sinks, layouts/formatters, filters, async wrappers, and JSON output. | +| [Spring Boot logging](https://docs.spring.io/spring-boot/reference/features/logging.html) | Not a logging backend, but a very common deployment experience. Operators set `logging.level.*`, `logging.file.*`, and `logging.config` through properties, env vars, or command-line args. | The XDK should have simple override keys for the common cases and a structured config file for the full backend graph. | +| [java.util.logging](https://docs.oracle.com/en/java/javase/26/docs/api/java.logging/java/util/logging/LogManager.html) | The JDK built-in logger. Many libraries still emit through it, and bridges are common. | Hierarchical names and handler/formatter concepts are familiar, but the XDK should avoid JUL's global `LogManager` and flat properties-only limitations. | +| [Commons Logging](https://commons.apache.org/logging/index.html), JBoss Logging, and bridges | Facades exist because libraries should not force an application backend. | `@Inject Logger` should give Ecstasy the same library/application separation without classpath discovery. | +| [logstash-logback-encoder](https://github.com/logfellow/logstash-logback-encoder) and cloud appenders | This is how many SLF4J/Logback services produce structured JSON for Elastic, Cloud Logging, CloudWatch, or Azure Monitor. | JSON and cloud output must be first-class backend concerns, not message parsing hacks. | + +For Go, the comparison point is different. [`log/slog`](https://pkg.go.dev/log/slog) +does not define a `logback.xml`-style configuration file. A Go program usually parses +its own flags, environment variables, or config file, constructs a handler graph in +`main`, and installs it with `slog.SetDefault`. Dynamic levels are represented by +`slog.LevelVar`; redaction and field rewriting are represented by +`HandlerOptions.ReplaceAttr`; fanout is a handler wrapper such as `slog.MultiHandler` +or a third-party/custom handler. + +## Java Logback baseline + +A typical Logback config combines: + +- appenders: where output goes; +- layouts/encoders: how events are rendered; +- loggers: category-specific level and appender rules; +- root: default level and destinations; +- scan/reload: optional config-file hot reload. + +```xml + + + + %d %-5level [%thread] %logger - %msg %kvp%n + + + + + logs/app.jsonl + + + + + 8192 + + + + + + + + + + + + +``` + +The important point is that caller code does not mention any of this. It just logs. + +## Proposed Ecstasy JSON shape + +Ecstasy does not need to copy Logback XML. A JSON or Ecstasy-native config object is a +better fit for the XDK because the backend graph is already typed and programmatic. + +```json +{ + "version": 1, + "logging": { + "rootLevel": "Info", + "sinks": [ + { + "name": "console", + "type": "console", + "format": "text" + }, + { + "name": "json", + "type": "json", + "destination": {"type": "file", "path": "logs/app.jsonl"}, + "redactedKeys": ["authorization", "password", "token"], + "fields": { + "time": "time", + "level": "severity", + "logger": "logger", + "message": "message", + "mdc": "mdc", + "markers": "markers", + "exception": "exception", + "source": "source" + } + }, + { + "name": "async-json", + "type": "async", + "delegate": "json", + "capacity": 8192, + "onOverflow": "drop-newest" + } + ], + "loggers": [ + { + "name": "com.acme.payments", + "level": "Debug", + "sinks": ["async-json", "console"] + } + ] + } +} +``` + +This is the same information as the Logback XML, but the names line up with XDK +objects: sinks, levels, destinations, redaction, field names, and logger-name routing. + +## Recommended XDK configuration inputs + +The POC docs use JSON for the full backend graph. XML is familiar from Logback, but JSON +fits the XDK better because it maps directly to typed Ecstasy config objects and to the +same JSON tooling used for service configuration. A production container should accept +these inputs in a predictable order: + +1. Built-in defaults: console output, root `Info`, no cloud sink. +2. Packaged `logging.json` or `slogging.json` next to the application. +3. External file from `--logging.config=...`, `--slogging.config=...`, + `XTC_LOGGING_CONFIG`, or `XTC_SLOGGING_CONFIG`. +4. Environment/property overrides for deployment-specific values. +5. Command-line overrides for the last-mile operator change. + +For the eventual single XDK API, only the `logging.*` namespace should remain. While +this branch contains both POC libraries, the examples use `logging.*` for +`lib_logging` and `slogging.*` for `lib_slogging` so reviewers can compare them without +name collisions. + +Common property overrides should look like this: + +```properties +logging.config=/etc/acme/logging.json +logging.rootLevel=Info +logging.logger.com.acme.payments.level=Debug +logging.logger.com.acme.audit.level=Info +logging.sink.file-json.path=/var/log/acme/payments.jsonl +logging.sink.cloud.provider=gcp +logging.sink.cloud.project=acme-prod +logging.sink.cloud.service=payments +logging.sink.async-cloud.capacity=8192 +logging.redactedKeys=authorization,password,token +``` + +The same settings should be accepted as environment variables in deployment systems that +prefer env configuration: + +```bash +XTC_LOGGING_CONFIG=/etc/acme/logging.json +XTC_LOGGING_ROOT_LEVEL=Info +XTC_LOGGING_LOGGER_COM_ACME_PAYMENTS_LEVEL=Debug +XTC_LOGGING_SINK_CLOUD_PROVIDER=gcp +XTC_LOGGING_SINK_CLOUD_PROJECT=acme-prod +XTC_LOGGING_SINK_CLOUD_SERVICE=payments +XTC_LOGGING_REDACTED_KEYS=authorization,password,token +``` + +Command-line overrides should be reserved for operational changes: + +```bash +xtc run app.xtc \ + --logging.config=/etc/acme/logging.json \ + --logging.level.root=Warn \ + --logging.level.com.acme.payments=Debug \ + --logging.sink.cloud.provider=gcp \ + --logging.sink.cloud.project=acme-prod +``` + +The slog-shaped POC uses the same deployment idea with handler vocabulary: + +```properties +slogging.config=/etc/acme/slogging.json +slogging.rootLevel=Info +slogging.rule.component.com.acme.payments.level=Debug +slogging.handler.file-json.path=/var/log/acme/payments.jsonl +slogging.handler.cloud.provider=aws +slogging.handler.cloud.logGroup=/acme/payments +slogging.handler.async-cloud.capacity=8192 +slogging.attr.service=payments +``` + +```bash +xtc run app.xtc \ + --slogging.config=/etc/acme/slogging.json \ + --slogging.level.root=Info \ + --slogging.rule.component.com.acme.payments=Debug \ + --slogging.handler.cloud.provider=aws +``` + +The rule is: JSON describes the graph; properties, env vars, and command-line args patch +the graph. They should not become a second full configuration language. + +## What is implemented vs proposed + +The examples below deliberately mix shipped POC types with the next layer that would +make logging a production XDK subsystem. The boundary should be explicit: + +| Layer | Status in this branch | Recommended XDK home | +|---|---|---| +| Caller facade: `Logger`, levels, message formatting/attrs, MDC/context, markers where applicable | Implemented in `lib_logging` and `lib_slogging`. | The winning API becomes the first-class `lib_logging` module. The losing POC should not remain a peer facade. | +| Backend SPI: `LogSink` for `lib_logging`, `Handler` for `lib_slogging` | Implemented. | Stays in core `lib_logging`, because custom backends need the SPI without importing a config system. | +| Base backend primitives with no deployment dependencies | Implemented: `ConsoleLogSink`, `JsonLogSink`, `CompositeLogSink`, `HierarchicalLogSink`, `AsyncLogSink`, `MemoryLogSink`; slog equivalents include `TextHandler`, `JSONHandler`, `AsyncHandler`, `MemoryHandler`, `BoundHandler`. | Stays in core `lib_logging` if the dependency footprint remains small. | +| File destinations, rolling files, cloud destinations, provider clients | Not implemented here. The examples use names such as `FileJsonLogSink`, `CloudLogSink`, and `CloudHandler` as sketches. | Optional first-class XDK backend libraries, for example `lib_logging_file` and `lib_logging_cloud`, or provider-specific modules such as `lib_logging_gcp`, `lib_logging_aws`, and `lib_logging_azure` if dependencies are heavy. | +| JSON config schema, config parser, env/property/CLI override merge, file watching, hot reload services | Not implemented here. The examples use `LoggingConfig`, `SLoggingConfig`, `ConfiguredLogSink`, and `ConfiguredHandler` as sketches. | A first-class XDK config library, for example `lib_logging_config`, importing `logging`, `json`, and whichever backend modules it can instantiate. | +| Runtime defaults: automatic logger naming and automatic source-location capture | Partially sketched. Runtime injection works for the manual interpreter demo; automatic naming/source capture remain compiler/runtime work. | Runtime/compiler, not the logging library itself. The library already has explicit fields/methods for the lowered values. | + +If the slog-shaped API wins, the module split should be the same after renaming the +chosen facade to `lib_logging`: core logger/handler types in `lib_logging`, config +loading in `lib_logging_config`, and optional file/cloud handlers in backend libraries. + +## Log4j 2 and Spring Boot comparison + +Log4j 2 proves that XML is not required for a serious backend. The same appender/logger +graph can be written in JSON: + +```json +{ + "Configuration": { + "Appenders": { + "Console": { + "name": "STDOUT", + "PatternLayout": {"pattern": "%d [%t] %p %c - %m%n"} + }, + "File": { + "name": "JSON", + "fileName": "logs/app.jsonl", + "JsonTemplateLayout": {} + } + }, + "Loggers": { + "Logger": { + "name": "com.acme.payments", + "level": "debug", + "AppenderRef": [{"ref": "JSON"}, {"ref": "STDOUT"}] + }, + "Root": { + "level": "info", + "AppenderRef": {"ref": "STDOUT"} + } + } + } +} +``` + +Spring Boot proves that operators also need simple overrides for common cases: + +```properties +logging.level.root=warn +logging.level.com.acme.payments=debug +logging.file.name=/var/log/acme/app.log +logging.config=classpath:logback-spring.xml +``` + +The Ecstasy proposal should copy that operational convenience without copying Spring or +Logback internals. A minimal XDK equivalent is: + +```properties +logging.level.root=Warn +logging.level.com.acme.payments=Debug +logging.sink.file-json.path=/var/log/acme/app.jsonl +logging.config=/etc/acme/logging.json +``` + +That is enough for "turn on debug for this component now" while the JSON file remains +the source of truth for multi-sink, cloud, redaction, and formatter decisions. + +## How this maps to `lib_logging` + +The current POC already has the backend building blocks: + +| Config concept | Status | Type / future home | +|---|---|---| +| Console appender | Implemented | `ConsoleLogSink` in `lib_logging` | +| JSON encoder / JSON appender | Implemented for console-style JSON lines | `JsonLogSink` + `JsonLogSinkOptions` in `lib_logging` | +| Async appender | Implemented | `AsyncLogSink` in `lib_logging` | +| Multiple appenders | Implemented | `CompositeLogSink` in `lib_logging` | +| Per-logger level tree | Implemented | `HierarchicalLogSink` in `lib_logging` | +| Test/list appender | Implemented | `MemoryLogSink` in `lib_logging` | +| File output / rolling file output | Proposed | `FileJsonLogSink` or `RollingFileLogSink` in `lib_logging_file` | +| Cloud output | Proposed | `CloudLogSink` plus provider clients in `lib_logging_cloud` or provider modules | +| Config JSON loader and override merge | Proposed | `LoggingConfig` and loader services in `lib_logging_config` | +| Dynamic reload wrapper | Proposed | `ConfiguredLogSink` / `ReloadableLogSink` in `lib_logging_config` | + +A config loader can build the sink graph from JSON: + +```ecstasy +LogSink console = new ConsoleLogSink(Info); + +LogSink json = new JsonLogSink(new JsonLogSinkOptions( + Info, + ["authorization", "password", "token"])); + +LogSink asyncJson = new AsyncLogSink(json, 8192); + +LogSink paymentsFanout = new CompositeLogSink([asyncJson, console]); + +HierarchicalLogSink configured = new HierarchicalLogSink(paymentsFanout, Info); +configured.setLevel("com.acme.payments", Debug); +``` + +Dynamic reload should be handled by a stable service owned by the host container. The +shape is: + +```ecstasy +service ReloadableLogSink(LogSink initial) + implements LogSink { + private LogSink current = initial; + + void replace(LogSink next) { + current = next; + } + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return current.isEnabled(loggerName, level, marker); + } + + @Override + void log(LogEvent event) { + current.log(event); + } +} +``` + +The runtime injects `BasicLogger(..., reloadable)`. When `logging.json` changes, the +host parses the new file, builds a new `LogSink` graph, and calls `reloadable.replace`. +Existing injected loggers keep working because their sink object is stable. + +Adding a sink dynamically is the same operation: build a new graph containing the new +sink and swap it into the reloadable wrapper. A richer `ConfiguredLogSink` service could +also expose direct methods: + +```ecstasy +configured.addSink("audit-json", new AsyncLogSink(new JsonLogSink(auditOptions))); +configured.attachSink("com.acme.audit", "audit-json"); +configured.setLevel("com.acme.audit", Info); +``` + +Those methods belong in a future configuration module, not in the caller-facing +`Logger` API. + +## Self-contained `lib_logging` dynamic backend sketch + +This example shows the production shape reviewers should expect from the +SLF4J/Logback-shaped path: + +- root level is `Info`; +- `com.acme.payments` is set to `Debug`; +- payment logs tee to a local file and a cloud logging sink; +- audit logs can be added dynamically without changing caller code; +- the active backend can be rebuilt from `logging.json` and swapped at runtime. + +Everything in this sketch that implements `LogSink` uses the already-shipped extension +point. `ConsoleLogSink`, `JsonLogSink`, `CompositeLogSink`, `HierarchicalLogSink`, and +`AsyncLogSink` exist today. `FileJsonLogSink`, `CloudLogSink`, `CloudLogClient`, +`LoggingConfig`, and `ConfiguredLogSink` are proposed first-class backend/config types; +they should not be added to application code. + +```json +{ + "version": 1, + "logging": { + "rootLevel": "Info", + "sinks": [ + {"name": "console", "type": "console"}, + { + "name": "file-json", + "type": "file", + "path": "logs/payments.jsonl", + "format": "json", + "redactedKeys": ["authorization", "password", "token"] + }, + { + "name": "cloud", + "type": "cloud", + "provider": "gcp", + "project": "acme-prod", + "service": "payments", + "resource": {"type": "k8s_container"}, + "traceKeys": {"trace": "traceId", "span": "spanId"} + }, + { + "name": "async-cloud", + "type": "async", + "capacity": 8192, + "delegate": "cloud" + }, + { + "name": "payments-tee", + "type": "composite", + "sinks": ["file-json", "async-cloud", "console"] + } + ], + "loggers": [ + {"name": "com.acme.payments", "level": "Debug", "sinks": ["payments-tee"]}, + {"name": "com.acme.audit", "level": "Info", "sinks": ["async-cloud"]} + ] + } +} +``` + +The proposed cloud sink API shape is deliberately small: + +```ecstasy +interface CloudLogClient { + void write(CloudLogEntry entry); +} + +const CloudLogEntry( + String provider, + String severity, + String message, + Map payload, + Map labels = [], + String? trace = Null, + String? spanId = Null, + ); + +const CloudLogSink(CloudLogClient client, String provider, String service) + implements LogSink { + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return True; + } + + @Override + void log(LogEvent event) { + Map payload = new ListMap(); + payload.put("logger", event.loggerName); + payload.put("message", event.message); + payload.putAll(event.keyValues); + payload.put("mdc", event.mdcSnapshot); + + Map labels = new ListMap(); + for (Marker marker : event.markers) { + labels.put(marker.name, "true"); + } + + client.write(new CloudLogEntry( + provider, + event.level.name, + event.message, + payload, + labels, + trace = event.mdcSnapshot.getOrNull("traceId"), + spanId = event.mdcSnapshot.getOrNull("spanId"))); + } +} +``` + +A future `lib_logging_config` module would build and own the dynamic graph: + +```ecstasy +service ConfiguredLogSink(LogSink initial) + implements LogSink { + private LogSink current = initial; + + void reload(LoggingConfig config, CloudLogClient cloud) { + LogSink fileJson = new FileJsonLogSink(config.file("file-json")); + LogSink cloudSink = new CloudLogSink(cloud, config.provider, config.service); + LogSink asyncCloud = new AsyncLogSink(cloudSink, config.capacity("async-cloud")); + LogSink tee = new CompositeLogSink([fileJson, asyncCloud, new ConsoleLogSink()]); + + HierarchicalLogSink next = new HierarchicalLogSink(tee, config.rootLevel); + next.setLevel("com.acme.payments", Debug); + next.setLevel("com.acme.audit", Info); + + current = next; + } + + void addAuditCloudSink(CloudLogClient auditCloud) { + LogSink audit = new AsyncLogSink(new CloudLogSink(auditCloud, "gcp", "audit")); + // A production implementation would rebuild a named sink graph here; this sketch + // shows the operation belongs to the backend service, not caller code. + current = new CompositeLogSink([current, audit]); + } + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return current.isEnabled(loggerName, level, marker); + } + + @Override + void log(LogEvent event) { + current.log(event); + } +} +``` + +The cloud client is provider-specific, but the sink boundary is not: + +- **GCP Cloud Logging:** map `Level` to `severity`, `message` to the displayed message, + MDC/key-values to JSON payload, markers to labels, and trace/span IDs from MDC. +- **AWS CloudWatch Logs:** emit one JSON event to a log group/log stream or stdout for + the platform collector; keep level, logger, request ID, and trace ID as indexed fields. +- **Azure Monitor / Application Insights:** map level to severity, message to trace text, + MDC/key-values/markers to custom properties, and trace/span IDs to operation context. + +The caller sees none of this: + +```ecstasy +@Inject Logger logger; +@Inject MDC mdc; + +mdc.put("requestId", request.id); +mdc.put("traceId", request.traceId); + +logger.named("com.acme.payments.stripe") + .atInfo() + .addKeyValue("paymentId", payment.id) + .addKeyValue("amount", payment.amount) + .log("payment processed"); +``` + +## How this maps to Go slog / `lib_slogging` + +Go `log/slog` does not define a standard configuration-file format comparable to +`logback.xml`. Go programs usually construct a `Handler` graph from application config +or environment variables. The standard API provides `Handler`, `HandlerOptions`, +`TextHandler`, `JSONHandler`, `WithAttrs`, and `WithGroup`; configuration is ordinary Go +code around those pieces. + +The POC already has the slog-shaped core pieces: + +| Config concept | Status | Type / future home | +|---|---|---| +| Logger facade and attrs | Implemented | `slogging.Logger`, `Attr`, `Record`, `Level` in `lib_slogging` | +| Handler SPI | Implemented | `Handler` and `BoundHandler` in `lib_slogging` | +| Text and JSON output | Implemented | `TextHandler`, `JSONHandler`, `HandlerOptions` in `lib_slogging` | +| Async wrapper | Implemented | `AsyncHandler` in `lib_slogging` | +| Test/discard handlers | Implemented | `MemoryHandler`, `NopHandler` in `lib_slogging` | +| Request logger context | Implemented | `LoggerContext` in `lib_slogging` | +| Handler fanout | Proposed | `CompositeHandler` in core `lib_logging` if slog wins, or in `lib_logging_config` if kept config-only | +| File output / rolling file output | Proposed | `FileJSONHandler` or `RollingFileHandler` in `lib_logging_file` | +| Cloud output | Proposed | `CloudHandler` plus provider clients in `lib_logging_cloud` or provider modules | +| Attr-based dynamic level routing | Proposed | `AttrLevelRoutingHandler` in `lib_logging_config` if slog wins | +| Config JSON loader and override merge | Proposed | `SLoggingConfig`-equivalent config objects in `lib_logging_config` if slog wins | + +The equivalent Ecstasy JSON for the slog-shaped POC would configure handlers instead of +sinks: + +```json +{ + "version": 1, + "slogging": { + "handler": { + "type": "async", + "capacity": 8192, + "delegate": { + "type": "json", + "rootLevel": "Info", + "redactedKeys": ["authorization", "password", "token"], + "includeSource": true, + "fields": { + "time": "time", + "level": "severity", + "message": "message", + "source": "source", + "exception": "exception" + } + } + }, + "context": { + "attrs": [ + {"key": "service", "value": "payments"} + ] + } + } +} +``` + +Programmatic construction looks like this: + +```ecstasy +Handler json = new JSONHandler(new HandlerOptions( + Level.Info, + ["authorization", "password", "token"])); + +Handler async = new AsyncHandler(json, 8192); +Logger logger = new Logger(async) + .with([Attr.of("service", "payments")]); +``` + +Dynamic reload uses the same stable-wrapper idea: + +```ecstasy +service ReloadableHandler(Handler initial) + implements Handler { + private Handler current = initial; + + void replace(Handler next) { + current = next; + } + + @Override + Boolean enabled(Level level) = current.enabled(level); + + @Override + void handle(Record record) { + current.handle(record); + } + + @Override + Handler withAttrs(Attr[] attrs) { + return new BoundHandler(this, attrs); + } + + @Override + Handler withGroup(String name) { + return new BoundHandler(this, name); + } +} +``` + +The important difference from Logback is category routing. slog does not have a +first-class logger-name hierarchy. You can still configure handlers by attrs: + +```json +{ + "rules": [ + {"match": {"component": "com.acme.payments"}, "level": "Debug"} + ] +} +``` + +but that is an application/backend convention, not a built-in Logback-compatible +longest-prefix logger tree. + +## Self-contained `lib_slogging` dynamic handler sketch + +The slog-shaped version is the same backend idea with different vocabulary: + +- configure a handler graph instead of a sink graph; +- use attrs for service/component/request context; +- tee records to file and cloud through a composite handler; +- apply level rules by attr matching, not logger-name prefix matching. + +As above, the core handler API is implemented. `JSONHandler`, `TextHandler`, +`AsyncHandler`, `HandlerOptions`, `Attr`, and `Logger.with(...)` exist today. +`CompositeHandler`, `FileJSONHandler`, `CloudHandler`, `AttrLevelRoutingHandler`, and +the JSON/property/CLI config loader are proposed additions. If the slog shape wins, +they should graduate under `lib_logging` / `lib_logging_config`, not remain under a +permanent `lib_slogging` name. + +```json +{ + "version": 1, + "slogging": { + "rootLevel": "Info", + "handlers": [ + { + "name": "file-json", + "type": "file", + "path": "logs/payments.jsonl", + "format": "json", + "redactedKeys": ["authorization", "password", "token"] + }, + { + "name": "cloud", + "type": "cloud", + "provider": "aws", + "logGroup": "/acme/payments", + "service": "payments", + "traceKeys": {"trace": "traceId", "span": "spanId"} + }, + { + "name": "async-cloud", + "type": "async", + "capacity": 8192, + "delegate": "cloud" + }, + { + "name": "tee", + "type": "composite", + "handlers": ["file-json", "async-cloud"] + } + ], + "rules": [ + {"match": {"component": "com.acme.payments"}, "level": "Debug"}, + {"match": {"audit": true}, "level": "Info"} + ], + "context": { + "attrs": [ + {"key": "service", "value": "payments"} + ] + } + } +} +``` + +The proposed handler additions: + +```ecstasy +const CompositeHandler(Handler[] handlers) + implements Handler { + + @Override + Boolean enabled(Level level) { + for (Handler handler : handlers) { + if (handler.enabled(level)) { + return True; + } + } + return False; + } + + @Override + void handle(Record record) { + for (Handler handler : handlers) { + if (handler.enabled(record.level)) { + handler.handle(record); + } + } + } + + @Override + Handler withAttrs(Attr[] attrs) { + return new CompositeHandler(handlers.map(h -> h.withAttrs(attrs))); + } + + @Override + Handler withGroup(String name) { + return new CompositeHandler(handlers.map(h -> h.withGroup(name))); + } +} + +const CloudHandler(CloudLogClient client, String provider, String service) + implements Handler { + + @Override + Boolean enabled(Level level) = True; + + @Override + void handle(Record record) { + Map payload = new ListMap(); + payload.put("message", record.message); + for (Attr attr : record.attrs) { + payload.put(attr.key, attr.value); + } + + client.write(new CloudLogEntry( + provider, + record.level.label, + record.message, + payload, + labels = ["service" = service], + trace = payload.getOrNull("traceId")?.toString(), + spanId = payload.getOrNull("spanId")?.toString())); + } + + @Override + Handler withAttrs(Attr[] attrs) = new BoundHandler(this, attrs); + + @Override + Handler withGroup(String name) = new BoundHandler(this, name); +} +``` + +Dynamic reload again uses a stable handler: + +```ecstasy +service ConfiguredHandler(Handler initial) + implements Handler { + private Handler current = initial; + + void reload(SLoggingConfig config, CloudLogClient cloud) { + Handler fileJson = new FileJSONHandler(config.file("file-json")); + Handler cloudHandler = new AsyncHandler( + new CloudHandler(cloud, config.provider, config.service), + config.capacity("async-cloud")); + + Handler tee = new CompositeHandler([fileJson, cloudHandler]); + current = new AttrLevelRoutingHandler(tee, config.rules, config.rootLevel) + .withAttrs([Attr.of("service", config.service)]); + } + + @Override + Boolean enabled(Level level) = current.enabled(level); + + @Override + void handle(Record record) { + current.handle(record); + } + + @Override + Handler withAttrs(Attr[] attrs) { + return new BoundHandler(this, attrs); + } + + @Override + Handler withGroup(String name) { + return new BoundHandler(this, name); + } +} +``` + +Caller code is still simple: + +```ecstasy +@Inject slogging.Logger logger; + +slogging.Logger payments = logger.with([ + Attr.of("component", "com.acme.payments"), + Attr.of("requestId", request.id), + Attr.of("traceId", request.traceId), +]); + +payments.info("payment processed", [ + Attr.of("paymentId", payment.id), + Attr.of("amount", payment.amount), +]); +``` + +This is cloud-compatible for the same reason as the `lib_logging` version: every field +that cloud providers care about is present as structured data. The trade-off is routing: +the slog-shaped backend can route by attrs, but it does not provide Logback's +standardized hierarchical logger category model. + +## Equivalence table + +| Logback idea | `lib_logging` equivalent | Go slog / `lib_slogging` equivalent | +|---|---|---| +| `` | `LogSink` implementation | `Handler` implementation | +| `` fanout | `CompositeLogSink` | Handler fanout wrapper; Go 1.26 adds `slog.MultiHandler`, and an Ecstasy `CompositeHandler` would be the same shape. | +| `` | `HierarchicalLogSink.setLevel(name, level)` | Attr-based handler rule, e.g. `component == "..."`; not built into slog. | +| `` | `rootLevel` in `ConsoleLogSink`, `JsonLogSinkOptions`, or `HierarchicalLogSink` | `HandlerOptions.rootLevel` | +| `AsyncAppender` | `AsyncLogSink` | `AsyncHandler` | +| JSON encoder | `JsonLogSink` | `JSONHandler` | +| `%X{requestId}` MDC rendering | `MDC` rendered by a sink | `Logger.with([Attr.of("requestId", id)])` or `LoggerContext` | +| `MarkerFilter` | marker-aware `LogSink` | attr filter such as `audit == true` | +| `scan="true"` reload | `ReloadableLogSink.replace(...)` in a config module | `ReloadableHandler.replace(...)` in a config module | + +## Recommendation + +For the canonical `lib_logging` path, use an Ecstasy JSON configuration file or typed +Ecstasy config object rather than Logback XML. Keep the model familiar: + +- named sinks/appenders; +- root level; +- per-logger overrides; +- async wrappers; +- JSON/text formatting options; +- redaction; +- reload by replacing a stable sink service's delegate. + +For the slog-shaped path, document that configuration is handler-graph construction. +It can be driven by the same JSON file style, but it should not pretend to have Logback +logger categories unless the handler explicitly implements attr-based routing. + +The user-facing rule stays the same in both designs: caller code depends on `Logger`; +configuration code owns sinks/handlers. + +--- + +Previous: [`lazy-logging.md`](lazy-logging.md) | Next: [`custom-sinks.md`](custom-sinks.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/custom-handlers.md b/doc/logging/usage/custom-handlers.md new file mode 100644 index 0000000000..f8369e8cf4 --- /dev/null +++ b/doc/logging/usage/custom-handlers.md @@ -0,0 +1,191 @@ +# Writing a custom `Handler` + +> **Audience:** engineers writing a new `Handler` for `lib_slogging`. + +`Handler` is the backend extension point for `lib_slogging`, the same way `LogSink` +is the backend extension point for `lib_logging`. + +Official Go reference: [`slog.Handler`](https://pkg.go.dev/log/slog#Handler). + +## The contract + +```ecstasy +interface Handler { + Boolean enabled(Level level); + void handle(Record record); + Handler withAttrs(Attr[] attrs); + Handler withGroup(String name); +} +``` + +The first two methods are the same basic shape as `LogSink`: cheap enabled check, +then emit the record. The second two methods are the slog-specific part. They are +called when user code derives a logger: + +```ecstasy +Logger requestLog = logger.with([Attr.of("requestId", req.id)]); +Logger grouped = requestLog.withGroup("payments"); +``` + +A handler that only counts/drops records may return `this` from both derivation +methods because attrs do not affect its behavior. A normal emitting handler should +either return `new BoundHandler(this, attrs/name)` for correct default semantics or +pre-render attrs/group prefixes into a handler-specific derived instance so each later +`handle(record)` does less work. + +## `const` or `service`? + +Same rule as `LogSink`: + +| Shape | Use it for | Examples | +|---|---|---| +| `const` | Stateless forwarders and fixed-configuration renderers. | `TextHandler`, `JSONHandler`, `NopHandler`. | +| `service` | Shared mutable state or external resources. | `MemoryHandler`, `AsyncHandler`, file writers, network writers. | + +## Counting records by level + +```ecstasy +service CountingHandler + implements Handler { + + public/private Map counts = new HashMap(); + + @Override + Boolean enabled(Level level) { + return True; + } + + @Override + void handle(Record record) { + counts.process(record.level, e -> { + e.value = e.exists ? e.value + 1 : 1; + return Null; + }); + } + + @Override + Handler withAttrs(Attr[] attrs) = this; + + @Override + Handler withGroup(String name) = this; +} +``` + +Used like: + +```ecstasy +CountingHandler handler = new CountingHandler(); +Logger logger = new Logger(handler); + +logger.info("a"); +logger.warn("b"); + +assert handler.counts[Level.Info] == 1; +assert handler.counts[Level.Warn] == 1; +``` + +The same code exists as a unit-test helper in +[`lib_slogging/src/test/x/SLoggingTest/CountingHandler.x`](../../../lib_slogging/src/test/x/SLoggingTest/CountingHandler.x). + +## Reusing the handler contract checks + +The test suite includes a small `HandlerContract`, modelled on Go's +[`testing/slogtest`](https://pkg.go.dev/testing/slogtest), that third-party handlers +can reuse: + +```ecstasy +MemoryHandler handler = new MemoryHandler(); + +HandlerContract.assertWithAttrsPrepend(handler, () -> handler.records); +handler.reset(); +HandlerContract.assertWithGroupNests(handler, () -> handler.records); +``` + +Your own handler test can pass the handler instance plus a snapshot function that +returns the records it emitted. The contract checks that derived attrs are prepended +and groups are nested before the final handler observes the record. + +## Handler options and async wrapping + +The shipped text and JSON handlers accept `HandlerOptions`, which covers the +production knobs this POC needs without inventing a config-file format: + +```ecstasy +Handler handler = new JSONHandler(new HandlerOptions( + Level.Info, + ["authorization", "password"])); + +Logger logger = new Logger(new AsyncHandler(handler)); +``` + +`HandlerOptions` controls the root threshold, source inclusion, field names, and +key-based redaction. `AsyncHandler` owns the bounded queue and preserves `withAttrs` +/ `withGroup` semantics by wrapping the derived delegate handler. + +## Filtering audit events + +In the slog model, categories are attrs, not markers: + +```ecstasy +service AuditFilteringHandler(Handler delegate) + implements Handler { + + @Override + Boolean enabled(Level level) { + return delegate.enabled(level); + } + + @Override + void handle(Record record) { + for (Attr attr : record.attrs) { + if (attr.key == "audit" && attr.value == True) { + delegate.handle(record); + return; + } + } + } + + @Override + Handler withAttrs(Attr[] attrs) { + return new AuditFilteringHandler(delegate.withAttrs(attrs)); + } + + @Override + Handler withGroup(String name) { + return new AuditFilteringHandler(delegate.withGroup(name)); + } +} +``` + +Call site: + +```ecstasy +logger.info("user signed in", [ + Attr.of("audit", True), + Attr.of("user", userId), +]); +``` + +The SLF4J-shaped equivalent would use a marker. The slog-shaped equivalent keeps the +category in the same `Attr` channel as every other structured field. + +## Operational rules + +| Rule | Reason | +|---|---| +| Keep `enabled` cheap. | It runs before record allocation on every logging call. | +| Keep normal backend failures inside the handler. | A failed remote endpoint should lead to drop, buffer, or fallback behavior, not an application failure. | +| Do not parse `record.message` to recover fields. | Structured data belongs in `Attr` values. | +| Do not add SLF4J-style placeholder formatting to the core handler path. | Migration sugar can live in an adapter; the slog core stays message-plus-attrs. | + +## What a production destination handler still needs + +The shipped `JSONHandler` renders through `lib_json`, preserves nested groups, honors +redaction options, includes source metadata when configured, and represents exceptions +structurally. A production cloud/file/network handler built on the same contract would +add destination configuration (`Console`, file, stream, socket, HTTP exporter), +retry/drop policy, batching, and backend-specific timestamp or severity formatting. + +--- + +Previous: [`custom-sinks.md`](custom-sinks.md) | Next: [`../lib-logging-vs-lib-slogging.md`](../lib-logging-vs-lib-slogging.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/custom-sinks.md b/doc/logging/usage/custom-sinks.md new file mode 100644 index 0000000000..f118a2a688 --- /dev/null +++ b/doc/logging/usage/custom-sinks.md @@ -0,0 +1,270 @@ +# Writing a custom `LogSink` + +> **Audience:** engineers writing a new `LogSink` for `lib_logging` (file, network, cloud, test capture). + +`LogSink` is the API/impl boundary for `lib_logging`. Anything below it is replaceable +without touching caller code. This document walks through writing a custom sink end to +end. + +## The contract + +```ecstasy +interface LogSink { + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null); + void log(LogEvent event); +} +``` + +That's it. Everything else — message substitution, MDC capture, marker filtering, +and disabled-level fast paths — happens in `BasicLogger` above the sink. + +| Method | Responsibility | +|---|---| +| `isEnabled` | Hot-path threshold/category check. It runs once per log statement and must stay cheap. | +| `log` | Emit the already-built event. `LogEvent.message` is substituted, `mdcSnapshot` is captured, and structured data is available in `markers`, `exception`, `keyValues`, and `mdcSnapshot`. | + +## `const` or `service`? + +`BasicLogger` is a `const` and holds its sink in a `LogSink` field. Ecstasy requires +that field to be `Passable`, so every implementation must be either `immutable` (a +`const`) or a `service`. Pick one of the two: + +- **`const`** — for stateless forwarders / pure adapters. Configuration is fixed at + construction (e.g. a `rootLevel` value passed in once). The sink owns no mutable state + shared across fibers. Examples in `lib_logging`: `NoopLogSink`, `ConsoleLogSink`. + Cheap to construct, cheap to pass; methods run on the caller's fiber. +- **`service`** — for sinks that genuinely have shared mutable state: an event buffer, + a counter map, an open file handle, a worker queue. The service mailbox handles + concurrent ingress for free. Examples: `MemoryLogSink`, `AsyncLogSink`, + `HierarchicalLogSink`, and the `FileLogSink` example below. + +If you find yourself reaching for `synchronized` blocks, you want a `service`. If your +sink is "given a `Console`/`Writer`/`Function`, format and forward" — you want a +`const`. The full rule, with reference examples from the wider XDK / platform +ecosystem, is in `../design/design.md` ("Sink type: `const` vs `service`"). + +## A worked example: a counting sink + +The simplest non-trivial sink — counts events per level. Useful for metrics export. + +```ecstasy +service CountingLogSink + implements LogSink { + + public/private Map counts = new HashMap(); + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return True; + } + + @Override + void log(LogEvent event) { + counts.process(event.level, e -> { + e.value = e.exists ? e.value + 1 : 1; + return Null; + }); + } +} +``` + +Wiring it up directly in tests or examples is just constructor injection: + +```ecstasy +CountingLogSink sink = new CountingLogSink(); +Logger logger = new BasicLogger("my.module", sink); +logger.info("hello"); +logger.warn("uh oh"); +assert sink.counts[Info] == 1; +assert sink.counts[Warn] == 1; +``` + +In a hosted application, the same sink becomes the active backend by registering it as +the `Logger` resource supplier for that container. The application code still receives +`@Inject Logger logger;`; only the host's resource map changes. + +## A more realistic example: writing to a file + +```ecstasy +service FileLogSink + implements LogSink { + + @Inject FileSystem fs; + + public/private Level rootLevel = Info; + + private File file; + private Writer writer; + + construct(Path path, Level rootLevel = Info) { + this.rootLevel = rootLevel; + this.file = fs.fileFor(path).ensure(); + this.writer = file.appender(); + } + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= rootLevel.severity; + } + + @Override + void log(LogEvent event) { + StringBuffer buf = new StringBuffer(); + buf.append(event.timestamp.toString()) + .append(' ') + .append(event.level.name.padRight(' ', 5)) + .append(' ') + .append(event.loggerName) + .append(": ") + .append(event.message) + .append('\n'); + writer.print(buf.toString()); + if (Exception e ?= event.exception) { + writer.print(e.toString()).print('\n'); + } + } +} +``` + +Notes: +- `fs` is injected — same pattern as `Console`. The sink doesn't pick file paths; the + caller does. +- The constructor opens the file once. Sinks are expected to be services, so the + writer state is service-local and thread-safe. +- The `level.severity >= rootLevel.severity` pattern is the canonical level filter. + +## Per-logger thresholds (logback-style) + +Many sinks want different thresholds for different logger names — the classic +`com.example.foo at DEBUG, root at INFO` pattern. + +The shipped [`HierarchicalLogSink`](../../../lib_logging/src/main/x/logging/HierarchicalLogSink.x) +implements this longest-prefix policy. The sketch below shows the same mechanics in +one place for readers who want to write a custom variant. + +```ecstasy +service HierarchicalLogSink + implements LogSink { + + @Inject Console console; + + public/private Map perLogger = new HashMap(); + public/private Level rootLevel = Info; + + void setLevel(String prefix, Level level) { + perLogger.put(prefix, level); + } + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= effectiveLevel(loggerName).severity; + } + + @Override + void log(LogEvent event) { + // ...same as ConsoleLogSink, with thresholds applied. + } + + /** + * Walk the logger name from longest prefix to shortest, returning the first match. + * "com.example.foo.bar" matches "com.example.foo" before "com.example". + */ + private Level effectiveLevel(String loggerName) { + String name = loggerName; + while (True) { + if (Level lvl := perLogger.get(name)) { + return lvl; + } + Int dot = name.lastIndexOf('.'); + if (dot <= 0) { + return rootLevel; + } + name = name[0 ..< dot]; + } + } +} +``` + +This is roughly what Logback's `LoggerContext` does internally. Wrapping it as a sink +keeps the rest of the library unaware. + +## A composite sink + +[`CompositeLogSink`](../../../lib_logging/src/main/x/logging/CompositeLogSink.x) is +the built-in equivalent of attaching multiple Logback appenders to one logger: + +```ecstasy +LogSink sink = new CompositeLogSink([ + new ConsoleLogSink(), + new FileLogSink(/var/log/app.log), +]); +``` + +Each delegate still gets its own `isEnabled(...)` decision, so a JSON audit sink and a +human-readable console sink can sit behind the same logger without sharing policy. + +## Filtering by marker + +Markers exist precisely so that sinks can decide based on category, not on message text. + +```ecstasy +service MarkerFilteringLogSink(LogSink delegate, Marker required) + implements LogSink { + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return marker?.contains(required) : False; + } + + @Override + void log(LogEvent event) { + for (Marker m : event.markers) { + if (m.contains(required)) { + delegate.log(event); + return; + } + } + } +} +``` + +Pair this with a `TeeLogSink` to e.g. send everything to the console but only +`AUDIT`-marked events to a separate audit file. + +## Structured / JSON output + +The base library ships [`JsonLogSink`](../../../lib_logging/src/main/x/logging/JsonLogSink.x), +rendered through `lib_json`, with options for field names, redaction, inclusion of MDC +/ markers / key-values / source, and root threshold: + +```ecstasy +LogSink sink = new JsonLogSink(new JsonLogSinkOptions( + Info, + ["authorization", "password"], + "***", + True, True, True, True, + "time", "level", "logger", "message", + "mdc", "markers", "exception", "source")); +``` + +Caller code stays unchanged: `logger.atInfo().addKeyValue("password", "...").log(...)` +emits a structured value, and the sink decides whether to redact it. + +## Operational rules + +| Rule | Reason | +|---|---| +| Keep `isEnabled` close to `O(1)`. | It runs before every enabled or disabled log event; no I/O belongs there. | +| Do not let normal backend failures escape from `log`. | Logging should not break the caller. Drop, buffer, or fall back to stderr when a remote endpoint or file is unavailable. | +| Do not block callers on slow output. | Network, encryption, and disk-heavy paths should sit behind `AsyncLogSink` or a destination-specific async sink. | +| Render `event.message` as-is. | `MessageFormatter` has already performed `{}` substitution and trailing-exception promotion. | +| Render `event.mdcSnapshot`; do not read `MDC` again. | `BasicLogger` captured the immutable per-fiber context before the event crossed the sink boundary. | +| Frame production output. | Line-oriented or otherwise framed output keeps `tail -f`, log shippers, and cloud collectors predictable. | + +The sink can also rely on three upstream guarantees: disabled events are filtered +before formatting, MDC is captured in the event, and SLF4J-style trailing exceptions +have already been promoted to `event.exception`. + +--- + +Previous: [`configuration.md`](configuration.md) | Next: [`custom-handlers.md`](custom-handlers.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/ecstasy-vs-java-examples.md b/doc/logging/usage/ecstasy-vs-java-examples.md new file mode 100644 index 0000000000..0acadf7b46 --- /dev/null +++ b/doc/logging/usage/ecstasy-vs-java-examples.md @@ -0,0 +1,383 @@ +# Ecstasy `lib_logging` vs Java SLF4J — side by side + +> **Audience:** SLF4J users porting code to Ecstasy. Per-feature side-by-side. + +For every API in `lib_logging` this document shows a working SLF4J 2.x example in Java +first, then the equivalent in Ecstasy. The point is to make adoption frictionless: anyone +who has used SLF4J should be able to skim once and be productive. + +## 1. Hello world + +**Java (SLF4J):** + +```java +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class Demo { + private static final Logger log = LoggerFactory.getLogger(Demo.class); + + public static void main(String[] args) { + log.info("Hello {}", "world"); + } +} +``` + +**Ecstasy (`lib_logging`):** + +```ecstasy +module Demo { + package log import logging.xtclang.org; + + void run() { + @Inject log.Logger logger; + logger.info("Hello {}", ["world"]); + } +} +``` + +The equivalent of the SLF4J static-factory line is to inject once and derive the full +logger name you want: + +```ecstasy +@Inject log.Logger root; +log.Logger logger = root.named("Demo"); +``` + +## 2. Level checks (avoid expensive arg construction) + +**Java:** + +```java +if (log.isDebugEnabled()) { + log.debug("computed result: {}", expensiveSerialize(thing)); +} +``` + +**Ecstasy:** + +```ecstasy +if (logger.debugEnabled) { + logger.debug("computed result: {}", [expensiveSerialize(thing)]); +} +``` + +For one-line expensive values, prefer lazy suppliers instead of a guard: + +**Kotlin (`kotlin-logging`):** + +```kotlin +logger.debug { "computed result: ${expensiveSerialize(thing)}" } +``` + +**Java (SLF4J 2.x fluent supplier):** + +```java +log.atDebug() + .addArgument(() -> expensiveSerialize(thing)) + .log("computed result: {}"); +``` + +**Ecstasy (`lib_logging`):** + +```ecstasy +logger.debug(() -> $"computed result: {expensiveSerialize(thing)}"); + +logger.atDebug() + .addLazyArgument(() -> expensiveSerialize(thing)) + .log("computed result: {}"); +``` + +## 3. Logging an exception + +**Java:** + +```java +try { + process(req); +} catch (Exception e) { + log.error("processing failed for {}", req.id(), e); +} +``` + +**Ecstasy:** + +```ecstasy +try { + process(req); +} catch (Exception e) { + logger.error("processing failed for {}", [req.id], cause=e); +} +``` + +Both follow the SLF4J **throwable-promotion rule**: a trailing `Exception`/`Throwable` +that has no matching placeholder is treated as the cause, not as a substitution. + +## 4. Markers + +**Java:** + +```java +import org.slf4j.Marker; +import org.slf4j.MarkerFactory; + +private static final Marker AUDIT = MarkerFactory.getMarker("AUDIT"); +log.info(AUDIT, "user {} signed in", userId); +``` + +**Ecstasy:** + +```ecstasy +@Inject log.MarkerFactory markers; +log.Marker AUDIT = markers.getMarker("AUDIT"); +logger.info("user {} signed in", [userId], marker=AUDIT); +``` + +## 5. MDC + +**Java:** + +```java +import org.slf4j.MDC; + +MDC.put("requestId", req.id()); +try { + log.info("handling request"); + handle(req); +} finally { + MDC.remove("requestId"); +} +``` + +**Ecstasy:** + +```ecstasy +@Inject log.MDC mdc; + +mdc.put("requestId", req.id); +try { + logger.info("handling request"); + handle(req); +} finally { + mdc.remove("requestId"); +} +``` + +## 6. Fluent event builder (SLF4J 2.x) + +**Java:** + +```java +log.atInfo() + .addMarker(AUDIT) + .addKeyValue("requestId", req.id()) + .addKeyValue("user", userId) + .setCause(e) + .log("payment {} failed for {}", paymentId, customer); +``` + +**Ecstasy:** + +```ecstasy +logger.atInfo() + .addMarker(AUDIT) + .addKeyValue("requestId", req.id) + .addKeyValue("user", userId) + .setCause(e) + .log("payment {} failed for {}", paymentId, customer); +``` + +When the level is disabled, the final `log(...)` call short-circuits before formatting, +MDC capture, `LogEvent` allocation, or lazy supplier evaluation. Eager `addArgument` and +`addKeyValue` expressions still run before the method call, as in Java; use +`addLazyArgument` / `addLazyKeyValue` for expensive values. + +## 7. Acquiring a logger by name vs by class + +**Java:** + +```java +Logger byName = LoggerFactory.getLogger("com.example.foo"); +Logger byClass = LoggerFactory.getLogger(MyClass.class); +``` + +**Ecstasy:** + +```ecstasy +@Inject log.Logger root; +log.Logger byName = root.named("com.example.foo"); +log.Logger byClass = root.named(MyClass.path); +``` + +`@Inject("com.example.foo") log.Logger logger;` was intentionally rejected. The runtime +registers one root logger under the normal `@Inject Logger logger;` key; per-name +loggers are derived by API, which keeps the injector simple and mirrors SLF4J's +`LoggerFactory.getLogger(...)` idiom. + +## 8. Configuration: changing the root level + +**Java (`logback.xml`):** + +```xml + + + + + +``` + +**Java (programmatic, Logback):** + +```java +import ch.qos.logback.classic.Level; +import ch.qos.logback.classic.Logger; +import org.slf4j.LoggerFactory; + +((Logger) LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME)).setLevel(Level.DEBUG); +``` + +**Ecstasy (explicit construction with the default `ConsoleLogSink`):** + +```ecstasy +log.LogSink sink = new log.ConsoleLogSink(log.Level.Debug); +log.Logger logger = new log.BasicLogger("com.example", sink); +``` + +For richer per-logger routing use `HierarchicalLogSink` programmatically today. A +future configured backend can add file-based config analogous to Logback's +`JoranConfigurator`; see `../future/logback-integration.md`. + +## 9. Custom appender / sink + +**Java (Logback):** + +```java +public class CountingAppender extends AppenderBase { + private final AtomicLong count = new AtomicLong(); + + @Override + protected void append(ILoggingEvent e) { + count.incrementAndGet(); + } + + public long getCount() { return count.get(); } +} +``` + +Wire it via `logback.xml`: + +```xml + + +``` + +**Ecstasy (`lib_logging`):** + +```ecstasy +service CountingSink + implements log.LogSink { + + public/private Int count = 0; + + @Override + Boolean isEnabled(String name, log.Level level, log.Marker? marker = Null) = True; + + @Override + void log(log.LogEvent event) { + ++count; + } +} +``` + +Wire it by replacing the injected default sink — see `../usage/custom-sinks.md` for the runtime +side of the story. + +## 10. NOP / silent logger (for libraries that opt out) + +**Java:** + +```java +// add slf4j-nop to the classpath, OR: +Logger silent = NOPLogger.NOP_LOGGER; +``` + +**Ecstasy:** + +```ecstasy +log.LogSink silent = new log.NoopLogSink(); +log.Logger logger = new log.BasicLogger("com.example", silent); +``` + +## 11. Capturing events in a test + +**Java (Logback `ListAppender`):** + +```java +ListAppender capture = new ListAppender<>(); +capture.start(); +((Logger) LoggerFactory.getLogger("com.example")).addAppender(capture); + +systemUnderTest.run(); + +assertThat(capture.list) + .hasSize(1) + .extracting(ILoggingEvent::getFormattedMessage) + .containsExactly("processed 42 records"); +``` + +**Ecstasy (`MemoryLogSink`):** + +```ecstasy +log.MemoryLogSink capture = new log.MemoryLogSink(); +log.Logger logger = new log.BasicLogger("com.example", capture); + +systemUnderTest.run(logger); + +assert capture.events.size == 1; +assert capture.events[0].message == "processed 42 records"; +``` + +## 12. Throwable promotion + +**Java:** + +```java +log.warn("cleanup failed", new IOException("disk full")); +// SLF4J knows the Throwable is the cause, not a {} substitution. +``` + +**Ecstasy:** + +```ecstasy +logger.warn("cleanup failed", cause=new IOException("disk full")); +``` + +The `MessageFormatter.format` method also enforces the same rule: if the last entry of +`arguments` is an `Exception` and there is no matching placeholder, it is promoted to +the cause. + +## 13. Parameterized message edge cases + +**Java:** + +```java +log.info("{} of {}", current, total); // both filled +log.info("plain message", arg); // arg ignored — no {} +log.info("{} {} {}", "a", "b"); // last {} stays literal +log.info("escaped \\{}", "x"); // emits "escaped {}" +``` + +**Ecstasy:** + +```ecstasy +logger.info("{} of {}", [current, total]); +logger.info("plain message", [arg]); +logger.info("{} {} {}", ["a", "b"]); +logger.info("escaped \\{}", ["x"]); +``` + +The semantics are the same — `MessageFormatter` is a port of SLF4J's reference behaviour. + +--- + +Related: [`slf4j-parity.md`](slf4j-parity.md), [`injected-logger-example.md`](injected-logger-example.md). Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/injected-logger-example.md b/doc/logging/usage/injected-logger-example.md new file mode 100644 index 0000000000..f40973992a --- /dev/null +++ b/doc/logging/usage/injected-logger-example.md @@ -0,0 +1,299 @@ +# `@Inject Logger` end-to-end — a working example + +> **Audience:** new users. Start here if you've never touched `lib_logging` before. + +This document shows the canonical way an Ecstasy application uses `lib_logging`. The +goal is to be the file someone reads when they ask "okay, but what does it actually +look like in real code?" + +The accompanying executable SLF4J-shaped sample lives at +[`manualTests/src/main/x/TestLogging.x`](../../../manualTests/src/main/x/TestLogging.x). +The slog-shaped comparison sample is +[`manualTests/src/main/x/TestSLogging.x`](../../../manualTests/src/main/x/TestSLogging.x). + +Run them side by side with: + +```shell +./gradlew :manualTests:runOne -PtestName=TestLogging --console=plain +./gradlew :manualTests:runOne -PtestName=TestSLogging --console=plain +``` + +> **Status (2026-05).** Runtime injection of `@Inject Logger logger;` is wired for +> the manual interpreter demo. The injected value is a `BasicLogger`, so per-fiber +> `MDC` survives injection. This POC keeps the public API independent of the temporary +> runtime wiring. + +## API at a glance + +There are exactly two acquisition shapes: + +```ecstasy +@Inject Logger logger; // root logger via injection +Logger payments = logger.named("payments"); // per-name child via the API +``` + +There is **no** `@Inject("payments") Logger logger;` form. Per-name loggers are +*derived* from the injected one, the same way SLF4J users write +`LoggerFactory.getLogger(MyClass.class)` once at the top of a class. + +The examples below `import` the public lib_logging types unqualified by adding +type-level imports next to the `package log import …` alias: + +```ecstasy +package log import logging.xtclang.org; +import log.Logger; +import log.MarkerFactory; +import log.MDC; +import log.Marker; +``` + +You can also keep the `log.` prefix everywhere — both forms compile — but unqualified +reads better in real code, and that's the form below. + +## The smallest possible app + +```ecstasy +module HelloLogging { + package log import logging.xtclang.org; + import log.Logger; + + void run() { + @Inject Logger logger; + logger.info("hello, {}", ["world"]); + } +} +``` + +That's it. There is no factory call, no static initializer, no configuration file. The +runtime container hands the application a `Logger`; the default sink (`ConsoleLogSink`) +emits the line to the platform `Console`. Output: + +``` +2026-04-29T11:23:45.012Z [] INFO logger: hello, world +``` + +## What MDC is, before we use it + +Mapped Diagnostic Context — `MDC` for short — is a small per-fiber string→string +scratchpad that travels with the calling fiber but does not appear in any +function signature. The pattern is: at a request boundary, the framework writes +a few keys (`requestId`, `user`, `tenant`); every subsequent log call inside +that request automatically carries them; at the end of the request the +framework clears them. Without MDC you would either thread `requestId` through +every method as an argument, or lose it. + +It is the same idea as Logback's `MDC` and Log4j 2's `ThreadContext`. + +```ecstasy +@Inject MDC mdc; + +mdc.put("requestId", "r_42"); +logger.info("processing"); // sink sees requestId=r_42 in the event +logger.info("validated"); // same +mdc.clear(); // request boundary +``` + +Three things to know: + +1. **Scope is the calling fiber.** Bindings flow into child fibers spawned + from this one, but mutations from a child do not leak back to the parent. + This is the right propagation for request handlers — child workers see the + parent's context but cannot corrupt it. +2. **Sinks read it, callers write it.** A sink decides whether to render MDC + at all (the JSON sink emits it as `{"mdc": {...}}`; the console sink + appends `[mdc=k=v,…]`); the calling code never has to remember which + sinks care. +3. **Clear at the boundary.** Long-lived fibers should `mdc.remove(key)` or + `mdc.clear()` at the end of the unit of work, otherwise context bleeds + into unrelated events. + +When MDC is *not* the right tool: if the value is data the call needs, pass it +as an argument or a structured key/value pair. MDC is for cross-cutting +identifiers that every event in a request should carry without the calling +code having to know. + +## A more realistic app + +The example below is closer to what real code looks like. It demonstrates: +- a *named* logger acquired by injection; +- per-level emission with parameterized messages; +- attaching an exception via the `cause` parameter; +- using a marker for categorical routing; +- using MDC for request-scoped context (per the section above); +- the SLF4J 2.x fluent event builder. + +```ecstasy +module PaymentService { + package log import logging.xtclang.org; + import log.Logger; + import log.MarkerFactory; + import log.MDC; + import log.Marker; + + @Inject Logger logger; + @Inject MarkerFactory markers; + @Inject MDC mdc; + + Marker audit = markers.getMarker("AUDIT"); + + void run() { + processPayment("p_123", 4200, "EUR", "user_42"); + } + + void processPayment(String paymentId, Int amount, String currency, String userId) { + // Per-request context that should appear on every event below. + mdc.put("paymentId", paymentId); + mdc.put("user", userId); + + try { + logger.info("processing payment of {} {}", [amount, currency]); + + validate(paymentId, amount); + charge(paymentId, amount, currency); + + // Structured + categorical: tag with AUDIT, attach typed KV pairs. + logger.atInfo() + .addMarker(audit) + .addKeyValue("amount", amount) + .addKeyValue("currency", currency) + .log("payment completed"); + + } catch (Exception e) { + logger.error("payment failed: {}", [paymentId], cause=e); + } finally { + mdc.remove("paymentId"); + mdc.remove("user"); + } + } + + void validate(String paymentId, Int amount) { + if (amount <= 0) { + throw new IllegalArgumentException($"amount must be positive: {amount}"); + } + if (logger.debugEnabled) { + logger.debug("validated {} (amount={})", [paymentId, amount]); + } + } + + void charge(String paymentId, Int amount, String currency) { + // ... (calls into payment-processor service) + } +} +``` + +When run with the default `ConsoleLogSink` and root level `Info`, this prints (for a +successful payment): + +``` +2026-04-29T11:23:45.012Z [main] INFO PaymentService: processing payment of 4200 EUR +2026-04-29T11:23:45.034Z [main] INFO PaymentService: payment completed [marker=AUDIT] +``` + +The `validate` call's `debug` line is elided because `Debug < Info`, and crucially the +`logger.debugEnabled` guard keeps the parameter array `[paymentId, amount]` from being +constructed at all. + +## Scaling up — multiple loggers, one per module/class + +In a typical app each scope just has its own `logger` — inject once, use directly. +You only reach for `Logger.named(String)` when you need the *same* enclosing scope to +emit under multiple names. The most common case is one logger per class within a +module: + +```ecstasy +module BillingService { + package log import logging.xtclang.org; + import log.Logger; + + @Inject Logger logger; // injected once for the whole module + + service Invoicer { + Logger logger = BillingService.logger.named("billing.Invoicer"); + + void issue(Invoice inv) { + logger.info("issuing invoice {}", [inv.id]); + // ... + } + } + + service Charger { + Logger logger = BillingService.logger.named("billing.Charger"); + + void charge(Invoice inv) { + logger.info("charging {} for {}", [inv.customer, inv.total]); + // ... + } + } +} +``` + +Each service holds its own `logger` field — call sites still read `logger.info(...)`, +no rename. Both share the injected one's sink, so a configuration-driven sink +(`lib_logging_logback`, future) can set `billing.Charger` to `Debug` while keeping +`billing.Invoicer` at `Info`. This matches SLF4J's `LoggerFactory.getLogger(MyClass)` +idiom one-to-one. + +## Without injection — `LoggerFactory` + +For static initialisers or anywhere injection isn't in scope: + +```ecstasy +module Util { + package log import logging.xtclang.org; + + static log.Logger logger = log.LoggerFactory.getLogger("util.Helper"); + + static void greet(String name) { + logger.info("hello, {}", [name]); + } +} +``` + +`LoggerFactory.getLogger` is itself a service that consults an injected default +`LogSink`, so the per-container override property still holds. + +## Without injection — explicit construction + +Anywhere injection isn't in scope (static initialisers, unit-test fixtures), construct +the logger by hand exactly like the unit tests do: + +```ecstasy +module Today { + package log import logging.xtclang.org; + import log.BasicLogger; + import log.ConsoleLogSink; + import log.Logger; + import log.LogSink; + + void run() { + LogSink sink = new ConsoleLogSink(); + Logger logger = new BasicLogger("Today", sink); + + logger.info("hello, {}", ["world"]); + } +} +``` + +`LoggerFactory.getLogger(...)` is also available for the static-init case. + +## Cheat sheet + +| You want to… | Write | +|---|---| +| Inject the root logger | `@Inject Logger logger;` | +| Get a per-name logger | `Logger payments = logger.named("payments");` | +| Log info | `logger.info("msg")` | +| Log info with args | `logger.info("msg {}", [arg])` | +| Log info with exception | `logger.info("msg", cause=e)` | +| Log info with marker | `logger.info("msg", marker=AUDIT)` | +| Cheap level check | `if (logger.infoEnabled) { ... }` | +| Fluent builder | `logger.atInfo().addMarker(AUDIT).log("msg")` | +| Get a logger by name (no injection) | `LoggerFactory.getLogger("foo")` | +| Attach context for this request | `mdc.put("requestId", id);` | + +That covers ~95% of real-world logging use. + + +--- + +Previous: [`../README.md`](../README.md) | Next: [`structured-logging.md`](structured-logging.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/lazy-logging.md b/doc/logging/usage/lazy-logging.md new file mode 100644 index 0000000000..73f00af38c --- /dev/null +++ b/doc/logging/usage/lazy-logging.md @@ -0,0 +1,175 @@ +# Lazy logging + +> **Audience:** anyone whose log calls have non-trivial argument cost. Mandatory reading if you use `debug` heavily. + +Lazy logging is a core requirement for the XDK logging API, not a future nice-to-have. +Disabled log calls must not build expensive messages, serialize payloads, or compute +structured values just to be dropped by the level check. + +## The requirement + +The accepted logging API **MUST** support one-line lazy logging: + +- message construction runs only after the backend accepts the level; +- structured value construction runs only after the backend accepts the level; +- callers can still use explicit `if (logger.debugEnabled)` guards for multi-statement work; +- the lazy call shape is familiar to Java, Kotlin, Go, and Scala users. + +## Why placeholder formatting is not enough + +SLF4J `{}` formatting defers string substitution, but it does not defer argument +evaluation: + +```java +log.debug("payload {}", expensiveSerialize(payload)); +``` + +`expensiveSerialize(payload)` runs before Java calls `debug(...)`, even when DEBUG is +off. The same is true in Ecstasy for eager arguments: + +```ecstasy +logger.debug("payload {}", [expensiveSerialize(payload)]); +``` + +The `MessageFormatter` work is skipped for disabled events, but the caller expression +already ran. Lazy logging closes that gap. + +## Industry shapes + +| Ecosystem | Lazy mechanism | Design lesson for Ecstasy | +|---|---|---| +| Kotlin logging | `logger.debug { "payload ${expensive()}" }`; inline blocks avoid allocation in Kotlin/JVM. | The call site should stay one line. | +| Java SLF4J 2.x | `log.atDebug().addArgument(() -> expensive()).log("payload {}")` and `log(() -> "...")` on the builder. | Suppliers belong behind the enabled check. | +| Java JUL | `logger.log(Level.FINE, () -> expensiveMessage())`. | Supplier messages are already a standard Java pattern. | +| Go `log/slog` | `Handler.Enabled` runs before record construction; expensive values can implement `slog.LogValuer`. | Attribute-first APIs need lazy values, not message formatting. | +| Scala / Rust / C++ | Macros expand logging calls into guarded code. | Compiler elision is powerful but should not be required for v0. | + +## `lib_logging`: SLF4J/Kotlin-style suppliers + +Use a lazy message supplier when the whole message is expensive: + +```kotlin +// Kotlin +logger.debug { "payload ${serializer.dump(payload)}" } +``` + +```java +// Java JUL / supplier-style APIs +logger.log(Level.FINE, () -> "payload " + serializer.dump(payload)); +``` + +```ecstasy +// Ecstasy lib_logging +logger.debug(() -> $"payload {serializer.dump(payload)}"); +``` + +Use the fluent builder when the message template is cheap but one argument or structured +field is expensive: + +```java +// Java SLF4J 2.x +log.atDebug() + .addArgument(() -> serializer.dump(payload)) + .addKeyValue("requestId", requestId) + .log("payload {}"); +``` + +```ecstasy +// Ecstasy lib_logging +logger.atDebug() + .addLazyArgument(() -> serializer.dump(payload)) + .addKeyValue("requestId", requestId) + .log("payload {}"); +``` + +For expensive structured fields: + +```ecstasy +logger.atDebug() + .addLazyKeyValue("payload", () -> serializer.dump(payload)) + .log("debug payload attached"); +``` + +### How it is implemented + +`BasicLogger` checks `LogSink.isEnabled(name, level, marker)` before invoking a +`MessageSupplier`. `BasicEventBuilder` freezes markers first so service sinks can see the +primary marker during enablement, then checks the sink. Only if enabled does it invoke +lazy message, argument, or key/value suppliers, run `MessageFormatter`, snapshot MDC, and +allocate `LogEvent`. + +Ecstasy cannot overload `addArgument(Object)` with `addArgument(function Object())` +because functions are also objects. The POC therefore uses explicit method names: +`addLazyArgument` and `addLazyKeyValue`. That is slightly less Java-like, but it is clear +at the call site and avoids ambiguous overload rules. + +## `lib_slogging`: slog-style lazy attrs + +Do not use `MessageFormatter` for slogging. The slog model has a complete message plus +structured attrs, so the lazy equivalent is a message supplier and lazy attribute values: + +```go +// Go slog-style idea +logger.Debug("payload", "json", expensiveValueThatMayResolveLazily) +``` + +```ecstasy +// Ecstasy lib_slogging +logger.debug("payload", [ + Attr.lazy("json", () -> serializer.dump(payload)), +]); +``` + +Lazy message construction is also available: + +```ecstasy +logger.debug(() -> $"payload {serializer.dump(payload)}", [ + Attr.of("requestId", requestId), +]); +``` + +`Attr.lazy` is the POC's compact equivalent of Go slog's `LogValuer`: a value wrapper +that resolves only for enabled records. `Logger.emit` performs `Handler.enabled(level)` +first, then resolves lazy attrs, constructs the `Record`, and calls `Handler.handle`. +`BoundHandler` also resolves lazy attrs attached by `logger.with(...)` when an enabled +record is handled. + +One Ecstasy-specific rule matters: `Attr` is a `const`, so the lazy supplier must capture +only passable state. Capturing a service reference or immutable value is fine. Capturing +and mutating an ordinary local variable is not, because the supplier carrier is frozen. +For mutable request state, resolve to a stable value before `Attr.lazy`, capture a service, +or use the direct lazy message overload when the deferred computation does not need to +be stored inside an `Attr`. + +## When to still use explicit guards + +Lazy suppliers are for one-line expensive message/value construction. Use explicit level +guards when the logging side work is multi-step or has side effects: + +```ecstasy +if (logger.debugEnabled) { + DebugSnapshot snapshot = diagnostics.snapshot(order); + logger.debug("order diagnostics {}", [snapshot.summary]); + metrics.debugSnapshotCount++; +} +``` + +The same rule applies to slogging: + +```ecstasy +if (logger.debugEnabled) { + PayloadDump dump = diagnostics.dump(payload); + logger.debug("payload dump", [Attr.of("dump", dump.text)]); +} +``` + +## Compiler optimization path + +The current POC uses runtime function values. That is enough for API semantics and is +the same conceptual shape as Java suppliers. Later compiler work can optimize single-use +non-escaping closures, or add Kotlin-style trailing-block syntax if Ecstasy wants that +surface. The API does not depend on that work. + +--- + +Previous: [`structured-logging.md`](structured-logging.md) | Next: [`configuration.md`](configuration.md) → | Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/platform-and-examples-adaptation.md b/doc/logging/usage/platform-and-examples-adaptation.md new file mode 100644 index 0000000000..9eb058a7d4 --- /dev/null +++ b/doc/logging/usage/platform-and-examples-adaptation.md @@ -0,0 +1,282 @@ +# How `lib_logging` would change `~/src/platform` and `~/src/examples` + +> **Audience:** anyone deciding what migrating real Ecstasy code (the `~/src/platform` repo) costs. + +This is a forward-looking doc surveying two real Ecstasy code bases that already exist +side by side with the XDK in this dev environment, and showing how each would adopt +`lib_logging` if it shipped as part of the XDK today. The point is to validate the API +choices against actual code, not invented snippets. + +The repos surveyed: + +- `~/src/platform/` — the XTC platform host (kernel, account/host managers, OAuth + proxy, CLI tools). +- `~/src/examples/` — the demo apps (CardGame, welcome, shopping, chess-game). + +## What they do today + +Both repos use `@Inject Console console;` and write log-shaped lines manually. They +have invented their own ad-hoc logging vocabularies, and they do not all agree. + +### `~/src/platform/` + +The platform code logs heavily and has crystallised a convention: every log line is +prefixed `Info :`, `Warn :`, or `Error:` and timestamped with a helper from +`common.logTime`. + +`platform/common/src/main/x/common.x:47`: + +```ecstasy +static String logTime(Time? time = Null) { ... } +``` + +Used in `platform/kernel/src/main/x/kernel.x` ~13 times like this: + +```ecstasy +console.print($"{common.logTime($)} Info : Starting the AccountManager..."); +console.print($|{common.logTime($)} Warn : Failed to load the ProxyManager; new \ + instance will be initialized + ); +``` + +`platform/auth/src/main/x/auth/OAuthProvider.x` has 12 such call sites: + +```ecstasy +console.print($"Error: Authentication request to {provider} has timed out"); +console.print("Error: Authentication request rejected: too many attempts"); +console.print($"Info : Requesting authorization from {provider}"); +``` + +`platform/host/src/main/x/host/HostManager.x` collects deferred messages into an +in-memory error list and replays them later: + +```ecstasy +errors.add($|Info : Deployment "{webAppInfo.deployment}" is already active); +// ... +errors.reportAll(msg -> console.print(msg)); +``` + +This is what every codebase that has *almost* invented a logging library looks like. + +### `~/src/examples/` + +The demo apps print to the console in the same way but without the `Info :`/`Warn :` +prefix. Each `Eights` game turn does: + +```ecstasy +console.print($"{player.name} plays {card}"); +console.print($"{player.name} cannot play and passes."); +console.print($"{player.name} draws {card}"); +``` + +These aren't really log lines — they're program output. But they share the same shape +as the platform's `console.print` calls and would benefit from the same separation +between "intentional user-facing output" (`Console`) and "diagnostic event" +(`Logger`). + +## What would change with `lib_logging` + +Three categories of change, depending on what each `console.print` is actually for. + +### 1. Diagnostic logging → `Logger` + +This is the platform repo's bread and butter. The migration is mechanical: + +**Before** (`platform/kernel/src/main/x/kernel.x:149`): + +```ecstasy +@Inject Console console; + +void start() { + console.print($"{common.logTime($)} Info : Starting the AccountManager..."); + // ... + if (!loaded) { + console.print($|{common.logTime($)} Warn : Failed to load the ProxyManager; new \ + instance will be initialized + ); + } + if (errored) { + errors.reportAll(msg -> console.print(msg)); + } +} +``` + +**After**: + +```ecstasy +@Inject log.Logger rootLogger; +log.Logger logger = rootLogger.named("platform.kernel"); + +void start() { + logger.info("Starting the AccountManager..."); + // ... + if (!loaded) { + logger.warn("Failed to load the ProxyManager; new instance will be initialized"); + } + if (errored) { + errors.reportAll(msg -> logger.error(msg)); + } +} +``` + +What we get: +- The timestamp goes away from the call site (the sink renders it). +- `Info :`/`Warn :`/`Error:` text disappears (the sink renders it). +- `common.logTime` becomes dead code; same for any duplicate of it elsewhere. +- The platform host can choose at deploy time whether to render plain text, JSON, or + ship to a remote aggregator — *without changing any caller*. + +### 2. The deferred-error pattern → markers + a memory sink + +The pattern in `host/HostManager.x` — collect errors into a list, replay them later — +maps cleanly onto the `MemoryLogSink` shipped in `lib_logging`. Each `errors.add(...)` +becomes `auditLogger.error(...)` against a logger whose sink is a memory-backed buffer +that the host drains on completion. + +```ecstasy +log.MemoryLogSink deploymentLog = new log.MemoryLogSink(); +log.Logger deployLogger = new log.BasicLogger("host.deploy", deploymentLog); + +// ... do the work ... + +// On completion: +for (log.LogEvent event : deploymentLog.events) { + console.print(event.message); +} +``` + +If categorical filtering matters ("show only audit events to ops"), markers do that: + +```ecstasy +log.Marker AUDIT = markers.getMarker("AUDIT"); +deployLogger.atInfo() + .addMarker(AUDIT) + .log("Deployment \"{}\" is already active", webAppInfo.deployment); +``` + +### 3. Genuine user-facing output → stays on `Console` + +Examples like `~/src/examples/CardGame/src/main/x/cardGame/Eights.x`: + +```ecstasy +console.print($"{player.name} plays {card}"); +``` + +This is *user output*, not a log event. It stays on `Console`. The split is healthy: +`Console` is for "things the user is intentionally meant to see on stdout"; `Logger` is +for "diagnostic events the operator might be interested in". A logging library that is +shaped like SLF4J makes that distinction obvious. + +Mixed cases — say, a CLI that wants progress output during normal runs but verbose +diagnostics under `--verbose` — naturally split into "console for the progress lines, +logger for the diagnostics". CLI tools at any scale benefit from the split. + +## Worked migration: `auth/OAuthProvider.x` + +This is the worst-offending file in the platform repo by call-site count. Twelve +`console.print` lines that all begin with `Info :` / `Error:`. After migration: + +```ecstasy +service OAuthProvider { + @Inject log.Logger rootLogger; + log.Logger logger = rootLogger.named("platform.auth.oauth"); + + void timeout(String provider) { + logger.error("Authentication request to {} has timed out", [provider]); + } + + void rejected() { + logger.error("Authentication request rejected: too many attempts"); + } + + void requesting(String provider) { + logger.info("Requesting authorization from {}", [provider]); + } + + void crossSiteForgery() { + logger.error("Cross-site forgery detected"); + } + + // ... +} +``` + +The lines are shorter, the level is now structured (a sink can filter on it), the file +no longer threads `Console` through every method, and the operator gets to pick +whether the output goes to stdout, a file, or a remote sink. + +## What lands in the platform repo's `common.x` + +Today `common.x` exports `logTime`. Post-migration, that helper goes away, but the +package gains a small canonical `Logger` per subsystem: + +```ecstasy +package common { + package log import logging.xtclang.org; + + @Inject log.Logger rootLogger; + + log.Logger kernelLogger = rootLogger.named("platform.kernel"); + log.Logger authLogger = rootLogger.named("platform.auth"); + log.Logger hostLogger = rootLogger.named("platform.host"); + log.Logger proxyLogger = rootLogger.named("platform.proxy"); + log.Logger cliLogger = rootLogger.named("platform.cli"); +} +``` + +…or each subsystem injects its own logger directly without going through `common`. +Either is fine; the point is that *every* subsystem now has a named logger and there +is no string-prefix convention to maintain. + +## What this means concretely for each repo + +### Platform repo + +- **Files touched:** roughly 10 — anywhere a `console.print($"... Info :")` / + `console.print($"... Error:")` exists. +- **Net code change:** roughly -50 lines (no more timestamp prefix, no more level + string, often no more `console` injection in files that only used it for logging). +- **Configuration model:** the base `ConsoleLogSink`, `JsonLogSink`, + `HierarchicalLogSink`, `CompositeLogSink`, and `AsyncLogSink` cover the common + programmatic cases now. A future configured backend adds file/network destinations, + rolling files, filters, and hot reload. +- **Migration risk:** very low. Each call-site is independent; migrate file by file. +- **Side benefit:** `MemoryLogSink` replaces every ad-hoc `errors.add(...)` / + `errors.reportAll(...)` pattern with one library helper. + +### Examples repo + +- **Files touched:** few. Most `console.print` calls are genuine user output; only + the *non-game-output* messages (test setup, error reporting from the CardGame + shuffler, validation messages in `shopping/cart_api.x`) become logger calls. +- **Net code change:** roughly neutral. The intent is clarity, not LOC. +- **Educational value:** the examples become canonical demonstrations of the API/impl + split — `welcome` could ship with `ConsoleLogSink` by default and `MemoryLogSink` + in tests, side by side. That's a useful lesson for a "new to Ecstasy" reader. + +## Open question + +`lib_logging` is itself an XDK lib (`logging.xtclang.org`). The platform and examples +repos consume the XDK as a published artifact. To use `lib_logging` they need: + +- The next XDK release to ship with the new module included (already done in this + branch — see `xdk/build.gradle.kts` and `xdk/settings.gradle.kts`). +- Each repo's `build.gradle.kts` to add no new dependencies (the XDK already includes + it transitively) — they just `package log import logging.xtclang.org;` and use it. + +That's it. There is no separate Maven coordinate or jar to add. This is the upside of +shipping logging as part of the XDK: every Ecstasy program everywhere gets it for +free. + +## Recommendation + +When `lib_logging` v0 lands and the runtime injection wiring is in place, do a single +PR against the platform repo migrating `kernel.x`, `auth/OAuthProvider.x`, and +`host/HostManager.x` to `Logger`. That covers ~80% of the platform's logging surface +in one chunk and validates the API end-to-end on real code. The examples repo can +follow opportunistically, file by file — there is no urgency, and the gains there are +educational rather than operational. + +--- + +Migration survey. Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/slf4j-parity.md b/doc/logging/usage/slf4j-parity.md new file mode 100644 index 0000000000..077ccb6965 --- /dev/null +++ b/doc/logging/usage/slf4j-parity.md @@ -0,0 +1,131 @@ +# SLF4J 2.x → lib_logging mapping + +> **Audience:** Java/Kotlin/Scala engineers familiar with SLF4J 2.x. Reference doc — scan once, return on demand. + +This document inventories every SLF4J 2.x type and major method that user code is likely +to touch, and shows the equivalent in `lib_logging`. The intent is that an SLF4J user can +scan this once and know everything they need. + +## Acquiring a logger + +| SLF4J 2.x (Java) | lib_logging (Ecstasy) | +|---|---| +| `LoggerFactory.getLogger(MyClass.class)` | `@Inject Logger logger;` then `Logger LOG = logger.named(MyClass.path);` | +| `LoggerFactory.getLogger("com.example.foo")` | `logger.named("com.example.foo")` *(or `LoggerFactory.getLogger("com.example.foo")` when the non-injection factory is wired to a default sink)* | +| (no equivalent) | `@Inject Logger logger;` *(injects a single root logger)* | + +`@Inject("com.example.foo") Logger logger;` was considered and rejected. SLF4J +doesn't have a parameterized injection annotation either; `LoggerFactory.getLogger` +is its idiom and `Logger.named(String)` is the direct Ecstasy equivalent. + +`Logger.named(String)` takes the full logger name. It does not append a suffix to the +current logger name; write `logger.named("billing.Charger")`, not +`logger.named("billing").named("Charger")`, unless the second call also supplies the +full name. + +## `Logger` core methods + +SLF4J's `Logger` defines five families (one per level) each with ~10 overloads. We collapse +each family to one canonical signature using Ecstasy default arguments, plus the fluent +builder for unusual shapes. + +| SLF4J 2.x (Java) | lib_logging (Ecstasy) | +|---|---| +| `void info(String msg)` | `info(message)` | +| `void info(String format, Object arg)` | `info(message, [arg])` | +| `void info(String format, Object arg1, Object arg2)` | `info(message, [arg1, arg2])` | +| `void info(String format, Object... args)` | `info(message, args)` | +| `void info(String msg, Throwable t)` | `info(message, cause=t)` | +| `void info(Marker marker, String msg)` | `info(message, marker=m)` | +| `void info(Marker marker, String format, Object... args)` | `info(message, args, marker=m)` | +| `void info(Marker marker, String msg, Throwable t)` | `info(message, cause=t, marker=m)` | +| `boolean isInfoEnabled()` | `infoEnabled` (property) | +| `boolean isInfoEnabled(Marker marker)` | `isEnabled(Info, marker=m)` | +| `LoggingEventBuilder atInfo()` | `atInfo()` | +| `LoggingEventBuilder atLevel(Level level)` | `atLevel(level)` | +| `String getName()` | `name` (property) | + +Same applies to `trace`, `debug`, `warn`, `error`. + +## `Level` enum + +| SLF4J `org.slf4j.event.Level` | lib_logging `Level` | +|---|---| +| `TRACE` | `Trace` | +| `DEBUG` | `Debug` | +| `INFO` | `Info` | +| `WARN` | `Warn` | +| `ERROR` | `Error` | +| (n/a — sink-side only) | `Off` | + +## `Marker` + +| SLF4J 2.x (Java) | lib_logging (Ecstasy) | +|---|---| +| `MarkerFactory.getMarker("AUDIT")` | `MarkerFactory.getMarker("AUDIT")` | +| `MarkerFactory.getDetachedMarker("X")` | `MarkerFactory.getDetachedMarker("X")` | +| `marker.add(child)`, `marker.remove(child)` | `marker.add(child)`, `marker.remove(child)` | +| `marker.hasReferences()` | `marker.hasReferences` | +| `marker.iterator()` | `marker.references` | +| `marker.contains(other)` | `marker.contains(other)` | +| `marker.contains("NAME")` | `marker.containsName("NAME")` | +| `marker.getName()` | `marker.name` | + +## `MDC` + +| SLF4J 2.x (Java) | lib_logging (Ecstasy) | +|---|---| +| `MDC.put("k", "v")` | `mdc.put("k", "v")` | +| `MDC.get("k")` | `mdc.get("k")` | +| `MDC.remove("k")` | `mdc.remove("k")` | +| `MDC.clear()` | `mdc.clear()` | +| `MDC.getCopyOfContextMap()` | `mdc.copyOfContextMap` | + +In Ecstasy the MDC is acquired by injection (`@Inject MDC mdc;`) rather than via a static +class. The semantics — per-fiber/thread scratchpad readable by sinks — are the same. + +## `LoggingEventBuilder` (SLF4J 2.x fluent API) + +| SLF4J 2.x (Java) | lib_logging (Ecstasy) | +|---|---| +| `b.setMessage("...")` | `b.setMessage("...")` | +| `b.addArgument(x)` | `b.addArgument(x)` | +| `b.addMarker(m)` | `b.addMarker(m)` | +| `b.setCause(e)` | `b.setCause(e)` | +| `b.addKeyValue("k", v)` | `b.addKeyValue("k", v)` | +| `b.log()` | `b.log()` | +| `b.log("msg")` | `b.log("msg")` | +| `b.log("fmt", a)` | `b.log("fmt", a)` | + +## Message format + +SLF4J's `{}` placeholder semantics are reproduced verbatim by `MessageFormatter.format`: + +- Each unescaped `{}` consumes the next argument's `toString()`. +- Excess arguments are dropped; excess placeholders are left literal. +- `\{}` is an escape; the literal `{}` is emitted. +- `\\{}` is an escaped backslash followed by a real placeholder. +- The last argument, if it is an `Exception` and there is no remaining placeholder, is + promoted to the cause. (SLF4J calls this "throwable promotion".) + +## What we don't replicate + +- SLF4J's static `LoggerFactory` returns logger instances unconditionally; ours is a + service that accesses an injected sink. Behaviorally identical for users. +- SLF4J's `EventRecodingLogger` / `SubstituteLoggerFactory` machinery for handling + pre-init log calls — we don't have an init race because injection resolves at scope + entry. +- SLF4J's `Marker.iterator()` is a Java `Iterator`; ours is an Ecstasy + `Iterator`. Identical concept. + +## What we add + +- Native injection (`@Inject Logger logger;`). SLF4J has nothing like this; it relies on + the static factory. Injection makes per-container sink override trivial — see + `../design/design.md` for the documented override pattern. +- `Off` level, useful for sink configuration. SLF4J expresses this through level checks + in `Logger.isEnabledFor(...)`. + +--- + +Reference doc. Related: [`ecstasy-vs-java-examples.md`](ecstasy-vs-java-examples.md). Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/slog-parity.md b/doc/logging/usage/slog-parity.md new file mode 100644 index 0000000000..9d66714a08 --- /dev/null +++ b/doc/logging/usage/slog-parity.md @@ -0,0 +1,182 @@ +# Go `log/slog` → `lib_slogging` mapping + +> **Audience:** Go engineers familiar with `log/slog`. Reference doc — scan once, return on demand. + +This document is the slog-shaped companion to +[`slf4j-parity.md`](slf4j-parity.md). It maps the Go `log/slog` API to the Ecstasy +POC and calls out the differences that are intentional. + +Official reference: Go [`log/slog`](https://pkg.go.dev/log/slog). + +## Acquiring a logger + +| Go `log/slog` | `lib_slogging` | +|---|---| +| `slog.Default()` / top-level `slog.Info(...)` | Not modelled in v0. Ecstasy should prefer injected resources over process-global defaults. | +| `slog.New(handler)` | `new Logger(handler)` | +| `logger.With("requestId", id)` | `logger.with([Attr.of("requestId", id)])` | +| `logger.WithGroup("payments")` | `logger.withGroup("payments")` | +| `slog.NewTextHandler(w, opts)` | `new TextHandler(level)` or `new TextHandler(new HandlerOptions(...))`; output goes through `@Inject Console`. | +| `slog.NewJSONHandler(w, opts)` | `new JSONHandler(level)` or `new JSONHandler(new HandlerOptions(...))`; renders through `lib_json` and writes through `@Inject Console`. | +| framework/request context | `LoggerContext.bind(logger)` / `LoggerContext.currentOr(root)` | + +Runtime `@Inject slogging.Logger logger;` is wired in this branch. The native injector +resolves resources by `(name, requested type)`, so both `logging.Logger logger` and +`slogging.Logger logger` can use the familiar default resource name `logger`. + +Important distinction: Go `slog` has `Logger.With(...)` and `Logger.WithGroup(...)`, +but it does **not** have SLF4J/Logback-style hierarchical logger names. `WithGroup` +qualifies attribute keys for output; it is not the same thing as a logger category such +as `com.acme.payments.stripe` with a Logback longest-prefix level rule. + +## `Logger` methods + +| Go `log/slog` | `lib_slogging` | +|---|---| +| `logger.Debug(msg, args...)` | `logger.debug(message, [Attr.of("k", v)])` | +| `logger.Info(msg, args...)` | `logger.info(message, attrs)` | +| `logger.Warn(msg, args...)` | `logger.warn(message, attrs)` | +| `logger.Error(msg, args...)` | `logger.error(message, attrs, cause=e)` | +| `logger.Log(ctx, level, msg, args...)` | `logger.log(level, message, attrs)` | +| `logger.LogAttrs(ctx, level, msg, attrs...)` | Same `logger.log(level, message, attrs)` path; attrs are already typed. | +| `logger.Enabled(ctx, level)` | `logger.enabled(level)` | +| `AddSource` output | `logger.logAt(level, message, sourceFile, sourceLine, attrs)` | +| lazy message helpers used by wrappers | `logger.debug(() -> expensiveMessage(), attrs)` and the same shape for `info` / `warn` / `error` / `log` / `logAt` | + +The intentional call-shape differences are small: + +| Difference | Rationale | +|---|---| +| Ecstasy uses explicit `Attr.of("key", value)` values instead of alternating `"key", value` pairs. | This removes Go's bad-key case and keeps attrs typed before they reach the handler. | +| Common logging calls do not carry a context parameter. | Ecstasy can use `LoggerContext` (`SharedContext`) for framework/request propagation without adding a context argument to every log call. | +| `error(...)` accepts `cause=` as well as attrs. | `Exception` is first-class in Ecstasy, and handlers/tests often need direct access to the cause. | + +## `Attr` + +| Go `log/slog` | `lib_slogging` | +|---|---| +| `slog.String("user", user)` | `Attr.of("user", user)` | +| `slog.Int("count", count)` | `Attr.of("count", count)` | +| `slog.Bool("audit", true)` | `Attr.of("audit", True)` | +| `slog.Any("err", err)` | `Attr.of("err", e)` or `cause=e` | +| `slog.Group("user", ...)` | `Attr.group("user", [...])` | +| values implementing `slog.LogValuer` | `Attr.lazy("key", () -> expensiveValue)` | + +Go has many typed helper functions because it has no common `Object` parent. Ecstasy +does, so a single `Attr.of` is enough for the POC. A production handler can still +render strings, numbers, booleans, times, durations, exceptions, and groups differently. + +`Attr.lazy` is the POC's `LogValuer` equivalent. `Logger` calls `Handler.enabled(level)` +first, then resolves lazy attrs before constructing the `Record`. Because `Attr` is a +`const`, the supplier must capture passable state: immutable values and service +references are fine; mutating an ordinary local variable from inside the supplier is not. + +## `Handler` + +| Go `log/slog.Handler` | `lib_slogging.Handler` | +|---|---| +| `Enabled(context.Context, Level) bool` | `enabled(Level)` | +| `Handle(context.Context, Record) error` | `handle(Record)` | +| `WithAttrs([]Attr) Handler` | `withAttrs(Attr[])` | +| `WithGroup(string) Handler` | `withGroup(String)` | + +The extra derivation methods are the key architectural difference from +`lib_logging.LogSink`. They let a handler pre-bind attrs or a group once when a +logger is derived, instead of doing that work on every record. The shipped handlers use +`BoundHandler` for the default semantics, and tests include a small +`HandlerContract` helper modelled on Go's `testing/slogtest`. + +`AsyncHandler` can wrap any handler when output is slow: + +```ecstasy +Logger logger = new Logger(new AsyncHandler(new JSONHandler())); +``` + +## No markers or hierarchical names + +Two SLF4J/Logback concepts are intentionally not part of the slog API: + +| SLF4J / Logback concept | Go slog way | `lib_slogging` way | What you lose | +|---|---|---|---| +| `LoggerFactory.getLogger("com.acme.payments")` plus Logback prefix config | `slog.New(handler).With("component", "com.acme.payments")` | `logger.with([Attr.of("component", "com.acme.payments")])` | No built-in logger name field and no standard longest-prefix level tree. A handler can filter attrs, but that is custom policy. | +| `MarkerFactory.getMarker("AUDIT")` and marker filters | `logger.Info("...", "audit", true)` | `logger.info("...", [Attr.of("audit", True)])` | No marker identity, marker containment DAG, or marker-specific enabled check. | + +That trade-off is deliberate. The slog model keeps one structured channel (`Attr`) and +lets handlers decide how to render or filter it. The cost is that Java/Logback teams do +not get direct equivalents for marker filters or category-prefix configuration. + +## Context and source metadata + +The default Ecstasy style is explicit: + +```ecstasy +Logger reqLog = logger.with([Attr.of("requestId", req.id)]); +reqLog.info("processing", [Attr.of("path", req.path)]); +``` + +When framework code needs a request logger to flow through code that does not accept a +logger parameter, bind it with `LoggerContext`: + +```ecstasy +LoggerContext context = new LoggerContext(); + +using (context.bind(reqLog)) { + worker.process^(); +} + +Logger log = context.currentOr(logger); +log.info("inside worker"); +``` + +Source metadata is explicit for now: + +```ecstasy +logger.logAt(Level.Info, "processing", "PaymentService.x", 42, + [Attr.of("requestId", req.id)]); +``` + +That fills `Record.sourceFile` and `Record.sourceLine`, and `JSONHandler` renders it +under the `"source"` object. Automatic call-site capture can target this method later. + +## Levels + +| Go `log/slog` | `lib_slogging` | +|---|---| +| `slog.LevelDebug` (`-4`) | `Level.Debug` | +| `slog.LevelInfo` (`0`) | `Level.Info` | +| `slog.LevelWarn` (`4`) | `Level.Warn` | +| `slog.LevelError` (`8`) | `Level.Error` | +| custom `slog.Level(2)` | `new Level(2, "NOTICE")` | + +This is one place where the slog model is materially different from SLF4J: +levels are open. That is more extensible, but less canonical. + +## Message formatting + +There is no slog equivalent to SLF4J's `MessageFormatter`. This is intentional: + +```ecstasy +// SLF4J-shaped +logger.info("processed {} records", [count]); + +// slog-shaped +logger.info("processed records", [Attr.of("count", count)]); +``` + +The slog design keeps the human message stable and puts variable data in attrs. Reusing +`lib_logging.MessageFormatter` in `lib_slogging` would add positional arguments next +to attrs and remove the model's main simplification. If migration sugar is useful +later, it should live in an adapter module, not in the core slog API. + +## Deferred or intentionally omitted + +| Area | Current branch behavior | +|---|---| +| Process-global functions | Not modelled. Ecstasy should prefer injected resources over `slog.Info(...)`-style globals. | +| Output destinations | `TextHandler` and `JSONHandler` write to injected `Console`; a production backend can add stream/file/socket/HTTP handlers. | +| Automatic source capture | `logAt(...)` is explicit today and is the method future compiler/runtime sugar can target. | +| Handler prefix caching | `BoundHandler` provides correct derivation semantics. High-performance handlers can override `withAttrs` / `withGroup` to cache serialized prefixes. | + +--- + +Reference doc for Go slog migrators. Related: [`../api-cross-reference.md`](../api-cross-reference.md). Up: [`../README.md`](../README.md) diff --git a/doc/logging/usage/structured-logging.md b/doc/logging/usage/structured-logging.md new file mode 100644 index 0000000000..21099eded0 --- /dev/null +++ b/doc/logging/usage/structured-logging.md @@ -0,0 +1,425 @@ +# Structured logging in the logging POC + +> **Audience:** anyone shipping Ecstasy code that needs to emit JSON or talk to a structured log aggregator. + +This document explains structured logging as a requirement, then shows how the same +event looks in Java SLF4J/Logback, Go `log/slog`, Ecstasy `lib_logging`, and Ecstasy +`lib_slogging`. + +"Structured" means a log event carries typed key/value fields in addition to a +human-readable message. Downstream systems can query fields directly instead of +regexing text. + +## Fast path + +The logging proposal treats structured fields as a hard requirement, not an optional +formatter trick: + +- `lib_logging` carries structured fields as `LogEvent.keyValues`, request context as + `MDC`, and categories as `Marker`; `JsonLogSink` renders those fields directly. +- `lib_slogging` carries the same information as `Attr` values; `JSONHandler` renders + attrs and nested groups directly. +- Both designs keep the human message stable and put machine-readable fields beside it, + so cloud logging, search, alerting, and audit pipelines do not need to parse text. + +The examples immediately below show the same payment event in Java SLF4J, Go `slog`, +Ecstasy `lib_logging`, and Ecstasy `lib_slogging`. The later sections explain the +implementation layers and migration rationale. + +## Why structured logging is mandatory now + +A modern logging framework is not just a better `print`. It feeds systems such as +Cloud Logging, CloudWatch, Azure Monitor, OpenSearch, Splunk, Grafana Loki, +OpenTelemetry collectors, alert rules, audit pipelines, and incident dashboards. Those +systems need stable fields. + +Unstructured text is still useful for a human reading a terminal: + +```text +payment processed p_123 amount=4200 currency=EUR merchant=m_42 +``` + +It is weak as production telemetry. Every consumer has to rediscover the schema by +parsing the message. A harmless wording change breaks dashboards and alerts. + +A structured event keeps the human message stable and carries machine-readable fields: + +```json +{ + "time": "2026-04-29T11:23:45.012Z", + "level": "INFO", + "logger": "com.example.PaymentService", + "message": "payment processed", + "paymentId": "p_123", + "amount": 4200, + "currency": "EUR", + "merchantId": "m_42", + "mdc": {"requestId": "r_abc"} +} +``` + +That is queryable: `paymentId="p_123"`, `amount > 1000`, `merchantId="m_42"`, +`requestId="r_abc"`, `level >= ERROR`. Any XDK logging design that cannot carry this +without parsing message text is not production-ready. + +## Same event in the four APIs + +### Java + SLF4J 2.x + +```java +log.atInfo() + .setMessage("payment processed") + .addKeyValue("paymentId", payment.id()) + .addKeyValue("amount", payment.amount()) + .addKeyValue("currency", payment.currency()) + .addKeyValue("merchantId", merchant.id()) + .log(); +``` + +This requires a Logback encoder/appender that reads SLF4J 2.x key/value pairs, for +example a JSON/logstash/cloud encoder. A plain PatternLayout usually renders only the +message unless configured to include key/value data. + +### Go + `log/slog` + +```go +logger.Info("payment processed", + "paymentId", payment.ID, + "amount", payment.Amount, + "currency", payment.Currency, + "merchantId", merchant.ID) +``` + +Go's built-in `JSONHandler` renders attrs as JSON fields. `With(...)` carries +request-wide fields: + +```go +requestLog := logger.With("requestId", req.ID) +requestLog.Info("payment processed", "paymentId", payment.ID) +``` + +### Ecstasy + `lib_logging` + +```ecstasy +logger.atInfo() + .setMessage("payment processed") + .addKeyValue("paymentId", payment.id) + .addKeyValue("amount", payment.amount) + .addKeyValue("currency", payment.currency) + .addKeyValue("merchantId", merchant.id) + .log(); +``` + +Context that applies to many events lives in MDC: + +```ecstasy +@Inject MDC mdc; +mdc.put("requestId", request.id); + +logger.atInfo() + .addKeyValue("paymentId", payment.id) + .log("payment processed"); +``` + +`JsonLogSink` renders MDC and key/value pairs as structured fields. + +### Ecstasy + `lib_slogging` + +```ecstasy +logger.info("payment processed", [ + Attr.of("paymentId", payment.id), + Attr.of("amount", payment.amount), + Attr.of("currency", payment.currency), + Attr.of("merchantId", merchant.id), +]); +``` + +Request-wide fields are attached by deriving a logger: + +```ecstasy +Logger requestLog = logger.with([Attr.of("requestId", request.id)]); +requestLog.info("payment processed", [Attr.of("paymentId", payment.id)]); +``` + +`JSONHandler` renders attrs as JSON fields and preserves nested `Attr.group(...)` +values. + +## How SLF4J 2.x does structured logging + +SLF4J 1.x had no structured-logging story at the API level. You either: + +- abused the message text (`log.info("event=login user={} ip={}", user, ip)`), which works + but every consumer has to parse it back out; or +- depended directly on `logback-classic` and added structured fields through + `org.slf4j.Marker` subclasses or a custom `Encoder`. + +SLF4J 2.x added **first-class key/value pairs** via the fluent `LoggingEventBuilder`. +The relevant types: + +| Type | Role | +|---|---| +| `org.slf4j.spi.LoggingEventBuilder` | Builder returned by `logger.atInfo()` / `atLevel(...)`. Provides `addKeyValue(String, Object)`. | +| `org.slf4j.event.KeyValuePair` | Tiny value type holding `(String key, Object value)`. | +| `org.slf4j.spi.LoggingEventAware` | Marker interface for sinks that want the raw `LoggingEvent` (including KV pairs) instead of just the formatted message. | +| `org.slf4j.event.LoggingEvent.getKeyValuePairs()` | Sink-side accessor. | + +A typical SLF4J 2.x structured log call: + +```java +log.atInfo() + .setMessage("payment processed") + .addKeyValue("paymentId", payment.id()) + .addKeyValue("amount", payment.amount()) + .addKeyValue("currency", payment.currency()) + .addKeyValue("merchantId", merchant.id()) + .log(); +``` + +A Logback encoder that understands KV pairs (e.g. `LogstashEncoder` from the +`logstash-logback-encoder` library) emits this as JSON: + +```json +{ + "@timestamp": "2026-04-29T11:23:45.012Z", + "level": "INFO", + "logger": "com.example.PaymentService", + "message": "payment processed", + "paymentId": "p_123", + "amount": 4200, + "currency": "EUR", + "merchantId": "m_42" +} +``` + +The PatternLayout-based encoders (the default `ch.qos.logback.classic.encoder.PatternLayoutEncoder`) +*don't* surface KV pairs — they only see the formatted message. So in practice, structured +logging in the SLF4J/Logback world means: **caller emits with `addKeyValue`, plus a +KV-aware encoder on the sink side**. + +## Three layers in the SLF4J approach + +It's worth pulling these apart because the same three layers exist in `lib_logging`: + +1. **The call-site API** — how the caller expresses "this event carries these fields". + In SLF4J 2.x: `LoggingEventBuilder.addKeyValue`. KV pairs are typed `Object`, not + `String`, so the caller doesn't have to pre-format. + +2. **The event data model** — how the event carries the fields between the call site + and the sink. In SLF4J: `LoggingEvent.getKeyValuePairs()` returning `List`. + +3. **The sink/encoder** — what actually renders or forwards the structured data. In + SLF4J/Logback: a KV-aware `Encoder` (LogstashEncoder, GelfEncoder, etc.) or a custom + `Appender` that consumes `getKeyValuePairs()` directly. + +Most SLF4J users only see layer 1. Library authors and ops engineers see all three. + +## How `lib_logging` maps onto this + +The same three layers, with the same separation of concerns: + +### Layer 1 — call site + +`LoggingEventBuilder` already has `addKeyValue(String key, Object value)`, mirroring SLF4J: + +```ecstasy +logger.atInfo() + .setMessage("payment processed") + .addKeyValue("paymentId", payment.id) + .addKeyValue("amount", payment.amount) + .addKeyValue("currency", payment.currency) + .addKeyValue("merchantId", merchant.id) + .log(); +``` + +This is the public, stable API — caller code committed to this signature today will not +break when richer sinks arrive. + +### Layer 2 — event data model + +The `LogEvent` const carries a `keyValues` field analogous to SLF4J's +`getKeyValuePairs()`: + +```ecstasy +const LogEvent( + String loggerName, + Level level, + String message, + Time timestamp, + Marker[] markers = [], + Exception? exception = Null, + Object[] arguments = [], + Map mdcSnapshot = [], + String threadName = "", + String? sourceFile = Null, + Int sourceLine = -1, + Map keyValues = [], + ); +``` + +`BasicEventBuilder` accumulates KV pairs in a local `ListMap`, freezes a snapshot, and +passes them through when `log()` materializes the event. Duplicate keys currently use +"last value wins" semantics; the tests pin that behavior down. + +### Layer 3 — sink + +`LogSink` already gets the whole `LogEvent`, so a sink that wants to render KV pairs +just reads `event.keyValues`. The default `ConsoleLogSink` is intentionally simple: it +appends KV pairs as `{key=value, key=value}` after the message, in the same spirit as +Logback's simple text layouts. Structured sinks can render the same fields as real JSON. + +The shipped `JsonLogSink` renders every event as a single JSON object per line: + +```json +{"timestamp":"2026-04-29T11:23:45.012Z","level":"INFO","logger":"com.example.PaymentService","message":"payment processed","paymentId":"p_123","amount":4200,"currency":"EUR","merchantId":"m_42","mdc":{"requestId":"r_abc"}} +``` + +That's a separate `LogSink`, and the runtime chooses whether it's the active sink. +Caller code is unchanged. + +### Configuring `JsonLogSink` + +```ecstasy +LogSink sink = new JsonLogSink(new JsonLogSinkOptions( + Info, + ["authorization", "password"], + "***", + True, True, True, True, + "time", "level", "logger", "message", + "mdc", "markers", "exception", "source")); +``` + +`JsonLogSink` renders through `lib_json`, preserves MDC/markers/key-values/source, and +redacts configured keys before output. + +## How `lib_slogging` maps onto this + +The slog-shaped library collapses message arguments, markers, and per-event key/value +pairs into one concept: `Attr`. + +### Layer 1 — call site + +```ecstasy +logger.info("payment processed", [ + Attr.of("paymentId", payment.id), + Attr.of("amount", payment.amount), + Attr.of("currency", payment.currency), + Attr.of("merchantId", merchant.id), +]); +``` + +For request-scoped fields, derive a logger once: + +```ecstasy +Logger requestLog = logger.with([ + Attr.of("requestId", request.id), + Attr.of("tenant", request.tenant), +]); +``` + +Groups are explicit nested attrs: + +```ecstasy +requestLog.info("payment processed", [ + Attr.group("payment", [ + Attr.of("id", payment.id), + Attr.of("amount", payment.amount), + Attr.of("currency", payment.currency), + ]), +]); +``` + +### Layer 2 — event data model + +`Record` carries the completed message and `Attr[]`: + +```ecstasy +const Record( + Time time, + Level level, + String message, + Attr[] attrs = [], + Exception? exception = Null, + String? sourceFile = Null, + Int sourceLine = -1, + String threadName = "", + ); +``` + +There is no separate marker channel and no separate message-argument channel. That is +the point of the slog design. + +### Layer 3 — handler + +`JSONHandler` renders attrs directly as JSON fields, preserving groups as nested JSON +objects and applying `HandlerOptions` redaction: + +```ecstasy +Handler handler = new JSONHandler(new HandlerOptions( + Level.Info, + ["authorization", "password"])); +Logger logger = new Logger(handler); +``` + +This is conceptually closer to Go's built-in `slog.NewJSONHandler` than to SLF4J's +facade-plus-encoder split. + +## What about `MDC` vs key/value pairs? + +SLF4J has _two_ structured-data channels: + +- **MDC** — process-/fiber-local, set by surrounding code, carried on every event in the + scope. Used for context that doesn't change per call: request ID, tenant, user. +- **Key/value pairs** — per-call, attached to one specific event. Used for facts about + *this* specific log entry: payment ID, amount, target. + +`lib_logging` keeps the same split. `MDC` is its own service; `LoggingEventBuilder.addKeyValue` +is per-event. A structured sink should render both — typically MDC under an `"mdc"` +sub-object and KV pairs at the top level, matching what `LogstashEncoder` does. + +`lib_slogging` keeps the slog shape: request-scoped data is normally attached by +deriving a logger with `with(attrs)`, and per-event data is passed as attrs on the +specific log call. `LoggerContext` is available only when framework code needs +implicit propagation. + +## Migration story for SLF4J users adopting `lib_logging` + +The smallest possible change to a structured-logging-heavy codebase moving from SLF4J to +`lib_logging` is _none_: `atInfo()`/`addKeyValue`/`log()` is identical. Only the import +changes, and the LogSink choice on the deployment side. + +The bigger story — encoder/appender wiring, log aggregator integration — is handled by +the structured sink (`JsonLogSink` in the simple case, a future configured backend in +the complex case). See `../future/logback-integration.md`. + +## Migration story for Go slog users adopting `lib_slogging` + +The conceptual migration is also direct: + +| Go slog | Ecstasy `lib_slogging` | +|---|---| +| `logger.Info("payment processed", "paymentId", id)` | `logger.info("payment processed", [Attr.of("paymentId", id)])` | +| `logger.With("requestId", id)` | `logger.with([Attr.of("requestId", id)])` | +| `logger.WithGroup("payment")` | `logger.withGroup("payment")` | +| `slog.Group("payment", "id", id)` | `Attr.group("payment", [Attr.of("id", id)])` | +| `slog.NewJSONHandler(w, opts)` | `new JSONHandler(new HandlerOptions(...))` | + +The main Ecstasy difference is that attrs are explicit `Attr` values, not alternating +`key, value` arguments. That avoids Go's malformed-argument case and keeps the record +typed before it reaches the handler. + +## What to build first vs. later + +For the next production-oriented cut, the right order is: + +1. Decide whether duplicate structured keys should remain "last value wins" (`Map`) or + preserve duplicates (`KeyValuePair[]`, closer to SLF4J 2.x). +2. Add cloud-oriented layouts that map MDC, markers, and key/value pairs to the field + names expected by GCP / AWS / Azure shippers. + +Everything beyond that — typed value formatting, schema validation, OpenTelemetry +integration, log-correlation IDs propagated across services — is sink-side. The API +is already shaped to support it without changes. + +--- + +Previous: [`injected-logger-example.md`](injected-logger-example.md) | Next: [`lazy-logging.md`](lazy-logging.md) → | Up: [`../README.md`](../README.md) diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 79dd21e570..2ddc2636a0 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -137,6 +137,8 @@ xdk-crypto = { group = "org.xtclang", name = "lib-crypto" } xdk-ecstasy = { group = "org.xtclang", name = "lib-ecstasy" } xdk-json = { group = "org.xtclang", name = "lib-json" } xdk-jsondb = { group = "org.xtclang", name = "lib-jsondb" } +xdk-logging = { group = "org.xtclang", name = "lib-logging" } +xdk-slogging = { group = "org.xtclang", name = "lib-slogging" } xdk-net = { group = "org.xtclang", name = "lib-net" } xdk-oodb = { group = "org.xtclang", name = "lib-oodb" } xdk-sec = { group = "org.xtclang", name = "lib-sec" } diff --git a/javatools/src/main/java/org/xvm/runtime/NativeContainer.java b/javatools/src/main/java/org/xvm/runtime/NativeContainer.java index d3f41633a5..d8b902806e 100644 --- a/javatools/src/main/java/org/xvm/runtime/NativeContainer.java +++ b/javatools/src/main/java/org/xvm/runtime/NativeContainer.java @@ -108,17 +108,19 @@ public NativeContainer(Runtime runtime, ModuleRepository repository) { // ----- initialization ------------------------------------------------------------------------ private ConstantPool loadNativeTemplates() { - ModuleStructure moduleRoot = f_repository.loadModule(ECSTASY_MODULE); - ModuleStructure moduleTurtle = f_repository.loadModule(TURTLE_MODULE); - ModuleStructure moduleNative = f_repository.loadModule(NATIVE_MODULE); + var moduleRoot = f_repository.loadModule(ECSTASY_MODULE); + var moduleTurtle = f_repository.loadModule(TURTLE_MODULE); + var moduleNative = f_repository.loadModule(NATIVE_MODULE); if (moduleRoot == null || moduleTurtle == null || moduleNative == null) { throw new IllegalStateException("Native libraries are missing"); } // "root" is a merge of "native" module into the "system" - FileStructure fileRoot = new FileStructure(moduleRoot, true); + var fileRoot = new FileStructure(moduleRoot, true); fileRoot.merge(moduleTurtle, true, false); + mergeOptionalModule(fileRoot, LOGGING_MODULE); + mergeOptionalModule(fileRoot, SLOGGING_MODULE); fileRoot.merge(moduleNative, true, false); fileRoot.linkModules(f_repository, true); @@ -373,11 +375,147 @@ private void initResources(ConstantPool pool) { xBasicHashCollector templateHashCollector = xBasicHashCollector.INSTANCE; TypeConstant typeHashCollector = templateHashCollector.getCanonicalType(); addResourceSupplier(new InjectionKey("hash", typeHashCollector), templateHashCollector::ensureCollector); + + // +++ logging.Logger, logging.MDC, and slogging.Logger + // TODO: This is not intended as permanent code. It is a hack for the POC of + // injected SLF4J and slog logging and will likely be implemented differently + // and in a more robust manner for the real logging implementation in the XDK. + registerLoggingResources(pool); + registerSLoggingResources(pool); + } + + private void registerLoggingResources(ConstantPool pool) { + if (!moduleAvailable(LOGGING_MODULE)) { + return; + } + + var typeLogger = typeFor(pool, LOGGING_MODULE, "Logger"); + var typeBasic = typeFor(pool, LOGGING_MODULE, "BasicLogger"); + var typeMDC = typeFor(pool, LOGGING_MODULE, "MDC"); + + // Both POC logger types ("logger" + lib_logging.Logger and "logger" + slogging.Logger) + // share the resource name "logger". `addResourceSupplier` permits this because each + // entry's InjectionKey is keyed on (name, type), and `getInjectable` disambiguates + // by type when more than one supplier is registered under one name. + addResourceSupplier(new InjectionKey("logger", typeLogger), + (frame, hOpts) -> ensureLogger(frame, typeBasic, inferLoggerName(frame, "logger"))); + addResourceSupplier(new InjectionKey("mdc", typeMDC), + (frame, hOpts) -> ensureConst(frame, typeMDC)); + } + + private void registerSLoggingResources(ConstantPool pool) { + if (!moduleAvailable(SLOGGING_MODULE)) { + return; + } + + var typeLogger = typeFor(pool, SLOGGING_MODULE, "Logger"); + addResourceSupplier(new InjectionKey("logger", typeLogger), + (frame, hOpts) -> ensureConst(frame, typeFor(frame.poolContext(), + SLOGGING_MODULE, "Logger"))); + } + + private boolean moduleAvailable(String moduleName) { + return f_repository.loadModule(moduleName) != null; + } + + private TypeConstant typeFor(ConstantPool pool, String moduleName, String className) { + var module = pool.ensureModuleConstant(moduleName); + return pool.ensureTerminalTypeConstant(pool.ensureClassConstant(module, className)); + } + + /** + * Construct an instance of `logging.xtclang.org.BasicLogger` named `name`. The + * `BasicLogger(String)` convenience constructor wires a fresh `ConsoleLogSink`, so the + * native side does not have to thread a sink handle through. + */ + private ObjectHandle ensureLogger(Frame frame, TypeConstant typeLogger, String name) { + ClassTemplate template = getTemplate(typeLogger.getSingleUnderlyingClass(true)); + ConstantPool pool = getConstantPool(); + MethodStructure ctor = template.getStructure().findConstructor(pool.typeString()); + ClassComposition clz = template.getCanonicalClass(); + + ObjectHandle[] ahArgs = new ObjectHandle[ctor.getMaxVars()]; + ahArgs[0] = xString.makeHandle(name); + + switch (template.construct(frame, ctor, clz, null, ahArgs, Op.A_STACK)) { + case Op.R_NEXT: + return frame.popStack(); + + case Op.R_CALL: + return new DeferredCallHandle(frame.m_frameNext); + + case Op.R_EXCEPTION: + return new DeferredCallHandle(frame.clearException()); + + default: + throw new IllegalStateException(); + } + } + + /** + * Derive the default logger name for a canonical `logging.Logger` injection. When + * the compiler only supplies the field-name fallback (`logger`), use the enclosing + * method namespace as the logger name. If a future compiler supplies a resource name + * such as the module/class directly, the runtime keeps that explicit name. + */ + private String inferLoggerName(Frame frame, String sName) { + if ("logger".equals(sName) && frame.f_function != null) { + var idMethod = frame.f_function.getIdentityConstant(); + var idNamespace = idMethod.getNamespace(); + if (idNamespace != null) { + var sPath = idNamespace.getPathString(); + if (sPath != null && !sPath.isEmpty()) { + return sPath; + } + + var idModule = idMethod.getModuleConstant(); + if (idModule != null) { + var sModule = idModule.getName(); + if (sModule != null && !sModule.isEmpty()) { + return sModule; + } + } + } + } + return sName; + } + + /** + * Construct an instance of a parameter-less `const` from the lib_logging module via its + * default constructor. Used by the [MDC] supplier; suitable for any other future + * lib_logging const that wants the same default-construction pattern. + */ + private ObjectHandle ensureConst(Frame frame, TypeConstant typeConst) { + ClassTemplate template = getTemplate(typeConst.getSingleUnderlyingClass(true)); + MethodStructure ctor = template.getStructure().findConstructor(); + ClassComposition clz = template.getCanonicalClass(); + + switch (template.construct(frame, ctor, clz, null, + new ObjectHandle[ctor.getMaxVars()], Op.A_STACK)) { + case Op.R_NEXT: + return frame.popStack(); + + case Op.R_CALL: + return new DeferredCallHandle(frame.m_frameNext); + + case Op.R_EXCEPTION: + return new DeferredCallHandle(frame.clearException()); + + default: + throw new IllegalStateException(); + } } /** * Add a native resource supplier for an injection. * + * Maintains two indexes in lockstep: `f_mapResources` keyed by the full + * `(name, type)` `InjectionKey` (the canonical store), and `f_mapResourceNames` + * keyed by name only (the lookup fast path used by `getInjectable`). Most names + * map to a single key; the list-valued shape exists to support cases like the + * two POC logger libraries, which deliberately register distinct types under the + * same resource name "logger". + * * @param key the injection key * @param supplier the resource supplier */ @@ -385,7 +523,7 @@ private void addResourceSupplier(InjectionKey key, InjectionSupplier supplier) { assert !f_mapResources.containsKey(key); f_mapResources.put(key, supplier); - f_mapResourceNames.put(key.f_sName, key); + f_mapResourceNames.computeIfAbsent(key.f_sName, k -> new ArrayList<>()).add(key); } public ObjectHandle ensureOSStorage(Frame frame, ObjectHandle hOpts) { @@ -673,21 +811,31 @@ public ConstantPool getConstantPool() { @Override public ObjectHandle getInjectable(Frame frame, String sName, TypeConstant type, ObjectHandle hOpts) { - InjectionKey key = f_mapResourceNames.get(sName); - if (key == null) { - // for "Nullable" types the NativeContainer can only supply a trivial result; - // anything better than that must be done naturally by a container that hosts the - // calling container - return type.isNullable() ? xNullable.NULL : null; + // Look up by name first via `f_mapResourceNames` — O(1) average — then disambiguate + // by type. The list-per-name shape exists so that two suppliers can share a resource + // name when their types differ; this is what lets `lib_logging.Logger` and + // `slogging.Logger` both register under the name "logger" (see registerLoggingResources + // / registerSLoggingResources). For most names the list has exactly one entry, so the + // inner loop is constant work. + List candidates = f_mapResourceNames.get(sName); + if (candidates != null) { + for (var key : candidates) { + if (matchesType(key.f_type, type)) { + return f_mapResources.get(key).supply(frame, hOpts); + } + } } - // check for equality first, but allow "congruency", "duck type" equality as well or - // sans-Nullable equivalency - TypeConstant typeResource = key.f_type; - return typeResource.equals(type) || typeResource.isEquivalent(type) - || typeResource.isEquivalent(type.removeNullable()) - ? f_mapResources.get(key).supply(frame, hOpts) - : null; + // for "Nullable" types the NativeContainer can only supply a trivial result; + // anything better than that must be done naturally by a container that hosts the + // calling container + return type.isNullable() ? xNullable.NULL : null; + } + + private boolean matchesType(TypeConstant typeResource, TypeConstant typeRequest) { + return typeResource.equals(typeRequest) + || typeResource.isEquivalent(typeRequest) + || typeResource.isEquivalent(typeRequest.removeNullable()); } @Override @@ -728,6 +876,8 @@ public FileStructure createFileStructure(ModuleStructure moduleApp) { // TODO CP/GG: that needs to be reworked (for now the order is critical) fileApp.merge(m_moduleTurtle, false, false); fileApp.merge(f_repository.loadModule("crypto.xtclang.org"), true, false); + mergeOptionalModule(fileApp, LOGGING_MODULE); + mergeOptionalModule(fileApp, SLOGGING_MODULE); fileApp.merge(f_repository.loadModule("net.xtclang.org"), true, false); fileApp.merge(f_repository.loadModule("web.xtclang.org"), true, false); fileApp.merge(m_moduleNative, false, false); @@ -738,6 +888,13 @@ public FileStructure createFileStructure(ModuleStructure moduleApp) { return fileApp; } + private void mergeOptionalModule(FileStructure file, String sModule) { + ModuleStructure module = f_repository.loadModule(sModule); + if (module != null) { + file.merge(module, true, false); + } + } + // ----- helpers ------------------------------------------------------------------------------- @@ -850,6 +1007,8 @@ public String toString() { private static final String ECSTASY_MODULE = Constants.ECSTASY_MODULE; private static final String TURTLE_MODULE = Constants.TURTLE_MODULE; private static final String NATIVE_MODULE = Constants.NATIVE_MODULE; + private static final String LOGGING_MODULE = "logging.xtclang.org"; + private static final String SLOGGING_MODULE = "slogging.xtclang.org"; private static final String TURTLE_PREFIX = "mack."; private static final int TURTLE_LENGTH = TURTLE_PREFIX.length(); private static final String NATIVE_PREFIX = "_native."; @@ -877,12 +1036,16 @@ public String toString() { private final Map f_mapIdByName = new ConcurrentHashMap<>(); /** - * Map of resource names for a name based lookup. + * Map of resources that are injectable from this container, keyed by their InjectionKey. */ - private final Map f_mapResourceNames = new HashMap<>(); + private final Map f_mapResources = new HashMap<>(); /** - * Map of resources that are injectable from this container, keyed by their InjectionKey. + * Name-only index over `f_mapResources`. Lets `getInjectable` do an O(1) average + * lookup by `sName` and then disambiguate by type, instead of scanning every + * supplier on every injection. The list-valued shape supports more than one + * supplier sharing a name (e.g. the two POC logger libraries both registering + * under "logger" with different types). */ - private final Map f_mapResources = new HashMap<>(); -} \ No newline at end of file + private final Map> f_mapResourceNames = new HashMap<>(); +} diff --git a/javatools_bridge/src/main/x/_native.x b/javatools_bridge/src/main/x/_native.x index 95423b58ae..4cb020f9b5 100644 --- a/javatools_bridge/src/main/x/_native.x +++ b/javatools_bridge/src/main/x/_native.x @@ -10,4 +10,4 @@ module _native.xtclang.org { package libcrypto import crypto.xtclang.org; package libnet import net.xtclang.org; package libweb import web.xtclang.org; -} \ No newline at end of file +} diff --git a/lib_logging/README.md b/lib_logging/README.md new file mode 100644 index 0000000000..f87b4ec08f --- /dev/null +++ b/lib_logging/README.md @@ -0,0 +1,19 @@ +# lib_logging + +SLF4J 2.x-shaped injectable logging library for Ecstasy. Acquired by injection: + +```ecstasy +@Inject Logger logger; +logger.info("processed {} records in {}ms", [count, elapsed]); +``` + +Named loggers, levels, parameterized `{}` messages, exception attachment, +markers, MDC, and the SLF4J 2.x fluent event builder. The default +`ConsoleLogSink` writes to the platform `Console`; richer sinks (JSON, async, +fanout, hierarchical-level, file/network) plug in behind the same API. + +## Documentation + +All design docs live at the repo root under [`doc/logging/`](../doc/logging). +Start at [`doc/logging/README.md`](../doc/logging/README.md) — it carries the +goals, requirements, status, reading paths, and the full doc index. diff --git a/lib_logging/build.gradle.kts b/lib_logging/build.gradle.kts new file mode 100644 index 0000000000..5f2bcc475a --- /dev/null +++ b/lib_logging/build.gradle.kts @@ -0,0 +1,11 @@ +plugins { + alias(libs.plugins.xtc) +} + +dependencies { + xdkJavaTools(libs.javatools) + xtcModule(libs.xdk.ecstasy) + xtcModule(libs.xdk.json) + xtcModuleTest(libs.javatools.bridge) + xtcModuleTest(libs.xdk.xunit.engine) +} diff --git a/lib_logging/src/main/x/logging.x b/lib_logging/src/main/x/logging.x new file mode 100644 index 0000000000..f2923ed41f --- /dev/null +++ b/lib_logging/src/main/x/logging.x @@ -0,0 +1,78 @@ +/** + * Corresponds, as a whole module, to the `slf4j-api` jar in the Java ecosystem — plus a + * minimum viable binding (analogous to `slf4j-simple`) so the library is usable out of + * the box. The base module also includes the production building blocks that normally + * sit behind an SLF4J facade: async forwarding, multi-sink fanout, hierarchical logger + * thresholds, and JSON-Lines rendering with redaction knobs. A future + * `lib_logging_logback` module can add external configuration-file loading on top of + * those primitives. + * + * Ecstasy logging library. + * + * The logging library is intentionally shaped to be _instantly familiar_ to anyone who has used + * SLF4J 2.x in the Java ecosystem: named loggers, level checks, parameterized messages with `{}` + * placeholders, exception attachment, markers, MDC, and the SLF4J 2.x fluent event builder API. + * + * The primary entry point is injection: + * + * @Inject Logger logger; + * + * Per-name loggers are derived from the injected logger: + * + * Logger payments = logger.named("com.example.payments"); + * + * Acquiring a logger without injection is also supported via `LoggerFactory`: + * + * Logger logger = LoggerFactory.getLogger("com.example.thing"); + * + * # API / Implementation boundary + * + * The public API consists of: + * - [Logger] — the user-facing facade + * - [Level] — log severities + * - [Marker], [MarkerFactory] + * - [MDC] — mapped diagnostic context + * - [LogEvent] — an immutable record of a single log call + * - [LogSink] — the SPI that backends implement + * - [LoggingEventBuilder] — SLF4J 2.x style fluent API + * - [LoggerFactory] — non-injection acquisition path + * + * The implementation side of that boundary contains: + * - [ConsoleLogSink] — default sink, writes through `@Inject Console` + * - [NoopLogSink] — drops every event + * - [MemoryLogSink] — captures events in memory; useful in tests + * - [JsonLogSink] — JSON-Lines sink rendered by `lib_json` + * - [CompositeLogSink] — Logback-style multi-appender fanout + * - [HierarchicalLogSink] — Logback-style longest-prefix level routing + * - [AsyncLogSink] — bounded async wrapper for slow sinks + * + * A future `lib_logging_logback` module is expected to ship configuration-file loading, + * richer appenders, layouts, filters, and hot reload — see + * `doc/logging/future/logback-integration.md`. + * + * For official SLF4J API links mapped back to these Ecstasy types, see + * `doc/logging/api-cross-reference.md`. + */ +module logging.xtclang.org { + package json import json.xtclang.org; + + /** + * Lazy message supplier used by [Logger] and [LoggingEventBuilder]. + * + * The logging implementation invokes this function only after the active [LogSink] + * accepts the event's level (and primary marker). This mirrors Java `Supplier` + * logging APIs and Kotlin logging blocks, where expensive message construction stays + * behind the level check. + */ + typedef function String() as MessageSupplier; + + /** + * Lazy structured value supplier used by the fluent builder for positional `{}` arguments + * and structured key/value pairs. + * + * The supplier is resolved immediately before [LogEvent] construction, never for a disabled + * event. The returned value must obey the same service-boundary rules as any other log + * argument or structured value. + */ + typedef function Object() as ObjectSupplier; +} diff --git a/lib_logging/src/main/x/logging/AsyncLogSink.x b/lib_logging/src/main/x/logging/AsyncLogSink.x new file mode 100644 index 0000000000..b94da753ad --- /dev/null +++ b/lib_logging/src/main/x/logging/AsyncLogSink.x @@ -0,0 +1,139 @@ +/** + * Logback-style async wrapper for any [LogSink]. + * + * The caller pays only the normal [isEnabled] check and the enqueue operation. A + * background fiber drains the queue and invokes the delegate sink. The [LogEvent] has + * already captured MDC and source metadata before it reaches this sink, so delayed + * emission preserves the caller's context. + * + * This is the POC equivalent of Logback's `AsyncAppender`: + * + * + * 8192 + * + * + * + * Ecstasy equivalent: + * + * LogSink sink = new AsyncLogSink(new JsonLogSink(), capacity=8192); + */ +service AsyncLogSink(LogSink delegate, Int capacity) implements LogSink { + + /** + * Convenience: bounded queue with room for 1024 events. + */ + construct(LogSink delegate) { + construct AsyncLogSink(delegate, 1024); + } + + /** + * Events waiting for the background drain fiber. + */ + private LogEvent[] queue = new LogEvent[]; + + /** + * True while a drain fiber has been scheduled or is running. + */ + private Boolean draining = False; + + /** + * True after [close] has been called. + */ + private Boolean closed = False; + + /** + * Count of events dropped because the queue was full or the sink was closed. + */ + public/private Int droppedCount = 0; + + /** + * Current queue depth. Intended for tests and operational probes. + */ + @RO Int pending.get() = queue.size; + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return delegate.isEnabled(loggerName, level, marker); + } + + @Override + void log(LogEvent event) { + if (closed || queue.size >= capacity) { + ++droppedCount; + return; + } + + queue.add(event); + if (!draining) { + draining = True; + drain^(); + } + } + + /** + * Synchronously drain all currently queued events. + */ + void flush() { + drain(); + } + + /** + * Stop accepting new events. By default this drains pending events first. + */ + void close(Boolean flush = True) { + closed = True; + if (flush) { + drain(); + } else { + droppedCount += queue.size; + queue.clear(); + draining = False; + } + } + + /** + * Drain the queue on this service's fiber. + * + * Two safety properties live here, both worth understanding before changing this code. + * + * 1. **`draining` must be reset on every exit, including exceptions.** + * The whole point of this wrapper is to insulate callers from a misbehaving + * delegate. If a delegate `log(event)` throws, we still need `draining = False` + * afterwards — otherwise `log()` will keep enqueueing events but never schedule + * another drain (because it sees `draining == True`), the queue grows to + * `capacity`, and from then on every event is silently dropped via + * `droppedCount`. The `try/finally` guarantees the flag is cleared. + * + * 2. **Per-event failure is contained.** + * `LogSink.log` is contractually non-throwing (see `LogSink.x`), but a buggy + * custom sink can still throw. We catch per event so that one bad event + * increments `droppedCount` and the rest of the batch still drains. The + * alternative — letting the exception escape and rely on (1) to recover — would + * drop every queued event behind the throwing one. That's a bigger surprise to + * operators than a single counter tick. + * + * The loop uses a swap-out batch instead of `queue.delete(0)` per event. `delete(0)` + * is O(n) on `Array`, so the original loop was O(n²) per drain. The swap takes the + * mutable buffer in one step, then iterates it; new events arriving during the inner + * loop land in a fresh `queue` and are picked up by the outer `while`. Service-fiber + * serialisation means no other `log()` call can run between our read of `queue` and + * our reassignment, so no event is lost. + */ + private void drain() { + try { + while (!queue.empty) { + LogEvent[] batch = queue; + queue = new LogEvent[]; + for (LogEvent event : batch) { + try { + delegate.log(event); + } catch (Exception e) { + ++droppedCount; + } + } + } + } finally { + draining = False; + } + } +} diff --git a/lib_logging/src/main/x/logging/BasicEventBuilder.x b/lib_logging/src/main/x/logging/BasicEventBuilder.x new file mode 100644 index 0000000000..60ba21b860 --- /dev/null +++ b/lib_logging/src/main/x/logging/BasicEventBuilder.x @@ -0,0 +1,262 @@ +/** + * Corresponds to `org.slf4j.spi.DefaultLoggingEventBuilder` — the SLF4J 2.x default + * implementation of the fluent builder. Accumulates state, then materializes a single + * call to the underlying logger. + * + * Default `LoggingEventBuilder` implementation. Accumulates state then forwards to the + * underlying `Logger`'s `log(...)` method. + * + * # Lazy values + * + * Both positional `{}` arguments and structured key/value pairs can be supplied either + * eagerly (the value is already computed) or lazily (a function that runs only after the + * level/marker check accepts the event). Lazy entries are stored as a [LazyValue] + * wrapper inside the same `args` / `keyValues` collections — there are no parallel + * boolean flag arrays to keep in sync. The resolver at `frozenArgs()` / `frozenKVs()` + * does `value.is(LazyValue) ? lv.resolve() : value`. This is also why + * `addLazyArgument` / `addLazyKeyValue` are explicit method names rather than overloads + * of `addArgument` / `addKeyValue`: in Ecstasy a `function Object()` is itself an + * `Object`, so an overload that accepted a supplier could not be reliably distinguished + * from one that accepted the supplier as a literal value. + */ +class BasicEventBuilder(BasicLogger logger, Level level) + implements LoggingEventBuilder { + + private String? message = Null; + private MessageSupplier? lazyMessage = Null; + private Object[] args = new Object[]; + private Marker[] markers = new Marker[]; + private Exception? cause = Null; + private Map keyValues = new ListMap(); + + /** + * Replace the pending message pattern. Mirrors SLF4J + * `LoggingEventBuilder.setMessage`. + */ + @Override + LoggingEventBuilder setMessage(String message) { + this.message = message; + this.lazyMessage = Null; + return this; + } + + /** + * Replace the pending message with a lazily computed message. + */ + @Override + LoggingEventBuilder setMessage(MessageSupplier message) { + this.message = Null; + this.lazyMessage = message; + return this; + } + + /** + * Append one positional `{}` argument. + */ + @Override + LoggingEventBuilder addArgument(Object value) { + args.add(value); + return this; + } + + /** + * Append one lazily computed positional `{}` argument. The supplier runs only after + * the sink accepts the event; see the class doc comment for why this is a separate + * method rather than an overload of `addArgument`. + */ + @Override + LoggingEventBuilder addLazyArgument(ObjectSupplier value) { + args.add(new LazyValue(value)); + return this; + } + + /** + * Attach one marker. Repeated calls produce a multi-marker event. + */ + @Override + LoggingEventBuilder addMarker(Marker marker) { + markers.add(marker); + return this; + } + + /** + * Attach an explicit cause. This wins over SLF4J throwable promotion from + * `arguments`. + */ + @Override + LoggingEventBuilder setCause(Exception cause) { + this.cause = cause; + return this; + } + + /** + * Attach one structured key/value pair for structured sinks. Duplicate keys use + * last-value-wins semantics in the current `Map` representation. + */ + @Override + LoggingEventBuilder addKeyValue(String key, Object value) { + keyValues.put(key, value); + return this; + } + + /** + * Attach one lazily computed structured key/value pair. Stored as a [LazyValue] + * inside the same `keyValues` map; resolved at `frozenKVs()` once the event is + * confirmed enabled. + */ + @Override + LoggingEventBuilder addLazyKeyValue(String key, ObjectSupplier value) { + keyValues.put(key, new LazyValue(value)); + return this; + } + + /** + * Emit using the message set by `setMessage`, if one was supplied. + */ + @Override + void log() { + if (String m ?= message) { + emit(m); + } else if (MessageSupplier supplier ?= lazyMessage) { + emit(supplier); + } + } + + /** + * Emit using `message` as the final message pattern. + */ + @Override + void log(String message) { + emit(message); + } + + /** + * Emit using a lazily computed final message pattern. + */ + @Override + void log(MessageSupplier message) { + emit(message); + } + + /** + * Convenience: append one argument and emit. + */ + @Override + void log(String format, Object arg) { + addArgument(arg); + emit(format); + } + + /** + * Convenience: append two arguments and emit. + */ + @Override + void log(String format, Object arg1, Object arg2) { + addArgument(arg1); + addArgument(arg2); + emit(format); + } + + /** + * Convenience: append all supplied arguments and emit. + */ + @Override + void log(String format, Object[] args) { + for (Object arg : args) { + addArgument(arg); + } + emit(format); + } + + /** + * Emit an eager message after the level/marker check. Lazy args and key/value suppliers are + * intentionally resolved only after this check succeeds. + */ + private void emit(String message) { + Marker[] frozenMarkers = frozenMarkers(); + if (enabled(frozenMarkers)) { + logger.emitWith(level, message, frozenArgs(), cause, frozenMarkers, frozenKVs()); + } + } + + /** + * Emit a lazy message after the level/marker check. This is the builder equivalent of + * SLF4J 2.x `log(Supplier)` and Kotlin logging's lambda block. + */ + private void emit(MessageSupplier message) { + Marker[] frozenMarkers = frozenMarkers(); + if (enabled(frozenMarkers)) { + logger.emitWith(level, message(), frozenArgs(), cause, frozenMarkers, frozenKVs()); + } + } + + /** + * Check the sink using the first marker as the primary marker, matching [BasicLogger.emitWith]. + */ + private Boolean enabled(Marker[] markers) { + Marker? primary = markers.empty ? Null : markers[0]; + return logger.isEnabled(level, primary); + } + + /** + * Materialise the accumulated markers as an immutable, freezing each marker so it can + * cross the sink service boundary. Empty case returns the canonical empty constant. + */ + private Marker[] frozenMarkers() { + if (markers.empty) { + return []; + } + Marker[] frozen = new Array(markers.size); + for (Marker m : markers) { + frozen.add(m.freeze()); + } + return frozen.toArray(Constant); + } + + /** + * Materialise the accumulated structured key/value map as a const-mode (immutable) map + * so the resulting `LogEvent` can cross the sink boundary. Lazy entries (stored as + * [LazyValue] wrappers) are resolved here, after the level/marker check has confirmed + * the event will actually be emitted. SLF4J's reference impl realises a `KeyValuePair` + * list at the same point; here it's a `Map` on `LogEvent.keyValues`. + */ + private Map frozenKVs() { + if (keyValues.empty) { + return []; + } + ListMap snapshot = new ListMap(); + for ((String k, Object v) : keyValues) { + snapshot.put(k, resolve(v)); + } + return snapshot.makeImmutable(); + } + + /** + * Convert the mutable accumulating buffer to a constant-mode (immutable) array so + * it can cross service boundaries on its way to the sink. Lazy entries are resolved + * here for the same reason as in [frozenKVs]. Equivalent to SLF4J's `EventArgArray` + * materialisation step. + */ + private Object[] frozenArgs() { + if (args.empty) { + return []; + } + + Object[] snapshot = new Array(args.size); + for (Object value : args) { + snapshot.add(resolve(value)); + } + return snapshot.toArray(Constant); + } + + /** + * Unwrap a [LazyValue] if present; pass plain values through unchanged. Centralises + * the discrimination so both `frozenArgs` and `frozenKVs` share one rule. + */ + private static Object resolve(Object value) { + if (LazyValue lazy := value.is(LazyValue)) { + return lazy.resolve(); + } + return value; + } +} diff --git a/lib_logging/src/main/x/logging/BasicLogger.x b/lib_logging/src/main/x/logging/BasicLogger.x new file mode 100644 index 0000000000..a8cf1de74e --- /dev/null +++ b/lib_logging/src/main/x/logging/BasicLogger.x @@ -0,0 +1,260 @@ +/** + * Corresponds, role-wise, to a typical SLF4J binding's `Logger` implementation — + * e.g. `org.slf4j.simple.SimpleLogger` (slf4j-simple), `ch.qos.logback.classic.Logger` + * (Logback). Like `org.slf4j.helpers.AbstractLogger`, this class collapses many caller- + * facing overloads down to a single emission path. + * + * Default `Logger` implementation: a thin forwarder over a `LogSink`. + * + * Holds only the logger's name and a reference to the active sink. All level checks, formatting, + * MDC capture, and event construction happens here so that sinks themselves remain dumb. + * + * `BasicLogger` is a `const`, not a `class` or `service`. Method calls therefore execute on + * the caller's fiber, which is essential for [MDC]: a service wrapper would create a new + * fiber per call, and `mdc.copyOfContextMap` invoked from inside that fiber would not see + * tokens registered by the caller. Being a `const` also lets the runtime hand a + * `BasicLogger` straight back from `@Inject Logger logger;` — no bridge service is needed. + * + * User code does not normally instantiate this directly; it is what the runtime hands you in + * response to `@Inject Logger logger;` and what `LoggerFactory.getLogger` returns. + */ +const BasicLogger(String name, LogSink sink, LoggerRegistry? registry) + implements Logger { + + /** + * Convenience: name only. Wires a fresh [ConsoleLogSink] and no registry. This is + * the constructor the runtime calls for `@Inject Logger logger;`. + */ + construct(String name) { + construct BasicLogger(name, new ConsoleLogSink(), Null); + } + + /** + * Convenience: `(name, sink)`. No registry — `named(child)` allocates fresh on each + * call. Used by tests and ad-hoc construction. + * + * The convenience forms are explicit (rather than `registry = Null` defaults on the + * primary constructor) because cross-module default-argument resolution on `const` + * constructors does not always synthesise the shorter form: callers from a + * different module hit "Unresolvable function `construct(String, LogSink)`" + * otherwise. + */ + construct(String name, LogSink sink) { + construct BasicLogger(name, sink, Null); + } + + @Inject Clock clock; + @Inject MDC mdc; + + @Override + @RO Boolean traceEnabled.get() = sink.isEnabled(name, Trace); + @Override + @RO Boolean debugEnabled.get() = sink.isEnabled(name, Debug); + @Override + @RO Boolean infoEnabled.get() = sink.isEnabled(name, Info); + @Override + @RO Boolean warnEnabled.get() = sink.isEnabled(name, Warn); + @Override + @RO Boolean errorEnabled.get() = sink.isEnabled(name, Error); + + @Override + Boolean isEnabled(Level level, Marker? marker = Null) { + return sink.isEnabled(name, level, marker); + } + + @Override + Logger named(String name) { + // `named` is a *replacement*, not a concatenation: the caller supplies the full + // logger name they want. This matches how SLF4J users typically write + // `LoggerFactory.getLogger("com.example.PaymentService")` and the existing + // `NamedLoggerTest` contract. With a registry attached, repeated calls with the + // same name return the same instance. + if (LoggerRegistry r ?= registry) { + return r.ensure(name); + } + return new BasicLogger(name, sink); + } + + @Override + void trace(String message, Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + emit(Trace, message, arguments, cause, marker); + } + + @Override + void trace(MessageSupplier message, Exception? cause = Null, Marker? marker = Null) { + emitLazy(Trace, message, cause, marker); + } + + @Override + void debug(String message, Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + emit(Debug, message, arguments, cause, marker); + } + + @Override + void debug(MessageSupplier message, Exception? cause = Null, Marker? marker = Null) { + emitLazy(Debug, message, cause, marker); + } + + @Override + void info(String message, Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + emit(Info, message, arguments, cause, marker); + } + + @Override + void info(MessageSupplier message, Exception? cause = Null, Marker? marker = Null) { + emitLazy(Info, message, cause, marker); + } + + @Override + void warn(String message, Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + emit(Warn, message, arguments, cause, marker); + } + + @Override + void warn(MessageSupplier message, Exception? cause = Null, Marker? marker = Null) { + emitLazy(Warn, message, cause, marker); + } + + @Override + void error(String message, Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + emit(Error, message, arguments, cause, marker); + } + + @Override + void error(MessageSupplier message, Exception? cause = Null, Marker? marker = Null) { + emitLazy(Error, message, cause, marker); + } + + @Override + void log(Level level, String message, Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + emit(level, message, arguments, cause, marker); + } + + @Override + void log(Level level, MessageSupplier message, Exception? cause = Null, Marker? marker = Null) { + emitLazy(level, message, cause, marker); + } + + @Override + void logAt(Level level, String message, String sourceFile, Int sourceLine, + Object[] arguments = [], Exception? cause = Null, Marker? marker = Null) { + Marker[] markers = frozen(marker); + emitWith(level, message, arguments, cause, markers, [], + sourceFile=sourceFile, sourceLine=sourceLine); + } + + @Override + void logAt(Level level, MessageSupplier message, String sourceFile, Int sourceLine, + Exception? cause = Null, Marker? marker = Null) { + emitLazy(level, message, cause, marker, sourceFile=sourceFile, sourceLine=sourceLine); + } + + @Override + LoggingEventBuilder atTrace() = builderFor(Trace); + @Override + LoggingEventBuilder atDebug() = builderFor(Debug); + @Override + LoggingEventBuilder atInfo() = builderFor(Info); + @Override + LoggingEventBuilder atWarn() = builderFor(Warn); + @Override + LoggingEventBuilder atError() = builderFor(Error); + @Override + LoggingEventBuilder atLevel(Level level) = builderFor(level); + + /** + * Construct an event and hand it to the sink. The fast-path level check is the first thing + * that happens — when disabled, no formatting work is done. + * + * The single-marker entry point used by per-level methods (`info`/`warn`/...) wraps the + * optional `marker` into a `Marker[]` so the rest of the pipeline only deals with one + * shape. Multi-marker events come through [BasicEventBuilder], which calls [emitWith] + * directly with an already-frozen `Marker[]`. + */ + private void emit(Level level, String message, Object[] arguments, Exception? cause, Marker? marker) { + Marker[] markers = frozen(marker); + emitWith(level, message, arguments, cause, markers, []); + } + + /** + * Lazy-message equivalent of [emit]. The supplier runs only after the sink accepts the + * level/primary-marker check; when disabled, the caller's expensive message construction + * never executes. This is the Ecstasy analogue of Java/Kotlin supplier/block logging. + */ + private void emitLazy(Level level, MessageSupplier message, Exception? cause, Marker? marker, + String? sourceFile = Null, Int sourceLine = -1) { + Marker[] markers = frozen(marker); + Marker? primary = markers.empty ? Null : markers[0]; + if (!sink.isEnabled(name, level, primary)) { + return; + } + + emitWith(level, message(), [], cause, markers, [], + sourceFile=sourceFile, sourceLine=sourceLine); + } + + /** + * Freeze an optional single marker so it can safely cross a service sink boundary. + */ + private static Marker[] frozen(Marker? marker) { + if (Marker m ?= marker) { + return [m.freeze()]; + } + return []; + } + + /** + * Internal entry point used by [BasicEventBuilder] to deliver an event that may carry + * structured key/value pairs and/or multiple markers. Per-level methods on `Logger` + * funnel through [emit] which wraps a single optional marker into a one-element array + * before calling here, so this is the one method that sees the full multi-marker shape. + * + * Service-boundary semantics: Ecstasy only lets immutable values, `const` values, + * service references, or already-frozen `Freezable`s cross from one service to + * another. `BasicMarker` is a mutable class (it accumulates child references via + * `add`), so handing a caller-supplied marker straight to a sink — when the sink IS + * a service (e.g. `service MemoryLogSink`) — fails with "mutable object cannot be + * used as an argument to a service call". The two callers ([emit] above and + * [BasicEventBuilder.frozenMarkers]) are both responsible for freezing their markers + * before they reach this method, so we can pass `markers` to the sink as-is. The + * sink-side `isEnabled` fast path therefore sees frozen markers too. + */ + void emitWith(Level level, String message, Object[] arguments, Exception? cause, + Marker[] markers, Map keyValues, + String? sourceFile = Null, Int sourceLine = -1) { + // v0 policy on `arguments`: no defensive copy. Per `open-questions.md` Q11 the + // caller is contractually required not to mutate the array between the return of + // `info(...)` and any (possibly async) sink consuming the resulting `LogEvent`. + // Matches SLF4J's posture; `BasicEventBuilder` already freezes its accumulating + // `args` before calling here, so the fluent path is safe by construction. + // Reconsider if/when async sinks become a default. + + // The SPI's level check is single-marker (the first one is treated as "primary"), + // matching SLF4J 1.x's `Logger.isEnabledFor(Marker)`. Sinks that want richer + // multi-marker filtering inspect `event.markers` directly inside `log()`. + Marker? primary = markers.empty ? Null : markers[0]; + if (!sink.isEnabled(name, level, primary)) { + return; + } + (String formatted, Exception? promoted) = MessageFormatter.format(message, arguments); + Exception? finalCause = cause ?: promoted; + sink.log(new LogEvent( + loggerName = name, + level = level, + message = formatted, + timestamp = clock.now, + markers = markers, + exception = finalCause, + arguments = arguments, + mdcSnapshot = mdc.copyOfContextMap, + threadName = "", // TODO(runtime): expose current-fiber identity + sourceFile = sourceFile, + sourceLine = sourceLine, + keyValues = keyValues, + )); + } + + private LoggingEventBuilder builderFor(Level level) { + return new BasicEventBuilder(this, level); + } +} diff --git a/lib_logging/src/main/x/logging/BasicMarker.x b/lib_logging/src/main/x/logging/BasicMarker.x new file mode 100644 index 0000000000..d6da894f45 --- /dev/null +++ b/lib_logging/src/main/x/logging/BasicMarker.x @@ -0,0 +1,107 @@ +/** + * Corresponds to `org.slf4j.helpers.BasicMarker` — the small in-memory `Marker` + * implementation that ships with `slf4j-api`. + * + * Minimal default `Marker` implementation. Stores child references in a list. Implements + * the [Freezable] contract by freezing its child list on `freeze`; once frozen, `add` and + * `remove` will throw — matching the standard Ecstasy `Freezable` posture. + */ +class BasicMarker(String name) + implements Marker { + + private Marker[] children = new Marker[]; + + /** + * Add a direct child reference unless it is already present. + */ + @Override + void add(Marker reference) { + if (reference.contains(this)) { + throw new IllegalArgument( + $"Adding marker reference {reference.name} would create a cycle with {name}"); + } + if (!children.contains(reference)) { + children.add(reference); + } + } + + /** + * Remove a direct child reference by marker equality. + */ + @Override + Boolean remove(Marker reference) { + return children.removeIfPresent(reference); + } + + /** + * True when this marker has direct children. + */ + @Override + @RO Boolean hasReferences.get() { + return !children.empty; + } + + /** + * Iterator over direct child markers. + */ + @Override + @RO Iterator references.get() { + return children.iterator(); + } + + /** + * Transitive marker containment check by marker. + */ + @Override + Boolean contains(Marker other) { + if (this.name == other.name) { + return True; + } + for (Marker child : children) { + if (child.contains(other)) { + return True; + } + } + return False; + } + + /** + * Transitive marker containment check by name. + */ + @Override + Boolean containsName(String name) { + if (this.name == name) { + return True; + } + for (Marker child : children) { + if (child.containsName(name)) { + return True; + } + } + return False; + } + + // ---- Freezable ------------------------------------------------------------------------ + + @Override + immutable Marker freeze(Boolean inPlace = False) { + if (this.is(immutable)) { + return this; + } + if (inPlace) { + // Freeze each child in place and the children array as well, then make this immutable. + for (Int i : 0 ..< children.size) { + children[i] = children[i].freeze(inPlace=True); + } + children = children.freeze(inPlace=True); + return makeImmutable(); + } + // Out-of-place: produce a fresh frozen copy. + BasicMarker copy = new BasicMarker(name); + for (Marker child : children) { + copy.children.add(child.freeze(inPlace=False)); + } + copy.children = copy.children.freeze(inPlace=True); + return copy.makeImmutable(); + } +} diff --git a/lib_logging/src/main/x/logging/CompositeLogSink.x b/lib_logging/src/main/x/logging/CompositeLogSink.x new file mode 100644 index 0000000000..338a09612c --- /dev/null +++ b/lib_logging/src/main/x/logging/CompositeLogSink.x @@ -0,0 +1,43 @@ +/** + * Logback-style multi-appender sink. + * + * `CompositeLogSink` fans out each enabled event to every delegate sink that accepts + * the event's logger/level/primary-marker tuple. This is the Ecstasy equivalent of + * attaching multiple Logback appenders to one logger. + * + * Java/Logback equivalent: + * + * + * + * + * + * + * Ecstasy equivalent: + * + * LogSink sink = new CompositeLogSink([ + * new ConsoleLogSink(), + * new JsonLogSink() + * ]); + */ +const CompositeLogSink(LogSink[] sinks) + implements LogSink { + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + for (LogSink sink : sinks) { + if (sink.isEnabled(loggerName, level, marker)) { + return True; + } + } + return False; + } + + @Override + void log(LogEvent event) { + for (LogSink sink : sinks) { + if (sink.isEnabled(event.loggerName, event.level, event.marker)) { + sink.log(event); + } + } + } +} diff --git a/lib_logging/src/main/x/logging/ConsoleLogSink.x b/lib_logging/src/main/x/logging/ConsoleLogSink.x new file mode 100644 index 0000000000..957d317c88 --- /dev/null +++ b/lib_logging/src/main/x/logging/ConsoleLogSink.x @@ -0,0 +1,149 @@ +/** + * Corresponds to `org.slf4j.simple.SimpleLogger`'s output behaviour (slf4j-simple) and + * Logback's `ch.qos.logback.core.ConsoleAppender` paired with a basic `PatternLayout`. + * The role is the same: a no-config sink that puts events on stdout/stderr in a fixed, + * human-readable line format. + * + * The default sink: writes one line per event to the platform `Console`. + * + * Format: + * + * 2026-04-29T11:23:45.012Z [thread-name] INFO com.example.foo: hello world + * + * If the event has an exception, the stack trace is appended on subsequent lines. + * + * # Why this sink is a `const`, not a `service` + * + * `ConsoleLogSink` carries no mutable state of its own. Its threshold (`rootLevel`) is + * fixed at construction; every other piece of "state" is either an injected service + * reference (`@Inject Console console`) or constructed transiently inside `log()`. + * + * That makes it a forwarder, not an event collector — structurally identical to + * `lib_ecstasy/src/main/x/ecstasy/io/ConsoleAppender.x` and `ConsoleLog.x`, both of + * which are `class`/`const`-shaped wrappers over an injected `Console`. + * + * Sinks that *do* hold mutable shared state (e.g. `MemoryLogSink` collecting events, + * `AsyncLogSink` owning a worker queue, `HierarchicalLogSink` owning mutable level + * configuration, or a future `FileLogSink` owning a `Writer`) must remain `service`. + * The rule of thumb is documented in + * `doc/logging/design/design.md` ("Sink type: `const` vs `service`"). + * + * Configuration is intentionally minimal: a single `rootLevel` threshold applied to every + * logger. Per-logger routing belongs in [HierarchicalLogSink]; multi-destination routing + * belongs in [CompositeLogSink]. + */ +const ConsoleLogSink(Level rootLevel) + implements LogSink { + + /** + * No-arg convenience: `new ConsoleLogSink()` resolves to threshold `Info`. Explicit + * (rather than `Level rootLevel = Info` default on the primary constructor) because + * cross-module default-argument resolution on `const` constructors does not always + * synthesise the zero-arg form — callers from a different module otherwise hit + * "Unresolvable function `construct()`." + */ + construct() { + construct ConsoleLogSink(Info); + } + + @Inject Console console; + + /** + * Cheap root-threshold check. The default sink intentionally has no per-logger or + * marker-specific configuration; those belong in richer sinks. + */ + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= rootLevel.severity; + } + + /** + * Render a `LogEvent` as one human-readable line. This default output is stable + * enough for demos/tests, not a replacement for a Logback-style layout system. + */ + @Override + void log(LogEvent event) { + StringBuffer buf = new StringBuffer(); + buf.append(event.timestamp.toString()) + .append(" [").append(event.threadName).append("] ") + .append(event.level.name.leftJustify(5, ' ')) + .append(' ') + .append(event.loggerName) + .append(": ") + .append(event.message); + + appendMarkers(buf, event.markers); + appendMdc(buf, event.mdcSnapshot); + appendKeyValues(buf, event.keyValues); + + console.print(buf.toString()); + + if (Exception e ?= event.exception) { + // TODO(impl): pretty-print stack frames; for now rely on Exception.toString(). + console.print(e.toString()); + } + } + + /** + * Append Logback-style marker text, e.g. `[marker=AUDIT]` or `[markers=AUDIT,SECURITY]`. + */ + private void appendMarkers(StringBuffer buf, Marker[] markers) { + if (markers.empty) { + return; + } + + buf.append(markers.size == 1 ? " [marker=" : " [markers="); + Boolean first = True; + for (Marker marker : markers) { + first = appendSeparator(buf, first, ","); + buf.append(marker.name); + } + buf.append(']'); + } + + /** + * Append the MDC snapshot captured before the event reached the sink. + */ + private void appendMdc(StringBuffer buf, Map mdc) { + if (mdc.empty) { + return; + } + + buf.append(" [mdc="); + Boolean first = True; + for ((String key, String value) : mdc) { + first = appendSeparator(buf, first, ","); + buf.append(key).append('=').append(value); + } + buf.append(']'); + } + + /** + * Append structured key/value pairs in the compact text layout. + */ + private void appendKeyValues(StringBuffer buf, Map values) { + if (values.empty) { + return; + } + + buf.append(" {"); + Boolean first = True; + for ((String key, Object value) : values) { + first = appendSeparator(buf, first, ", "); + buf.append(key).append('=').append(value.toString()); + } + buf.append('}'); + } + + /** + * Append `separator` after the first item in a small rendered list. + * + * @return False, so callers can write `first = appendSeparator(...)`. + */ + private Boolean appendSeparator(StringBuffer buf, Boolean first, String separator) { + if (!first) { + buf.append(separator); + } + return False; + } +} diff --git a/lib_logging/src/main/x/logging/HierarchicalLogSink.x b/lib_logging/src/main/x/logging/HierarchicalLogSink.x new file mode 100644 index 0000000000..16a8379b55 --- /dev/null +++ b/lib_logging/src/main/x/logging/HierarchicalLogSink.x @@ -0,0 +1,81 @@ +/** + * Logback-style per-logger threshold wrapper. + * + * The sink keeps a mutable map of logger-name prefixes to levels. The longest matching + * prefix wins; when no prefix matches, [rootLevel] applies. Events that pass the + * threshold are forwarded to [delegate]. + * + * This is backend behavior, not pure SLF4J facade behavior. It corresponds to Logback's + * configured logger tree: + * + * + * + * + * Ecstasy equivalent: + * + * HierarchicalLogSink sink = new HierarchicalLogSink(new ConsoleLogSink(), Info); + * sink.setLevel("com.acme.payments", Debug); + */ +service HierarchicalLogSink(LogSink delegate, Level rootLevel) + implements LogSink { + + /** + * Convenience: root threshold `Info`. + */ + construct(LogSink delegate) { + construct HierarchicalLogSink(delegate, Info); + } + + /** + * Mutable per-prefix configuration, e.g. `"com.example.payments" -> Debug`. + */ + private Map levels = new HashMap(); + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= effectiveLevel(loggerName).severity + && delegate.isEnabled(loggerName, level, marker); + } + + @Override + void log(LogEvent event) { + if (isEnabled(event.loggerName, event.level, event.marker)) { + delegate.log(event); + } + } + + /** + * Configure a logger-name prefix. The longest prefix wins. + */ + void setLevel(String loggerPrefix, Level level) { + levels.put(loggerPrefix, level); + } + + /** + * Remove a logger-name prefix override. + */ + void clearLevel(String loggerPrefix) { + levels.remove(loggerPrefix); + } + + /** + * Resolve the effective threshold for `loggerName`. + */ + Level effectiveLevel(String loggerName) { + String name = loggerName; + while (True) { + if (Level level := levels.get(name)) { + return level; + } + + if (Int dot := name.lastIndexOf('.')) { + if (dot <= 0) { + return rootLevel; + } + name = name[0 ..< dot]; + } else { + return rootLevel; + } + } + } +} diff --git a/lib_logging/src/main/x/logging/JsonLogSink.x b/lib_logging/src/main/x/logging/JsonLogSink.x new file mode 100644 index 0000000000..ec22efb057 --- /dev/null +++ b/lib_logging/src/main/x/logging/JsonLogSink.x @@ -0,0 +1,147 @@ +import json.Doc; +import json.JsonArray; +import json.JsonObject; +import json.Printer; + +/** + * Production-oriented JSON-lines sink for the canonical SLF4J-shaped API. + * + * It renders through `lib_json`, preserves MDC, markers, structured key/value pairs, + * exceptions, and explicit source metadata, and supports key-based redaction through + * [JsonLogSinkOptions]. + * + * This is backend/encoder behavior, not part of SLF4J itself. The Java comparison point + * is Logback with a JSON encoder such as `logstash-logback-encoder`: + * + * + * + * Ecstasy equivalent: + * + * LogSink sink = new JsonLogSink(new JsonLogSinkOptions(Info, + * ["authorization", "password"])); + */ +const JsonLogSink(JsonLogSinkOptions options) + implements LogSink { + + /** + * Convenience: Info threshold, all standard fields enabled, no redacted keys. + */ + construct() { + construct JsonLogSink(new JsonLogSinkOptions()); + } + + /** + * Convenience: configure only the root threshold. + */ + construct(Level rootLevel) { + construct JsonLogSink(new JsonLogSinkOptions(rootLevel)); + } + + @Inject Console console; + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= options.rootLevel.severity; + } + + @Override + void log(LogEvent event) { + console.print(render(event)); + } + + /** + * Render the event as one compact JSON object. + */ + String render(LogEvent event) { + return Printer.DEFAULT.render(toJson(event)); + } + + /** + * Convert the event into a JSON object. Exposed for tests and custom wrappers. + */ + JsonObject toJson(LogEvent event) { + JsonObject obj = json.newObject(); + obj.put(options.timeKey, event.timestamp.toString()); + obj.put(options.levelKey, event.level.name); + obj.put(options.loggerKey, event.loggerName); + obj.put(options.messageKey, event.message); + + if (options.includeMarkers && !event.markers.empty) { + JsonArray markers = json.newArray(); + for (Marker marker : event.markers) { + markers.add(marker.name); + } + obj.put(options.markersKey, markers.toArray(Constant)); + } + + if (options.includeMdc && !event.mdcSnapshot.empty) { + JsonObject mdc = json.newObject(); + for ((String key, String value) : event.mdcSnapshot) { + mdc.put(key, redacted(key, value)); + } + obj.put(options.mdcKey, mdc.makeImmutable()); + } + + if (options.includeKeyValues && !event.keyValues.empty) { + for ((String key, Object value) : event.keyValues) { + obj.put(key, redacted(key, value)); + } + } + + if (Exception e ?= event.exception) { + obj.put(options.exceptionKey, exceptionJson(e)); + } + + if (options.includeSource) { + if (String file ?= event.sourceFile) { + JsonObject source = json.newObject(); + source.put("file", file); + if (event.sourceLine >= 0) { + source.put("line", event.sourceLine.toIntLiteral()); + } + obj.put(options.sourceKey, source.makeImmutable()); + } + } + + return obj.makeImmutable(); + } + + private Doc redacted(String key, Object value) { + return options.redacts(key) ? options.redaction : toJsonValue(value); + } + + private Doc toJsonValue(Object value) { + if (String s := value.is(String)) { + return s; + } + if (Boolean b := value.is(Boolean)) { + return b; + } + if (IntLiteral n := value.is(IntLiteral)) { + return n; + } + if (FPLiteral n := value.is(FPLiteral)) { + return n; + } + if (IntNumber n := value.is(IntNumber)) { + return n.toIntLiteral(); + } + if (FPNumber n := value.is(FPNumber)) { + return n.toFPLiteral(); + } + if (Exception e := value.is(Exception)) { + return exceptionJson(e); + } + return value.toString(); + } + + private JsonObject exceptionJson(Exception e) { + JsonObject obj = json.newObject(); + obj.put("type", &e.class.name); + obj.put("message", e.message); + if (Exception cause ?= e.cause) { + obj.put("cause", exceptionJson(cause)); + } + return obj.makeImmutable(); + } +} diff --git a/lib_logging/src/main/x/logging/JsonLogSinkOptions.x b/lib_logging/src/main/x/logging/JsonLogSinkOptions.x new file mode 100644 index 0000000000..cda62c9ade --- /dev/null +++ b/lib_logging/src/main/x/logging/JsonLogSinkOptions.x @@ -0,0 +1,112 @@ +/** + * Configuration for [JsonLogSink]. + * + * This is intentionally small but production-oriented: it covers thresholding, + * field naming, MDC/marker/source inclusion, and key-based redaction without + * committing the XDK to a full Logback XML/JSON configuration language. + */ +const JsonLogSinkOptions( + /** + * Lowest event level accepted by the sink. + */ + Level rootLevel, + /** + * MDC or key/value keys whose values should be replaced with [redaction]. + */ + String[] redactedKeys, + /** + * Replacement value emitted for redacted keys. + */ + String redaction, + /** + * True to include [LogEvent.mdcSnapshot]. + */ + Boolean includeMdc, + /** + * True to include [LogEvent.markers]. + */ + Boolean includeMarkers, + /** + * True to include fluent `addKeyValue(...)` payloads. + */ + Boolean includeKeyValues, + /** + * True to include [LogEvent.sourceFile] / [LogEvent.sourceLine] when present. + */ + Boolean includeSource, + /** + * Field name for the timestamp. + */ + String timeKey, + /** + * Field name for the level. + */ + String levelKey, + /** + * Field name for the logger name. + */ + String loggerKey, + /** + * Field name for the rendered message. + */ + String messageKey, + /** + * Field name for MDC data. + */ + String mdcKey, + /** + * Field name for marker names. + */ + String markersKey, + /** + * Field name for structured exception data. + */ + String exceptionKey, + /** + * Field name for source metadata. + */ + String sourceKey, + ) { + + /** + * Default production-safe options. + */ + construct() { + construct JsonLogSinkOptions(Info, [], "***", + True, True, True, True, + "time", "level", "logger", "message", + "mdc", "markers", "exception", "source"); + } + + /** + * Convenience: custom root threshold. + */ + construct(Level rootLevel) { + construct JsonLogSinkOptions(rootLevel, [], "***", + True, True, True, True, + "time", "level", "logger", "message", + "mdc", "markers", "exception", "source"); + } + + /** + * Convenience: custom root threshold and redaction keys. + */ + construct(Level rootLevel, String[] redactedKeys) { + construct JsonLogSinkOptions(rootLevel, redactedKeys, "***", + True, True, True, True, + "time", "level", "logger", "message", + "mdc", "markers", "exception", "source"); + } + + /** + * True iff the supplied key should be replaced with [redaction]. + */ + Boolean redacts(String key) { + for (String candidate : redactedKeys) { + if (candidate == key) { + return True; + } + } + return False; + } +} diff --git a/lib_logging/src/main/x/logging/LazyValue.x b/lib_logging/src/main/x/logging/LazyValue.x new file mode 100644 index 0000000000..804d8eae46 --- /dev/null +++ b/lib_logging/src/main/x/logging/LazyValue.x @@ -0,0 +1,36 @@ +/** + * Internal carrier for lazily-computed positional arguments and key/value pairs. + * + * The fluent builder ([LoggingEventBuilder]) needs to distinguish "this slot holds an + * already-computed value" from "this slot holds a function to invoke after the level + * check passes." Storing the supplier function directly in an `Object[]` does not work + * by itself, because in Ecstasy `function Object()` is itself an `Object`, so a runtime + * `value.is(...)` check cannot reliably tell a real `Object` value apart from a supplier + * the caller wanted invoked. Wrapping the supplier in this dedicated type makes the + * discrimination explicit: arguments and key/value pairs are stored as `Object[]` / + * `Map`; the resolver step in [BasicEventBuilder] does + * `value.is(LazyValue) ? lv.resolve() : value`. + * + * # Why this is a `class`, not a `const` + * + * The slog-shaped sibling library has the same wrapper at `slogging/LazyValue.x` and it + * *is* a `const` there, because slog's `Attr` is a `const` and so its `value` field must + * be `Passable`. Here the wrapper lives only in `BasicEventBuilder` (a `class`) and is + * resolved into a plain `Object` before crossing any service boundary, so the + * passability constraint does not apply. Crucially, making this a `class` means the + * captured supplier closure is NOT auto-frozen on construction — so a caller can write + * + * @Volatile Int calls = 0; + * builder.addLazyArgument(() -> { ++calls; return value; }).log("..."); + * + * and the `++calls` mutation works as expected. A `const` wrapper would freeze the + * closure environment and `++calls` would throw `ReadOnly` (which is exactly what the + * slog side documents as a constraint on `Attr.lazy` callers). + */ +class LazyValue(ObjectSupplier supplier) { + + /** + * Evaluate the deferred value. + */ + Object resolve() = supplier(); +} diff --git a/lib_logging/src/main/x/logging/Level.x b/lib_logging/src/main/x/logging/Level.x new file mode 100644 index 0000000000..f46231acb1 --- /dev/null +++ b/lib_logging/src/main/x/logging/Level.x @@ -0,0 +1,31 @@ +/** + * Corresponds to `org.slf4j.event.Level` (SLF4J 2.x), `ch.qos.logback.classic.Level`, + * `org.apache.logging.log4j.Level` (Log4j 2), `java.util.logging.Level` (JUL — different + * names but the same idea). + * + * The severity levels used by the logging API. Identical in spirit to SLF4J / Logback / Log4j2: + * `Trace` is the most verbose, `Error` is the most severe. `Off` is reserved for sink-side + * configuration (i.e. "log nothing for this category") and is never used as the level of an + * actual `LogEvent`. + * + * The numeric `severity` field is exposed so sinks can do cheap threshold comparisons: + * + * if (event.level.severity >= threshold.severity) { + * // emit + * } + */ +enum Level(Int severity) { + Trace(10), + Debug(20), + Info (30), + Warn (40), + Error(50), + Off (Int.MaxValue); + + /** + * True iff an event at this level should be emitted given the supplied `threshold`. + */ + Boolean enabledAtThreshold(Level threshold) { + return this.severity >= threshold.severity; + } +} diff --git a/lib_logging/src/main/x/logging/LogEvent.x b/lib_logging/src/main/x/logging/LogEvent.x new file mode 100644 index 0000000000..d09cfa4bd1 --- /dev/null +++ b/lib_logging/src/main/x/logging/LogEvent.x @@ -0,0 +1,48 @@ +/** + * Corresponds to `org.slf4j.event.LoggingEvent` (SLF4J) and + * `ch.qos.logback.classic.spi.ILoggingEvent` / `LoggingEvent` (Logback). + * + * An immutable record of a single log call. Sinks receive `LogEvent` instances; layouts and + * appenders can read every field freely without worrying about caller-side mutation. + * + * The `mdcSnapshot` is captured at the moment of the + * log call so that asynchronous sinks see the context as it was when the event was raised, not + * as it is when the event is finally written. + */ +const LogEvent( + String loggerName, + Level level, + String message, + Time timestamp, + // The markers attached to this event. Empty when no markers were supplied. The + // per-level `Logger.info(..., marker=X)` API supplies at most one; the fluent + // `LoggingEventBuilder.addMarker(...)` accumulates many. Mirrors SLF4J 2.x's + // `LoggingEvent.getMarkers(): List`. + Marker[] markers = [], + Exception? exception = Null, + Object[] arguments = [], + Map mdcSnapshot = [], + String threadName = "", + // Optional source metadata. The default `(Null, -1)` means "not captured". + // `Logger.logAt(...)` populates these fields explicitly; future compiler/runtime + // call-site capture should lower into that same API instead of changing sinks. + String? sourceFile = Null, + Int sourceLine = -1, + // Structured key/value pairs accumulated through `LoggingEventBuilder.addKeyValue`. + // Mirrors SLF4J 2.x's `KeyValuePair` list on `LoggingEvent`. Sinks that don't care + // about structured payloads can ignore this field; sinks that do (JSON layouts, + // logback's structured-arguments encoder, future `lib_logging_logback`) read it + // alongside `message`. + Map keyValues = [], + ) { + + /** + * Convenience accessor for the first marker, or `Null` when none. Sinks that only + * surface a single category line (`[marker=NAME]`) can read this without + * iterating; richer sinks should iterate [markers] directly. Mirrors SLF4J 1.x's + * `LoggingEvent.getMarker()` for one-marker callers. + */ + @RO Marker? marker.get() { + return markers.empty ? Null : markers[0]; + } +} diff --git a/lib_logging/src/main/x/logging/LogSink.x b/lib_logging/src/main/x/logging/LogSink.x new file mode 100644 index 0000000000..2788158f83 --- /dev/null +++ b/lib_logging/src/main/x/logging/LogSink.x @@ -0,0 +1,70 @@ +/** + * Corresponds conceptually to a Logback `Appender`: a per-destination backend with its + * own level/filter decision. This is not part of pure SLF4J; it is the backend boundary + * that a real SLF4J deployment would normally get from Logback, Log4j 2, or another + * implementation. + * + * Mapping multiple `LogSink`s onto one logger (Logback's "appender attached to logger" + * model) is the job of [CompositeLogSink]. + * + * The Service-Provider Interface for logging backends. A `LogSink` is the only thing a + * `Logger` ever talks to; everything else (level checks, message formatting, marker + * filtering, MDC capture) is implemented above the sink and is sink-agnostic. + * + * # Why this is the API/impl boundary + * + * Anything _above_ this interface is the public, stable API that user code depends on. + * Anything _below_ this interface is replaceable: `ConsoleLogSink` for the default + * platform-controlled console output, `MemoryLogSink` in tests, [JsonLogSink] for + * structured JSON-Lines output, [CompositeLogSink] and [HierarchicalLogSink] for + * Logback-style backend policy, or a future file/network/cloud sink. All of those are + * interchangeable behind this interface. + * + * # Implementing a custom sink + * + * The contract is small: + * - `isEnabled` is the cheap fast-path. It must return promptly and may be called once per + * log statement. Implementations doing per-logger configuration (e.g. logback) consult + * their own configuration tree here. + * - `log` receives a fully-formed `LogEvent`. The message has already had `{}` placeholders + * substituted; the `mdcSnapshot` has already been captured. + * + * See `doc/logging/usage/custom-sinks.md` for worked examples. + * + * # Choosing between `const` and `service` for an implementation + * + * `BasicLogger` is a `const` and references its sink through a `LogSink` field. That + * means every concrete implementation must be `Passable` — i.e. either `immutable` + * (`const`) or a `service`. Pick one of the two: + * + * - `const` — for stateless forwarders / pure adapters whose configuration is fixed + * at construction. Examples in this library: [NoopLogSink], + * [ConsoleLogSink]. Cheap to construct; methods run on the caller's fiber. + * - `service` — for sinks that carry mutable state shared across fibers: event + * buffers, hit counters, file/socket writers, async worker queues. + * Examples: [MemoryLogSink], [HierarchicalLogSink], [AsyncLogSink], + * and a future `FileLogSink`. + * + * The full rule, with reference examples from the platform/xunit codebases (e.g. + * `service ConsoleExecutionListener`, `service ErrorLog`), is in + * `doc/logging/design/design.md` under "Sink type: `const` vs `service`". + */ +interface LogSink { + + /** + * Cheap level check. Should be safe to call on every log statement. May consult marker- + * specific or logger-specific configuration to decide. + * + * @param loggerName the name of the logger asking + * @param level the level of the message about to be emitted + * @param marker an optional marker the caller has attached + */ + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null); + + /** + * Emit the supplied event. Must not throw under normal conditions; sinks that experience + * an internal failure (disk full, remote endpoint unreachable) are expected to degrade + * gracefully — typically by writing to standard error — rather than propagate. + */ + void log(LogEvent event); +} diff --git a/lib_logging/src/main/x/logging/Logger.x b/lib_logging/src/main/x/logging/Logger.x new file mode 100644 index 0000000000..689e0f38e2 --- /dev/null +++ b/lib_logging/src/main/x/logging/Logger.x @@ -0,0 +1,232 @@ +/** + * Corresponds to `org.slf4j.Logger`. The same role is played by + * `ch.qos.logback.classic.Logger` (Logback's concrete class), `org.apache.logging.log4j.Logger` + * (Log4j 2), and `java.util.logging.Logger` (JUL). + * + * The user-facing logging facade. + * + * Modeled directly on `org.slf4j.Logger`: per-level methods (`trace`, `debug`, `info`, `warn`, + * `error`), parameterized messages with `{}` placeholders, optional `Exception` cause, optional + * `Marker`, and the SLF4J 2.x fluent event builder via `atInfo()` / `atLevel(...)`. + * + * Acquisition is by injection: + * + * @Inject Logger logger; // root logger + * Logger log = logger.named("com.example.foo"); // named logger + * + * or via the `LoggerFactory` static accessor. + * + * Implementations are free, but the canonical implementation in this module is a thin + * forwarder over a `LogSink` (see `BasicLogger`); user code never sees it directly. + */ +interface Logger { + + /** + * The logger's name. Sinks use this for routing decisions. + */ + @RO String name; + + /** + * Return a logger with the supplied `name` that shares this logger's sink. Mirrors the + * SLF4J idiom `LoggerFactory.getLogger(MyClass.class)` declared once at the top of a + * class: + * + * @Inject Logger logger; // root logger, fixed name + * static Logger PaymentLogger = logger.named("payments"); // per-class derivative + * + * `name` is the full logger name, not a suffix appended to this logger's name. + * Implementations may intern by name; `BasicLogger` does so when it was created by a + * `LoggerRegistry` and otherwise returns a fresh logger. + */ + Logger named(String name); + + /** + * Cheap `Trace` level check. + */ + @RO Boolean traceEnabled; + /** + * Cheap `Debug` level check. + * + * Use this for multi-statement guarded work. For single expensive messages or values, + * prefer the lazy supplier overloads (`logger.debug(() -> "...")` or + * `logger.atDebug().addArgument(() -> value).log("...")`), which perform this same + * enabled check before invoking the supplied function. + */ + @RO Boolean debugEnabled; + /** + * Cheap `Info` level check. + */ + @RO Boolean infoEnabled; + /** + * Cheap `Warn` level check. + */ + @RO Boolean warnEnabled; + /** + * Cheap `Error` level check. + */ + @RO Boolean errorEnabled; + + /** + * Cheap level check. Equivalent to the per-level shortcuts above; the explicit form is + * preferred when the level is determined dynamically. + */ + Boolean isEnabled(Level level, Marker? marker = Null); + + // ---- Per-level emission ------------------------------------------------------------------- + + /** + * Emit a `Trace` event using SLF4J `{}` placeholder formatting. + */ + void trace(String message, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit a `Trace` event whose complete message is built lazily after the level check. + * + * This is the Ecstasy equivalent of Java `Logger.log(Level, Supplier)` and + * Kotlin logging blocks such as `logger.trace { "..." }`. + */ + void trace(MessageSupplier message, + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit a `Debug` event using SLF4J `{}` placeholder formatting. + */ + void debug(String message, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit a `Debug` event whose complete message is built lazily after the level check. + */ + void debug(MessageSupplier message, + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit an `Info` event using SLF4J `{}` placeholder formatting. + */ + void info (String message, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit an `Info` event whose complete message is built lazily after the level check. + */ + void info (MessageSupplier message, + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit a `Warn` event using SLF4J `{}` placeholder formatting. + */ + void warn (String message, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit a `Warn` event whose complete message is built lazily after the level check. + */ + void warn (MessageSupplier message, + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit an `Error` event using SLF4J `{}` placeholder formatting. + */ + void error(String message, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit an `Error` event whose complete message is built lazily after the level check. + */ + void error(MessageSupplier message, + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit at a runtime-chosen level. Equivalent to `if (level == Info) info(...) else if ...`. + */ + void log(Level level, + String message, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit at a runtime-chosen level with lazy message construction. + */ + void log(Level level, + MessageSupplier message, + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit at a runtime-chosen level with explicit source metadata. + * + * This mirrors the source-aware entry point on the slog-shaped library and gives a + * future compiler/runtime call-site capture feature a stable lowering target. Normal + * application code should keep using [log] / [info] / [atInfo] until such sugar exists. + */ + void logAt(Level level, + String message, + String sourceFile, + Int sourceLine, + Object[] arguments = [], + Exception? cause = Null, + Marker? marker = Null); + + /** + * Emit at a runtime-chosen level with explicit source metadata and lazy message construction. + * + * This is the lowering target for future compiler-provided source capture when the caller + * also wants Kotlin-style lazy message construction. + */ + void logAt(Level level, + MessageSupplier message, + String sourceFile, + Int sourceLine, + Exception? cause = Null, + Marker? marker = Null); + + // ---- Fluent (SLF4J 2.x style) builder ------------------------------------------------------ + + /** + * Begin a fluent log event at `Trace` level. Example: + * + * logger.atTrace() + * .addMarker(MarkerFactory.getMarker("INTERNAL")) + * .addKeyValue("user", userId) + * .setCause(e) + * .log("processing failed for {}", input); + */ + LoggingEventBuilder atTrace(); + /** + * Begin a fluent log event at `Debug` level. + */ + LoggingEventBuilder atDebug(); + /** + * Begin a fluent log event at `Info` level. + */ + LoggingEventBuilder atInfo(); + /** + * Begin a fluent log event at `Warn` level. + */ + LoggingEventBuilder atWarn(); + /** + * Begin a fluent log event at `Error` level. + */ + LoggingEventBuilder atError(); + /** + * Begin a fluent log event at a runtime-selected level. + */ + LoggingEventBuilder atLevel(Level level); +} diff --git a/lib_logging/src/main/x/logging/LoggerFactory.x b/lib_logging/src/main/x/logging/LoggerFactory.x new file mode 100644 index 0000000000..e3fe0c540c --- /dev/null +++ b/lib_logging/src/main/x/logging/LoggerFactory.x @@ -0,0 +1,52 @@ +/** + * Corresponds to `org.slf4j.LoggerFactory` (the static accessor) and + * `org.slf4j.ILoggerFactory` (the interface that bindings implement). The escape-hatch + * accessor for code that cannot use `@Inject`. + * + * Static accessor for `Logger` instances, for code that cannot use `@Inject`. + * + * Mirrors `org.slf4j.LoggerFactory`. The first call into `getLogger` resolves the active + * `LogSink` (via injection) and caches it; subsequent calls return name-keyed `Logger` + * instances backed by that sink. + * + * Library code that wants `@Inject`-free acquisition should typically use: + * + * static Logger logger = LoggerFactory.getLogger("com.example.foo"); + * + * or, when a class object is in scope: + * + * static Logger logger = LoggerFactory.getLogger(MyClass); + */ +service LoggerFactory { + + /** + * The active default backend. This is intended to be supplied by the host + * container; if no richer sink is registered, the runtime can fall back to the + * console sink. + */ + @Inject LogSink defaultSink; + + /** + * Lazily-created name cache scoped to the injected sink. Mirrors SLF4J's + * `ILoggerFactory` identity-stability rule without creating JVM-global state. + */ + private @Lazy LoggerRegistry registry.calc() { + return new LoggerRegistry(defaultSink); + } + + /** + * Get the logger for `name`, creating it on first access. Repeated calls with the + * same name return the identical `Logger`; `getLogger("a.b")` and + * `getLogger("a").named("b")` return the same instance. + */ + Logger getLogger(String name) { + return registry.ensure(name); + } + + /** + * Convenience: `getLogger(clz.path)`. + */ + Logger getLogger(Class clz) { + return getLogger(clz.path); + } +} diff --git a/lib_logging/src/main/x/logging/LoggerRegistry.x b/lib_logging/src/main/x/logging/LoggerRegistry.x new file mode 100644 index 0000000000..872cc2b0f3 --- /dev/null +++ b/lib_logging/src/main/x/logging/LoggerRegistry.x @@ -0,0 +1,51 @@ +/** + * Corresponds to the role played by `org.slf4j.ILoggerFactory` impls (Logback's + * `LoggerContext`, Log4j 2's `LoggerContext`): the central name-keyed cache that + * makes repeated `getLogger("a.b")` / `logger.named("a.b")` calls return the same + * `Logger` instance. + * + * A name-keyed intern cache for `Logger` instances. `BasicLogger.named(name)` consults + * an attached registry — when present — and returns the cached logger for that exact + * full name; without a registry, each call allocates a fresh logger. + * + * # Why interning belongs in a separate service + * + * `BasicLogger` is a `const` (so the runtime can hand it back from `@Inject Logger` + * without a service-boundary hop, see `BasicLogger.x`). A `const` cannot carry the + * mutable hash map a cache requires — the field would not be `Passable`. Splitting the + * cache out into a `service` keeps the logger immutable while still giving SLF4J's + * "same name → same Logger instance" semantics. + * + * # Scope + * + * Each registry is bound to a single [LogSink]. Loggers from different sinks live in + * different registries (or in no registry at all): `LoggerFactory` constructs one + * registry per active sink internally; user code that wants identity-stable loggers + * across an explicit non-default sink can construct its own: + * + * service ListLogSink testSink = new ListLogSink(); + * LoggerRegistry registry = new LoggerRegistry(testSink); + * Logger root = registry.ensure("root"); + * assert &root == ®istry.ensure("root"); // same instance + * assert &root.named("a.b") == ®istry.ensure("a.b"); + */ +service LoggerRegistry(LogSink sink) { + + private Map cache = new HashMap(); + + /** + * Return the logger for `name`, creating it on first access. Repeated calls with the + * same `name` return the identical instance. + */ + Logger ensure(String name) { + return cache.computeIfAbsent(name, () -> new BasicLogger(name, sink, this)); + } + + /** + * Discard all cached loggers. Primarily useful for tests that want a clean slate; + * production code typically never calls this. + */ + void reset() { + cache.clear(); + } +} diff --git a/lib_logging/src/main/x/logging/LoggingEventBuilder.x b/lib_logging/src/main/x/logging/LoggingEventBuilder.x new file mode 100644 index 0000000000..c9b25460c2 --- /dev/null +++ b/lib_logging/src/main/x/logging/LoggingEventBuilder.x @@ -0,0 +1,99 @@ +/** + * Corresponds to `org.slf4j.spi.LoggingEventBuilder` (SLF4J 2.x). The default SLF4J impl is + * `org.slf4j.spi.DefaultLoggingEventBuilder`; we ship `BasicEventBuilder` in the same role. + * + * Fluent log-event builder, mirroring `org.slf4j.spi.LoggingEventBuilder` from SLF4J 2.x. + * + * The builder accumulates state then materializes a single `LogEvent` on `log(...)`. The + * level check happens at `log(...)`, after markers have been attached, so marker-aware + * sinks still get a chance to enable or suppress the event. If the sink rejects the event, + * no formatting, MDC snapshot, or `LogEvent` allocation happens. + * + * The builder supports eager values and supplier-valued lazy values. Lazy suppliers are + * resolved only after the sink accepts the level/primary-marker check. Java SLF4J can model + * this as `addArgument(Supplier)`; Ecstasy's `function Object()` is itself an `Object`, so + * the POC uses explicit `addLazyArgument` / `addLazyKeyValue` names to keep overload resolution + * unambiguous. See `doc/logging/usage/lazy-logging.md`. + * + * Example: + * + * logger.atInfo() + * .addMarker(MarkerFactory.getMarker("AUDIT")) + * .addKeyValue("requestId", req.id) + * .setCause(e) + * .log("payment {} succeeded for {}", paymentId, customer); + */ +interface LoggingEventBuilder { + + /** + * Set (or replace) the message pattern. Equivalent to passing the message to `log(...)`. + */ + LoggingEventBuilder setMessage(String message); + + /** + * Set (or replace) a lazily computed message. The supplier runs only if the event is enabled. + */ + LoggingEventBuilder setMessage(MessageSupplier message); + + /** + * Append an argument used to substitute the next `{}` placeholder in the message. + */ + LoggingEventBuilder addArgument(Object value); + + /** + * Append a lazily computed `{}` argument. The supplier runs only if the event is enabled. + */ + LoggingEventBuilder addLazyArgument(ObjectSupplier value); + + /** + * Attach a marker. Repeated calls add multiple markers. + */ + LoggingEventBuilder addMarker(Marker marker); + + /** + * Attach an exception to the event. + */ + LoggingEventBuilder setCause(Exception cause); + + /** + * Attach a key/value pair, intended for structured-logging sinks. Sinks that don't + * understand structured KV pairs (e.g. `ConsoleLogSink`) may render them as `key=value` + * after the message or simply ignore them. + */ + LoggingEventBuilder addKeyValue(String key, Object value); + + /** + * Attach a lazily computed key/value pair. The supplier runs only if the event is enabled. + */ + LoggingEventBuilder addLazyKeyValue(String key, ObjectSupplier value); + + /** + * Materialize and emit. After `log` is called the builder must not be reused. + */ + void log(); + + /** + * Convenience: same as `setMessage(message).log()`. + */ + void log(String message); + + /** + * Convenience: same as `setMessage(message).log()`, with lazy message construction. + */ + void log(MessageSupplier message); + + /** + * Convenience: same as `setMessage(format).addArgument(arg).log()`. + */ + void log(String format, Object arg); + + /** + * Convenience: same as `setMessage(format).addArgument(arg1).addArgument(arg2).log()`. + */ + void log(String format, Object arg1, Object arg2); + + /** + * Convenience: append all `args` then `log()`. + */ + void log(String format, Object[] args); +} diff --git a/lib_logging/src/main/x/logging/MDC.x b/lib_logging/src/main/x/logging/MDC.x new file mode 100644 index 0000000000..c0e6b383aa --- /dev/null +++ b/lib_logging/src/main/x/logging/MDC.x @@ -0,0 +1,129 @@ +/** + * Corresponds to `org.slf4j.MDC` (and the underlying `org.slf4j.spi.MDCAdapter`). + * Logback's `LogbackMDCAdapter` and Log4j 2's `ThreadContext` cover the same concept. + * + * Mapped Diagnostic Context — a per-fiber string-keyed scratchpad that sinks can include in + * formatted output. Bindings made by one fiber are visible to its callees and to fibers it + * spawns (matching SLF4J's `InheritableThreadLocal`-style propagation), but mutations from + * a child fiber do **not** bleed back to the parent: each fiber's `put` / `remove` / `clear` + * affects only its own logical-thread-of-execution. + * + * Implementation: backed by a single static [ecstasy.SharedContext] holding an `immutable Map`. Every + * mutating operation copy-on-writes a new immutable map and rebinds it via + * `ecstasy.SharedContext.withValue`. Because the runtime stores `ecstasy.SharedContext` tokens on the fiber + * (not the service), and child fibers receive their own copy of the parent's token map at + * spawn time, the child's `withValue` registers on the child's chain only — the parent's + * binding stays untouched. The shared *value* (the immutable map) is safe because there is + * no mutation to leak. + * + * `MDC` is intentionally a `const`, not a `service`. Methods on a `const` execute on the + * caller's service, which is where the fiber's token chain lives. If `MDC` were a service, + * each call would cross into a fresh service-side fiber and `withValue` would register on + * that fiber instead of the caller's — invisible to the caller after the call returns. + * + * Usage matches SLF4J's flat `put` / `remove` / `clear` API: + * + * mdc.put("requestId", id); + * try { + * logger.info("processing"); + * } finally { + * mdc.remove("requestId"); + * } + * + * The token chain grows by one entry per mutation. For typical request-scoped MDC usage + * (a handful of `put`s at the request boundary, a `clear` at the end) this is bounded by + * the request fiber's lifetime. Long-lived fibers that mutate MDC frequently will retain a + * proportional chain. + */ +const MDC { + + /** + * The fiber-local backing store. The default value `[]` is the empty immutable map, so + * `currentMap()` never has to special-case the unbound state. + */ + /** + * The empty immutable map used as the default value of [mapContext] when no fiber + * has yet bound an MDC entry. Naming it lets the static initializer infer the type + * argument for `SharedContext`. + */ + private static immutable Map emptyMap = []; + + private static ecstasy.SharedContext> mapContext = + new ecstasy.SharedContext("mdc", emptyMap); + + /** + * Read the current fiber's view. Always returns an immutable map; the caller can hand + * the result to a sink without further copying. + */ + private immutable Map currentMap() { + return mapContext.hasValue() ?: []; + } + + /** + * Build the next immutable map by copying entries other than `dropKey` and optionally + * appending `(addKey, addValue)`. One allocation per mutation; O(n) in current size. + */ + private immutable Map derive(String? dropKey, String? addKey, String? addValue) { + HashMap next = new HashMap(); + for ((String k, String v) : currentMap()) { + if (k != dropKey) { + next.put(k, v); + } + } + if (String key ?= addKey, String value ?= addValue) { + next.put(key, value); + } + return next.makeImmutable(); + } + + /** + * Store a value under `key`. Setting `Null` removes the key. + * + * Skips the copy-on-write derivation when the existing entry already matches the + * incoming value. Each MDC mutation otherwise costs one fresh `HashMap`-build + + * `makeImmutable` + `SharedContext.withValue` token; the no-op fast path matters in + * code paths that reassert request context on every call (a common pattern in + * cross-cutting middleware that re-`put`s `requestId` defensively). Symmetrical to + * the existing fast path in [remove]. + */ + void put(String key, String? value) { + if (value == Null) { + remove(key); + return; + } + if (String existing := currentMap().get(key), existing == value) { + return; + } + mapContext.withValue(derive(key, key, value)); + } + + /** + * Read the value for `key`, or `Null` if no value is set on the calling fiber. + */ + String? get(String key) = currentMap().getOrNull(key); + + /** + * Remove `key` from the calling fiber's context. No-op if no value is set. + */ + void remove(String key) { + if (currentMap().contains(key)) { + mapContext.withValue(derive(key, Null, Null)); + } + } + + /** + * Remove all entries from the calling fiber's context. + */ + void clear() { + if (!currentMap().empty) { + mapContext.withValue([]); + } + } + + /** + * An immutable snapshot of the current fiber's context map. Sinks call this when + * emitting an event so the captured state is independent of any subsequent mutation. + * Returns the empty map if no value has been set on the calling fiber. + */ + @RO Map copyOfContextMap.get() = currentMap(); +} diff --git a/lib_logging/src/main/x/logging/Marker.x b/lib_logging/src/main/x/logging/Marker.x new file mode 100644 index 0000000000..b0686d11ca --- /dev/null +++ b/lib_logging/src/main/x/logging/Marker.x @@ -0,0 +1,96 @@ +/** + * Corresponds to `org.slf4j.Marker` (SLF4J), with the same semantics as + * `ch.qos.logback.classic.spi.Marker` (Logback) and the marker concept in Log4j 2. + * + * A `Marker` is a named tag that can be attached to a log call to give downstream filters and + * appenders a structured way to route or suppress events without having to grep on message + * content. Mirrors `org.slf4j.Marker`. + * + * Markers can be composed: a marker may have references to other markers (forming a DAG), so + * that a single specific marker can be matched against a broader category. `contains` does the + * transitive check. + * + * # XDK-idiomatic interface design + * + * `Marker` extends three core XDK interfaces: + * + * - **[Freezable]** — markers are carried on `LogEvent` (a `const`); a `const` can only carry + * freezable fields. Anything that may end up on an event must be freezable. This is also + * the property that lets a `Marker` cross a service boundary safely (e.g. when the active + * `LogSink` is a service): the runtime auto-freezes shareable values at the boundary. + * - **[Stringable]** — markers are routinely rendered into formatted log lines and JSON + * payloads. Implementing `Stringable` lets layouts and encoders pre-size their buffers via + * `estimateStringLength()` and avoid extra `toString()` allocations on the hot path. Other + * XDK libs (e.g. `Path`, `URI`, `IPAddress`) follow the same pattern. + * - **[Hashable]** — sinks and filters frequently key off markers (a marker-aware filter + * keeps a `Set` of "interesting" categories). `Hashable` makes that natural and + * makes markers usable as `Map` keys without bridging through their name. + * + * The default implementation in this module ([BasicMarker]) is intentionally minimal — the + * in-memory marker shipped here exists primarily so that user code that *uses* markers + * compiles and runs against the default sink (which mostly ignores them). Real filtering is + * the job of richer sinks (see `doc/logging/future/logback-integration.md`). + */ +interface Marker + extends Freezable + extends Stringable + extends Hashable { + + /** + * The marker's name. Stable identifier; markers with the same name are considered equal. + */ + @RO String name; + + /** + * Add a child marker as a reference. Idempotent: adding the same reference twice is a + * no-op. Throws `ReadOnly` on a frozen marker. + */ + void add(Marker reference); + + /** + * Remove a previously added reference. Returns True iff the reference was present. + * Throws `ReadOnly` on a frozen marker. + */ + Boolean remove(Marker reference); + + /** + * True iff this marker has any child references. + */ + @RO Boolean hasReferences; + + /** + * The direct child references of this marker. + */ + @RO Iterator references; + + /** + * Transitive containment check: True iff this marker is `other`, or has a (possibly + * indirect) reference to a marker named the same as `other`. + */ + Boolean contains(Marker other); + + /** + * Transitive containment check by name. + */ + Boolean containsName(String name); + + // ---- Stringable ----------------------------------------------------------------------- + + @Override + Int estimateStringLength() = name.size; + + @Override + Appender appendTo(Appender buf) = name.appendTo(buf); + + // ---- Hashable ------------------------------------------------------------------------- + + @Override + static Int64 hashCode(CompileType value) { + return value.name.hashCode(); + } + + @Override + static Boolean equals(CompileType value1, CompileType value2) { + return value1.name == value2.name; + } +} diff --git a/lib_logging/src/main/x/logging/MarkerFactory.x b/lib_logging/src/main/x/logging/MarkerFactory.x new file mode 100644 index 0000000000..ecc8197f29 --- /dev/null +++ b/lib_logging/src/main/x/logging/MarkerFactory.x @@ -0,0 +1,72 @@ +/** + * Corresponds to `org.slf4j.MarkerFactory` (the static-method facade in SLF4J) plus + * `org.slf4j.IMarkerFactory` (the interface SLF4J's facade delegates to). Logback's + * marker-aware filters consume the same marker objects. + * + * Factory for `Marker` instances. Mirrors `org.slf4j.MarkerFactory`. + * + * `getMarker(name)` is _interning_: two calls with the same name return the same + * instance. `getDetachedMarker(name)` always returns a fresh instance, useful when a + * caller wants a marker that is not visible to other callers. + * + * # Why a `class`, not a `service` + * + * The natural Ecstasy answer to "make this thread-safe" is `service`. We deliberately + * do not pick that here, because of an interaction between three facts: + * + * - `BasicMarker` is a *mutable* `class` — `add(child)` builds the DAG in place. + * - The SLF4J idiom is `factory.getMarker("X").add(child)`, which mutates the + * interned marker directly after retrieval. + * - Ecstasy auto-freezes any `Freezable` value that crosses a service boundary on + * the way out. + * + * Together, those rules mean a `service MarkerFactory.getMarker(...)` would return a + * frozen `BasicMarker` to its caller, and the next `marker.add(...)` call would throw + * `ReadOnly`. That contradicts the SLF4J pattern and breaks every existing test and + * `manualTests/TestLogging.x` call site. + * + * # Thread-safety contract + * + * Because this is a `class` with a mutable internal `HashMap`, **a single + * `MarkerFactory` instance is not safe to share across fibers**. The supported pattern + * is one factory per component (per service, per request, per test). The existing + * test suite (`MarkerFactoryTest`, `MarkerTest`) and `manualTests/TestLogging.x` follow + * this pattern: each scope constructs its own factory. + * + * If a future user genuinely needs a process-global, fiber-safe marker registry, the + * right shape is a separate `service MarkerRegistry` whose API folds DAG mutation + * into service methods (e.g. `addReference(parentName, childName)`) so the in-service + * `BasicMarker` instances stay live and never cross the freeze boundary. That is + * tracked in `doc/logging/roadmap.md` rather than retrofitted onto this type, because + * collapsing both shapes into one type either gives up the SLF4J ergonomics here or + * gives up safe global sharing there. + */ +class MarkerFactory { + + private Map markers = new HashMap(); + + /** + * Get (creating if necessary) the canonical marker with the supplied name. + * Subsequent calls with the same `name` return the same marker. + * + * Thread safety: the underlying `HashMap.computeIfAbsent` is not concurrent. Use + * one `MarkerFactory` per fiber/component (see the class doc comment). + */ + Marker getMarker(String name) { + return markers.computeIfAbsent(name, () -> new BasicMarker(name)); + } + + /** + * Construct a fresh marker that is not registered in the factory's interning map. + */ + Marker getDetachedMarker(String name) { + return new BasicMarker(name); + } + + /** + * True iff a marker with the supplied name has previously been interned. + */ + Boolean exists(String name) { + return markers.contains(name); + } +} diff --git a/lib_logging/src/main/x/logging/MemoryLogSink.x b/lib_logging/src/main/x/logging/MemoryLogSink.x new file mode 100644 index 0000000000..fd4e023ccc --- /dev/null +++ b/lib_logging/src/main/x/logging/MemoryLogSink.x @@ -0,0 +1,63 @@ +/** + * Corresponds to Logback's `ch.qos.logback.core.read.ListAppender` — the standard + * test-helper appender that captures events in an in-memory list for later assertion. + * + * A test-oriented sink: captures every emitted event in an in-memory list. Suitable for unit + * tests that want to assert on what a logger emitted. + * + * MemoryLogSink sink = new MemoryLogSink(); + * Logger logger = new BasicLogger("test", sink); + * logger.info("processed {}", 42); + * assert sink.events.size == 1; + * assert sink.events[0].level == Info; + * + * # Why this sink is a `service`, not a `const` + * + * It accumulates events. The `events[]` array is mutated on every `log()` call, and + * the same instance is typically shared across many fibers (the logger under test plus + * the assertion code that reads back the events). That is structurally identical to + * `service ConsoleExecutionListener` in `lib_xunit_engine` and `service ErrorLog` in + * `platform/common` — both are stateful event collectors. See + * `doc/logging/design/design.md` ("Sink type: `const` vs `service`") for the full rule. + */ +service MemoryLogSink + implements LogSink { + + /** + * Threshold below which events are dropped. Defaults to `Trace` so tests see everything. + */ + public/private Level rootLevel = Trace; + + /** + * Mutable backing storage for captured events. Internal — must not escape the service + * boundary, because Ecstasy forbids returning a mutable array from a service call. + * External callers read [events] instead, which returns an immutable snapshot. + */ + private LogEvent[] eventList = new LogEvent[]; + + /** + * The captured events, in emission order. Each access returns a fresh immutable + * snapshot of the current state — the backing array is internal so it can stay + * mutable for `add` / `clear`, while crossing the service boundary safely. + */ + @RO LogEvent[] events.get() { + return eventList.toArray(Constant); + } + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= rootLevel.severity; + } + + @Override + void log(LogEvent event) { + eventList.add(event); + } + + /** + * Discard all captured events. + */ + void reset() { + eventList.clear(); + } +} diff --git a/lib_logging/src/main/x/logging/MessageFormatter.x b/lib_logging/src/main/x/logging/MessageFormatter.x new file mode 100644 index 0000000000..18142b839f --- /dev/null +++ b/lib_logging/src/main/x/logging/MessageFormatter.x @@ -0,0 +1,124 @@ +/** + * Corresponds to `org.slf4j.helpers.MessageFormatter` — the SLF4J helper that turns a + * pattern + arguments into a substituted string and, by SLF4J convention, promotes a + * trailing `Throwable` argument into a separate cause. + * + * `{}`-placeholder substitution, contract-compatible with SLF4J's `MessageFormatter`. + * + * Rules, copied from SLF4J for behavioural parity: + * - Each unescaped `{}` is replaced by the next argument's `toString()`. + * - Excess arguments beyond the placeholder count are ignored (subject to throwable + * promotion below). + * - Excess placeholders beyond the argument count are left as-is in the output. + * - A literal `{}` in the message is escaped as `\{}` — the backslash is consumed and + * the `{}` is emitted unchanged. + * - A literal backslash before an unrelated `{}` is escaped as `\\{}` — both + * backslashes are consumed (one literal output) and the placeholder *is* substituted. + * - Throwable promotion: if the last argument is an `Exception` and no placeholder + * consumes it, it is returned as the promoted cause rather than appended to the + * message. This matches SLF4J's behaviour and is intentionally distinct from the + * explicit `cause=` parameter on the per-level methods, which always wins over a + * promoted-from-args throwable (`BasicLogger.emit` enforces that ordering). + */ +static service MessageFormatter { + + private static Char DELIM_START = '{'; + private static Char DELIM_STOP = '}'; + private static Char ESCAPE_CHAR = '\\'; + + /** + * Substitute placeholders in `message` with `args`. Returns the formatted message and + * the promoted-throwable (if any). + */ + (String formatted, Exception? cause) format(String message, Object[] args) { + // Determine throwable promotion: if the trailing argument is an Exception, treat it + // as the candidate cause. The actual decision (promote vs consume as placeholder + // value) is made below based on placeholder consumption. + Int argCount = args.size; + Boolean hasTrailingExn = argCount > 0 && args[argCount - 1].is(Exception); + Int argLimit = hasTrailingExn ? argCount - 1 : argCount; + Exception? promoted = Null; + + StringBuffer buf = new StringBuffer(message.size + 32); + Int argI = 0; + Int i = 0; + Int n = message.size; + + while (i < n) { + Char c = message[i]; + + if (c != DELIM_START) { + buf.append(c); + ++i; + continue; + } + + // saw '{' — peek for '}' + if (i + 1 >= n || message[i + 1] != DELIM_STOP) { + buf.append(c); + ++i; + continue; + } + + // we have a `{}` at position i..i+1; check for escapes immediately before + Boolean escapedDelim = False; // \{} — literal {} + Boolean doubledEscape = False; // \\{} — literal \, then real placeholder + + if (i >= 1 && message[i - 1] == ESCAPE_CHAR) { + if (i >= 2 && message[i - 2] == ESCAPE_CHAR) { + doubledEscape = True; + } else { + escapedDelim = True; + } + } + + if (escapedDelim) { + // remove the escape char we already appended, then emit literal "{}" + buf.truncate(buf.size - 1); + buf.append(DELIM_START); + buf.append(DELIM_STOP); + i += 2; + continue; + } + + if (doubledEscape) { + // we already appended both backslashes; remove the trailing one (the second + // one) so the rendered text has a single literal '\' before the (real) + // substituted argument. + buf.truncate(buf.size - 1); + } + + // real placeholder — substitute or leave literal if we've run out of args + if (argI < argLimit) { + buf.append(safeToString(args[argI])); + ++argI; + } else { + buf.append(DELIM_START); + buf.append(DELIM_STOP); + } + i += 2; + } + + // Throwable promotion: the trailing exception is promoted only if no placeholder + // would have consumed it. argI counted only argLimit, so the trailing Exception is + // never substituted in here; it's always either promoted or dropped. + if (hasTrailingExn) { + promoted = args[argCount - 1].as(Exception); + } + + return (buf.toString(), promoted); + } + + /** + * Defensive `toString` that catches exceptions from the argument's own `toString()` — + * matches SLF4J's policy of never letting a malformed argument tear down the whole log + * statement. + */ + private static String safeToString(Object o) { + try { + return o.toString(); + } catch (Exception e) { + return "[FAILED toString(): " + e.text + "]"; + } + } +} diff --git a/lib_logging/src/main/x/logging/NoopLogSink.x b/lib_logging/src/main/x/logging/NoopLogSink.x new file mode 100644 index 0000000000..2dc72cac72 --- /dev/null +++ b/lib_logging/src/main/x/logging/NoopLogSink.x @@ -0,0 +1,33 @@ +/** + * Corresponds to `org.slf4j.helpers.NOPLogger` (and the binding `slf4j-nop`) plus Logback's + * `ch.qos.logback.core.helpers.NOPAppender`. A sink that intentionally produces no output. + * + * A sink that silently drops every event. Useful when: + * - a library wants to default to "no output" unless the embedding application opts in; + * - a test wants to suppress noise without rewriting calling code; + * - benchmarks want to measure caller-side overhead minus sink cost. + * + * `isEnabled` always returns False so callers that respect the level check can skip + * building arguments and snapshotting MDC entirely. + * + * # Why this sink is a `const` + * + * `NoopLogSink` is purely stateless — `isEnabled` always returns `False` and `log` is + * intentionally empty. There is nothing to mutate, nothing to share, nothing to fan in + * from many fibers. That is the canonical case for `const`: cheap to construct, cheap + * to pass across service boundaries, no scheduler overhead. See `doc/logging/design/design.md` + * ("Sink type: `const` vs `service`") for the full rule. + */ +const NoopLogSink + implements LogSink { + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return False; + } + + @Override + void log(LogEvent event) { + // intentionally empty + } +} diff --git a/lib_logging/src/test/x/LoggingTest.x b/lib_logging/src/test/x/LoggingTest.x new file mode 100644 index 0000000000..aebcfb1cb8 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest.x @@ -0,0 +1,19 @@ +/** + * Unit tests for the `logging.xtclang.org` library. + * + * The tests build small, in-memory `LogSink` implementations (a list-backed sink, a + * counting sink) and assert that: + * - the level check fast-path elides emission of disabled events; + * - the per-level methods (`trace`, `debug`, `info`, `warn`, `error`) route correctly; + * - markers and exceptions arrive at the sink intact; + * - the fluent event builder accumulates state and drops disabled events at emission; + * - `MessageFormatter` substitutes `{}` placeholders. + * + * Each test class targets one cohesive area; submodules are picked up automatically by + * the xunit engine. + */ +module LoggingTest { + package json import json.xtclang.org; + package logging import logging.xtclang.org; + package xunit import xunit.xtclang.org; +} diff --git a/lib_logging/src/test/x/LoggingTest/AsyncLogSinkTest.x b/lib_logging/src/test/x/LoggingTest/AsyncLogSinkTest.x new file mode 100644 index 0000000000..ecdaad8bb8 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/AsyncLogSinkTest.x @@ -0,0 +1,35 @@ +import logging.AsyncLogSink; +import logging.BasicLogger; +import logging.Logger; + +/** + * Tests the Logback-style async wrapper. + */ +class AsyncLogSinkTest { + + @Test + void shouldDrainQueuedEventsToDelegate() { + ListLogSink delegate = new ListLogSink(); + AsyncLogSink async = new AsyncLogSink(delegate, 10); + Logger logger = new BasicLogger("async.test", async); + + logger.info("queued"); + async.flush(); + + assert delegate.events.size == 1; + assert delegate.events[0].message == "queued"; + } + + @Test + void shouldDropEventsAfterCloseWithoutFlush() { + ListLogSink delegate = new ListLogSink(); + AsyncLogSink async = new AsyncLogSink(delegate, 10); + Logger logger = new BasicLogger("async.closed", async); + + async.close(flush=False); + logger.info("dropped"); + + assert async.droppedCount == 1; + assert delegate.events.empty; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/CompositeLogSinkTest.x b/lib_logging/src/test/x/LoggingTest/CompositeLogSinkTest.x new file mode 100644 index 0000000000..7ee9002fd4 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/CompositeLogSinkTest.x @@ -0,0 +1,39 @@ +import logging.BasicLogger; +import logging.CompositeLogSink; +import logging.Logger; + +/** + * Tests multi-appender fan-out. + */ +class CompositeLogSinkTest { + + @Test + void shouldFanOutToAllEnabledDelegates() { + ListLogSink first = new ListLogSink(); + ListLogSink second = new ListLogSink(); + Logger logger = new BasicLogger("composite.test", + new CompositeLogSink([first, second])); + + logger.warn("fanout"); + + assert first.events.size == 1; + assert second.events.size == 1; + assert first.events[0].message == "fanout"; + assert second.events[0].message == "fanout"; + } + + @Test + void shouldSkipDisabledDelegates() { + ListLogSink first = new ListLogSink(); + ListLogSink second = new ListLogSink(); + second.setLevel(logging.Level.Error); + + Logger logger = new BasicLogger("composite.filtered", + new CompositeLogSink([first, second])); + + logger.warn("only first"); + + assert first.events.size == 1; + assert second.events.empty; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/CountingSink.x b/lib_logging/src/test/x/LoggingTest/CountingSink.x new file mode 100644 index 0000000000..adc0d3844f --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/CountingSink.x @@ -0,0 +1,27 @@ +import logging.Level; +import logging.LogEvent; +import logging.LogSink; +import logging.Marker; + +/** + * Test-only sink: counts events by level. Useful as a reference example for users + * writing their own custom sinks — see `doc/logging/usage/custom-sinks.md`. + */ +service CountingSink + implements LogSink { + + public/private Map counts = new HashMap(); + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return True; + } + + @Override + void log(LogEvent event) { + counts.process(event.level, e -> { + e.value = e.exists ? e.value + 1 : 1; + return Null; + }); + } +} diff --git a/lib_logging/src/test/x/LoggingTest/CountingSinkTest.x b/lib_logging/src/test/x/LoggingTest/CountingSinkTest.x new file mode 100644 index 0000000000..2e7abec45d --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/CountingSinkTest.x @@ -0,0 +1,25 @@ +import logging.BasicLogger; +import logging.Logger; + +/** + * Demonstrates that a custom `LogSink` can be plugged in and observed end-to-end. + * The sink is the per-level `CountingSink` defined alongside; this test class drives + * it through a `BasicLogger` and asserts the counts come out right. + */ +class CountingSinkTest { + + @Test + void shouldCountPerLevel() { + CountingSink sink = new CountingSink(); + Logger logger = new BasicLogger("counter", sink); + + logger.info ("a"); + logger.info ("b"); + logger.warn ("c"); + logger.error("d"); + + assert sink.counts[logging.Level.Info] == 2; + assert sink.counts[logging.Level.Warn] == 1; + assert sink.counts[logging.Level.Error] == 1; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/EmissionTest.x b/lib_logging/src/test/x/LoggingTest/EmissionTest.x new file mode 100644 index 0000000000..300c5dbc9c --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/EmissionTest.x @@ -0,0 +1,54 @@ +import logging.BasicLogger; +import logging.Logger; + +/** + * End-to-end emission tests using `BasicLogger` over a `ListLogSink`. Verifies that + * each per-level method routes the right `Level` and that the sink receives the + * correct event content. + */ +class EmissionTest { + + @Test + void shouldRouteInfo() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("emission.test", sink); + + logger.info("hello"); + + assert sink.events.size == 1; + assert sink.events[0].level == logging.Level.Info; + assert sink.events[0].loggerName == "emission.test"; + assert sink.events[0].message == "hello"; + } + + @Test + void shouldRouteAllLevels() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("all.levels", sink); + + logger.trace("t"); + logger.debug("d"); + logger.info ("i"); + logger.warn ("w"); + logger.error("e"); + + assert sink.events.size == 5; + assert sink.events[0].level == logging.Level.Trace; + assert sink.events[1].level == logging.Level.Debug; + assert sink.events[2].level == logging.Level.Info; + assert sink.events[3].level == logging.Level.Warn; + assert sink.events[4].level == logging.Level.Error; + } + + @Test + void shouldCarryException() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("with.cause", sink); + Exception boom = new Exception("boom"); + + logger.error("failure", cause=boom); + + assert sink.events.size == 1; + assert sink.events[0].exception == boom; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/FluentBuilderTest.x b/lib_logging/src/test/x/LoggingTest/FluentBuilderTest.x new file mode 100644 index 0000000000..4bfd8569b2 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/FluentBuilderTest.x @@ -0,0 +1,61 @@ +import logging.BasicLogger; +import logging.BasicMarker; +import logging.Logger; +import logging.Marker; + +/** + * Tests for the SLF4J 2.x style fluent event builder reachable via `logger.atInfo()`, + * `logger.atLevel(level)`, etc. Confirms that: + * - chaining accumulates state and a final `log(...)` materialises a single event; + * - the builder respects the underlying logger's level threshold; + * - markers, causes, and arguments survive the round-trip. + */ +class FluentBuilderTest { + + @Test + void shouldBuildAndEmitInfoEvent() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("fluent", sink); + + logger.atInfo() + .setMessage("processed") + .addArgument(42) + .log(); + + assert sink.events.size == 1; + assert sink.events[0].level == logging.Level.Info; + assert sink.events[0].message == "processed"; + } + + @Test + void shouldCarryMarkerAndCauseThroughBuilder() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("fluent.full", sink); + Marker audit = new BasicMarker("AUDIT"); + Exception cause = new Exception("network down"); + + logger.atError() + .addMarker(audit) + .setCause(cause) + .log("emit failed"); + + assert sink.events.size == 1; + assert sink.events[0].level == logging.Level.Error; + assert sink.events[0].marker?.name == "AUDIT" : assert; + assert sink.events[0].exception == cause; + } + + @Test + void shouldShortCircuitWhenLevelDisabled() { + ListLogSink sink = new ListLogSink(); + sink.setLevel(logging.Level.Warn); + Logger logger = new BasicLogger("fluent.disabled", sink); + + logger.atDebug() + .setMessage("expensive computation result") + .addArgument("would-have-been-formatted") + .log(); + + assert sink.events.size == 0; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/HierarchicalLogSinkTest.x b/lib_logging/src/test/x/LoggingTest/HierarchicalLogSinkTest.x new file mode 100644 index 0000000000..d2936f7efd --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/HierarchicalLogSinkTest.x @@ -0,0 +1,38 @@ +import logging.BasicLogger; +import logging.HierarchicalLogSink; +import logging.Logger; + +/** + * Tests Logback-style longest-prefix level configuration. + */ +class HierarchicalLogSinkTest { + + @Test + void shouldUseRootLevelWhenNoPrefixMatches() { + ListLogSink delegate = new ListLogSink(); + HierarchicalLogSink sink = new HierarchicalLogSink(delegate, logging.Level.Info); + Logger logger = new BasicLogger("other.component", sink); + + logger.debug("drop"); + logger.info("keep"); + + assert delegate.events.size == 1; + assert delegate.events[0].message == "keep"; + } + + @Test + void shouldUseLongestMatchingPrefix() { + ListLogSink delegate = new ListLogSink(); + HierarchicalLogSink sink = new HierarchicalLogSink(delegate, logging.Level.Warn); + Logger logger = new BasicLogger("payments.stripe.checkout", sink); + + sink.setLevel("payments", logging.Level.Info); + sink.setLevel("payments.stripe", logging.Level.Debug); + + logger.debug("debug stripe"); + logger.trace("drop trace"); + + assert delegate.events.size == 1; + assert delegate.events[0].message == "debug stripe"; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/JsonLogSinkTest.x b/lib_logging/src/test/x/LoggingTest/JsonLogSinkTest.x new file mode 100644 index 0000000000..4d75c64532 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/JsonLogSinkTest.x @@ -0,0 +1,55 @@ +import json.Doc; +import json.JsonObject; +import json.Parser; + +import logging.BasicLogger; +import logging.JsonLogSink; +import logging.JsonLogSinkOptions; +import logging.Logger; + +/** + * Tests the production JSON sink's semantic document shape. + */ +class JsonLogSinkTest { + + @Test + void shouldRenderStructuredEventWithSourceAndRedaction() { + JsonLogSink sink = new JsonLogSink( + new JsonLogSinkOptions(logging.Level.Info, ["card"])); + ListLogSink capture = new ListLogSink(); + Logger logger = new BasicLogger("json.payment", capture); + + logger.atInfo() + .addKeyValue("requestId", "r_42") + .addKeyValue("card", "4111111111111111") + .log("charged"); + + JsonObject obj = parseObject(sink.render(capture.events[0])); + + assert obj["logger"] == "json.payment"; + assert obj["message"] == "charged"; + assert obj["requestId"] == "r_42"; + assert obj["card"] == "***"; + } + + @Test + void shouldRenderLogAtSourceMetadata() { + JsonLogSink sink = new JsonLogSink(); + ListLogSink list = new ListLogSink(); + Logger probe = new BasicLogger("json.source", list); + + probe.logAt(logging.Level.Warn, "slow", "PaymentService.x", 42); + + JsonObject obj = parseObject(sink.render(list.events[0])); + assert obj["source"].is(JsonObject); + JsonObject source = obj["source"].as(JsonObject); + assert source["file"] == "PaymentService.x"; + assert source["line"] == 42; + } + + private JsonObject parseObject(String rendered) { + Doc doc = new Parser(rendered.toReader()).parseDoc(); + assert doc.is(JsonObject); + return doc.as(JsonObject); + } +} diff --git a/lib_logging/src/test/x/LoggingTest/LazyLoggingTest.x b/lib_logging/src/test/x/LoggingTest/LazyLoggingTest.x new file mode 100644 index 0000000000..28165281af --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/LazyLoggingTest.x @@ -0,0 +1,92 @@ +import logging.BasicLogger; +import logging.Logger; + +/** + * Verifies lazy logging semantics: supplier bodies must run only after the sink accepts the + * level/primary-marker check. This covers the Kotlin-style direct-message API and the + * SLF4J 2.x-style fluent builder. + */ +class LazyLoggingTest { + + @Test + void shouldNotEvaluateLazyMessageWhenLevelDisabled() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("lazy.message.disabled", sink); + @Volatile Int calls = 0; + + sink.setLevel(logging.Level.Info); + logger.debug(() -> { + ++calls; + return "expensive debug message"; + }); + + assert calls == 0; + assert sink.events.empty; + } + + @Test + void shouldEvaluateLazyMessageWhenEnabled() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("lazy.message.enabled", sink); + @Volatile Int calls = 0; + + logger.info(() -> { + ++calls; + return "expensive info message"; + }); + + assert calls == 1; + assert sink.events.size == 1; + assert sink.events[0].message == "expensive info message"; + } + + @Test + void shouldNotEvaluateLazyBuilderValuesWhenLevelDisabled() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("lazy.builder.disabled", sink); + @Volatile Int argCalls = 0; + @Volatile Int kvCalls = 0; + + sink.setLevel(logging.Level.Info); + logger.atDebug() + .addLazyArgument(() -> { + ++argCalls; + return "argument"; + }) + .addLazyKeyValue("requestId", () -> { + ++kvCalls; + return "r_42"; + }) + .log("value {}"); + + assert argCalls == 0; + assert kvCalls == 0; + assert sink.events.empty; + } + + @Test + void shouldEvaluateLazyBuilderValuesWhenEnabled() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("lazy.builder.enabled", sink); + @Volatile Int argCalls = 0; + @Volatile Int kvCalls = 0; + + logger.atInfo() + .addLazyArgument(() -> { + ++argCalls; + return "argument"; + }) + .addLazyKeyValue("requestId", () -> { + ++kvCalls; + return "r_42"; + }) + .log("value {}"); + + assert argCalls == 1; + assert kvCalls == 1; + assert sink.events.size == 1; + assert sink.events[0].message == "value argument"; + assert sink.events[0].arguments[0].as(String) == "argument"; + assert sink.events[0].keyValues.getOrNull("requestId").as(String) == "r_42"; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/LevelCheckTest.x b/lib_logging/src/test/x/LoggingTest/LevelCheckTest.x new file mode 100644 index 0000000000..69974d02ee --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/LevelCheckTest.x @@ -0,0 +1,55 @@ +import logging.BasicLogger; +import logging.Logger; + +/** + * Verifies the level-check fast path: events below the sink's threshold are dropped + * before any sink-side `log()` work happens, and the per-level `*Enabled` properties + * reflect the threshold faithfully. + */ +class LevelCheckTest { + + @Test + void shouldDropEventsBelowThreshold() { + ListLogSink sink = new ListLogSink(); + sink.setLevel(logging.Level.Warn); + Logger logger = new BasicLogger("threshold", sink); + + logger.trace("ignored"); + logger.debug("ignored"); + logger.info ("ignored"); + logger.warn ("kept"); + logger.error("kept"); + + assert sink.events.size == 2; + assert sink.events[0].level == logging.Level.Warn; + assert sink.events[1].level == logging.Level.Error; + } + + @Test + void shouldReportEnabledFlagsConsistentlyWithThreshold() { + ListLogSink sink = new ListLogSink(); + sink.setLevel(logging.Level.Info); + Logger logger = new BasicLogger("flags", sink); + + assert !logger.traceEnabled; + assert !logger.debugEnabled; + assert logger.infoEnabled; + assert logger.warnEnabled; + assert logger.errorEnabled; + } + + @Test + void shouldDropEverythingAtOff() { + ListLogSink sink = new ListLogSink(); + sink.setLevel(logging.Level.Off); + Logger logger = new BasicLogger("silenced", sink); + + logger.trace("nope"); + logger.debug("nope"); + logger.info ("nope"); + logger.warn ("nope"); + logger.error("nope"); + + assert sink.events.size == 0; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/LevelTest.x b/lib_logging/src/test/x/LoggingTest/LevelTest.x new file mode 100644 index 0000000000..97a4b7550c --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/LevelTest.x @@ -0,0 +1,35 @@ +import logging.Level; + +/** + * Tests for the `Level` enum and its severity ordering. + */ +class LevelTest { + + @Test + void shouldOrderBySeverity() { + assert logging.Level.Trace.severity < logging.Level.Debug.severity; + assert logging.Level.Debug.severity < logging.Level.Info.severity; + assert logging.Level.Info.severity < logging.Level.Warn.severity; + assert logging.Level.Warn.severity < logging.Level.Error.severity; + assert logging.Level.Error.severity < logging.Level.Off.severity; + } + + @Test + void shouldEnableEventsAtOrAboveThreshold() { + assert logging.Level.Info.enabledAtThreshold(logging.Level.Info); + assert logging.Level.Warn.enabledAtThreshold(logging.Level.Info); + assert logging.Level.Error.enabledAtThreshold(logging.Level.Info); + } + + @Test + void shouldDisableEventsBelowThreshold() { + assert !logging.Level.Trace.enabledAtThreshold(logging.Level.Info); + assert !logging.Level.Debug.enabledAtThreshold(logging.Level.Info); + } + + @Test + void shouldSilenceEverythingAtOff() { + assert !logging.Level.Trace.enabledAtThreshold(logging.Level.Off); + assert !logging.Level.Error.enabledAtThreshold(logging.Level.Off); + } +} diff --git a/lib_logging/src/test/x/LoggingTest/ListLogSink.x b/lib_logging/src/test/x/LoggingTest/ListLogSink.x new file mode 100644 index 0000000000..90738d9645 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/ListLogSink.x @@ -0,0 +1,52 @@ +import logging.Level; +import logging.LogEvent; +import logging.LogSink; +import logging.Marker; + +/** + * A test-only sink that captures every event in a linked list. Used by the test cases + * here as a stand-in for a "real" sink — it lets tests assert exactly what the + * `Logger` chose to emit, in order, without any formatting nondeterminism. + * + * `rootLevel` defaults to `Trace` so by default every emitted event is captured. + * + * Modelled as a `service` because it carries mutable state (`events[]`) shared across + * the fiber-under-test and the assertion code. See `doc/logging/design/design.md` + * ("Sink type: `const` vs `service`"). + */ +service ListLogSink + implements LogSink { + + public/private Level rootLevel = logging.Level.Trace; + + /** + * Mutable backing list. Internal — Ecstasy forbids returning a mutable array from a + * service call, so we expose [events] as an immutable snapshot below. + */ + private LogEvent[] eventList = new LogEvent[]; + + /** + * Immutable snapshot of the captured events, in emission order. Each access copies. + */ + @RO LogEvent[] events.get() { + return eventList.toArray(Constant); + } + + @Override + Boolean isEnabled(String loggerName, Level level, Marker? marker = Null) { + return level.severity >= rootLevel.severity; + } + + @Override + void log(LogEvent event) { + eventList.add(event); + } + + void setLevel(Level level) { + rootLevel = level; + } + + void reset() { + eventList.clear(); + } +} diff --git a/lib_logging/src/test/x/LoggingTest/MDCTest.x b/lib_logging/src/test/x/LoggingTest/MDCTest.x new file mode 100644 index 0000000000..75df2706e7 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/MDCTest.x @@ -0,0 +1,134 @@ +import logging.BasicLogger; +import logging.Logger; +import logging.MDC; + +/** + * Tests for [MDC]. Covers the SLF4J-style flat API (`put` / `get` / `remove` / `clear`) + * and the snapshot-on-emit path that lands MDC contents on `LogEvent.mdcSnapshot`. + * + * Per-fiber isolation across spawned services is exercised end-to-end by the manualTests + * `TestLogging.x` demo; that path needs runtime injection and a spawned service, which is + * outside the scope of these in-process unit tests. + * + * Each test calls `mdc.clear()` at the start because the test harness does not reset MDC + * state between tests and the underlying `SharedContext` is static. + */ +class MDCTest { + + @Test + void shouldStartEmpty() { + @Inject MDC mdc; + mdc.clear(); + + assert mdc.copyOfContextMap.empty; + assert mdc.get("anything") == Null; + } + + @Test + void shouldRoundTripPutAndGet() { + @Inject MDC mdc; + mdc.clear(); + + mdc.put("requestId", "r_42"); + assert mdc.get("requestId") == "r_42"; + } + + @Test + void shouldOverwriteOnRepeatedPut() { + @Inject MDC mdc; + mdc.clear(); + + mdc.put("attempt", "1"); + mdc.put("attempt", "2"); + assert mdc.get("attempt") == "2"; + assert mdc.copyOfContextMap.size == 1; + } + + @Test + void shouldRemoveKey() { + @Inject MDC mdc; + mdc.clear(); + + mdc.put("user", "u_7"); + mdc.put("session", "s_9"); + mdc.remove("user"); + + assert mdc.get("user") == Null; + assert mdc.get("session") == "s_9"; + } + + @Test + void shouldTreatPutNullAsRemove() { + @Inject MDC mdc; + mdc.clear(); + + mdc.put("key", "value"); + mdc.put("key", Null); + assert mdc.get("key") == Null; + } + + @Test + void shouldClearAllEntries() { + @Inject MDC mdc; + mdc.clear(); + + mdc.put("a", "1"); + mdc.put("b", "2"); + mdc.put("c", "3"); + mdc.clear(); + + assert mdc.copyOfContextMap.empty; + } + + @Test + void shouldAttachSnapshotToLogEvent() { + @Inject MDC mdc; + mdc.clear(); + + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("mdc.attach", sink); + + mdc.put("requestId", "r_99"); + mdc.put("user", "u_3"); + + logger.info("processing"); + + assert sink.events.size == 1; + assert sink.events[0].mdcSnapshot.size == 2; + assert sink.events[0].mdcSnapshot.getOrNull("requestId") == "r_99"; + assert sink.events[0].mdcSnapshot.getOrNull("user") == "u_3"; + } + + @Test + void shouldSnapshotIndependentlyOfSubsequentMutation() { + @Inject MDC mdc; + mdc.clear(); + + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("mdc.snap", sink); + + mdc.put("phase", "before"); + logger.info("first"); + + mdc.put("phase", "after"); + logger.info("second"); + + assert sink.events.size == 2; + assert sink.events[0].mdcSnapshot.getOrNull("phase") == "before"; + assert sink.events[1].mdcSnapshot.getOrNull("phase") == "after"; + } + + @Test + void shouldEmitEmptyMapWhenMDCIsClear() { + @Inject MDC mdc; + mdc.clear(); + + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("mdc.empty", sink); + + logger.info("no context"); + + assert sink.events.size == 1; + assert sink.events[0].mdcSnapshot.empty; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/MarkerFactoryTest.x b/lib_logging/src/test/x/LoggingTest/MarkerFactoryTest.x new file mode 100644 index 0000000000..c8614c71fe --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/MarkerFactoryTest.x @@ -0,0 +1,43 @@ +import logging.Marker; +import logging.MarkerFactory; + +/** + * Tests for `MarkerFactory` interning semantics. SLF4J users expect + * `MarkerFactory.getMarker("AUDIT")` to return a canonical marker for that name, while + * `getDetachedMarker(...)` returns an unregistered, one-off marker. + */ +class MarkerFactoryTest { + + @Test + void shouldInternMarkersByName() { + MarkerFactory factory = new MarkerFactory(); + + Marker first = factory.getMarker("AUDIT"); + Marker second = factory.getMarker("AUDIT"); + + assert &first == &second; + assert first.name == "AUDIT"; + } + + @Test + void shouldTrackInternedMarkerExistence() { + MarkerFactory factory = new MarkerFactory(); + + assert !factory.exists("SECURITY"); + + factory.getMarker("SECURITY"); + + assert factory.exists("SECURITY"); + } + + @Test + void shouldCreateDetachedMarkersOutsideTheInternCache() { + MarkerFactory factory = new MarkerFactory(); + + Marker detached = factory.getDetachedMarker("AUDIT"); + + assert !factory.exists("AUDIT"); + Marker interned = factory.getMarker("AUDIT"); + assert &detached != &interned; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/MarkerTest.x b/lib_logging/src/test/x/LoggingTest/MarkerTest.x new file mode 100644 index 0000000000..140aa23a09 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/MarkerTest.x @@ -0,0 +1,108 @@ +import logging.BasicLogger; +import logging.BasicMarker; +import logging.Logger; +import logging.Marker; + +import xunit.assertions.assertThrows; + +/** + * Tests for `Marker` semantics: equality by name, transitive containment, and end-to- + * end propagation through a `Logger` to a sink. + */ +class MarkerTest { + + @Test + void shouldContainSelf() { + Marker m = new BasicMarker("AUDIT"); + assert m.contains(m); + assert m.containsName("AUDIT"); + } + + @Test + void shouldContainTransitively() { + Marker security = new BasicMarker("SECURITY"); + Marker audit = new BasicMarker("AUDIT"); + Marker breach = new BasicMarker("BREACH"); + + security.add(audit); + audit.add(breach); + + assert security.contains(audit); + assert security.contains(breach); // transitive + assert security.containsName("BREACH"); + } + + @Test + void shouldNotContainUnrelated() { + Marker a = new BasicMarker("A"); + Marker b = new BasicMarker("B"); + assert !a.contains(b); + assert !a.containsName("B"); + } + + @Test + void shouldRejectSelfReference() { + Marker a = new BasicMarker("A"); + + IllegalArgument e = assertThrows(() -> a.add(a)); + assert e.message.indexOf("cycle"); + } + + @Test + void shouldRejectReciprocalReference() { + Marker a = new BasicMarker("A"); + Marker b = new BasicMarker("B"); + + a.add(b); + + IllegalArgument e = assertThrows(() -> b.add(a)); + assert e.message.indexOf("cycle"); + } + + @Test + void shouldPropagateMarkerOnEvent() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("with.marker", sink); + Marker audit = new BasicMarker("AUDIT"); + + logger.info("authenticated", marker=audit); + + assert sink.events.size == 1; + assert sink.events[0].markers.size == 1; + assert sink.events[0].markers[0].name == "AUDIT"; + // Backwards-compat single-marker accessor. + assert sink.events[0].marker?.name == "AUDIT" : assert; + } + + @Test + void shouldPropagateMultipleMarkersThroughFluentBuilder() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("multi.marker", sink); + Marker audit = new BasicMarker("AUDIT"); + Marker security = new BasicMarker("SECURITY"); + + logger.atInfo() + .addMarker(audit) + .addMarker(security) + .log("login attempt"); + + assert sink.events.size == 1; + assert sink.events[0].markers.size == 2; + assert sink.events[0].markers[0].name == "AUDIT"; + assert sink.events[0].markers[1].name == "SECURITY"; + // The single-marker compat accessor returns the first one. + assert sink.events[0].marker?.name == "AUDIT" : assert; + } + + @Test + void shouldEmitNoMarkersWhenNoneAttached() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("no.marker", sink); + + logger.info("plain message"); + + assert sink.events.size == 1; + assert sink.events[0].markers.empty; + assert sink.events[0].marker == Null; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/MessageFormatterTest.x b/lib_logging/src/test/x/LoggingTest/MessageFormatterTest.x new file mode 100644 index 0000000000..3ee2e36366 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/MessageFormatterTest.x @@ -0,0 +1,91 @@ +import logging.MessageFormatter; + +/** + * Tests for [MessageFormatter] — the SLF4J-shaped `{}` substitution + throwable promotion + * helper. Cases are lifted from the SLF4J reference suite to keep behavioural parity + * obvious. + */ +class MessageFormatterTest { + + @Test + void shouldReturnPatternUnchangedWhenNoArgs() { + (String formatted, Exception? cause) = MessageFormatter.format("hello world", []); + assert formatted == "hello world"; + assert cause == Null; + } + + @Test + void shouldSubstituteSinglePlaceholder() { + (String formatted, Exception? cause) = MessageFormatter.format("hello {}", ["world"]); + assert formatted == "hello world"; + assert cause == Null; + } + + @Test + void shouldSubstituteMultiplePlaceholders() { + (String formatted, _) = MessageFormatter.format("a={} b={} c={}", ["1", "2", "3"]); + assert formatted == "a=1 b=2 c=3"; + } + + @Test + void shouldLeaveExcessPlaceholdersLiteral() { + (String formatted, _) = MessageFormatter.format("a={} b={}", ["only-one"]); + assert formatted == "a=only-one b={}"; + } + + @Test + void shouldDropExcessArguments() { + (String formatted, _) = MessageFormatter.format("a={}", ["used", "extra"]); + assert formatted == "a=used"; + } + + @Test + void shouldEscapePlaceholderWithSingleBackslash() { + // "\{}" — backslash is consumed, "{}" is emitted literally + (String formatted, _) = MessageFormatter.format("escaped: \\{}", ["X"]); + assert formatted == "escaped: {}"; + } + + @Test + void shouldKeepLiteralBackslashWithDoubleEscape() { + // "\\{}" — both backslashes consumed (one literal '\'), placeholder substitutes + (String formatted, _) = MessageFormatter.format("\\\\{}", ["X"]); + assert formatted == "\\X"; + } + + @Test + void shouldHandleAdjacentPlaceholders() { + (String formatted, _) = MessageFormatter.format("{}{}{}", ["a", "b", "c"]); + assert formatted == "abc"; + } + + @Test + void shouldPromoteTrailingException() { + Exception boom = new Exception("boom"); + (String formatted, Exception? cause) = + MessageFormatter.format("op failed: {}", ["disk", boom]); + assert formatted == "op failed: disk"; + assert cause == boom; + } + + @Test + void shouldPromoteTrailingExceptionWithoutPlaceholder() { + Exception boom = new Exception("boom"); + (String formatted, Exception? cause) = + MessageFormatter.format("no placeholder", [boom]); + assert formatted == "no placeholder"; + assert cause == boom; + } + + @Test + void shouldHandleStrayBraces() { + (String formatted, _) = MessageFormatter.format("a { b } c {} d", ["X"]); + assert formatted == "a { b } c X d"; + } + + @Test + void shouldHandleEmptyPattern() { + (String formatted, _) = MessageFormatter.format("", ["unused"]); + assert formatted == ""; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/NamedLoggerTest.x b/lib_logging/src/test/x/LoggingTest/NamedLoggerTest.x new file mode 100644 index 0000000000..ce4ff648c6 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/NamedLoggerTest.x @@ -0,0 +1,98 @@ +import logging.BasicLogger; +import logging.Logger; +import logging.LoggerRegistry; + +/** + * Tests for `Logger.named(String)` — the SLF4J `LoggerFactory.getLogger(class)` analogue. + * The derived logger must: + * - carry the supplied `name`; + * - share the parent's `sink`, so `LogEvent.loggerName` reflects the derived name on + * events the *child* emits while events emitted from the parent keep the parent's + * name; + * - obey the parent's level threshold (because the threshold lives on the sink). + */ +class NamedLoggerTest { + + @Test + void shouldDeriveLoggerWithSuppliedName() { + ListLogSink sink = new ListLogSink(); + Logger root = new BasicLogger("root", sink); + Logger child = root.named("payments"); + + assert child.name == "payments"; + } + + @Test + void shouldRouteChildEventsThroughSharedSink() { + ListLogSink sink = new ListLogSink(); + Logger root = new BasicLogger("root", sink); + Logger child = root.named("payments"); + + root.info("from root"); + child.info("from child"); + + assert sink.events.size == 2; + assert sink.events[0].loggerName == "root"; + assert sink.events[1].loggerName == "payments"; + } + + @Test + void shouldRespectParentSinkLevelThreshold() { + ListLogSink sink = new ListLogSink(); + sink.setLevel(logging.Level.Warn); + Logger root = new BasicLogger("root", sink); + Logger child = root.named("noisy"); + + // The child's level threshold is its sink's threshold (which is the parent's sink), + // so a Debug event from the child is dropped just like one from the parent. + child.debug("muted"); + child.warn ("audible"); + + assert sink.events.size == 1; + assert sink.events[0].message == "audible"; + assert sink.events[0].loggerName == "noisy"; + } + + @Test + void shouldChainNaming() { + ListLogSink sink = new ListLogSink(); + Logger root = new BasicLogger("root", sink); + Logger a = root.named("a"); + Logger ab = a.named("a.b"); + + assert a.name == "a"; + assert ab.name == "a.b"; + + ab.info("hi"); + assert sink.events.size == 1; + assert sink.events[0].loggerName == "a.b"; + } + + @Test + void shouldInternChildrenWhenRegistryAttached() { + ListLogSink sink = new ListLogSink(); + LoggerRegistry registry = new LoggerRegistry(sink); + + Logger root = registry.ensure("root"); + Logger rootAgain = registry.ensure("root"); + // Two registry lookups for the same name return the same instance. + assert &root == &rootAgain; + + // `named` consults the registry when one is attached, so identity is also + // stable when reaching the same logger by different paths. + Logger ab1 = root.named("a.b"); + Logger ab2 = registry.ensure("a.b"); + assert &ab1 == &ab2; + } + + @Test + void shouldNotInternWhenRegistryAbsent() { + ListLogSink sink = new ListLogSink(); + Logger root = new BasicLogger("root", sink); // no registry + + // Without a registry, each `named` call allocates fresh. + Logger a1 = root.named("a"); + Logger a2 = root.named("a"); + assert &a1 != &a2; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/SourceLocationTest.x b/lib_logging/src/test/x/LoggingTest/SourceLocationTest.x new file mode 100644 index 0000000000..9a795d61a3 --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/SourceLocationTest.x @@ -0,0 +1,31 @@ +import logging.BasicLogger; +import logging.Logger; + +/** + * Tests explicit source-location capture in the SLF4J-shaped facade. + */ +class SourceLocationTest { + + @Test + void shouldPopulateSourceFieldsWhenUsingLogAt() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("source.test", sink); + + logger.logAt(logging.Level.Warn, "slow payment", "PaymentService.x", 77); + + assert sink.events.size == 1; + assert sink.events[0].sourceFile == "PaymentService.x"; + assert sink.events[0].sourceLine == 77; + } + + @Test + void shouldLeaveSourceFieldsEmptyForNormalLogCalls() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("source.normal", sink); + + logger.info("normal"); + + assert sink.events[0].sourceFile == Null; + assert sink.events[0].sourceLine == -1; + } +} diff --git a/lib_logging/src/test/x/LoggingTest/StructuredLoggingTest.x b/lib_logging/src/test/x/LoggingTest/StructuredLoggingTest.x new file mode 100644 index 0000000000..065719c83c --- /dev/null +++ b/lib_logging/src/test/x/LoggingTest/StructuredLoggingTest.x @@ -0,0 +1,78 @@ +import logging.BasicLogger; +import logging.Logger; + +/** + * Bare-minimum SLF4J 2.x structured-logging test: verifies that + * `LoggingEventBuilder.addKeyValue(...)` actually flows through to a `LogEvent.keyValues` + * map that sinks can read. Mirrors SLF4J's KV-on-builder behaviour without taking on its + * full `KeyValuePair` data model — for v0 a `Map` is enough. + */ +class StructuredLoggingTest { + + @Test + void shouldCarrySingleKeyValueThroughBuilder() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("kv.single", sink); + + logger.atInfo() + .addKeyValue("requestId", "r_42") + .log("payment processed"); + + assert sink.events.size == 1; + assert sink.events[0].message == "payment processed"; + assert sink.events[0].keyValues.size == 1; + assert sink.events[0].keyValues.getOrNull("requestId").as(String) == "r_42"; + } + + @Test + void shouldCarryMultipleKeyValuesAndPreserveOrder() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("kv.many", sink); + + Int amount = 100; + logger.atInfo() + .addKeyValue("requestId", "r_99") + .addKeyValue("userId", "u_7") + .addKeyValue("amount", amount) + .log("checkout completed"); + + assert sink.events.size == 1; + assert sink.events[0].keyValues.size == 3; + + // ListMap preserves insertion order; this matches SLF4J's KeyValuePair list ordering. + String[] orderedKeys = sink.events[0].keyValues.keys.toArray(); + assert orderedKeys[0] == "requestId"; + assert orderedKeys[1] == "userId"; + assert orderedKeys[2] == "amount"; + + assert sink.events[0].keyValues.getOrNull("amount").as(Int) == 100; + } + + @Test + void shouldEmitEmptyKeyValuesWhenBuilderHasNone() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("kv.empty", sink); + + logger.atInfo().log("no extras"); + + assert sink.events.size == 1; + assert sink.events[0].keyValues.empty; + } + + @Test + void shouldOverwriteValueOnDuplicateKey() { + ListLogSink sink = new ListLogSink(); + Logger logger = new BasicLogger("kv.dup", sink); + + Int first = 1; + Int second = 2; + logger.atInfo() + .addKeyValue("attempt", first) + .addKeyValue("attempt", second) + .log("retry"); + + assert sink.events.size == 1; + assert sink.events[0].keyValues.size == 1; + assert sink.events[0].keyValues.getOrNull("attempt").as(Int) == 2; + } +} diff --git a/lib_slogging/README.md b/lib_slogging/README.md new file mode 100644 index 0000000000..3260d4f73a --- /dev/null +++ b/lib_slogging/README.md @@ -0,0 +1,33 @@ +# lib_slogging + +Go `log/slog`-shaped structured logging library for Ecstasy. Comparison POC +that lives beside [`lib_logging`](../lib_logging) so reviewers can pick between +two API shapes: + +```ecstasy +Logger logger = new Logger(new TextHandler()); + +Logger requestLog = logger.with([ + Attr.of("requestId", req.id), + Attr.of("user", req.userId), +]); + +requestLog.info("payment processed", [ + Attr.of("amount", amount), + Attr.of("currency", currency), +]); +``` + +Attribute-first events: no markers, no MDC, no `{}` interpolation in the base +shape. Categorisation, errors, and structured payload all travel as `Attr` +values. Backends implement the small `Handler` interface. + +> **POC name.** This module exists only to compare the slog shape against the +> SLF4J shape. Whichever wins ships as `lib_logging`; this module is not +> intended as a permanent second facade. + +## Documentation + +All design docs live at the repo root under [`doc/logging/`](../doc/logging). +Start at [`doc/logging/README.md`](../doc/logging/README.md) for goals, +requirements, status, reading paths, and the full doc index. diff --git a/lib_slogging/build.gradle.kts b/lib_slogging/build.gradle.kts new file mode 100644 index 0000000000..d1ae7ef9b3 --- /dev/null +++ b/lib_slogging/build.gradle.kts @@ -0,0 +1,12 @@ +plugins { + alias(libs.plugins.xtc) +} + +dependencies { + xdkJavaTools(libs.javatools) + xtcModule(libs.xdk.ecstasy) + xtcModule(libs.xdk.convert) + xtcModule(libs.xdk.json) + xtcModuleTest(libs.javatools.bridge) + xtcModuleTest(libs.xdk.xunit.engine) +} diff --git a/lib_slogging/src/main/x/slogging.x b/lib_slogging/src/main/x/slogging.x new file mode 100644 index 0000000000..273ad03c35 --- /dev/null +++ b/lib_slogging/src/main/x/slogging.x @@ -0,0 +1,101 @@ +/** + * Corresponds, as a whole module, to Go 1.21+'s `log/slog` package — plus a minimum + * viable handler so the library is usable out of the box. Provided as a parallel design + * exercise alongside [`logging.xtclang.org`][lib_logging] (the SLF4J-shaped library) so + * reviewers can compare the two designs side-by-side. + * + * Ecstasy structured-logging library, slog-shaped. + * + * The module is intentionally shaped to be _instantly familiar_ to anyone who has used + * `log/slog` in Go: a top-level `Logger` carrying a `Handler`, derived loggers via + * `Logger.with(attrs)`, optional context propagation via `LoggerContext`, no "marker" + * concept (categorisation is just attributes), open-ended integer-based `Level` values. + * + * The primary entry point is injection: + * + * @Inject Logger logger; + * + * Acquiring loggers without injection is supported by constructing one directly: + * + * Logger logger = new Logger(new TextHandler()); + * + * Derived loggers are pure construction: + * + * Logger requestLogger = logger.with(Map:[ + * "requestId" = req.id, + * "user" = req.userId, + * ]); + * + * # Status + * + * **This module is a working comparison POC.** The core API, level checks, derived + * loggers, groups, custom levels, lazy message suppliers, runtime injection, source + * metadata, context binding, and memory/text/JSON/no-op handlers are implemented and + * covered by unit tests. + * + * # API / Implementation boundary + * + * The public API consists of: + * - [Logger] — the user-facing concrete `const` + * - [Level] — open-ended severity (Int + String label) + * - [Attributes] — typedef for `Map` carried as structured data + * - [Record] — immutable record of a single log call (LogEvent equivalent) + * - [Handler] — the SPI that backends implement (LogSink equivalent) + * - [LoggerContext] — optional SharedContext helper for request-scoped loggers + * + * The implementation side of that boundary contains: + * - [TextHandler] — default human-readable text handler, writes via `@Inject Console` + * - [JSONHandler] — JSON-Lines structured handler, rendered by `lib_json` + * - [HandlerOptions] — threshold, redaction, source, and field-name options + * - [AsyncHandler] — bounded async wrapper for slow handlers + * - [BoundHandler] — derivation wrapper used by handlers that do not cache prefixes + * - [NopHandler] — drops every record + * - [MemoryHandler] — captures records in memory; useful in tests + * + * # See also + * + * doc/logging/lib-logging-vs-lib-slogging.md — the design comparison document + * doc/logging/api-cross-reference.md — official Go slog links mapped to these types + * doc/logging/open-questions.md — list of reviewer questions (Q-D6) + * lib_logging/src/main/x/logging.x — the SLF4J-shaped sibling library + */ +module slogging.xtclang.org { + package convert import convert.xtclang.org; + package json import json.xtclang.org; + + /** + * Lazy message supplier used by [Logger]. + * + * The logger invokes this function only after [Handler.enabled] accepts the record's + * level. + */ + typedef function AnyValue() as MessageSupplier; + + /** + * AnyValue is used to represent various values of specific types in log message attributes. + * + * An AnyValue is either: + * + * - a [PrimitiveValue] (Null, Boolean, String, integer or floating-point number, or byte + * array), + * - an array of AnyValue, + * - a Map. + * + * Arbitrary deep nesting of values for arrays and maps is allowed (essentially allows to + * represent an equivalent of a JSON object). + * Using array and map values may carry a higher performance overhead compared to primitive + * values. + */ + typedef PrimitiveValue | AnyValue[] | Map as AnyValue; + + /** + * The various primitive values allowed in an AnyValue. + */ + typedef Nullable | Boolean | String | IntNumber | IntLiteral | Float | FPLiteral | Byte[] + as PrimitiveValue; + + /** + * A map of attribute key-value pairs carried by resources, scopes, and data points. + */ + typedef Map as Attributes; +} diff --git a/lib_slogging/src/main/x/slogging/AsyncHandler.x b/lib_slogging/src/main/x/slogging/AsyncHandler.x new file mode 100644 index 0000000000..86f9498432 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/AsyncHandler.x @@ -0,0 +1,102 @@ +/** + * Async wrapper for a slog [Handler]. + * + * Records are fully constructed before they enter this handler, so delayed emission + * preserves attributes, groups, source metadata, and exceptions exactly as the caller + * produced them. + */ +service AsyncHandler(Handler delegate, Int capacity = 1024) + implements Handler { + + private Record[] queue = new Record[]; + private Boolean draining = False; + private Boolean closed = False; + + /** + * Count of records dropped because the queue was full or closed. + */ + public/private Int droppedCount = 0; + + /** + * Current queue depth. Intended for tests and operational probes. + */ + @RO Int pending.get() = queue.size; + + @Override + Boolean enabled(Level level) = delegate.enabled(level); + + @Override + void handle(Record record) { + if (closed || queue.size >= capacity) { + ++droppedCount; + return; + } + + queue.add(record); + if (!draining) { + draining = True; + drain^(); + } + } + + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new AsyncHandler(delegate.withAttributes(attributes), capacity); + + @Override + Handler withGroup(String name) + = name.empty ? this : new AsyncHandler(delegate.withGroup(name), capacity); + + /** + * Synchronously drain currently queued records. + */ + void flush() = drain(); + + /** + * Stop accepting new records. By default this drains pending records first. + */ + void close(Boolean flush = True) { + closed = True; + if (flush) { + drain(); + } else { + droppedCount += queue.size; + queue.clear(); + draining = False; + } + } + + /** + * Drain the queue on this service's fiber. + * + * The same two safety properties as `lib_logging.AsyncLogSink.drain` apply here — + * see that file's `drain` doc-comment for the full rationale. Briefly: + * + * 1. `draining` must be cleared on every exit path (`try/finally`) so a delegate + * exception cannot strand the wrapper in the "drain already scheduled" state, + * after which `handle()` would silently drop everything via `droppedCount`. + * 2. Per-record failure is contained so one bad record does not prevent the rest + * of the batch from draining. + * + * The swap-out batch (`batch = queue; queue = new Record[]`) replaces the original + * O(n) `queue.delete(0)` per record. Service-fiber serialisation guarantees the + * swap is atomic with respect to other `handle()` calls. + */ + private void drain() { + try { + while (!queue.empty) { + Record[] batch = queue; + queue = new Record[]; + for (Record record : batch) { + try { + delegate.handle(record); + } catch (Exception e) { + ++droppedCount; + } + } + } + } finally { + draining = False; + } + } +} diff --git a/lib_slogging/src/main/x/slogging/BoundHandler.x b/lib_slogging/src/main/x/slogging/BoundHandler.x new file mode 100644 index 0000000000..8cdd031ef4 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/BoundHandler.x @@ -0,0 +1,91 @@ +/** + * Handler derivation wrapper used by shipped handlers that do not maintain their own + * cached prefix representation. + * + * Go slog puts the power of `Logger.With` and `Logger.WithGroup` at the handler + * boundary: a derived logger contains a derived handler. A production handler can + * override [withAttributes] / [withGroup] to pre-render JSON fragments, allocate a cached text + * prefix, or fold attributes into a backend-specific context object. `BoundHandler` + * supplies the default implementation for handlers that only need correct semantics. + * + * The wrapper composes in the same order as Go slog: + * + * logger.with(Map:["env"="prod"]) + * .withGroup("payments") + * .info("charged", Map:["amount"=1099]); + * + * yields `env=prod` outside the `payments` group, while: + * + * logger.withGroup("payments") + * .with(Map:["env"="prod"]) + * .info("charged", Map:["amount"=1099]); + * + * yields both `env` and `amount` inside the `payments` group. + */ +const BoundHandler(Handler delegate, Attributes attributes = [], String? groupName = Null) + implements Handler { + + /** + * Fast-path level check. Bound attributes and groups never affect enablement. + */ + @Override + Boolean enabled(Level level) = delegate.enabled(level); + + /** + * Apply the bound attributes/group to the record, then forward to the delegate. + */ + @Override + void handle(Record record) { + Attributes merged = merge(attributes, record.attributes); + if (String group ?= groupName) { + if (!merged.empty) { + ListMap wrapped = new ListMap(); + wrapped.put(group, merged); + merged = wrapped.makeImmutable(); + } + } + + delegate.handle(new Record( + timestamp = record.timestamp, + message = record.message, + level = record.level, + attributes = merged, + exception = record.exception, + sourceFile = record.sourceFile, + sourceLine = record.sourceLine, + threadName = record.threadName, + )); + } + + /** + * Stack another attribute derivation around this handler. The wrapper order is what + * preserves the difference between `With(...).WithGroup(...)` and + * `WithGroup(...).With(...)`. + */ + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new BoundHandler(delegate=this, attributes=attributes); + + /** + * Stack another group derivation around this handler. + */ + @Override + Handler withGroup(String name) + = name.empty ? this : new BoundHandler(delegate=this, groupName=name); + + /** + * Immutable concatenation helper preserving insertion order. + */ + private Attributes merge(Attributes first, Attributes second) { + if (first.empty) { + return second; + } + if (second.empty) { + return first; + } + ListMap result = new ListMap(first.size + second.size); + result.putAll(first); + result.putAll(second); + return result.makeImmutable(); + } +} diff --git a/lib_slogging/src/main/x/slogging/Handler.x b/lib_slogging/src/main/x/slogging/Handler.x new file mode 100644 index 0000000000..340583b656 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/Handler.x @@ -0,0 +1,67 @@ +/** + * Corresponds to `log/slog.Handler` (`go.dev/src/log/slog/handler.go`). The SPI a + * logging backend implements — analogous to the SLF4J-shaped library's + * [logging.LogSink], but with a richer surface that lets the handler pre-resolve + * derivations. + * + * # The contract + * + * Boolean enabled(Level level) — cheap fast-path filter + * void handle(Record record) — emit the record + * Handler withAttributes(Attributes attributes) — derive a pre-bound handler + * Handler withGroup(String name) — derive a name-prefixed handler + * + * The two extra methods exist so that when user code does `logger.with(attributes)` once and + * then logs millions of records through the derived logger, the handler can do the + * "merge attributes into the namespace" work *once* at derivation time rather than on every + * `handle` call. + * + * Handlers that do not maintain their own derived representation can return + * [BoundHandler] from both derivation methods. Handlers with a native prefix/cache can + * return their own implementation instead. A true discard handler such as [NopHandler] + * can still return `this`. + * + * # Choosing between `const` and `service` for an implementation + * + * Same rule as `lib_logging`'s [logging.LogSink]: + * + * - `const` — for stateless forwarders / pure adapters. Examples: `TextHandler`, + * `NopHandler`, `JSONHandler`. Construction-time configuration only. + * - `service` — for handlers that carry mutable state shared across fibers. Examples: + * `MemoryHandler` (collects records), `AsyncHandler` (owns a worker + * queue), and a future `FileHandler` (owns a writer). + * + * `Logger` is a `const` and references its `handler` through this interface, so every + * concrete `Handler` must be `Passable` (`immutable` or a `service`). The full rule and + * the citations to `lib_xunit_engine` / `platform/common` are in `design.md` ("Sink + * type: `const` vs `service`") — same rule, both libraries. + */ +interface Handler { + + /** + * Cheap level check. Should be safe to call on every emission. + */ + Boolean enabled(Level level); + + /** + * Emit the supplied record. + * + * Must not throw under normal conditions; handlers that experience an internal + * failure (disk full, remote endpoint down) are expected to degrade gracefully — + * typically by writing to standard error — rather than propagate. + */ + void handle(Record record); + + /** + * Return a handler that carries the supplied attributes pre-bound. Use + * [BoundHandler] for the default semantics, or return a handler-specific derived + * instance that caches a rendered prefix. + */ + Handler withAttributes(Attributes attributes); + + /** + * Return a handler that namespaces subsequent attributes under the supplied group + * name. Use [BoundHandler] for the default semantics. + */ + Handler withGroup(String name); +} diff --git a/lib_slogging/src/main/x/slogging/HandlerOptions.x b/lib_slogging/src/main/x/slogging/HandlerOptions.x new file mode 100644 index 0000000000..bcebf461fa --- /dev/null +++ b/lib_slogging/src/main/x/slogging/HandlerOptions.x @@ -0,0 +1,56 @@ +/** + * Shared production options for text/JSON handlers. + * + * The slog API keeps caller semantics in `Logger`/`Record`/`Attributes`; formatting policy + * belongs in handlers. This const captures the first production knobs without turning + * the POC into a full configuration system. + */ +const HandlerOptions( + /** + * Lowest level a record must have to pass the handler. + */ + Level rootLevel, + /** + * Attribute keys whose values should be replaced with [redaction]. + */ + String[] redactedKeys, + /** + * Replacement value emitted for redacted attributes. + */ + String redaction, + /** + * True to render `Record.sourceFile` / `sourceLine` when present. + */ + Boolean includeSource, + /** + * Field name for the timestamp. + */ + String timeKey, + /** + * Field name for the level label. + */ + String levelKey, + /** + * Field name for the human message. + */ + String messageKey, + /** + * Field name for source metadata. + */ + String sourceKey, + /** + * Field name for structured exception data. + */ + String exceptionKey, + ) { + + construct(Level rootLevel = Level.Info, String[] redactedKeys = []) { + construct HandlerOptions(rootLevel, redactedKeys, "***", True, + "timestamp", "level", "msg", "source", "exception"); + } + + /** + * True iff the supplied key should be rendered as [redaction]. + */ + Boolean redacts(String key) = redactedKeys.contains(key); +} diff --git a/lib_slogging/src/main/x/slogging/JSONHandler.x b/lib_slogging/src/main/x/slogging/JSONHandler.x new file mode 100644 index 0000000000..5cea26c6c0 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/JSONHandler.x @@ -0,0 +1,193 @@ +import convert.formats.Base64Format; + +import json.Doc; +import json.JsonArray; +import json.JsonObject; +import json.Printer; + +/** + * Corresponds to `log/slog.JSONHandler` (`go.dev/src/log/slog/json_handler.go`). The + * structured-output handler — emits one compact JSON object per record on its own line. + * + * Rendering is delegated to `lib_json`, so strings are escaped by the platform JSON + * printer rather than by ad-hoc concatenation. Groups are preserved as nested objects, + * source metadata is emitted under `"source"`, and exceptions are represented + * structurally. + */ +const JSONHandler + implements Handler { + + /** + * Create a [JSONHandler]. + * + * @param options (optional) the handler's options + * @param groupName (optional) the handler's group name + * @param consumer (optional) the consumer that will process the [JsonObject] produced from + * the log [Record] (the default will print the json to the console) + */ + construct(HandlerOptions? options = Null, String groupName = "", JsonConsumer? consumer = Null) { + this.options = options ?: new HandlerOptions(); + this.groupName = groupName; + this.consumer = consumer ?: defaultConsumer; + } + + typedef function void (JsonObject) as JsonConsumer; + + HandlerOptions options; + + String groupName; + + JsonConsumer consumer; + + /** + * Cheap threshold check; no JSON work happens for disabled records. + */ + @Override + Boolean enabled(Level level) = level.enabledAtThreshold(options.rootLevel); + + /** + * Render and print one JSON line. + */ + @Override + void handle(Record record) = consumer(toJson(record)); + + private static void defaultConsumer(JsonObject obj) { + @Inject Console console; + console.print(Printer.DEFAULT.render(obj)); + } + + /** + * Convert a record into a JSON document. This is the handler's semantic core; `handle` + * only prints the rendered document. + */ + JsonObject toJson(Record record) { + JsonObject obj = json.newObject(); + IntLiteral nanos = (record.timestamp.epochPicos / 1000).toIntLiteral(); // epoch nanos + obj.put(options.timeKey, nanos); + obj.put(options.levelKey, record.level.label); + obj.put(options.messageKey, encodeAnyValue(record.message)); + + JsonObject attrTarget = groupName.empty ? obj : ensureObject(obj, groupName); + addAttributes(attrTarget, record.attributes); + + if (Exception e ?= record.exception) { + obj.put(options.exceptionKey, exceptionJson(e)); + } + + if (options.includeSource) { + if (String file ?= record.sourceFile) { + JsonObject source = json.newObject(); + source.put("file", file); + if (record.sourceLine >= 0) { + source.put("line", record.sourceLine.toIntLiteral()); + } + obj.put(options.sourceKey, source.makeImmutable()); + } + } + + if (!record.threadName.empty) { + obj.put("thread", record.threadName); + } + + return obj.makeImmutable(); + } + + /** + * Return a handler with pre-bound attributes. This implementation uses [BoundHandler]; + * a lower-level production sink could override this method to cache a serialized + * prefix instead. + */ + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new BoundHandler(delegate=this, attributes=attributes); + + /** + * Return a handler that nests subsequent attributes under `name`. + */ + @Override + Handler withGroup(String name) + = name.empty ? this : new BoundHandler(delegate=this, groupName=name); + + /** + * Add all attributes into a JSON object, preserving slog groups as nested objects. + */ + private void addAttributes(JsonObject obj, Attributes attributes) { + for ((String key, AnyValue value) : attributes) { + obj.put(key, options.redacts(key) ? options.redaction : encodeAnyValue(value)); + } + } + + /** + * Convert an [AnyValue] value to a JSON document. + */ + private Doc encodeAnyValue(AnyValue value) { + if (String s := value.is(String)) { + return s; + } + if (Boolean b := value.is(Boolean)) { + return b; + } + if (IntLiteral n := value.is(IntLiteral)) { + return n; + } + if (FPLiteral n := value.is(FPLiteral)) { + return n; + } + if (IntNumber n := value.is(IntNumber)) { + return n.toIntLiteral(); + } + if (FPNumber n := value.is(FPNumber)) { + return n.toFPLiteral(); + } + if (value.is(Byte[])) { + return Base64Format.Instance.encode(value); + } + if (value.is(AnyValue[])) { + JsonArray array = json.newArray(); + for (AnyValue element : value) { + array.add(encodeAnyValue(element)); + } + return array; + } + if (value.is(Map)) { + JsonObject entries = json.newObject(); + for ((String k, AnyValue v) : value) { + entries.put(k, encodeAnyValue(v)); + } + return entries; + } + assert as $"Unhandled AnyValue to JSON conversion {&value.type}"; + } + + /** + * Represent an exception structurally. Stack frames are intentionally omitted from + * the POC because `Exception.formatStackTrace()` is still a TODO in `lib_ecstasy`. + */ + private JsonObject exceptionJson(Exception e) { + JsonObject obj = json.newObject(); + obj.put("type", &e.class.name); + obj.put("message", e.message); + if (Exception cause ?= e.cause) { + obj.put("cause", exceptionJson(cause)); + } + return obj.makeImmutable(); + } + + /** + * Ensure a nested object path exists. Used only for the compatibility `groupName` + * constructor; normal grouping flows through [BoundHandler] as nested attribute maps. + */ + private JsonObject ensureObject(JsonObject root, String path) { + JsonObject target = root; + for (String part : path.split('.')) { + if (Doc existing := target.get(part), existing.is(JsonObject)) { + target = existing.as(JsonObject); + } else { + JsonObject child = json.newObject(); + target.put(part, child); + target = child; + } + } + return target; + } +} diff --git a/lib_slogging/src/main/x/slogging/Level.x b/lib_slogging/src/main/x/slogging/Level.x new file mode 100644 index 0000000000..8762750c12 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/Level.x @@ -0,0 +1,75 @@ +/** + * Corresponds to [Open Telemetry Logging data model Severity fields] + * (https://opentelemetry.io/docs/specs/otel/logs/data-model/#severity-fields) + * + * Severity is a plain integer; well-known constants are named (`DEBUG=5`, `INFO=9`, `WARN=13`, + * `ERROR=17`) but callers may construct intermediate or beyond-canonical levels. Smaller numerical + * values correspond to less severe events (such as debug events), larger numerical values + * correspond to more severe events (such as errors and critical events). + * + * The level model is an open integer line. The canonical four levels are spaced four apart so + * callers can interject custom levels (for example, adding `Notice` between `Info` and `Warn`, + * `Critical` past `Error`) without colliding with a library's choices. + * + * Level NOTICE = new Level(10, "NOTICE"); + * Level CRITICAL = new Level(18, "CRITICAL"); + * logger.log(NOTICE, "user signed in"); + * + * The four canonical names are exposed as `static` constants on this type. Comparisons go through + * `severity` directly. + * + * if (level.severity >= threshold.severity) { ... } + * + * @param severity numerical value of the severity + * @param label human-readable label for the severity + */ +const Level(Int severity, String label) + implements Orderable, Hashable { + /** + * A debugging level log event. + */ + static Level Debug = new Level( 5, "DEBUG"); + + /** + * An informational level log event. + */ + static Level Info = new Level( 9, "INFO"); + + /** + * A warning level log event. + */ + static Level Warn = new Level(13, "WARN"); + + /** + * An error level log event. + */ + static Level Error = new Level(17, "ERROR"); + + /** + * `True` iff a record at this level should be emitted given the supplied + * `threshold`. Matches the SLF4J library's [logging.Level.enabledAtThreshold] for + * caller convenience. + */ + Boolean enabledAtThreshold(Level threshold) = severity >= threshold.severity; + + /** + * Natural ordering is severity ordering. + */ + @Override + static Ordered compare(CompileType a, CompileType b) + = a.severity <=> b.severity; + + @Override + static Boolean equals(CompileType value1, CompileType value2) + = value1.severity == value2.severity; + + @Override + static Int hashCode(CompileType value) + = value.severity.hashCode(); + + @Override + Int estimateStringLength() = label.size; + + @Override + Appender appendTo(Appender buf) = label.appendTo(buf); +} diff --git a/lib_slogging/src/main/x/slogging/Logger.x b/lib_slogging/src/main/x/slogging/Logger.x new file mode 100644 index 0000000000..fc600b5ccf --- /dev/null +++ b/lib_slogging/src/main/x/slogging/Logger.x @@ -0,0 +1,247 @@ +/** + * Corresponds to `log/slog.Logger` (`go.dev/src/log/slog/logger.go`). The user-facing + * concrete logger. + * + * Unlike the SLF4J-shaped sibling library — where [logging.Logger] is an interface and + * [logging.BasicLogger] is its `const` implementation — `Logger` here is concrete from + * the start. Polymorphism lives at the [Handler] boundary; the `Logger` itself is a + * thin combinator that builds a [Record] and forwards it. + * + * `Logger` is a `const` so: + * - methods run on the caller's fiber (no service-boundary hop on the hot path); + * - derivation via [with] / [withGroup] is just handler construction, no mutation; + * - it is `Passable`, so it can flow across fibers (fields of services, parameters + * of service calls). + * + * # The user-facing API + * + * logger.debug("computed", Map:["ms"=12]); + * logger.debug(() -> expensiveMessage()); + * logger.info("processing", Map:["path"=req.path]); + * logger.warn("retrying", Map:["attempt"=3]); + * logger.error("failed", cause=e); + * logger.log(NOTICE, "user signed in"); + * + * Logger reqLog = logger.with(Map:[ + * "requestId" = req.id, + * "user" = req.userId, + * ]); + * + * Logger groupLog = logger.withGroup("payments"); + * groupLog.info("charged", Map:["amount"=1099]); + * // text handler: 2026-04-29 INFO charged payments.amount=1099 + * // json handler: {"level":"INFO","msg":"charged","payments":{"amount":1099}} + * + * # POC status + * + * The `log` method is implemented (level-check fast path, lazy supplier resolution, + * record construction, forward to handler), and [logAt] provides explicit source metadata + * for callers or future compiler/runtime sugar. Runtime injection for + * `@Inject slogging.Logger logger;` is registered by `NativeContainer`. There is no + * `LogAttributes(ctx, level, msg, attributes...)` form; [LoggerContext] is the Ecstasy-shaped + * optional context helper. + */ +const Logger + implements Orderable { + + @Inject Clock clock; + + /** + * Compatibility convenience: handler plus pre-bound attributes. The attributes are handed to + * [Handler.withAttributes] immediately, which is the slog contract and what allows a + * handler to pre-render or cache its own derived state. + */ + construct(Handler? handler = Null, Attributes attributes = []) { + Handler base = handler ?: new TextHandler(); + this.handler = attributes.empty ? base : base.withAttributes(attributes); + } + + Handler handler; + + /** + * Cheap `Debug` enabled check. Mirrors Go's `Logger.Enabled(ctx, slog.LevelDebug)` + * and the SLF4J-shaped library's `debugEnabled` property. + */ + @RO Boolean debugEnabled.get() = handler.enabled(Level.Debug); + + /** + * Cheap `Info` enabled check. Use for multi-statement guarded work; for one-line + * expensive values, prefer `logger.info(() -> "...")`. + */ + @RO Boolean infoEnabled.get() = handler.enabled(Level.Info); + + /** + * Cheap `Warn` enabled check. Delegates directly to the active handler. + */ + @RO Boolean warnEnabled.get() = handler.enabled(Level.Warn); + + /** + * Cheap `Error` enabled check. Delegates directly to the active handler. + */ + @RO Boolean errorEnabled.get() = handler.enabled(Level.Error); + + /** + * Runtime-level enabled check for custom levels. Equivalent to Go + * `Logger.Enabled(ctx, level)`, minus the `context.Context` parameter. + */ + Boolean enabled(Level level) = handler.enabled(level); + + /** + * Emit a `Debug` record. The message is already a complete human-readable string; + * structured values belong in `extra` attributes rather than `{}` placeholders. See + * `doc/logging/usage/slog-parity.md` § "Message formatting". + */ + void debug(String message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Debug, message, extra, cause); + + /** + * Emit a `Debug` record whose message is computed after the handler's enabled check. + */ + void debug(MessageSupplier message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Debug, message, extra, cause); + + /** + * Emit an `Info` record. + */ + void info(String message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Info, message, extra, cause); + + /** + * Emit an `Info` record whose message is computed after the handler's enabled check. + */ + void info(MessageSupplier message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Info, message, extra, cause); + + /** + * Emit a `Warn` record. + */ + void warn(String message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Warn, message, extra, cause); + + /** + * Emit a `Warn` record whose message is computed after the handler's enabled check. + */ + void warn(MessageSupplier message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Warn, message, extra, cause); + + /** + * Emit an `Error` record. `cause` is an Ecstasy convenience for a thrown exception. + */ + void error(String message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Error, message, extra, cause); + + /** + * Emit an `Error` record whose message is computed after the handler's enabled check. + */ + void error(MessageSupplier message, Attributes extra = Map:[], Exception? cause = Null) + = log(Level.Error, message, extra, cause); + + /** + * Open-level emission. Use for custom levels. + * + * The enabled check is first. If the handler rejects `level`, no record is + * constructed and no attributes are merged. This mirrors Go slog's + * `Handler.Enabled` fast path. + */ + void log(Level level, String message, Attributes extra = Map:[], Exception? cause = Null) + = emit(level, message, extra, cause, Null, -1); + + /** + * Open-level emission with lazy message construction. + */ + void log(Level level, MessageSupplier message, Attributes extra = Map:[], + Exception? cause = Null) + = emit(level, message, extra, cause, Null, -1); + + /** + * Open-level emission with explicit source metadata. + * + * Go `slog` can populate source information when configured with `AddSource`. This + * POC cannot ask the compiler/runtime for the current call site yet, so `logAt` + * provides a precise and testable explicit API. A later compiler helper can lower + * caller-site sugar into this method without changing the handler contract. + */ + void logAt(Level level, String message, String sourceFile, Int sourceLine, + Attributes extra = Map:[], Exception? cause = Null) + = emit(level, message, extra, cause, sourceFile, sourceLine); + + /** + * Open-level emission with explicit source metadata and lazy message construction. + */ + void logAt(Level level, MessageSupplier message, String sourceFile, Int sourceLine, + Attributes extra = Map:[], Exception? cause = Null) + = emit(level, message, extra, cause, sourceFile, sourceLine); + + /** + * Common emission path used by [log] and [logAt]. The handler owns any attributes + * attached by derived loggers; the record receives only call-time extras. + */ + private void emit(Level level, String message, Attributes extra, Exception? cause, + String? sourceFile, Int sourceLine) { + if (!handler.enabled(level)) { + return; + } + handler.handle(new Record( + timestamp = clock.now, + message = message, + level = level, + attributes = extra, + exception = cause, + sourceFile = sourceFile, + sourceLine = sourceLine, + )); + } + + /** + * Lazy-message equivalent of [emit]. The message is resolved only after the handler + * accepts the level, matching Go slog's `Enabled` fast path. + */ + private void emit(Level level, MessageSupplier message, Attributes extra, Exception? cause, + String? sourceFile, Int sourceLine) { + if (!handler.enabled(level)) { + return; + } + handler.handle(new Record( + timestamp = clock.now, + message = message(), + level = level, + attributes = extra, + exception = cause, + sourceFile = sourceFile, + sourceLine = sourceLine, + )); + } + + /** + * Return a derived logger that carries the supplied attributes. Equivalent to + * `slog.Logger.With(...)`. + * + * The attributes are not stored in `Logger`; they live in the derived [Handler]. That is + * the key slog design point: a handler can pre-resolve attributes once at derivation time + * instead of merging or rendering them on every emission. + * + * This is the slog replacement for SLF4J MDC in the core API: instead of hidden + * fiber-local state, code passes around a logger that visibly carries request or + * component attributes. See `doc/logging/usage/slog-parity.md`. + */ + Logger with(Attributes more) { + if (more.empty) { + return this; + } + return new Logger(handler.withAttributes(more)); + } + + /** + * Return a derived logger whose subsequent attributes are namespaced under + * `groupName`. Equivalent to `slog.Logger.WithGroup(...)`. + * + * The grouping state lives in the derived handler. Text handlers dot-flatten + * (`payments.amount`), while JSON handlers nest (`{"payments":{"amount":...}}`). + */ + Logger withGroup(String groupName) { + if (groupName.empty) { + return this; + } + return new Logger(handler.withGroup(groupName)); + } +} diff --git a/lib_slogging/src/main/x/slogging/LoggerContext.x b/lib_slogging/src/main/x/slogging/LoggerContext.x new file mode 100644 index 0000000000..2bfcb829ee --- /dev/null +++ b/lib_slogging/src/main/x/slogging/LoggerContext.x @@ -0,0 +1,49 @@ +import ecstasy.SharedContext; + +/** + * Optional context helper for code that wants Go-slog-style request propagation without + * passing a logger through every method signature. + * + * Go slog accepts `context.Context` on `LogAttributes` and `Handler.Enabled` / `Handle`. + * Ecstasy does not need to copy that exact API to be familiar: [SharedContext] already + * models a logical execution context that flows across service calls and child fibers. + * `LoggerContext` is the smallest explicit bridge: + * + * Logger requestLog = logger.with(Map:["requestId"=id]); + * using (loggerContext.bind(requestLog)) { + * worker.process^(); + * } + * + * if (Logger log := loggerContext.current()) { + * log.info("inside worker"); + * } + * + * This helper is intentionally separate from [Logger]. The recommended default remains + * explicit logger passing; `LoggerContext` is available for frameworks, request + * dispatchers, and adapter layers where implicit context is the least noisy API. + */ +const LoggerContext { + + /** + * The fiber-local backing store. No default value is supplied, so [current] is + * conditional and callers must decide their fallback policy explicitly. + */ + private static SharedContext context = new SharedContext("slogging.Logger"); + + /** + * Bind `logger` to the current logical execution context. + * + * Use the returned token with `using` so the previous logger is restored reliably. + */ + SharedContext.Token bind(Logger logger) = context.withValue(logger); + + /** + * Return the context logger when one is bound. + */ + conditional Logger current() = context.hasValue(); + + /** + * Return the context logger, or `fallback` when no context logger is bound. + */ + Logger currentOr(Logger fallback) = context.hasValue() ?: fallback; +} diff --git a/lib_slogging/src/main/x/slogging/MemoryHandler.x b/lib_slogging/src/main/x/slogging/MemoryHandler.x new file mode 100644 index 0000000000..542f390241 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/MemoryHandler.x @@ -0,0 +1,69 @@ +/** + * Test-oriented handler — captures every emitted record in an in-memory list for + * later assertion. Direct equivalent of `lib_logging`'s `MemoryLogSink`, and + * structurally identical to `slog`'s `slogtest`-style helpers in Go. + * + * MemoryHandler h = new MemoryHandler(); + * Logger logger = new Logger(h); + * logger.info("processed", Map:["count"=42]); + * assert h.records.size == 1; + * assert h.records[0].level == Level.Info; + * + * # Why this handler is a `service` + * + * It accumulates records. The backing array is mutated on every `handle()` call and + * the same instance is shared across the logger-under-test fiber and the assertion + * fiber. Same reasoning as `lib_logging`'s `MemoryLogSink`. See + * `doc/logging/design/design.md` ("Sink type: `const` vs `service`"). + */ +service MemoryHandler + implements Handler { + + public/private Level rootLevel = Level.Debug; + + /** + * Mutable backing storage. Internal — Ecstasy forbids returning a mutable array + * from a service call, so we expose [records] as an immutable snapshot below. + */ + private Record[] recordList = new Record[]; + + /** + * Immutable snapshot of the captured records, in emission order. Each access + * copies — same pattern as `lib_logging.MemoryLogSink.events`. + */ + @RO Record[] records.get() = recordList.toArray(Constant); + + /** + * Cheap threshold check. Defaults to `Debug`, matching Go slog's lowest canonical + * level and making tests capture everything unless they opt into a stricter level. + */ + @Override + Boolean enabled(Level level) = level.enabledAtThreshold(rootLevel); + + /** + * Capture the record in memory. This is the slog-shaped analogue of Logback's + * `ListAppender` and `lib_logging.MemoryLogSink`. + */ + @Override + void handle(Record record) = recordList.add(record); + + /** + * Derived loggers share this capture buffer, but [BoundHandler] applies the + * pre-bound attributes before the record reaches [handle]. + */ + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new BoundHandler(delegate=this, attributes=attributes); + + /** + * Group derived records while keeping the same capture buffer. + */ + @Override + Handler withGroup(String name) + = name.empty ? this : new BoundHandler(delegate=this, groupName=name); + + /** + * Discard all captured records. + */ + void reset() = recordList.clear(); +} diff --git a/lib_slogging/src/main/x/slogging/NopHandler.x b/lib_slogging/src/main/x/slogging/NopHandler.x new file mode 100644 index 0000000000..e9cf6d8896 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/NopHandler.x @@ -0,0 +1,41 @@ +/** + * Corresponds to slog's idiomatic "discard" handler — typically constructed as + * `slog.New(slog.NewTextHandler(io.Discard, nil))` in Go, but a single-purpose type is + * cleaner in Ecstasy. + * + * Stateless — drops every record. `enabled` always returns `False` so callers that + * respect the check can skip building attribute arrays entirely. + * + * # Why this handler is a `const` + * + * Identical reasoning to `lib_logging`'s `NoopLogSink`: no state, no shared mutation, + * pure forwarder. See `doc/logging/design/design.md` ("Sink type: `const` vs `service`"). + */ +const NopHandler + implements Handler { + + /** + * Always disabled so caller-side fast paths skip record construction. + */ + @Override + Boolean enabled(Level level) = False; + + /** + * Deliberately drops the record. In normal use this should not be reached because + * `enabled` returns `False`, but it remains a no-op for defensive simplicity. + */ + @Override + void handle(Record record) {} + + /** + * Deriving a no-op handler is still no-op. + */ + @Override + Handler withAttributes(Attributes attributes) = this; + + /** + * Grouping a no-op handler is still no-op. + */ + @Override + Handler withGroup(String name) = this; +} diff --git a/lib_slogging/src/main/x/slogging/Record.x b/lib_slogging/src/main/x/slogging/Record.x new file mode 100644 index 0000000000..8a8d6323a7 --- /dev/null +++ b/lib_slogging/src/main/x/slogging/Record.x @@ -0,0 +1,37 @@ +/** + * Corresponds to `log/slog.Record` (`go.dev/src/log/slog/record.go`). The immutable + * unit of "one log call's worth of data" handed to a [Handler]. + * + * Equivalent to the SLF4J-shaped library's [logging.LogEvent], but flattened: there is + * no separate `marker` or `keyValues` — categorisation and structured fields all live + * in `attributes`. + * + * The `attributes` map carries the structured data visible to the current handler call. + * Derived handlers may prepend attributes or wrap them in groups before forwarding to the + * final backend. This matches Go slog's `Handler.WithAttributes` / `WithGroup` model. + * + * Source-location capture (`sourceFile`, `sourceLine`) is opt-in. The default is + * `(Null, -1)`, meaning "not captured." [Logger.logAt] populates these fields + * explicitly; future compiler/runtime sugar can lower call-site metadata into that API. + * + * @param timestamp the time when the log event occurred as measured by the origin clock + * @param message value containing the body of the log record. Can be for example a + * human-readable string message (including multi-line) describing the event in a + * free form or it can be a structured data composed of arrays and maps of other + * values + * @param level the severity of the log event + * @param attributes additional information about the specific event occurrence. Attributes can vary + * for each occurrence of the event coming from the same source + * @param exception optional exception/cause of the log event + * @param sourceFile optional source file name + * @param sourceLine optional source line number + * @param threadName optional thread name + */ +const Record(Time timestamp, + AnyValue message, + Level level, + Attributes attributes = [], + Exception? exception = Null, + String? sourceFile = Null, + Int sourceLine = -1, + String threadName = ""); diff --git a/lib_slogging/src/main/x/slogging/TextHandler.x b/lib_slogging/src/main/x/slogging/TextHandler.x new file mode 100644 index 0000000000..ea254baadd --- /dev/null +++ b/lib_slogging/src/main/x/slogging/TextHandler.x @@ -0,0 +1,151 @@ +/** + * Corresponds to `log/slog.TextHandler` (`go.dev/src/log/slog/text_handler.go`). The + * default human-readable handler — emits one line per record in `key=value` format. + * + * Format: + * + * 2026-04-29T11:23:45.012Z INFO msg="processed request" requestId=r_42 user=u_3 + * + * If the record carries an exception, it is rendered on the following line. + * + * # Why this handler is a `const`, not a `service` + * + * Same rule as `lib_logging`'s `ConsoleLogSink`: stateless forwarder over `@Inject + * Console`. Threshold fixed at construction. No accumulation, no shared mutable state. + * See `doc/logging/design/design.md` ("Sink type: `const` vs `service`") for the full rule. + */ +const TextHandler + implements Handler { + + /** + * Create a [TextHandler]. + * + * @param options (optional) the handler's options + * @param groupName (optional) the handler's group name + * @param consumer (optional) the consumer that will process the String produced from the log + * [Record] (the default will print the line to the console) + */ + construct(HandlerOptions? options = Null, String groupName = "", TextConsumer? consumer = Null) { + this.options = options ?: new HandlerOptions(); + this.groupName = groupName; + this.consumer = consumer ?: defaultConsumer; + } + + typedef function void (String) as TextConsumer; + + /** + * Single-arg convenience: configurable threshold, no group prefix. + */ + construct(Level rootLevel) { + construct TextHandler(new HandlerOptions(rootLevel)); + } + + HandlerOptions options; + + String groupName; + + TextConsumer consumer; + + /** + * Cheap threshold check. This is intentionally only arithmetic on the level's + * severity; no formatting or attribute walking belongs on the fast path. + */ + @Override + Boolean enabled(Level level) = level.enabledAtThreshold(options.rootLevel); + + /** + * Render a record as one line of text. This is the POC equivalent of Go + * `slog.TextHandler.Handle`: the message is quoted and attributes are appended as + * `key=value` pairs. + */ + @Override + void handle(Record record) { + StringBuffer buf = new StringBuffer(); + buf.append(record.timestamp.toString()) + .append(' ') + .append(record.level.label.leftJustify(5, ' ')) + .append(' ') + .append(record.message); + + appendAttributes(buf, groupName, record.attributes); + appendSource(buf, record); + + consumer(buf.toString()); + + if (Exception e ?= record.exception) { + // TODO(impl): pretty-print stack frames; for now rely on Exception.toString(). + consumer(e.toString()); + } + } + + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new BoundHandler(delegate=this, attributes=attributes); + + @Override + Handler withGroup(String name) + = name.empty ? this : new BoundHandler(delegate=this, groupName=name); + + private static void defaultConsumer(String text) { + @Inject Console console; + console.print(text); + } + + /** + * Render one attr `key=value`, prefixing with the active group path if any. Nested + * groups (a value that is itself a `Map`) are flattened with a dot + * separator — matches `slog.TextHandler`. + */ + private void renderAttr(StringBuffer buf, String prefix, String key, AnyValue value) { + String fullKey = prefix.empty ? key : $"{prefix}.{key}"; + if (value.is(Map)) { + appendNestedAttributes(buf, fullKey, value); + } else { + buf.append(fullKey).append('=') + .append(options.redacts(key) ? options.redaction : value.toString()); + } + } + + /** + * Append top-level attributes. Each attr starts with a leading space because the base + * record fields have already been rendered. + */ + private void appendAttributes(StringBuffer buf, String prefix, Attributes attributes) { + for ((String key, AnyValue value) : attributes) { + buf.append(' '); + renderAttr(buf, prefix, key, value); + } + } + + /** + * Append children of an attr group. The first child continues in the current + * position; later children are separated with spaces. + */ + private void appendNestedAttributes(StringBuffer buf, String prefix, Map attributes) { + Boolean first = True; + for ((String key, AnyValue value) : attributes) { + if (!first) { + buf.append(' '); + } + first = False; + renderAttr(buf, prefix, key, value); + } + } + + /** + * Append source metadata when caller-supplied source capture is enabled. + */ + private void appendSource(StringBuffer buf, Record record) { + if (!options.includeSource) { + return; + } + + if (String file ?= record.sourceFile) { + buf.append(" source=") + .append(file); + if (record.sourceLine >= 0) { + buf.append(':').append(record.sourceLine); + } + } + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest.x b/lib_slogging/src/test/x/SLoggingTest.x new file mode 100644 index 0000000000..4f5fb58a37 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest.x @@ -0,0 +1,25 @@ +/** + * Unit tests for the `slogging.xtclang.org` library — the slog-shaped sibling of + * `lib_logging`. The structure mirrors `LoggingTest` so a reviewer comparing the two + * libraries can see equivalent test coverage at the same waterline. + * + * The tests build small, in-memory `Handler` implementations (`ListHandler`, + * `MemoryHandler`) and assert that: + * - the level check fast-path elides emission of disabled records; + * - the per-level methods (`debug`, `info`, `warn`, `error`) and the open `log(level, + * ...)` route correctly; + * - exceptions arrive at the handler intact; + * - `Logger.with(...)` accumulates always-on attributes that flow into every record; + * - `Logger.withGroup(...)` namespaces subsequent attributes; + * - `Logger.logAt(...)` populates explicit source metadata; + * - `LoggerContext` propagates request-scoped loggers; + * - `JSONHandler` renders parseable `lib_json` documents; + * - `HandlerContract` checks `withAttributes` / `withGroup` conformance; + * - nested attribute maps render as nested structure; + * - custom `Level` values comparable to / between the canonical four work. + */ +module SLoggingTest { + package json import json.xtclang.org; + package slogging import slogging.xtclang.org; + package xunit import xunit.xtclang.org; +} diff --git a/lib_slogging/src/test/x/SLoggingTest/AsyncHandlerTest.x b/lib_slogging/src/test/x/SLoggingTest/AsyncHandlerTest.x new file mode 100644 index 0000000000..8d117a651b --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/AsyncHandlerTest.x @@ -0,0 +1,34 @@ +import slogging.AsyncHandler; +import slogging.Logger; + +/** + * Tests the slog-shaped async handler wrapper. + */ +class AsyncHandlerTest { + + @Test + void shouldDrainQueuedRecordsToDelegate() { + ListHandler delegate = new ListHandler(); + AsyncHandler async = new AsyncHandler(delegate, 10); + Logger logger = new Logger(async); + + logger.info("queued"); + async.flush(); + + assert delegate.records.size == 1; + assert delegate.records[0].message == "queued"; + } + + @Test + void shouldDropRecordsAfterCloseWithoutFlush() { + ListHandler delegate = new ListHandler(); + AsyncHandler async = new AsyncHandler(delegate, 10); + Logger logger = new Logger(async); + + async.close(flush=False); + logger.info("dropped"); + + assert async.droppedCount == 1; + assert delegate.records.empty; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/CountingHandler.x b/lib_slogging/src/test/x/SLoggingTest/CountingHandler.x new file mode 100644 index 0000000000..58df074e8d --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/CountingHandler.x @@ -0,0 +1,35 @@ +import slogging.Attributes; +import slogging.BoundHandler; +import slogging.Handler; +import slogging.Level; +import slogging.Record; + +/** + * Test-only handler mirroring `LoggingTest.CountingSink`: counts records by level. + * This proves that a user-defined slog handler can replace the shipped handlers without + * changing caller code. + */ +service CountingHandler + implements Handler { + + public/private Map counts = new HashMap(); + + @Override + Boolean enabled(Level level) = True; + + @Override + void handle(Record record) { + counts.process(record.level, e -> { + e.value = e.exists ? e.value + 1 : 1; + return Null; + }); + } + + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new BoundHandler(delegate=this, attributes=attributes); + + @Override + Handler withGroup(String name) + = name.empty ? this : new BoundHandler(delegate=this, groupName=name); +} diff --git a/lib_slogging/src/test/x/SLoggingTest/CountingHandlerTest.x b/lib_slogging/src/test/x/SLoggingTest/CountingHandlerTest.x new file mode 100644 index 0000000000..a6ed7ae5a8 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/CountingHandlerTest.x @@ -0,0 +1,24 @@ +import slogging.Level; +import slogging.Logger; + +/** + * Demonstrates that a custom slog `Handler` can be plugged in and observed + * end-to-end, parallel to `LoggingTest.CountingSinkTest`. + */ +class CountingHandlerTest { + + @Test + void shouldCountPerLevel() { + CountingHandler handler = new CountingHandler(); + Logger logger = new Logger(handler); + + logger.info ("a"); + logger.info ("b"); + logger.warn ("c"); + logger.error("d"); + + assert handler.counts[Level.Info] == 2; + assert handler.counts[Level.Warn] == 1; + assert handler.counts[Level.Error] == 1; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/EmissionTest.x b/lib_slogging/src/test/x/SLoggingTest/EmissionTest.x new file mode 100644 index 0000000000..9021c61a2e --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/EmissionTest.x @@ -0,0 +1,64 @@ +import slogging.Level; +import slogging.Logger; + +/** + * Tests for the per-level emission methods. Mirrors `LoggingTest.EmissionTest`. + */ +class EmissionTest { + + @Test + void shouldRouteInfo() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + + logger.info("hello", Map:["count"=1]); + + assert handler.records.size == 1; + assert handler.records[0].level == Level.Info; + assert handler.records[0].message == "hello"; + assert handler.records[0].attributes.size == 1; + assert handler.records[0].attributes.contains("count"); + } + + @Test + void shouldRouteAllLevels() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + + logger.debug("d"); + logger.info ("i"); + logger.warn ("w"); + logger.error("e"); + + assert handler.records.size == 4; + assert handler.records[0].level == Level.Debug; + assert handler.records[1].level == Level.Info; + assert handler.records[2].level == Level.Warn; + assert handler.records[3].level == Level.Error; + } + + @Test + void shouldCarryException() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + Exception boom = new Exception("boom"); + + logger.error("failed", Map:["status"=500], cause=boom); + + assert handler.records.size == 1; + assert handler.records[0].exception?.text == "boom" : assert; + } + + @Test + void shouldRouteCustomLevelThroughLog() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + Level notice = new Level(10, "NOTICE"); + + logger.log(notice, "user signed in"); + + assert handler.records.size == 1; + assert handler.records[0].level == notice; + assert handler.records[0].message == "user signed in"; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/HandlerContract.x b/lib_slogging/src/test/x/SLoggingTest/HandlerContract.x new file mode 100644 index 0000000000..6c1f212b49 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/HandlerContract.x @@ -0,0 +1,63 @@ +import slogging.AnyValue; +import slogging.Attributes; +import slogging.Handler; +import slogging.Level; +import slogging.Record; + +/** + * Minimal slogtest-style contract helpers for third-party handlers. + * + * Go ships `testing/slogtest` so backend authors can prove that `WithAttributes` and + * `WithGroup` are implemented correctly. This helper is the Ecstasy POC equivalent: + * pass a handler and a snapshot function for the records it emitted. + */ +class HandlerContract { + + /** + * Verify that `withAttributes` prepends bound attributes to call-time attributes. + */ + static void assertWithAttributesPrepend(Handler root, function Record[] () records) { + Handler derived = root.withAttributes(Map:["requestId"="r_1"]); + derived.handle(sample(Map:["path"="/checkout"])); + + Record[] captured = records(); + assert captured.size == 1; + Attributes attributes = captured[0].attributes; + assert attributes.size == 2; + String[] keys = attributes.keys.toArray(); + assert keys[0] == "requestId"; + assert attributes["requestId"] == "r_1"; + assert keys[1] == "path"; + } + + /** + * Verify that `withGroup` nests subsequent attributes under the group name. + */ + static void assertWithGroupNests(Handler root, function Record[] () records) { + Handler grouped = root.withGroup("payments"); + grouped.handle(sample(Map:["amount"=1099])); + + Record[] captured = records(); + assert captured.size == 1; + Attributes attributes = captured[0].attributes; + assert attributes.size == 1; + AnyValue paymentsValue = attributes["payments"] ?: assert; + assert paymentsValue.is(Map); + + Map children = paymentsValue.as(Map); + assert children.size == 1; + assert children["amount"] == 1099; + } + + /** + * Shared sample record. + */ + private static Record sample(Attributes attributes) { + return new Record( + timestamp = new Time("2019-05-22T120123.456Z"), + message = "event", + level = Level.Info, + attributes = attributes, + ); + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/HandlerContractTest.x b/lib_slogging/src/test/x/SLoggingTest/HandlerContractTest.x new file mode 100644 index 0000000000..d8852986f3 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/HandlerContractTest.x @@ -0,0 +1,19 @@ +import slogging.MemoryHandler; + +/** + * Applies the contract helper to the shipped memory handler. Third-party handlers can + * copy this pattern for their own tests. + */ +class HandlerContractTest { + + @Test + void shouldValidateMemoryHandlerDerivations() { + MemoryHandler handler = new MemoryHandler(); + + HandlerContract.assertWithAttributesPrepend(handler, () -> handler.records); + + handler.reset(); + + HandlerContract.assertWithGroupNests(handler, () -> handler.records); + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/HandlerDerivationTest.x b/lib_slogging/src/test/x/SLoggingTest/HandlerDerivationTest.x new file mode 100644 index 0000000000..ee960ec87a --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/HandlerDerivationTest.x @@ -0,0 +1,37 @@ +import slogging.Logger; + +/** + * Tests that `Logger.with(...)` and `Logger.withGroup(...)` call into the handler + * derivation hooks. That hook is the main extra power in the slog handler contract + * compared with the two-method `LogSink` contract. + */ +class HandlerDerivationTest { + + @Test + void shouldCallHandlerWithAttributesWhenDerivingLogger() { + TrackingHandler handler = new TrackingHandler(); + Logger base = new Logger(handler); + + Logger derived = base.with(Map:["requestId"="r_1"]); + derived.info("processing"); + + assert handler.withAttributesCalls == 1; + assert handler.lastAttributes.size == 1; + assert handler.lastAttributes["requestId"] == "r_1"; + assert handler.records.size == 1; + } + + @Test + void shouldCallHandlerWithGroupWhenGroupingLogger() { + TrackingHandler handler = new TrackingHandler(); + Logger base = new Logger(handler); + + Logger grouped = base.withGroup("payments"); + grouped.info("charged", Map:["amount"=1099]); + + assert handler.withGroupCalls == 1; + assert handler.groups.size == 1; + assert handler.groups[0] == "payments"; + assert handler.records.size == 1; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/JSONHandlerTest.x b/lib_slogging/src/test/x/SLoggingTest/JSONHandlerTest.x new file mode 100644 index 0000000000..e072fb95f0 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/JSONHandlerTest.x @@ -0,0 +1,95 @@ +import json.Doc; +import json.JsonObject; +import json.Parser; + +import slogging.AnyValue; +import slogging.HandlerOptions; +import slogging.JSONHandler; +import slogging.Level; +import slogging.Record; + +/** + * Tests the shipped JSON handler's document shape. This is intentionally stricter than + * string contains checks: the rendered output must parse through `lib_json`. + */ +class JSONHandlerTest { + + @Test + void shouldRenderParseableJsonWithEscapingAndNestedGroups() { + JSONHandler handler = new JSONHandler(); + Exception boom = new Exception("bad \"card\""); + Map userAttributes = Map:["id"="u_3"]; + Record record = new Record( + timestamp = new Time("2019-05-22T120123.456Z"), + message = "charged \"ok\"", + level = Level.Info, + attributes = Map:[ + "amount" = 1099, + "user" = userAttributes, + ], + exception = boom, + sourceFile = "PaymentService.x", + sourceLine = 42, + ); + + JsonObject obj = handler.toJson(record); + + assert obj["level"] == "INFO"; + assert obj["msg"] == "charged \"ok\""; + assert obj["amount"] == 1099; + + assert obj["user"].is(JsonObject); + JsonObject user = obj["user"].as(JsonObject); + assert user["id"] == "u_3"; + + assert obj["exception"].is(JsonObject); + JsonObject exception = obj["exception"].as(JsonObject); + assert exception["message"] == "bad \"card\""; + + assert obj["source"].is(JsonObject); + JsonObject source = obj["source"].as(JsonObject); + assert source["file"] == "PaymentService.x"; + assert source["line"] == 42; + } + + @Test + void shouldRenderGroupedDerivedLoggerAsNestedJson() { + JSONHandler handler = new JSONHandler(); + Map paymentAttributes = Map:["amount"=1099]; + Record record = new Record( + timestamp = new Time("2019-05-22T120123.456Z"), + message = "charged", + level = Level.Info, + attributes = Map:["payments"=paymentAttributes], + ); + + JsonObject obj = handler.toJson(record); + + assert obj["payments"].is(JsonObject); + JsonObject payments = obj["payments"].as(JsonObject); + assert payments["amount"] == 1099; + } + + @Test + void shouldApplyHandlerOptionsRedaction() { + JSONHandler handler = new JSONHandler( + new HandlerOptions(Level.Info, ["token"])); + Record record = new Record( + timestamp = new Time("2019-05-22T120123.456Z"), + message = "auth", + level = Level.Info, + attributes = Map:["token"="secret", "user"="u_1"], + ); + + JsonObject obj = handler.toJson(record); + + assert obj["token"] == "***"; + assert obj["user"] == "u_1"; + } + + private JsonObject parseObject(String rendered) { + Doc doc = new Parser(rendered.toReader()).parseDoc(); + assert doc.is(JsonObject); + return doc.as(JsonObject); + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/LazyLoggingTest.x b/lib_slogging/src/test/x/SLoggingTest/LazyLoggingTest.x new file mode 100644 index 0000000000..5878971f83 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/LazyLoggingTest.x @@ -0,0 +1,41 @@ +import slogging.Level; +import slogging.Logger; + +/** + * Verifies slog-shaped lazy logging semantics. The logger must perform the handler + * enabled check before it invokes lazy message suppliers. + */ +class LazyLoggingTest { + + @Test + void shouldNotEvaluateLazyMessageWhenLevelDisabled() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + @Volatile Int calls = 0; + + handler.setLevel(Level.Info); + logger.debug(() -> { + ++calls; + return "expensive debug message"; + }); + + assert calls == 0; + assert handler.records.empty; + } + + @Test + void shouldEvaluateLazyMessageWhenEnabled() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + @Volatile Int calls = 0; + + logger.info(() -> { + ++calls; + return "expensive info message"; + }); + + assert calls == 1; + assert handler.records.size == 1; + assert handler.records[0].message == "expensive info message"; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/LevelCheckTest.x b/lib_slogging/src/test/x/SLoggingTest/LevelCheckTest.x new file mode 100644 index 0000000000..be4ab199ec --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/LevelCheckTest.x @@ -0,0 +1,37 @@ +import slogging.Level; +import slogging.Logger; + +/** + * Tests that the handler's `enabled(level)` fast-path drops events below threshold. + * Mirrors `LoggingTest.LevelCheckTest`. + */ +class LevelCheckTest { + + @Test + void shouldDropRecordsBelowThreshold() { + ListHandler handler = new ListHandler(); + handler.setLevel(Level.Warn); + Logger logger = new Logger(handler); + + logger.debug("d"); + logger.info ("i"); + logger.warn ("w"); + logger.error("e"); + + assert handler.records.size == 2; + assert handler.records[0].level == Level.Warn; + assert handler.records[1].level == Level.Error; + } + + @Test + void shouldReportEnabledFlagsConsistentlyWithThreshold() { + ListHandler handler = new ListHandler(); + handler.setLevel(Level.Info); + Logger logger = new Logger(handler); + + assert !logger.debugEnabled; + assert logger.infoEnabled; + assert logger.warnEnabled; + assert logger.errorEnabled; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/LevelTest.x b/lib_slogging/src/test/x/SLoggingTest/LevelTest.x new file mode 100644 index 0000000000..8fdc9e9a94 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/LevelTest.x @@ -0,0 +1,45 @@ +import slogging.Level; + +/** + * Tests for `Level` — the open-ended integer-severity model. + * + * Mirrors `LoggingTest.LevelTest` but exercises the slog-style additions: custom levels + * between or beyond the canonical four, ordered comparison via `severity`. + */ +class LevelTest { + + @Test + void shouldOrderCanonicalLevels() { + assert Level.Debug.severity < Level.Info.severity; + assert Level.Info.severity < Level.Warn.severity; + assert Level.Warn.severity < Level.Error.severity; + } + + @Test + void shouldSupportCustomLevels() { + // The canonical levels are spaced four apart precisely so callers can interject. + Level notice = new Level(10, "NOTICE"); + Level critical = new Level(18, "CRITICAL"); + + assert notice.severity > Level.Info.severity; + assert notice.severity < Level.Warn.severity; + assert critical.severity > Level.Error.severity; + } + + @Test + void shouldRespectThresholdSemantics() { + Level threshold = Level.Warn; + assert Level.Error.enabledAtThreshold(threshold); + assert Level.Warn.enabledAtThreshold(threshold); + assert !Level.Info.enabledAtThreshold(threshold); + assert !Level.Debug.enabledAtThreshold(threshold); + } + + @Test + void shouldRenderViaLabel() { + assert Level.Debug.toString() == "DEBUG"; + assert Level.Info.toString() == "INFO"; + assert Level.Warn.toString() == "WARN"; + assert Level.Error.toString() == "ERROR"; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/ListHandler.x b/lib_slogging/src/test/x/SLoggingTest/ListHandler.x new file mode 100644 index 0000000000..9909bb2e6c --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/ListHandler.x @@ -0,0 +1,54 @@ +import slogging.Attributes; +import slogging.BoundHandler; +import slogging.Handler; +import slogging.Level; +import slogging.Record; + +/** + * Test-only handler that captures every emitted record in a list. Used as a stand-in + * for a "real" handler so tests can assert exactly what the `Logger` chose to emit, in + * order. Mirrors the `ListLogSink` helper in `LoggingTest`. + * + * `rootLevel` defaults to `Debug` so by default every emitted record is captured. + * + * Modelled as a `service` because it carries mutable state (the records list) shared + * across the fiber-under-test and the assertion code. See + * `doc/logging/design/design.md` ("Sink type: `const` vs `service`"). + */ +service ListHandler + implements Handler { + + public/private Level rootLevel = Level.Debug; + + /** + * Mutable backing storage. Internal — Ecstasy forbids returning a mutable array + * from a service call, so we expose [records] as an immutable snapshot below. + */ + private Record[] recordList = new Record[]; + + /** + * Immutable snapshot of the captured records, in emission order. Each access + * copies — same pattern as `lib_logging.MemoryLogSink.events`. + */ + @RO Record[] records.get() = recordList.toArray(Constant); + + @Override + Boolean enabled(Level level) = level.enabledAtThreshold(rootLevel); + + @Override + void handle(Record record) = recordList.add(record); + + @Override + Handler withAttributes(Attributes attributes) + = attributes.empty ? this : new BoundHandler(delegate=this, attributes=attributes); + + @Override + Handler withGroup(String name) + = name.empty ? this : new BoundHandler(delegate=this, groupName=name); + + void setLevel(Level level) { + rootLevel = level; + } + + void reset() = recordList.clear(); +} diff --git a/lib_slogging/src/test/x/SLoggingTest/LoggerContextTest.x b/lib_slogging/src/test/x/SLoggingTest/LoggerContextTest.x new file mode 100644 index 0000000000..fd9598f651 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/LoggerContextTest.x @@ -0,0 +1,39 @@ +import slogging.Logger; +import slogging.LoggerContext; + +/** + * Tests the optional `SharedContext` helper for request-scoped sloggers. + */ +class LoggerContextTest { + + @Test + void shouldExposeLoggerInsideUsingScope() { + LoggerContext context = new LoggerContext(); + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler).with(Map:["requestId"="r_1"]); + + assert !context.current(); + + using (context.bind(logger)) { + assert Logger current := context.current(); + current.info("inside"); + } + + assert handler.records.size == 1; + assert handler.records[0].attributes.contains("requestId"); + assert handler.records[0].attributes["requestId"] == "r_1"; + assert !context.current(); + } + + @Test + void shouldReturnFallbackWhenNoLoggerIsBound() { + LoggerContext context = new LoggerContext(); + ListHandler handler = new ListHandler(); + Logger fallback = new Logger(handler); + + context.currentOr(fallback).info("fallback"); + + assert handler.records.size == 1; + assert handler.records[0].message == "fallback"; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/MemoryHandlerTest.x b/lib_slogging/src/test/x/SLoggingTest/MemoryHandlerTest.x new file mode 100644 index 0000000000..68d2657e32 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/MemoryHandlerTest.x @@ -0,0 +1,37 @@ +import slogging.Level; +import slogging.Logger; +import slogging.MemoryHandler; + +/** + * Tests the shipped `MemoryHandler` (the test-helper handler that's part of the + * library's own surface, parallel to `lib_logging.MemoryLogSink`). + */ +class MemoryHandlerTest { + + @Test + void shouldCaptureRecordWithAttributes() { + MemoryHandler handler = new MemoryHandler(); + Logger logger = new Logger(handler); + + logger.info("processed", Map:["count"=42]); + + assert handler.records.size == 1; + assert handler.records[0].level == Level.Info; + assert handler.records[0].message == "processed"; + assert handler.records[0].attributes.size == 1; + assert handler.records[0].attributes.contains("count"); + } + + @Test + void shouldResetClearsRecords() { + MemoryHandler handler = new MemoryHandler(); + Logger logger = new Logger(handler); + + logger.info("a"); + logger.info("b"); + assert handler.records.size == 2; + + handler.reset(); + assert handler.records.empty; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/NopHandlerTest.x b/lib_slogging/src/test/x/SLoggingTest/NopHandlerTest.x new file mode 100644 index 0000000000..3e5a855c64 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/NopHandlerTest.x @@ -0,0 +1,19 @@ +import slogging.Logger; +import slogging.NopHandler; + +/** + * Tests the shipped discard handler. Mirrors `lib_logging.NoopLogSink`: disabled level + * checks prevent records from being constructed or emitted. + */ +class NopHandlerTest { + + @Test + void shouldReportAllLevelsDisabled() { + Logger logger = new Logger(new NopHandler()); + + assert !logger.debugEnabled; + assert !logger.infoEnabled; + assert !logger.warnEnabled; + assert !logger.errorEnabled; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/SourceLocationTest.x b/lib_slogging/src/test/x/SLoggingTest/SourceLocationTest.x new file mode 100644 index 0000000000..9f5c4e912d --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/SourceLocationTest.x @@ -0,0 +1,33 @@ +import slogging.Level; +import slogging.Logger; + +/** + * Tests explicit source-location capture via `Logger.logAt(...)`. + */ +class SourceLocationTest { + + @Test + void shouldPopulateSourceFieldsWhenUsingLogAt() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + + logger.logAt(Level.Warn, "slow payment", "PaymentService.x", 77, + Map:["elapsedMs"=451]); + + assert handler.records.size == 1; + assert handler.records[0].sourceFile == "PaymentService.x"; + assert handler.records[0].sourceLine == 77; + assert handler.records[0].attributes.contains("elapsedMs"); + } + + @Test + void shouldLeaveSourceFieldsEmptyForNormalLogCalls() { + ListHandler handler = new ListHandler(); + Logger logger = new Logger(handler); + + logger.info("normal"); + + assert handler.records[0].sourceFile == Null; + assert handler.records[0].sourceLine == -1; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/TrackingHandler.x b/lib_slogging/src/test/x/SLoggingTest/TrackingHandler.x new file mode 100644 index 0000000000..fc05e9caf6 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/TrackingHandler.x @@ -0,0 +1,41 @@ +import slogging.Attributes; +import slogging.Handler; +import slogging.Level; +import slogging.Record; + +/** + * Test-only handler that records derivation hooks. It makes the `Handler.withAttributes` / + * `Handler.withGroup` part of the slog contract observable without depending on a + * concrete renderer. + */ +service TrackingHandler + implements Handler { + + public/private Int withAttributesCalls = 0; + public/private Int withGroupCalls = 0; + public/private Attributes lastAttributes = Map:[]; + public/private String[] groups = []; + private Record[] recordList = new Record[]; + + @RO Record[] records.get() = recordList.toArray(Constant); + + @Override + Boolean enabled(Level level) = True; + + @Override + void handle(Record record) = recordList.add(record); + + @Override + Handler withAttributes(Attributes attributes) { + ++withAttributesCalls; + lastAttributes = attributes.makeImmutable(); + return this; + } + + @Override + Handler withGroup(String name) { + ++withGroupCalls; + groups = (groups + [name]).toArray(Constant); + return this; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/WithGroupTest.x b/lib_slogging/src/test/x/SLoggingTest/WithGroupTest.x new file mode 100644 index 0000000000..4475e57294 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/WithGroupTest.x @@ -0,0 +1,68 @@ +import slogging.AnyValue; +import slogging.Attributes; +import slogging.Logger; + +/** + * Tests `Logger.withGroup(name)` — slog's namespace-prefix derivation. A grouped logger + * produces records whose attributes the *handler* renders under the supplied prefix. + * + * No equivalent exists in `lib_logging` (SLF4J's hierarchical logger names cover a + * different concern — the *logger's identity*, not the *attributes' namespace*). + */ +class WithGroupTest { + + @Test + void shouldReturnDistinctLoggerInstance() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler); + Logger grouped = base.withGroup("payments"); + + // The Logger is a const; "distinctness" here is about API shape: the two refer + // to different logger instances even when the underlying handler is the same. + assert &base != &grouped; + } + + @Test + void shouldGroupOnlySubsequentAttributes() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler).with(Map:["env"="prod"]); + Logger grouped = base.withGroup("payments"); + + grouped.info("charged", Map:["amount"=1099]); + + // Matches Go slog ordering: attributes bound before WithGroup stay outside the group; + // attributes supplied after WithGroup are nested under the group. + assert handler.records.size == 1; + Attributes attributes = handler.records[0].attributes; + assert attributes.size == 2; + String[] keys = attributes.keys.toArray(); + assert keys[0] == "env"; + assert keys[1] == "payments"; + AnyValue paymentsValue = attributes["payments"] ?: assert; + assert paymentsValue.is(Map); + + Map paymentAttributes = paymentsValue.as(Map); + assert paymentAttributes.size == 1; + assert paymentAttributes["amount"] == 1099; + } + + @Test + void shouldPutAttributesBoundAfterGroupInsideGroup() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler); + Logger grouped = base.withGroup("payments").with(Map:["currency"="SEK"]); + + grouped.info("charged", Map:["amount"=1099]); + + Attributes attributes = handler.records[0].attributes; + assert attributes.size == 1; + AnyValue paymentsValue = attributes["payments"] ?: assert; + assert paymentsValue.is(Map); + + Map paymentAttributes = paymentsValue.as(Map); + assert paymentAttributes.size == 2; + String[] paymentKeys = paymentAttributes.keys.toArray(); + assert paymentKeys[0] == "currency"; + assert paymentKeys[1] == "amount"; + } +} diff --git a/lib_slogging/src/test/x/SLoggingTest/WithTest.x b/lib_slogging/src/test/x/SLoggingTest/WithTest.x new file mode 100644 index 0000000000..2ec13f5823 --- /dev/null +++ b/lib_slogging/src/test/x/SLoggingTest/WithTest.x @@ -0,0 +1,81 @@ +import slogging.Attributes; +import slogging.Logger; + +/** + * Tests the slog `With(...)` derivation idiom: a logger derived from another carries + * the parent's attributes plus its own, and these flow into every record emitted via + * the derived logger. + * + * No direct equivalent in `lib_logging` (SLF4J uses MDC for this). The closest test in + * the SLF4J library is `MDCTest.shouldAttachSnapshotToLogEvent`. + */ +class WithTest { + + @Test + void shouldCarryAttachedAttributesIntoRecord() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler); + + Logger derived = base.with(Map:[ + "requestId" = "r_42", + "user" = "u_3", + ]); + + derived.info("processing"); + + assert handler.records.size == 1; + Attributes attributes = handler.records[0].attributes; + assert attributes.size == 2; + String[] keys = attributes.keys.toArray(); + assert keys[0] == "requestId" && attributes["requestId"] == "r_42"; + assert keys[1] == "user" && attributes["user"] == "u_3"; + } + + @Test + void shouldConcatenateAcrossWithChains() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler); + + Logger withA = base.with(Map:["a"=1]); + Logger withB = withA.with(Map:["b"=2]); + + withB.info("event"); + + assert handler.records.size == 1; + Attributes attributes = handler.records[0].attributes; + assert attributes.size == 2; + String[] keys = attributes.keys.toArray(); + assert keys[0] == "a"; + assert keys[1] == "b"; + } + + @Test + void shouldCombineAttachedAttributesWithCallTimeExtras() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler); + + Logger reqLog = base.with(Map:["requestId"="r_99"]); + + reqLog.info("processing", Map:["path"="/api"]); + + Attributes attributes = handler.records[0].attributes; + assert attributes.size == 2; + String[] keys = attributes.keys.toArray(); + assert keys[0] == "requestId"; + assert keys[1] == "path"; + } + + @Test + void shouldNotAffectParentLogger() { + ListHandler handler = new ListHandler(); + Logger base = new Logger(handler); + + Logger derived = base.with(Map:["k"="v"]); + + base.info("from base"); + derived.info("from derived"); + + assert handler.records[0].attributes.empty; + assert handler.records[1].attributes.size == 1; + } +} diff --git a/manualTests/src/main/x/TestLogging.x b/manualTests/src/main/x/TestLogging.x new file mode 100644 index 0000000000..bfc305cf25 --- /dev/null +++ b/manualTests/src/main/x/TestLogging.x @@ -0,0 +1,212 @@ +/** + * Manual end-to-end scenario for the SLF4J-shaped `lib_logging` POC. + * + * The unit tests prove the individual pieces; this module is the executable story a + * reviewer can run to see the real injection path and the major API features together: + * injected logger/MDC, named loggers, formatting, markers, lazy suppliers, fluent + * structured events, explicit source metadata, and Logback-style backend primitives. + */ +module TestLogging { + package log import logging.xtclang.org; + + import log.AsyncLogSink; + import log.BasicLogger; + import log.CompositeLogSink; + import log.ConsoleLogSink; + import log.HierarchicalLogSink; + import log.JsonLogSink; + import log.JsonLogSinkOptions; + import log.Level; + import log.LogEvent; + import log.Logger; + import log.LoggerRegistry; + import log.LogSink; + import log.Marker; + import log.MarkerFactory; + import log.MDC; + import log.MemoryLogSink; + import log.NoopLogSink; + + @Inject Console console; + + void run() { + console.print("=== TestLogging: SLF4J-shaped logging ==="); + runInjectedLogger(); + runMdcAndNamedLogger(); + runMarkersAndFluentBuilder(); + runBackendPrimitives(); + runRegistry(); + } + + /** + * Real runtime injection of `logging.Logger`. The default injected logger uses the + * familiar resource name `logger` and is backed by `ConsoleLogSink`. + */ + void runInjectedLogger() { + console.print("--- injected Logger: levels, formatting, lazy messages ---"); + + @Inject Logger logger; + assert logger.infoEnabled; + assert !logger.debugEnabled; + + logger.info("hello, {}", ["world"]); + logger.warn("disk space low: {} MB", [42]); + + @Volatile Int lazyCalls = 0; + logger.debug(() -> { + ++lazyCalls; + return "this debug message must not be built"; + }); + logger.atDebug() + .addLazyArgument(() -> { + ++lazyCalls; + return "this debug argument must not be built"; + }) + .log("debug {}"); + assert lazyCalls == 0; + + logger.info(() -> { + ++lazyCalls; + return "lazy info message built after the Info check"; + }); + assert lazyCalls == 1; + + try { + failingOperation(); + } catch (Exception e) { + logger.error("operation failed for {}", ["manual"], cause=e); + } + + logger.log(Level.Warn, "runtime-selected level {}", ["Warn"]); + logger.logAt(Level.Info, () -> "source-aware lazy message", + "TestLogging.x", 74); + } + + /** + * Injected MDC plus `Logger.named(...)`, the Ecstasy equivalent of deriving a + * per-class/per-component SLF4J logger. + */ + void runMdcAndNamedLogger() { + console.print("--- injected MDC and named logger ---"); + + @Inject Logger logger; + @Inject MDC mdc; + + Logger payments = logger.named("manual.logging.payments"); + + mdc.put("requestId", "r_manual"); + mdc.put("user", "u_7"); + payments.info("handling payment {}", ["p_123"]); + + assert mdc.copyOfContextMap.getOrNull("requestId") == "r_manual"; + + mdc.remove("user"); + assert !mdc.copyOfContextMap.contains("user"); + mdc.clear(); + } + + /** + * Marker DAGs and the SLF4J 2.x fluent builder against a memory sink so the manual + * scenario asserts the payload, not just the console output. + */ + void runMarkersAndFluentBuilder() { + console.print("--- markers and fluent structured event builder ---"); + + MemoryLogSink sink = new MemoryLogSink(); + Logger logger = new BasicLogger("manual.logging.fluent", sink); + MarkerFactory markers = new MarkerFactory(); + Marker audit = markers.getMarker("AUDIT"); + Marker security = markers.getMarker("SECURITY"); + Exception cause = new Exception("manual cause"); + + security.add(audit); + assert markers.exists("AUDIT"); + assert security.contains(audit); + + logger.atInfo() + .addMarker(security) + .setCause(cause) + .addArgument("checkout") + .addLazyArgument(() -> "lazy-arg") + .addKeyValue("requestId", "r_fluent") + .addLazyKeyValue("payload", () -> "lazy-payload") + .log("operation {} {}"); + + assert sink.events.size == 1; + LogEvent event = sink.events[0]; + assert event.level == Level.Info; + assert event.message == "operation checkout lazy-arg"; + assert event.marker?.contains(audit) : assert; + assert event.exception == cause; + assert event.arguments[1].as(String) == "lazy-arg"; + assert event.keyValues.getOrNull("requestId").as(String) == "r_fluent"; + assert event.keyValues.getOrNull("payload").as(String) == "lazy-payload"; + } + + /** + * The backend layer that gives this SLF4J-shaped facade Logback-style operational + * power: fanout, hierarchical level config, async wrapping, JSON/redaction, and nop. + */ + void runBackendPrimitives() { + console.print("--- Logback-style backend primitives ---"); + + MemoryLogSink first = new MemoryLogSink(); + MemoryLogSink second = new MemoryLogSink(); + Logger fanout = new BasicLogger("manual.logging.fanout", + new CompositeLogSink([first, second])); + fanout.warn("fanout event"); + assert first.events.size == 1; + assert second.events.size == 1; + + MemoryLogSink hierarchicalTarget = new MemoryLogSink(); + HierarchicalLogSink hierarchy = new HierarchicalLogSink( + hierarchicalTarget, Level.Warn); + hierarchy.setLevel("manual.logging.debug", Level.Debug); + + Logger muted = new BasicLogger("manual.logging.other.Component", hierarchy); + Logger debug = new BasicLogger("manual.logging.debug.Component", hierarchy); + muted.info("hidden by root Warn level"); + debug.debug("visible through prefix Debug override"); + assert hierarchicalTarget.events.size == 1; + assert hierarchicalTarget.events[0].message == "visible through prefix Debug override"; + + MemoryLogSink asyncTarget = new MemoryLogSink(); + AsyncLogSink async = new AsyncLogSink(asyncTarget, 8); + Logger asyncLogger = new BasicLogger("manual.logging.async", async); + asyncLogger.info("async event"); + async.flush(); + assert asyncTarget.events.size == 1; + + Logger jsonLogger = new BasicLogger("manual.logging.json", + new JsonLogSink(new JsonLogSinkOptions(Level.Debug, ["secret"]))); + jsonLogger.atInfo() + .addKeyValue("secret", "redact-me") + .addKeyValue("visible", "ok") + .log("json sink redaction demo"); + + Logger noop = new BasicLogger("manual.logging.noop", new NoopLogSink()); + assert !noop.errorEnabled; + noop.error("dropped by no-op sink"); + } + + /** + * Name-keyed logger interning, corresponding to SLF4J `ILoggerFactory` semantics. + */ + void runRegistry() { + console.print("--- logger registry interning ---"); + + LoggerRegistry registry = new LoggerRegistry(new MemoryLogSink()); + Logger one = registry.ensure("manual.logging.registry"); + Logger two = registry.ensure("manual.logging.registry"); + Logger child = one.named("manual.logging.registry.child"); + Logger child2 = registry.ensure("manual.logging.registry.child"); + + assert &one == &two; + assert &child == &child2; + child.info("registry child event"); + } + + void failingOperation() { + throw new Exception("boom"); + } +} diff --git a/manualTests/src/main/x/TestSLogging.x b/manualTests/src/main/x/TestSLogging.x new file mode 100644 index 0000000000..8b8399fe79 --- /dev/null +++ b/manualTests/src/main/x/TestSLogging.x @@ -0,0 +1,164 @@ +/** + * Manual end-to-end scenario for the Go `log/slog`-shaped `lib_slogging` POC. + * + * Run this next to `TestLogging` to compare the two proposed call shapes. This file + * exercises injected slog logger acquisition, attrs, derived loggers, groups, lazy + * message/attr suppliers, explicit source metadata, context binding, custom levels, + * and the handler backend SPI. + */ +module TestSLogging { + package slog import slogging.xtclang.org; + + import slog.Attributes; + import slog.AsyncHandler; + import slog.HandlerOptions; + import slog.JSONHandler; + import slog.Level; + import slog.Logger; + import slog.LoggerContext; + import slog.MemoryHandler; + import slog.NopHandler; + import slog.Record; + import slog.TextHandler; + + @Inject Console console; + + void run() { + console.print("=== TestSLogging: slog-shaped logging ==="); + runInjectedLogger(); + runDerivedLoggersAttrs(); + runHandlerPrimitives(); + runContextBinding(); + } + + /** + * Real runtime injection of `slogging.Logger`. The same resource name `logger` + * works because the injector also keys by requested type. + */ + void runInjectedLogger() { + console.print("--- injected slog Logger: levels, attributes, lazy messages ---"); + + @Inject Logger logger; + assert logger.infoEnabled; + assert !logger.debugEnabled; + + logger.info("hello", ["style"="slog"]); + + @Volatile Int lazyCalls = 0; + logger.debug(() -> { + ++lazyCalls; + return "this debug message must not be built"; + }); + assert lazyCalls == 0; + + logger.info(() -> { + ++lazyCalls; + return "lazy info message built after the Info check"; + }, ["scenario"="injected"]); + assert lazyCalls == 1; + + try { + failingOperation(); + } catch (Exception e) { + logger.error("operation failed", ["operation"="manual"], cause=e); + } + + Level notice = new Level(2, "NOTICE"); + logger.log(notice, "custom level", ["severity"=notice.severity]); + logger.logAt(Level.Warn, () -> "source-aware lazy message", + "TestSLogging.x", 64, ["source"="explicit"]); + } + + /** + * The central slog idea: derive loggers with attributes/groups instead of named logger + * categories and MDC. + */ + void runDerivedLoggersAttrs() { + console.print("--- derived loggers and groups ---"); + + MemoryHandler capture = new MemoryHandler(); + Logger root = new Logger(handler=capture); + + Logger payments = root.with(["requestId"="r_slog"]).withGroup("payments"); + + Attributes attributes = Map:["card"=["last4"="4242"], "amount"=1099]; + payments.info("charged", attributes); + + assert capture.records.size == 1; + + Record record = capture.records[0]; + assert record.message == "charged"; + assert record.attributes["requestId"] == "r_slog"; + assert var paymentAttrs := record.attributes.get("payments"); + assert paymentAttrs.is(Attributes); + assert paymentAttrs["amount"] == 1099; + assert var cardAttrs := paymentAttrs.get("card"); + assert cardAttrs.is(Attributes); + assert cardAttrs["last4"] == "4242"; + } + + /** + * Handler-side equivalents of Logback-style backend composition: memory capture, + * async wrapping, text/JSON rendering, redaction, and no-op output. + */ + void runHandlerPrimitives() { + console.print("--- slog handler primitives ---"); + + MemoryHandler asyncTarget = new MemoryHandler(); + AsyncHandler async = new AsyncHandler(asyncTarget, 8); + Logger asyncLogger = new Logger(async); + asyncLogger.info("async record", ["queued"=True]); + async.flush(); + assert asyncTarget.records.size == 1; + + Logger textLogger = new Logger(new TextHandler(Level.Debug)); + textLogger.debug("text handler debug", ["visible"=True]); + + Logger jsonLogger = new Logger(new JSONHandler( + new HandlerOptions(Level.Debug, ["secret"]))); + jsonLogger.info("json handler redaction", ["secret"="redact-me", "visible"="ok"]); + + Logger noop = new Logger(new NopHandler()); + assert !noop.errorEnabled; + noop.error("dropped by no-op handler"); + } + + /** + * Optional implicit context helper for framework/request propagation. + */ + void runContextBinding() { + console.print("--- LoggerContext binding ---"); + + MemoryHandler capture = new MemoryHandler(); + Logger root = new Logger(capture); + LoggerContext context = new LoggerContext(); + Logger bound = root.with(["context"="bound"]); + + using (context.bind(bound)) { + Logger current = context.currentOr(root); + current.info("from bound context"); + } + + assert capture.records.size == 1; + assert var value := capture.records[0].attributes.get("context"); + assert value.is(String) && value == "bound"; + } + + void failingOperation() { + throw new Exception("boom"); + } + + /** + * Service-backed lazy counter. `Attr.lazy` stores the supplier in a `const` attr, so + * captured mutable state must be passable; a service reference is the right manual + * test stand-in for a real provider object. + */ + service LazyCounter { + public/private Int calls = 0; + + String value(String result) { + ++calls; + return result; + } + } +} diff --git a/xdk/build.gradle.kts b/xdk/build.gradle.kts index 5d7b7a02bd..1734d8e542 100644 --- a/xdk/build.gradle.kts +++ b/xdk/build.gradle.kts @@ -111,6 +111,8 @@ dependencies { xtcModule(libs.xdk.crypto) xtcModule(libs.xdk.json) xtcModule(libs.xdk.jsondb) + xtcModule(libs.xdk.logging) + xtcModule(libs.xdk.slogging) xtcModule(libs.xdk.net) xtcModule(libs.xdk.oodb) xtcModule(libs.xdk.sec) diff --git a/xdk/settings.gradle.kts b/xdk/settings.gradle.kts index bda6186277..0427ec71c8 100644 --- a/xdk/settings.gradle.kts +++ b/xdk/settings.gradle.kts @@ -26,6 +26,8 @@ listOf( "lib_net", "lib_json", "lib_jsondb", + "lib_logging", + "lib_slogging", "lib_oodb", "lib_sec", "lib_web",