From 4d8e07d1b02e27e810bff9d6a4975c71433ca77a Mon Sep 17 00:00:00 2001 From: gnuxie Date: Tue, 7 Apr 2026 12:30:49 +0100 Subject: [PATCH] TDD experiment from 2510B cycle. --- exploration/25/TDD.md | 121 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 exploration/25/TDD.md diff --git a/exploration/25/TDD.md b/exploration/25/TDD.md new file mode 100644 index 0000000..2014218 --- /dev/null +++ b/exploration/25/TDD.md @@ -0,0 +1,121 @@ +# TDD Report + +## Context + +We need to get better with tests in the Draupnir project. Tests have been +neglected, it is unfortunate. The reason this has happened is because we haven't +found the right formula that works for us yet. All of our tests contain ugly +imperative brittle code that is the exact opposite of what we strive for in our +implementation source code. Tests lack declarative structure, and it is unclear +why they exist, why they are important, and what behaviour they are meant to be +testing. + +It's unclear if bad abstraction design in general leads to bad tests, and +whether changing our testing process would force cleaner abstractions. + +## Unknowns + +- Can we express the core invariants of a feature in a way that is: + - concise + - declarative + - stable over time + +- Does writing invariants + interface _before_ implementation improve: + - refactor safety + - design clarity + - contributor understanding + +- What is the cost overhead (time, cognitive load) of this style, and is it + acceptable for “hardened” features? + +- Can we prevent tests from ossifying accidental behaviour (implementation + quirks) rather than intended invariants? + +- Does this approach reduce bitrot on infrequently used paths (logging, edge + features) enough to justify itself? + +## Experiment + +### Hypothesis + +By designing from the outside first specifying invariants, then the public +interface, and only then the implementation. We will: + +- keep tests concise and declarative, +- prevent tests from pinning internal implementation details, +- and end up with higher-quality abstractions and more stable behaviour over + time. + +In this opinionated version of TDD: + +1. **Invariants**: We write down the behavioural laws of the + feature/abstraction. Each invariant must include a short explanation of _why_ + it exists, to make future negotiation (change/removal) possible. + +2. **Interface**: We design the public surface that makes those invariants + usable. This is the types/signatures/config shape. + +3. **Implementation**: We write the code that satisfies the interface while + making all invariants pass. + +### Evaluation plan + +- For this experiment, a feature/abstraction is only considered “hardened” if: + - its core invariants can be stated in a short, declarative form, and + - those invariants are enforced by tests. + +- If we cannot express the invariants declaratively, we treat this as a design + smell: + - either the abstraction is not yet well-formed and should remain “internal”, + - or we need to refactor the concept until the invariants become clear. + +- We will judge the experiment successful if, for the chosen pilot feature: + - refactors feel safer and faster, + - tests remain small and readable (no giant imperative scaffolding), + - and contributors can understand the behaviour by reading the invariants + + interface without diving into implementation. + +### Exploration points + +- We will design and use a utility to support work on projections in the policy + list subscription preview work + https://github.com/Gnuxie/matrix-protection-suite/pull/101. + +## Outcomes + +We developed a `SemanticType` abstraction that could document the behaviour of +an abstraction: + +> A semantic type is used to describe the expected behavior of a type beyond its +> shape. At the moment invariants can be specified through the `Law` method. +> This adds structure to important behavioural contracts that would otherwise be +> placed loosely in unit tests with no organisation. + +> Another important feature of the invariant law is that they describe why they +> are included in the sematic type which helps negotiating their removal if the +> underlying abstraction changes. + +> Critically the laws as opposed to tests are also defined against the interface +> and not any single concrete implementation. But all implementations can test +> their implementation against the semantic type. This brings the benefits of +> test driven development directly to the stage where the interface (and thus +> abstraction) is defined (or changed). Before any concrete implementation is +> even considered. + +- This worked but required a large amount of boiler plate to implement the + checks. Which might not be appropriate for all cases. There is value in + specifying the `what` and `why` of behaviour without having to provide a check + to verify it. + +- It's clear that wider use of the `SemanticType` abstraction would lead to + repetition in the checks. This is already visible in + https://github.com/Gnuxie/matrix-protection-suite/pull/101. + +- It's too early to really evaluate it but it at least "felt" cleaner, more + structured, and valuable than unit tests. Even if that's because it forces a + TDD approach to designing, and then implementing the abstraction. + +- There's a gap for how to apply `SemanticType` to existing abstractions in + order to break them or modify their behaviour without completely rewriting + them. This is important because obviously almost all of the abstractions in + our code base haven't been designed this way.