Computational sensor

Where a practice acts when an agent writes the code: a program deciding whether what was done holds. One cell of Böckeler's grid.

Examples

  • Tests, linters, type checkers, structural analysis Böckeler
  • Logs Böckeler
  • Browser Böckeler
  • Structural tests, such as a hook running ArchUnit tests on module boundaries Böckeler
  • Fitness functions
  • Dead-code detection
  • A gate that sends a failing check back to the agent, within a budget of attempts
  • A transition table the type checker holds exhaustive
  • Acceptance tests written before the code, frozen until a later step passes them
  • The files a step changed, reconciled with the files it planned to change
  • A snapshot of what the code does, taken before a refactoring
  • A ratchet: the accepted debt can only go down

The ones marked Böckeler are from Harness engineering for coding agent users; the others are added by this library.

Practices that act here

  • Acceptance test: A test of a specified acceptance criterion, written before the code from what the user should see, which the code must then pass.
  • Approval testing: A person approves an output once, and every later run must reproduce it until a new one is approved.
  • Architectural fitness function: An automated check that gives an objective verdict on one characteristic of the architecture, run like a test.
  • Characterization test: A test records what the code does today, right or not, so that a change to it cannot go unseen.
  • Contract test: One suite of tests runs against both a fake and the real implementation, so the fake the other tests use is known to behave like the real one.
  • Decision table testing: Every combination of a finite set of conditions is listed with its expected outcome, and each row becomes a test.
  • Equivalence partitioning and boundary value analysis: Inputs are split into classes the code should treat alike, and each class is tested once inside it and at each of its limits.
  • Fuzzing: A program is fed random or malformed input to find the inputs that make it crash or hang.
  • Golden master: The output of a reference run is kept, and every later run is compared with it to catch any change in behaviour.
  • Make illegal states unrepresentable: Types are shaped so that a state the program must never reach cannot be written, and the compiler refuses it.
  • Metamorphic testing: When the right output is unknown, two runs whose inputs are related are checked for outputs that relate the same way.
  • Mutation testing: Small faults are planted in the code, and a test suite that still passes on one of them has a gap to fill.
  • Parse, don't validate: Input is checked once, at the boundary, into a type that keeps what was checked, so the rest of the program never meets an invalid value.
  • Property-based testing: Generated inputs are checked against a property that must always hold, and a failing input is shrunk to the smallest one that still fails.
  • Ratchet: The violations a code base already has are accepted and recorded, and the check fails on any new one, so the count can only go down.
  • Separation of duties: The one who does the work is not the one who checks it, because a measure the doer controls stops measuring.
  • Smoke test: A quick run of the main paths of the whole system, end to end, to see that it works at all before deeper testing.
  • Snapshot testing: A test framework stores an output the first time a test runs and fails when a later run produces a different one.
  • Stateful (model-based) testing: Generated sequences of commands are run against the system and against a simple model of it, and the two must agree at every step.
  • Test sizes: A test is small, medium or large by what it touches, one process, one machine or beyond, which decides its speed and its flakiness better than unit or integration.
  • Unit testing: A test exercises a small part of the code, such as one function or class, fast enough to run on every change, with its collaborators real or replaced.
+ Propose a practice