---
title: "Pin today, fail first"
description: "Most of what we ship changes code that already works for businesses taking bookings every day. Before a feature touches that code, we write down what it does today as tests that must pass on the old commit, then tests for the new behaviour that must fail on it. A small script checks both against a copy of the base. Review findings get the same treatment: no fix until a test reproduces the finding."
url: "https://bookatu.com/engineering/pin-today-fail-first"
language: "en"
section: "Engineering"
category: "Engineering"
author: "The Bookatu engineering team"
published: "2026-10-28"
updated: "2026-10-28"
tags: ["testing", "characterization tests", "tdd", "code review", "engineering"]
---

# Pin today, fail first

Most of what we ship changes code that already works for businesses taking bookings every day. Before a feature touches that code, we write down what it does today as tests that must pass on the old commit, then tests for the new behaviour that must fail on it. A small script checks both against a copy of the base. Review findings get the same treatment: no fix until a test reproduces the finding.

> **The short version**
>
> - Before a feature touches existing code, we write characterization tests that pin what that code does today. They have to pass on the base commit, untouched.
> - Then we write tests for the new behaviour. They have to fail on the base commit, and why they fail matters as much as the fact that they do.
> - A short script copies the test files into a git worktree of the base commit and runs them there, one file at a time, so both claims are checked rather than remembered.
> - When a feature has to change something a characterization test pinned, the test is edited in place and marked CHANGED DELIBERATELY, with what it used to say and why. There are 101 of those markers across 49 test files.
> - Review findings are hypotheses. Each one is reproduced by a failing test before anybody fixes it. In our first release review, 10 of 19 findings were refuted before any fix was written.

Release 5 let a client book several services as one visit, a colour and then a cut, possibly with different people. It also let a business say which services go together. Both features run straight through the booking engine, the booking page, the staff New booking form, the public API and the MCP server: code that every business on Bookatu relies on, most of which has nothing to do with visits and never will. The risk in a feature like that is rarely the new code. It is the old code doing something slightly different afterwards, in a place nobody was looking.

So every feature starts the same way, with two sets of tests written before the code, each with a different job.

## Two kinds of test

The first kind says what the code does today. We call these characterization tests, after Michael Feathers, and they live in files named *-characterization.test.ts. There are over a hundred of them. Each opens with a header comment saying what it pins, and against which feature. This is the one written before visits existed:

```ts
/**
 * CHARACTERIZATION (S3-K5, rows 19 and 24): what one booking, and two separate
 * bookings of the same client on the same day, do TODAY, before a client can
 * book several services in one visit.
 *
 * The visit is off by default and every single-service path must stay exactly
 * as it is, so these are pinned on the commit before it:
 *
 *  - the engine offers a service only on the business's grid, and a client
 *    booking a time off it is refused;
 *  - a booking on the booking page is one appointment: one confirmation for the
 *    client, one "New booking" email for the business;
 *  ...
 */
```

The second kind says what the feature will do. Their headers start with NEW instead, and they describe the behaviour that does not exist yet. The two kinds are kept in separate files on purpose, because they are checked in opposite directions.

## Pinning today

A characterization test is not a claim that today's behaviour is right. It is a claim that today's behaviour is what it is, so that any change to it is a decision and not an accident. That is why the rule is that it must pass on the base commit with no help. If it does not pass there, it is describing what somebody believes the code does, which is a different and much less useful thing.

The visit work had one obvious danger. A colour of 45 minutes followed by a cut means the cut starts at a quarter to the hour, and our availability engine had only ever offered starts on the business's grid. Visits would need that relaxed. A single booking must not notice. So the first thing pinned was the grid itself:

```ts
it("lists starts every slot interval from opening, and refuses a client's time off the grid", async () => {
  const biz = await business("vchgrid", { slotIntervalMin: 30 });
  const day = dayKey(6);
  const [d] = await getAvailability({ orgId: biz.orgId, serviceId: biz.serviceId, durationMin: 30, fromKey: day, toKey: day });
  expect(d.open).toBe(true);
  expect(d.slots.slice(0, 3).map((s) => s.startMin)).toEqual([540, 570, 600]);
  expect(d.slots.every((s) => (s.startMin - 540) % 30 === 0)).toBe(true);
  const off = await resolveBookingSlot({ orgId: biz.orgId, serviceId: biz.serviceId, durationMin: 30, startAt: at(day, 555), staffId: biz.samId });
  expect(off.ok).toBe(false);
  const on = await resolveBookingSlot({ orgId: biz.orgId, serviceId: biz.serviceId, durationMin: 30, startAt: at(day, 570), staffId: biz.samId });
  expect(on).toMatchObject({ ok: true, staffId: biz.samId });
});
```

The combination rules got the same treatment, and one of their characterization tests is a good example of pinning scope rather than behaviour. Once a business can say that a 4 braids and a 6 braids are never booked together, the easy mistake is for that rule to leak beyond a single visit. So before the rules existed, we pinned that one client can book both, as two separate bookings on the same day, each with the same extra. That test still passes today. The rule applies inside a booking and nowhere else, and the test is the reason we know.

```ts
it("is offered with every service: both braids take the same extra, and one client books both", async () => {
  const s = await braids("cxboth");
  const day = dayKey(9);
  const four = await createBooking(
    { orgId: s.orgId, serviceId: s.fourId, startAt: at(day, 540), ...client("bria.both@example.com"), addOnIds: [s.beadsId] },
    "online",
  );
  const six = await createBooking(
    { orgId: s.orgId, serviceId: s.sixId, startAt: at(day, 720), ...client("bria.both@example.com"), addOnIds: [s.beadsId] },
    "online",
  );
  expect(four.ok && six.ok).toBe(true);
  expect(four.appointment!.customerId).toBe(six.appointment!.customerId);
  // ...
});
```

## Failing first, for the right reason

The new tests go in the other direction. Each must fail on the base commit, because a test of new behaviour that already passes is testing something that was already true. This is the visit's counterpart to the grid test above: with a 45 minute first service on a 30 minute grid, the visit starts on the grid and the second service starts at 9:45.

```ts
it("starts the next service off the grid when the one before is not a whole number of slots", async () => {
  const s = await salon("vavoffgrid", { slotIntervalMin: 30 }, { durationMin: 45 });
  const d = await startsOn(s, [{ serviceId: s.colourId }, { serviceId: s.cutId }]);
  expect(minutes(d).every((m) => (m - 540) % 30 === 0)).toBe(true);
  expect(planAt(d, 540)![1].startAt).toEqual(at(DAY, 585));
});
```

Both of these tests pass now, side by side, and that is the point. The single booking still lives on the grid. The visit's later service does not. They could only both pass because the engine was given a parameter that only the visit's search sets, and the characterization test is what would have gone red if that parameter had leaked.

Failing on the base is necessary and not sufficient. A test can be red for reasons that have nothing to do with the behaviour it describes, and a red you have not read is no better than a green you have not read. We wrote about the green version of this problem in Green for the wrong reason. The red version is the same mistake from the other side.

- An assertion that fails with today's value is the right reason. It means the test reaches the code it is about and the code says something else today.
- A missing module is a weak reason. The visit tests above import lib/visits-availability.ts, which did not exist on the base, so that file goes red before a single assertion runs. That proves the file is new. It says nothing about whether its assertions would catch a wrong answer, which is what breaking the code on purpose afterwards is for.
- A fixture that cannot be set up is the wrong reason. A test whose setup writes to a table the base does not have fails in its setup, and would fail the same way if the feature were never written.

## The red check

Saying a test passes or fails on the base is easy. Checking it means having the base commit somewhere you can run it, with the new test files in it and nothing else from the feature. We do that with a short shell script in our own tooling, outside the app. Its first lines say what it is for: characterization tests must pass there, and new-behaviour tests must fail there.

The technique is simple. The script makes a detached git worktree of the base commit, once, and links the working tree's node_modules into it so nothing is installed twice. Each run resets that worktree, copies in any new helper files under tests/ that are not tests themselves (a new fixture the new tests need), then copies in one test file at a time, runs vitest on just that file, records the result, and puts the worktree back the way it was before the next one. The heart of it:

```bash
git -C $TREE worktree add --detach $WT $BASE_REF
ln -s $TREE/node_modules $WT/node_modules
# ...
for f in "$@"; do
  mkdir -p "$WT/$(dirname $f)"; cp "$TREE/$f" "$WT/$f"
  name=$(echo $f | tr '/' '_')
  (cd $WT && npx vitest run "$f" --maxWorkers=2 > "$OUT/$name.txt" 2>&1); rc=$?
  # ...
  if [ $rc -ne 0 ]; then echo "RED   $f |$counts | $why"; else echo "GREEN $f |$counts"; fi
  git -C $WT checkout -q -- "$f" 2>/dev/null || rm -f "$WT/$f"
done
```

A few details earn their place.

- The output is one line per file, RED or GREEN, with the test counts. Every characterization file should say GREEN and every new file should say RED. Anything else has to be explained before the work goes on.
- Beside each RED goes the first error line from that file's output, picked by matching patterns like AssertionError, TypeError and Cannot find module. That is the line that tells you whether the red is the right red, and putting it on the summary line means it gets read.
- The full output of each file is kept, so a surprising line can be followed up without running anything again.
- Each base worktree is named for the base commit and a label. Several builders can run red checks against the same base at once without overwriting each other's copies.
- It runs the test files from the working tree against the base's code, so the comparison is exactly the tests that will be committed, against exactly the code they will be committed on top of.

## When today has to change

Sometimes the feature is supposed to change something a characterization test pinned. That is fine, and it happens often, because a characterization test pins today's behaviour whether or not today's behaviour is right. What is not fine is for the test to change quietly. The convention is to edit the assertion in place, keep the test in its characterization file, and leave a comment that starts CHANGED DELIBERATELY, naming the work that changed it, what it used to pin, and why the new value is the right one.

Some of those comments are a record of a bug we found by pinning it. The money screens were characterized before we built deposit reconciliation. When later work on what money was actually collected came through, one of the pinned figures turned out to be counting money that was still owed:

```ts
// CHANGED DELIBERATELY (T5-E3, requirement 18): "Collected in person" was
// the booked price less deposits ($150 - $25 = $125), which counted the $75
// still owed on the $100 visit as collected. It is now what was recorded
// on those visits in person: the $50 cash.
expect(text).toMatch(/Collected in person \$50\.00 cash, card terminal, bank & other, as recorded/);
```

Another came from consent law. A characterization test had pinned that recording a client's marketing consent minted a fresh unsubscribe token. That was today's behaviour, and the test held it exactly. It was also wrong, because a fresh token kills the unsubscribe link in every email already sent:

```ts
// CHANGED DELIBERATELY (CASL, row 14): this pinned a FRESH token
// (not.toBe("u_old")), which killed the unsubscribe link in every email
// already sent. The law wants that link to keep working (60 days at least).
expect(yes.unsubscribeToken).toBe("u_old");
```

The marker makes these searchable, and it makes them reviewable. A reviewer looking at a diff that edits a characterization file can see at once whether the change is deliberate and explained, or whether a test was simply edited until it went green. At the time of writing there are 101 of them across 49 files, and each one is a decision somebody can still read.

## Review findings are hypotheses

Every release goes through a review once it is built: several reviewers each taking an area of the code, plus a few lenses that cut across all of it, such as security, migrations and the seams between features. The rule for what comes back is the same as for a feature. No finding is fixed until a test reproduces it, and the test must fail on the code as it stands.

That rule throws a surprising amount away. In our first release review, 19 findings came back, and skeptics, whose job is to try to disprove each one, refuted 10 of them. Only the 9 that survived were fixed, each with a test that failed before the fix. Later reviews confirmed 20 of 27 findings and 15 of 24. A finding that cannot be turned into a failing test is either not a bug or not understood yet, and in neither case is a fix the next step.

One of the findings the release 5 review confirmed is a good example of the shape. The team can drag one service of a visit to a new time on its own, so a visit's services are not always back to back. Moving the whole visit kept each service's own gap from the first, which is right. But the check before the move laid the services out back to back, so the person doing the later service was checked at a time it would never be saved at, and could be double-booked at the time it was.

```ts
it("is refused when a later service's person is taken at the time it would be saved at", async () => {
  const s = await salon("vgapclient");
  const day = dayKey(10);
  const [first, second] = await gappedVisit(s, day, "lee.gap1@example.com");
  // Anna has somebody at 16:15. Moved to start at 12:00, the cut (4 hours
  // after the start) would be saved at 16:00-16:30, on top of them (a start
  // of its own, so the one-booking-per-person-and-start index cannot catch it).
  await insertBooking(tdb, s, { startAt: at(day, 975), staffId: s.annaId });

  const [d] = await getRescheduleAvailabilityAction(s.slug, first.id, day, day);
  expect(d.slots.map((x) => x.startMin)).not.toContain(720);
  expect(await rescheduleBookingAction(s.slug, first.id, at(day, 720).toISOString())).toMatchObject({ ok: false });
  expect((await reread(first.id)).startAt).toEqual(at(day, 600));
  expect((await reread(second.id)).startAt).toEqual(at(day, 840));
  expect(await annaOverlaps(s, at(day, 960), at(day, 990))).toHaveLength(1);
});
```

The colour is 90 minutes from 10:00 and the cut has been dragged to 14:00. Moved to 12:00, the old check asked whether the cut's person was free at 13:30, right after the colour, and she was. The move would then have saved the cut at 16:00, on top of somebody else. Against the unfixed code the first assertion fails, because 12:00 is offered, and that is the right red: an assertion about the behaviour, with today's value in it. The fix was to stop assuming:

```ts
/**
 * Each service's start, in minutes from the visit's first: the gaps it is
 * booked with, which are not back to back once the team has moved or resized
 * one service on its own. A whole-visit move keeps them, so it is checked at them.
 */
function bookedGaps(live: Appointment[]): number[] {
  const base = live[0].startAt.getTime();
  return live.map((r) => Math.round((r.startAt.getTime() - base) / 60_000));
}
```

Next to that test is a second refusal, for the team's own move, and a third test that a fix-first habit would probably not have produced: the visit still moves, with every service keeping its own gap, to a time each person is free at. A fix that refused every move of a gapped visit would have passed both refusals.

## What it costs

It is slower. Writing tests for behaviour you are not changing feels like overhead on the day, and the characterization files are large: they set up whole businesses on an in-process Postgres and render real pages. The red check is one more run of every new and pinned file against the base. The release 5 review commit ended with the full suite at 13,194 tests passing on a freshly migrated scratch database, and it is not a small suite to keep honest.

It also does not prove the tests are good. A red on the base says a test can tell the base from the feature. It does not say the test would notice the feature being subtly wrong, which is why we still break code on purpose after a fix to watch the right test fail.

What it buys is a specific kind of confidence. When the visit work landed, the single-service paths it ran through had been written down beforehand, by tests that had already passed on the code before it, and they still passed after. That is the closest thing we have to a promise that the businesses who never turn visits on will not notice that visits exist. For a feature that touches the booking engine, that promise is most of the job.