Method and lessons
A low-level project often gets things wrong, and sometimes silently. Eliott65 has set itself a few rules so that mistakes become visible, and it records the ones it has made.
On this page
The way of working
- Small versions. Each step is a numbered version (from 0.1 to 0.40 between 13 and 14 October 2026), with a log stating what was done, learnt and measured.
- A definition of “done”. Code compiled without warnings, all tests passing, documentation up to date, version incremented, and a physical measurement if the change affects the bus or timings.
- Written decisions. Every structural choice is the subject of a decision record (an ADR): the context, the decision, the options set aside, the consequences. See The decisions, in brief.
- Measure, then decide. The MMU in the bus had a criterion set in advance (at most a 20% loss); the measurement said −34%, and the idea was dropped.
Tests that do not take their own word for it
- Independent tools: disk images created by
mkfs.fatand checked byfsck.fat, plus a separately written FAT reader; a Python reference for the graphics, written from the specification and not from the code. - Counter-checks: an error is introduced on purpose (an inverted mirror, a wrongly extended colour, a wrong bit in the video, a forgotten give-back) and we check that the tests fail.
- The board as final judge: an acceptance suite runs on the real board, and a tool compares the emulator with the board.
The mistakes, and what they taught
The project’s log records its own mistakes. Here is a selection, with the rule that came out of each.
A failure hidden by a filter
A test on the board failed, but its output went through grep, whose exit code hid the failure: the
version went through without validation. Later, a failed build, hidden in the same way, led to
flashing the old firmware, and later still a filter looking for “error” in lower case let an
assembly error slip through.
Rule: set -o pipefail wherever output is filtered, case-insensitive filters, and validate only
after a zero exit code.
Documentation gets tested too
A chapter of the log was written after the tests had run, and two versions were committed without running them again: a documentation test was failing (a missing index link).
Rule: rerun the whole suite after writing documentation, and only commit on a zero return code.
Measurements taken on the wrong firmware
When flashing was failing silently (the 6502 was overwriting OpenOCD’s program), several measurements had been taken on the old firmware. They were discarded and taken again.
Rule: flashing fails loudly; only verified firmware is measured.
A “reasonable” timeout
The USB driver’s 50 ms timeout looked like a hardware fault. The standard allows 500 ms.
Rule: take timeouts from the standard, not by guesswork.
A “free” block that was not
The last block on the USB stick, thought to be empty, contained 489 non-zero bytes.
Rule: always back up before writing to a medium that is not ours.
Keystrokes in the wrong place
Some tests seemed to fail because the simulated keys were arriving in the wrong program, or in the middle of an input in progress.
Rule: check the state (which program is running, what the input contains) before typing.
The 6502’s carry
Twice, a 32-bit subtraction written in 6502 assembler gave negative durations: a comparison (CPY,
then CPX) placed inside the loop changed the carry and broke the chain of borrows of SBC. The
raw timestamps, read back through the probe, showed that the kernel was right and that the error was
in the measuring program.
The compiler, cleverer than expected
A function marked to run from RAM had been inlined by the compiler into another one that stayed in flash: core 1’s video loop had been running from flash from the start. Likewise, the text font had been placed in flash, which delayed video lines when core 0 was reading the USB stick.
Rule: after every build, a tool checks that no core 1 code is in flash.
The test that was wrong
In the graphics work, twice, the engine and the reference agreed with each other, against the hand-written case. It was the expectation that was wrong (a misplaced byte, tables put in the wrong place).
Lesson: an independent reference also protects against one’s own certainties.