Tiles and sprites: the graphics
Tiled backgrounds, moving sprites, up to 256 colours: without taking any memory from the 6502, and drawing each line in under 63.5 microseconds. The project’s longest piece of work.
On this page
Why tiles
A 320 × 240 image in 256 colours weighs 75 KiB: more than the 6502’s entire memory. Games consoles solved this problem long ago with tiles: a small set of patterns (8 × 8 or 16 × 16 pixels) is drawn once, and the background is just a map saying which pattern goes where. On top, sprites move freely.
With two layers, 256 tiles and 128 sprites, about 22 KiB is needed at 320 × 240. Memory is no longer the problem; the problem becomes drawing time.
The chosen model
The model is inspired by two well-known machines, without aiming for compatibility:
- from the VERA (the Commander X16’s video chip), flexible layers: tiles or bitmap, from 1 to 8 bits per pixel, 8 × 8 or 16 × 16 tiles;
- from the SNES, per-line effects and priorities.
| Element | Capacity |
|---|---|
| Resolution | 320 × 240 |
| Layers | 2, tiled or bitmap, independent scrolling |
| Sprites | 128, with depth and mirroring |
| Palette | 256 colours chosen from 4,096 |
| Per-line effects | scrolling and palette bank, line by line |
| Video memory | 32 KiB, expandable to 64 KiB |
Eliott65’s own idea: reading without copying
As with the text screen, graphics data can stay in the 6502 program’s memory: maps, bitmaps, sprite images, sprite table. The engine reads them there, without copying. The program moves a sprite by changing two bytes in its own memory, without a system call.
Alongside this, there is a 32 KiB video memory in the RP2040, loaded by kernel services (including directly from the USB stick, without going through the 6502). It can expand to 64 KiB by borrowing free frames from the multitasking pool, which are given back at the end.
GRAPHE.E65 (994 bytes): a 16-colour tile layer scrolling diagonally, its map in the program’s
memory, and 8 bouncing 16 × 16 sprites. Screenshot from the emulator.
VAGUE.E65: a 16-colour image, but a different palette bank on each line gives 241 colours on
screen; per-line scrolling makes the image ripple. Screenshot from the emulator.A reference to avoid mistakes
Before optimising anything, a reference was written in Python, from the specification, pixel by pixel. The firmware engine must produce exactly the same image. Three deliberately introduced errors (priority order, vertical mirroring, map wrap-around) are all detected.
Twice, the engine and the reference agreed against the hand-written test case: it was the expectation that was wrong.
The 63.5 µs wall
Each logical line must be ready within 63.5 µs. First run on the board:
| Program | Worst line | Budget |
|---|---|---|
| text (shell) | 61 µs | 63.5 µs |
| VAGUE | 140 µs | 63.5 µs |
| GRAPHE | 161 µs | 63.5 µs |
Two and a half times too slow. What the user saw: a garbled screen. The rendering was correct, but late, and the screen was putting out half-written lines.
“Shouldn’t we raise the core frequency?” No: the 252 MHz clock is the HDMI bit clock. Raising it would mean sending the pixels correspondingly faster: the number of cycles per line would stay the same. (The 720 × 480 signal adopted later, at 270 MHz, does give 7% more cycles per line: its line is longer, not faster.)
The way down, measurement by measurement
| Step | GRAPHE | VAGUE |
|---|---|---|
| start | 161 µs | 140 µs |
| engine reorganised (pages, loops per format, sprite list) | 123 | 86 |
| encoding tables in C | 123 | 86 |
| encoding in assembler | 100 | 63 |
| map read once per line | 91 | — |
| 4-bit tiles in dedicated C | 85 | — |
| 4-bit tiles in assembler | 61 | 63 |
The main culprit: the Cortex-M0+ has few registers, and the compiler was keeping variables on the stack, about 28 instructions per pixel for the encoding. Hand-written assembler does it in 7 to 12.
“I can see the diamonds bouncing, it’s fine”: GRAPHE on the board, in v0.36.
The limits, measured
Fourteen scenarios were then measured on the board, with a randomly filled video memory and every frame checked against the reference. The rendering is correct everywhere; only a few scenarios fitted within the budget: one 8 × 8 tile layer at 4 bits, a bitmap on its own, a few sprites per line. Two layers, 16 × 16 tiles, many sprites on the same line went over.
Two cores working together (v0.39 – v0.40)
Until then, core 1 did everything: draw the line, then encode it as an HDMI signal. The idea adopted: core 0 draws, core 1 encodes.
| Scenario | Core 1 alone | Working together | Missed lines |
|---|---|---|---|
| 8 × 8 tiles, 4 bits | 52 µs | 30 µs | 0 |
| two layers | 71 µs | 31 µs | 0 (201 before) |
| 16 × 16 tiles | 78 µs | 31 µs | 0 |
| bitmap + tiles | 82 µs | 31 µs | 0 |
| tiles + 16 sprites of 16 × 16 | 93 µs | 31 µs | 0 |
| tiles + 32 sprites of 32 × 32 | 192 µs | 30 µs | 30 |
(In the “working together” column, core 1’s time; core 0 has the eight-line buffer to smooth out its longer steps.)
Ten scenarios out of fourteen now fit, compared with four before.
Sprites in assembler (v0.41)
The RP2040’s two small banks of fast memory were full. Since only core 0 now runs the engine, and it already runs the kernel from flash, an experiment: leave the engine in flash. Measured on the board, it is as fast as from RAM in all fourteen scenarios (0 to 2 µs apart): the flash cache is enough. The space freed holds the sprite drawing in assembler.
| Scenario | Longest core 0 step | Lines missed per frame |
|---|---|---|
| 32 sprites 16 × 16 | 103 → 85 µs | 7 → 0 |
| 32 sprites 32 × 32 | 157 → 118 µs | 30 → 26 |
| 32 sprites 64 × 64 | 172 → 132 µs | 63 → 61 |
| two layers + 32 sprites | 122 → 104 µs | 13 → 10 |
Eleven scenarios out of fourteen fit. What remains: 32 large sprites on the same lines, and two layers with 32 sprites. The trade-off: graphics need a free page for the line buffer; without one, the kernel refuses them.
Sprite loops rewritten (v0.49)
A cycle-accurate measurement showed that the sprite pixel loop still cost 17 to 18 cycles per pixel: the byte was read again, and the column parity tested, for every pixel. Rewritten (advancing pointers, two pixels per byte in 4-bit, four pixels at a time in 8-bit), it drops to about 11 cycles per pixel. Two scenarios were added for mirroring, which the first fourteen did not cover.
| Scenario | Engine | Late lines per frame |
|---|---|---|
| 32 sprites 16 × 16 | 85 → 73 µs | 0 |
| 32 sprites 32 × 32 | 118 → 93 µs | 26 → 19 |
| 32 sprites 64 × 64 in 8-bit | 132 → 116 µs | 61 → 59 |
| two layers + 32 sprites 16 × 16 | 104 → 92 µs | 10 → 5 |
Thirteen scenarios out of sixteen fit, all correct. The work item is closed with these limits, in agreement with the user.
The price: system calls
Core 0 also serves system calls. A drawing step can therefore delay a call. Measured from the 6502’s side while displaying two layers: +0.7 µs on average, +34 µs at worst. A price judged acceptable: the cores have worked together by default since v0.40.
One long step remained: 104 µs with GRAPHE, at the start of each frame. The hypothesis: re-reading the table of 128 sprites (1 KiB) in one go. Since v0.42 it is done in slices of 16 sprites, one per pass of core 0, with the line buffer covering those passes. Longest step: 104 → 30 µs, hypothesis confirmed. It is also the worst delay a system call suffers because of drawing.
This measurement also revealed an old kernel flaw, unrelated to graphics: a program that made calls back to back could no longer be pre-empted. Fixed.
What is specific to Eliott65
Asked about the state of the art, the project looked at PicoDVI and PicoVGA, two well-known video libraries for the RP2040: HDMI encoding converges on the same solutions, as physics leaves little choice. What sets Eliott65 apart lies elsewhere: the kernel outside the 6502, graphics read in place from program memory, and video memory borrowed from processes when needed.
What next
- an avenue for the three scenarios that still overrun: core 1, free for about 29 µs per line, could draw the front sprites;
- the Hercules mode (720 × 350, 80 × 25 text): the 720 × 480 signal at 270 MHz has been the default since v0.52, the graphics part has existed since v0.53 and the text since v0.54, with its own 9 × 14 font since v0.58 (see Video);
- GRAPHE stopping after a few minutes at 270 MHz, with a candidate fix to be checked on the board (see The bus).
Sound, meanwhile, has made its way: see Sound.