Login
Track T: The test platform
Login

Glint's demonstrations from Phase 2 onward need a machine to boot on. Track T builds that machine from pinned upstream source, using only the BQ2 toolset: a reduced QEMU (the machine emulator) for aarch64 and x86_64, and U-Boot as the EFI firmware it runs. Building the test platform in PLAN.md gives the reasons and constraints. QEMU's own build runs Python code generators. The track replaces each with a Crystal port that reads the same input data. The track runs alongside Phase 1 and finishes before Phase 2, whose first boot (2.1) waits on its last slice.

Every generator port works from the data QEMU ships and the specification that describes that data, not from the Python program or its output. The specifications are docs/devel/decodetree.rst, qapi-code-gen.rst, kconfig.rst and tracing.rst in QEMU's source. The test of a port is whether QEMU builds, passes its own C tests and boots Glint. The slices run smallest first, then in the order QEMU's build needs their outputs, so each port is small and independently checkable before the next begins.

Provisional slices.

Slices

T.1 Inventory. This slice produces a list of every source file and every generated file needed for qemu-system-aarch64 -M virt and qemu-system-x86_64 -M q35, built with TCG (QEMU's own instruction emulator) and KVM (Linux's hardware virtualization), and with no graphical interface. For each generated file it records the input data it comes from and the generator that produces it. It comes first because it gives the track's true size, and the order of the later slices, before anything is ported. Measuring before building is Clarity.

T.2 hxtool and feature_to_c. This slice ports QEMU's two smallest generators. hxtool is a line filter that turns QEMU's .hx files into C option tables. feature_to_c embeds the gdb-xml/*.xml processor descriptions as C strings. Both are small enough to check by reading. They come right after the inventory to establish the pattern every later port follows: read the data, follow the spec, check by compiling QEMU. Starting with the least risky ports is Parsimony applied to the track's own order.

T.3 decodetree. This slice ports decodetree, which turns the 15 target/arm/tcg/*.decode files (compact descriptions of Arm instruction encodings) into C decoders, following decodetree.rst. Only the aarch64 build needs it, since x86's decoder is hand-written C. It is accepted when it parses all 15 files and target/arm/tcg compiles against its output. It comes after T.2 because it is the largest single port and benefits from the pattern T.2 set. A clean compile, followed by T.8's tests, is its Demonstrability.

T.4 minikconf. This slice ports minikconf, which selects the devices each machine includes from QEMU's Kconfig files and configs/devices/*/*.mak lists, following kconfig.rst. It reuses cleave's Kconfig parser where the two dialects agree, rather than writing a second parser. It comes after the decoder ports because the device selection decides which source files T.8 and T.9 compile, and that list is needed before the binaries can be built. Reusing cleave is Parsimony: each thing appears only once.

T.5 tracetool. This slice ports tracetool, which generates C tracing hooks from QEMU's trace-events files, following tracing.rst. It ports only the nop backend, which produces hooks that compile to nothing. The reduced QEMU needs the hooks to compile but has no use for tracing output. It comes here because the trace headers are included almost everywhere and must exist before T.8. Porting one backend instead of seven is Parsimony: solve the problem and no more.

T.6 qapi-gen. This slice ports qapi-gen, which generates the C types, visitors, command handlers and events of QMP (QEMU's control protocol) from the qapi/*.json schemas, following qapi-code-gen.rst. It is the last generator because it is the largest after decodetree and its output is consumed only by T.8 and T.9, so nothing earlier in the track waits on it. Working from QEMU's own schemas rather than imitating the Python is Loose Coupling to the data, not to the tool that happened to read it.

T.6d keymap-gen. This slice ports keycodemapdb's keymap-gen, which turns the key-code table in keymaps.csv into the C arrays that ui/input-keymap.c includes, one array per pair of code sets. QEMU's input core needs them even with no graphical interface, because QMP's send-key and the virtual keyboards translate through them. The CSV is the data and its header row names the columns, so the port reads the data, not the script. The first generator table missed it, and compiling the reduced binary in T.8a found it, which is the track's oracle doing its job (Demonstrability). It comes before T.8a finishes because T.8a cannot compile ui/input-keymap.c without it.

T.6e block-coroutine-wrapper. This slice ports the generator that writes the non-coroutine wrappers for block-layer functions marked co_wrapper or co_wrapper_mixed in QEMU's headers, following docs/devel/block-coroutine-wrapper.rst. The marked declarations are the data, and the document says what each wrapper must do: pack the arguments, run the coroutine, and poll until it finishes. Every block driver depends on the generated block-gen.c, so the reduced binary cannot link without it. Working from the marks and the document keeps the port to what the block layer asks for (Parsimony).

T.6f module_block and two shell scripts. This slice writes block/module_block.h, which tells the block layer which drivers live in loadable modules. With --disable-modules there are none, so the header holds an empty table. No specification exists, so the consumer in block.c and util/module.c defines what it must contain. The slice also ports QEMU's two small shell scripts, make-config-poison.sh and qemu-version.sh, to Crystal, as the build constraints prefer Crystal to shell. Doing the three together keeps a slice from holding a few lines each, and finishes the generator list T.8a needs (Parsimony).

T.7 GLib core and libfdt. This slice writes Crystal build descriptions for the two C libraries QEMU requires. One is GLib (core glib and gmodule only), whose own build also uses Meson and Python. The other is libfdt, the devicetree library from QEMU's bundled dtc subproject, which is plain C. These are build descriptions, not rewrites. It precedes T.8 and T.9 because both binaries link against these libraries. Building only the parts QEMU uses is Parsimony.

T.8 Reduced qemu-system-aarch64. This slice builds the aarch64 binary for the virt machine from a source list held as Crystal data. The list is taken from T.1's inventory, T.4's device selection and the outputs of T.2–T.6, linked against T.7's libraries. It is accepted when it passes the QEMU unit tests that run without Python. It comes after T.2–T.7 because it consumes all of them. A failing QEMU test is the oracle, not a comparison with what Python would have generated, which is Demonstrability.

T.9 Reduced qemu-system-x86_64. This slice builds the x86_64 binary for the q35 machine the same way as T.8, reusing everything except decodetree, which x86 does not need. It follows T.8 because aarch64 leads (plan rule 6), and because by then every shared part is proven. Its emulator supports AMD's SVM but not Intel's VMX, which is why the plan does SVM first. Reusing T.8's build with one machine changed is Parsimony.

T.10 U-Boot for both machines. This slice builds U-Boot, the EFI firmware Glint boots from, for both QEMU machines, using the qemu_arm64_defconfig and qemu-x86_64_defconfig configurations. Configuration and devicetree compilation are done by cleave's Kconfig and dtc, so no Python or flex/bison is needed. It follows T.8 and T.9 because firmware needs a machine to run on before it can be tested. It is mainline U-Boot, the EFI implementation Beagle seeks to support on its boards, so what Glint is tested against is what it should meet in 13.2–13.3. That is Clarity about what is tested.

T.11 The boot-test harness. This slice adds glint-run. It writes an ESP image (with Cleave::Fat16), boots it headless under T.8's or T.9's QEMU with T.10's U-Boot, and passes when an expected line appears on the serial console within a timeout. It comes last because it needs the binaries and the firmware, and it closes the track. Phase 2's first boot (2.1) uses it, and Phase 11's crash test (11.4) extends it. It is the plan's rung-3 Demonstrability, made into a command.

T.12 Builds that survive a moved source tree. Merged as check-in 116d0e26. A maintenance slice, added when the QEMU and U-Boot sources moved from RAM-backed /tmp to disk. Build symlinks are made by one helper that replaces a stale link, tools/uboot_build.cr defaults to build/, and the specs remove what they create under TMPDIR. Builds that survive their inputs moving, and a suite that leaves no trace, are Demonstrability.

T.13 A suite independent of order. Merged as check-in dfd919cc. A maintenance slice from the review of Track Z. The QEMU link specs need archives (libfdt.a, the GLib libraries) that a later spec builds, so the full suite passes only in file order. Each spec that needs an archive builds it through the same entry point the later spec uses, if it is missing. That makes any single spec, and any order, pass from a clean build directory, which is Demonstrability.

T.14 Boots that test the current build. Merged as check-in 16101670. A maintenance slice, found while 2.3 was re-recorded. bin/glint-run builds the images only when none exist, so it boots whatever image is in build/, and a stale one passes the boot checks. It rebuilds when an image is older than any source that goes into it, as tools/glint_build.cr names them. Then a boot always tests the code it claims to, which is Demonstrability.

T.15 A build tool that is never stale. Merged as check-in d0d2a777. A maintenance slice, found while 2.6 was rebased. T.14 rebuilds an image whose inputs changed, but the compiled bin/glint-build can itself predate a change to tools/zephyr/, and then generates old code. The tool now compares itself with its own sources on start, and rebuilds and re-runs itself when it is older. A build that cannot run with an old copy of its own rules is Demonstrability.

Track demo: the Phase 1 images boot under the project-built QEMU and U-Boot on both architectures, with everything built by BQ2 tools alone.

References