Source: Nix wrote half of my debugger, by Farid Zakaria
A test that fails some runs on your laptop can pass on someone else’s, because the difference is which thread ran when. Farid Zakaria’s Rewind VM makes such a run replayable, and most of the hard part came from Nix: a derivation already names the inputs of a build.
A flaky failure depends on the order threads ran in, and that order is not in the test. Rerun it on another machine and the schedule is different, so the bug may never show.
Two threads deposit into one bank account. Each one reads the balance, writes a line to the ledger, then stores the balance it read plus its deposit. If one thread runs between the other’s read and store, it writes a stale balance over the other’s deposit.
| teller 1 | teller 2 | balance |
|---|---|---|
| read 150 | 150 | |
| read 150 | 150 | |
| store 200 | 200 | |
| store 150 + 10 | 160 (teller 1’s deposit is gone) |
taskset, it lost money in none of 1,000: one core rarely switches threads in the middle of a deposit.A debugger that replays a run needs the program’s exact inputs, its debug symbols, its sources, the sources of every library under it, and a way for someone else to get all of that. A Nix derivation already is that list.
Source, compiler, libraries, kernel and the VM’s configuration. rewind nix realises them, packs the closure into a read-only erofs image and boots the VM on it.
Every run replays exactly. rewind show prints the command that makes a run again. Output NAR hashes are checked against the local store and the binary caches.
The build runs again under schedules that ask the guest kernel to reschedule at different steps, and the first failure is narrowed to one step: step 3237. The passing and failing runs are identical until that step.
nixpkgs builds with separateDebugInfo, and the debug outputs live on cache.nixos.org, so the source at any step is available, the Linux kernel included.
Same inputs, same schedule: the same run, step for step.
rewind where shows the source and stack at any step: thread 141 at deposit (bank.c:24).rewind gdb opens gdb on a fork of the run at any step; nothing gdb does changes the recording..rwd export, so the person you hand it to opens it with your notes on the timeline.Nix wrote half of my debugger by Farid Zakaria, the source for every fact on this page (text CC BY-SA).
A companion Short covers the same story; its link will be added here.
Sources: numbers from the source post (footnote 1; sections “Two tellers, one account”, “Free thing one”, “Free thing two”, “Who had the CPU”, “Check from here”, “Bookmarks”).