Copy a black box by measuring it

The problem

On a different OS he had used a program that processed playing audio in real time: sound felt farther away, lighter, less “on the ear”. He wanted the same on Linux, built on PipeWire. The catch: he had no copy. Only a friend’s laptop had it, so he could not change his own version and compare it with the reference whenever he wanted.

He could hear something was wrong but had no direction. The project sat dead for a while.

The turn

Stop guessing the parameters, techniques and algorithms inside. Study the behaviour of the filter itself. He did not need repeated access to the program to capture its responses: he generated one audio file on Linux, borrowed the laptop once, played the file through every mode and recorded the output.

The test file

Everything had to be in that one file, so nothing would need re-recording.

Order is illustrative; the contents are the post’s.

1. 1

   Eight sync clicks

   alignment

   Evenly spaced near the start, because playback and recording were started by hand.

2. 2

   Single-sample impulses

   three levels · left, right, both, opposite signs

   Everything after a lone nonzero sample was added by the filter: its decay, shape and phase delays. Silence between them lets each response finish.

3. 3

   Logarithmic sine sweep

   more energy per frequency than one sample

   Easier to separate from recording noise. Solved with a regularised division so near-zero input does not blow up noise:

   Ĥ(f) = Y(f) X*(f) / ( |X(f)|² + λ )
4. 4

   Steady tones

   frequencies and amplitudes

   Plus a 1000 Hz tone alternating between two levels, to see whether gain changed and how long it took.

5. 5

   Eight-frequency multitone

   new frequencies?

   Any frequency that pops up is a sign of non-linearity.

6. 6

   Three identical multitone copies

   time invariance

   Same input, should give the same output each time.

7. 7

   Noise, kept aside

   low, middle, high in sequence

   Not used for fitting. Used afterwards to test whether the extracted filter reproduces its recorded output.

The checks

A convolution filter is only the right model if the box is linear and time-invariant (LTI). The file tests each property directly:

• Scaling: F(ax) = aF(x). A louder impulse gives the same response, proportionally louder.
• Additivity: F(x₁+x₂) = F(x₁)+F(x₂). Both channels at once must equal the separate responses added; opposite signs, subtracted.
• Time invariance: the same input later gives the same response later.

For stereo, the box is four filters: each input channel feeds both outputs.

To align the recording he did not correlate raw clicks, since the filter smears each one over time. He summed squared samples over a window to get energy, cross-correlated the energy envelopes, and applied the same offset to both channels so the delay between them survived.

Results

The four filters showed the trick: sound also appeared in the opposite channel, quieter, slightly delayed, followed by a long tail. That was the out-of-head effect he had been trying to build by hand.

He tried increasingly long filters against the held-back noise and kept the full response, which predicted the noise better than shorter versions. The sweep-derived response agreed wherever it had enough input energy. Error was measured as ε = ‖y − ŷ‖ / ‖y‖, and compared with the difference between the identical repeats:

The model was as close to the recording as the recording was to itself. Loaded into his own Linux tool, Saq, the four responses gave the placement he had been chasing.

Use it on any black box

1. 1

   Design the inputs

   One session of access is enough if the probe file asks every question up front.

2. 2

   Check the assumptions first

   Scaling, additivity, repeats. Only then fit the model those assumptions allow.

3. 3

   Fit

   Impulses and sweeps, two routes to the same answer.

4. 4

   Validate on held-back input

   Predict something the fit never saw.

5. 5

   Compare error with repeat noise

   When model error matches repeat-to-repeat error, you are done.

When a check fails, as with the non-linear and time-varying modes, that is itself the finding: those modes need a different model, not a better filter.

Read more

Reverse engineering a commercial spatial audio effect, the source for every fact on this page.

Hacker News discussion.

Watch the Short: deemwar on YouTube.

Sources: sections “First attempt”, “Re-attempt”, “The assembly of tests”, “Recovering the filter”, “Output” of the source post.