rspektrum/README.md

rspektrum

rspektrum is an interactive spectrogram viewer for inspecting radio captures and arbitrary audio. It loads a WAV file, computes a short-time Fourier transform (STFT), and draws the result as a zoomable, pannable time–frequency image. Its distinguishing feature is support for mLnL annotations — labelled regions (TX frames, assertion outcomes, impairment fires, …) carried inside the WAV file itself — which it overlays on the measured spectrogram so you can compare what a modem intended to transmit against what actually hit the air.

You can box a time/frequency region, hear it back through a bandpass filter, and export either the picture (PNG) or the isolated audio (WAV). rspektrum runs three ways: a native desktop app (C + raylib), a headless command-line renderer, and a WebAssembly build in the browser.

Click for Video Demo

---

What it's for

The primary use case is reviewing captures from the mLink radio stack: a WAV recording of an over-the-air signal with an embedded mLnL chunk describing what the modem/daemon believed it was transmitting at each instant. rspektrum renders those annotations on top of the measured spectrogram, frame by frame, so intent and reality sit side by side.

It also works as a general-purpose spectrogram tool for plain WAVs with no annotations. See mlnl_chunk_spec.md for the annotation format.

---

Features

• STFT spectrogram — selectable colormaps, adjustable dB floor / dynamic range, absolute (dBFS) or relative amplitude scaling.
• mLnL annotation overlay — labelled boxes from the WAV's embedded annotation chunk; hover a box (or its region on the scope) for per-frame detail (sequence, channel, rate, scheduling offset…).
• Zoom & pan the time/frequency view.
• Region selection — box a time and frequency range with the mouse.
• Filtered playback — play just the selected region, band-limited to the selected frequency box via an FFT bandpass. What you hear is what you'd export.
• Waveform scope — toggleable time-domain view beneath the spectrum.
• Marker / ruler and a spectrum slice (PSD) readout.
• Export — save the view as a PNG, or the selected region as a WAV.
• Headless render mode — produce an annotated PNG from the CLI with no window, no GL, and no X server. Pure CPU; runs in CI, containers, or over SSH.
• Broad input — WAV directly (8/16-bit PCM, 32-bit float; stereo downmixed to mono); other formats transcoded via ffmpeg if it's on PATH. Drag-and-drop.
• Cross-platform — Linux/desktop, Windows, and a WebAssembly build.

---

Building

You need only make and a C compiler (gcc or clang) plus the X11/OpenGL development headers (see below). raylib is vendored in this repo and compiled from source — there is no separate raylib install step, no premake, no network access required. A plain clone builds:

make                # release -> bin/Release/rspektrum   (-O3 -ffast-math, AVX2/FMA)
make DEBUG=1        # debug   -> bin/Debug/rspektrum     (-g, no optimization)
make run            # build + launch
make test           # build + run the DSP correctness tests
make bench          # FFT benchmark over mlnl_samples.wav
make clean

Useful overrides: make CC=clang, or make ARCH=-march=native to tune for your own CPU (the default -march=x86-64-v3 targets any ~2013+ x86-64 chip; drop it with make ARCH= for an older CPU).

System dependencies

The compiler needs the X11 and OpenGL dev headers (the runtime libs are already present on any desktop; only the -dev/-devel packages are usually missing). The X11 extension libraries (Xrandr, Xinerama, Xcursor, Xi) are opened at runtime via dlopen, but their headers are still required to compile.

If make stops with an error like fatal error: X11/Xlib.h: No such file or GL/gl.h: No such file, install the dev packages for your distribution:

Distro	Command
Debian / Ubuntu / Mint	sudo apt install build-essential libx11-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev libgl1-mesa-dev
Fedora / RHEL / Rocky	sudo dnf install gcc make libX11-devel libXrandr-devel libXinerama-devel libXcursor-devel libXi-devel mesa-libGL-devel
Arch / Manjaro	sudo pacman -S base-devel libx11 libxrandr libxinerama libxcursor libxi mesa
openSUSE	sudo zypper install gcc make libX11-devel libXrandr-devel libXinerama-devel libXcursor-devel libXi-devel Mesa-libGL-devel
Alpine	sudo apk add build-base libx11-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev mesa-dev

On Debian/Ubuntu the single metapackage xorg-dev pulls in all of the X11 -dev packages above, if you'd rather not list them.

Run make check-deps to probe for the required headers before building — it prints the install hint for your platform if anything is missing.

Web (WebAssembly) build

./build_web.sh      # emscripten; emits the WebAssembly bundle to bin/web/

---

Usage (desktop GUI)

./bin/Release/rspektrum [input.wav]

Load a file by passing it on the command line, dragging a .wav onto the window, or pressing O for the file browser. Try the bundled sample:

./bin/Release/rspektrum mlnl_samples.wav    # in-repo WAV with an embedded mLnL chunk

Controls

Navigating. A left-drag pans by default and Ctrl+drag draws a selection box; Tab (or the rail's pan/select icons) swaps which one is bare, and Ctrl always means "the other one", so either mode does both without switching back. Middle-drag always pans.

Input	Action
LMB drag	Pan the view (Ctrl+drag to box-select)
Tab	Swap pan / select mode
Middle-drag / Alt+drag	Pan, regardless of mode
Mouse wheel	Zoom both axes (preserves aspect ratio)
Shift+wheel	Zoom the time axis only
Ctrl+wheel	Zoom the frequency axis only
Wheel on a scrollbar	Pan that axis (Shift to zoom it)
Space	Play / stop the selected region
Hover an annotation	Tooltip with that frame's mLnL detail; lists every overlapping frame under the cursor
N / Shift+N	Jump to the next / previous collision
O	Open file browser
P	Show / hide the waveform scope
M	Marker / ruler tool
S	Spectrum slice (PSD)
E	Export PNG
W	Export selection as WAV
Home	Reset view (fit all)
End	Zoom to start
F11	Toggle fullscreen
F1	About / help
Esc	Clear selection / close dialog

Layout

A menubar across the top holds one-shot actions (File — open, export PNG/WAV; View — reset/zoom, hide the icon rail, fullscreen; Annotations — jump to next collision; Help). Everything that toggles lives on the rail instead, so no control has two homes. Menu items are defined by naming a keyboard shortcut, so an item and its key can never drift apart, and items grey out under exactly the conditions that make the shortcut a no-op.

Down the left is a narrow icon rail — one column of square buttons, sized so it costs the spectrogram as little width as possible. Hover any icon for a tooltip. Left to right in function: play/stop and clear selection; pan/select mode; marker, spectrum slice, scope, grid, minimap; FFT size and colour/level popouts; annotations, collisions, and the timeline lane. The three settings popouts open beside the rail rather than widening it. View → Hide icon rail hands its width back to the spectrogram.

The minimap (top-right, toggled from the rail) is a thumbnail of the whole capture with the current view drawn on it — click or drag anywhere on it to scrub. Annotation density runs along its bottom edge and collisions along its top. The corner handle switches between two sizes. It is rendered once into a texture and only rebuilt when its content changes (new file, colormap, overlays toggled); panning and zooming just move the rectangle drawn on top, so navigation costs nothing.

Inspecting overlapping transmissions

When several stations are on the air at once their annotation boxes stack, and the one drawn last hides the rest. Two features address that:

• Hover any pile-up and the tooltip lists every frame under the cursor — one row per frame with its own colour swatch, led by the fields that actually tell them apart (node, frame name, position in the PTT, channel). Deep piles are capped with a +N more count.
• Collisions (sidebar toggle) highlights where transmissions genuinely overlap in both time and frequency. N / Shift+N, or the sidebar < prev / next > buttons, jump between them; each jump centres the region, keeps the current zoom unless the region needs more room, and reports its position (Collision 7/54 — 3 frames at 1284.95s).

A collision requires a real overlap in time and band, so two frames in different channels at the same instant are not flagged, and neither are zero-duration point markers (control, assertions), which annotate the run rather than occupy the air. Markers are drawn only across the band the overlap occupies, not the full frequency axis. Adjacent collisions merge into one region, so a busy stretch reads as a single span rather than dozens of bars.

---

Usage (headless render)

--render writes the spectrogram straight to a PNG with no window, no GL context, and no X server. It computes the STFT, colorizes the bitmap, bakes the annotation overlay onto it, and exports — all on the CPU — so it runs anywhere (CI, a bare SSH session, a container with no display):

./bin/Debug/rspektrum --render OUT.png INPUT.wav [options]

The output is the real spectrogram bitmap at native STFT resolution (not a screenshot of the UI), so it carries no sidebar/scope chrome — just the time–frequency image with the annotation overlay.

Flag	Effect
-r, --render OUT.png	Render to OUT.png and exit (no window/GL/X)
-a, --annotations	Force the annotation overlay on
--no-annotations	Force the overlay off
--annotation-opacity=V	Overlay strength 0..1 (default 0.5)
--annotation-kinds=LIST	Comma-separated kinds to draw (default: all)
--width N	Resize output to N px wide (default: native STFT size)
-h, --help	Usage

Annotation boxes are drawn outline + label only (no translucent fill): mLnL captures contain many overlapping full-band boxes whose fills would alpha-stack to opaque and bury the signal, so the outline marks each region while the spectrogram reads through.

# everything, brighter overlay
./bin/Debug/rspektrum --render /tmp/all.png mlnl_samples.wav --annotation-opacity=0.7

# only on-air frames and failed assertions
./bin/Debug/rspektrum --render /tmp/tx.png mlnl_samples.wav \
  --annotation-kinds=tx_frame,assertion_failed

Annotation kinds: tx_frame, tx_burst, control, channel_up, channel_down, assertion_passed, assertion_failed, impairment_fire, gain_change, unknown.

The hover tooltip only appears with a live mouse over a box, so it cannot show up in a static --render. To verify tooltip behaviour you need a real (or virtual) display driving the GUI — see below.

---

Driving the GUI headlessly (agents / CI)

The app can be run, screenshotted, and clicked on a virtual X display with no monitor or GPU (Mesa software GL under Xvfb). The full playbook lives in AGENTS.md; the working reference implementation is shot_input.sh.

The loop in one breath:

Xvfb :99 -screen 0 1280x800x24 >/tmp/xvfb.log 2>&1 &     # 1. fake screen
DISPLAY=:99 ./bin/Debug/rspektrum mlnl_samples.wav \
        >/tmp/app.log 2>&1 &                              # 2. run on it
sleep 2                                                    # 3. reach a steady frame
DISPLAY=:99 import -window root /tmp/shot.png             # 4. grab the frame

Prerequisites (Debian/Ubuntu): sudo apt-get install xvfb imagemagick xdotool (plus libgl1-mesa-dri and LIBGL_ALWAYS_SOFTWARE=1 if GL fails / frames are black). Synthesize input with xdotool against DISPLAY=:99 to exercise UI paths.

---

Technical notes

• STFT — Hann-windowed, 2048-point FFT with 50% overlap by default; frequency resolution sampleRate / fftSize Hz per bin. Amplitude in dB.
• Axes — X = time (s), Y = frequency (Hz, scaled to the file's Nyquist), colour = amplitude.
• Loading — the STFT overview is computed in one blocking pass behind the progress panel. It used to advance a fixed number of segments per frame, which made loading frame-paced rather than compute-bound: the frame limiter, not the FFT, set the speed, so a 478k-segment capture spent over a minute waiting between frames. The tell was that backgrounding the window — which skips presenting entirely — loaded the same file in seconds. Background work also continues while the window is unfocused, so a long capture can be left to finish behind another window.
• Long files — two things keep cost tied to what's on screen rather than to total duration. The spectrogram image is built for the visible segment range (capped at 8192 px wide), so a multi-hour capture renders at all — an unbounded full-file image exceeds the GPU texture limit and silently draws nothing — and zooming in genuinely re-renders at higher resolution instead of magnifying pixels. The scope draws from a precomputed min/max summary (1024-sample buckets) rather than rescanning every visible sample each frame, which on a 5.7-hour file is the difference between ~60 ms and ~0.07 ms per frame. Keeping both extremes per bucket means a single-sample transient still shows up when fully zoomed out.
• Time zoom limit — the tightest visible window is derived from the STFT hop (fftSize / HOP_RATIO samples), not from a fixed fraction of the file, so time resolution does not degrade as files get longer: a 30-minute recording zooms in just as far as a 30-second one. At 48 kHz / 2048-point FFT the floor is ~85 ms across the viewport; a smaller FFT zooms correspondingly tighter. Past that point there are no further STFT segments to show, so the view would only interpolate.
• Playback / WAV export share one processing path: the selected time span, FFT-bandpassed to the selected frequency box, peak-normalised.
• mLnL parsing — walks the WAV's RIFF chunks for the four-CC mLnL chunk (UTF-8 JSON Lines); unknown chunks are skipped, so annotated files stay standards-compliant audio everywhere else.

---

Source layout

src/
  spectrogram.c        # entry point, main loop, CLI args, headless render
  stft.c / fft.c       # STFT + FFT
  render.c             # spectrogram, annotations, tooltips, minimap, scope
  ui.c                 # menubar, icon rail + popouts, file browser
  primitives.c         # waveform scope + its min/max envelope summary
  audio.c              # WAV load (ffmpeg fallback), bandpass, playback, WAV export
  mlnl.c / mlnl.h      # mLnL annotation chunk parser
  platform_*.c         # per-OS shims (linux / win32 / web)

See raylib_for_desktop_applications.md for the performance / idle-CPU lessons behind the desktop build, and AGENTS.md for the headless-testing playbook.

Known rough edges — behaviour that is unspecified or awkward rather than simply broken — are tracked in known_bugs.md.