rspektrum/README.md

# rspektrum

**rspektrum** is an interactive spectrogram viewer for inspecting radio captures
and arbitrary audio. It loads a WAV file, computes a short-time Fourier transform
(STFT), and draws the result as a zoomable, pannable time–frequency image. Its
distinguishing feature is support for **mLnL annotations** — labelled regions
(TX frames, assertion outcomes, impairment fires, …) carried *inside* the WAV
file itself — which it overlays on the measured spectrogram so you can compare
what a modem *intended* to transmit against what actually hit the air.

You can box a time/frequency region, hear it back through a bandpass filter, and
export either the picture (PNG) or the isolated audio (WAV). rspektrum runs three
ways: a native desktop app (C + raylib), a headless command-line renderer, and a
WebAssembly build in the browser.

## [Click for Video Demo](https://nicecrew.tv/w/2w9Y5qvKuDz6mwrzAweryW)



![rspektrum spectrogram view with mLnL annotation overlay](resources/Screenshot.png)



---

## What it's for

The primary use case is reviewing captures from the **mLink** radio stack: a WAV
recording of an over-the-air signal with an embedded `mLnL` chunk describing what
the modem/daemon believed it was transmitting at each instant. rspektrum renders
those annotations on top of the measured spectrogram, frame by frame, so intent
and reality sit side by side.

It also works as a general-purpose spectrogram tool for plain WAVs with no
annotations. See [`mlnl_chunk_spec.md`](mlnl_chunk_spec.md) for the annotation
format.

---

## Features

- **STFT spectrogram** — selectable colormaps, adjustable dB floor / dynamic
  range, absolute (dBFS) or relative amplitude scaling.
- **mLnL annotation overlay** — labelled boxes from the WAV's embedded annotation
  chunk; hover a box (or its region on the scope) for per-frame detail (sequence,
  channel, rate, scheduling offset…).
- **Zoom & pan** the time/frequency view, with a **minimap** for navigating a
  long capture without zooming out and back in.
- **Collision detection** — finds transmissions that genuinely overlap in *both*
  time and frequency, and jumps between them.
- **Region selection** — box a time *and* frequency range with the mouse.
- **Filtered playback** — play just the selected region, band-limited to the
  selected frequency box via an FFT bandpass. What you hear is what you'd export.
  Transport controls (play/pause, rewind, loop) attach to the selection, and the
  playhead can be dragged to replay a subrange.
- **Waveform scope** — toggleable time-domain view beneath the spectrum.
- **Marker / ruler** and a **spectrum slice (PSD)** readout.
- **Export** — save the view as a PNG, or the selected region as a WAV.
- **Headless render mode** — produce an annotated PNG from the CLI with no
  window, no GL, and no X server. Pure CPU; runs in CI, containers, or over SSH.
- **Broad input** — WAV directly (8/16-bit PCM, 32-bit float; stereo downmixed to
  mono); other formats transcoded via `ffmpeg` if it's on `PATH`. Drag-and-drop.
- **Cross-platform** — Linux/desktop, Windows, and a WebAssembly build.

---

## Building

You need only **`make` and a C compiler** (`gcc` or `clang`) plus the X11/OpenGL
**development** headers (see below). raylib is vendored in this repo and compiled
from source — there is no separate raylib install step, no `premake`, no network
access required. A plain clone builds:

```bash
make                # release -> bin/Release/rspektrum   (-O3 -ffast-math, AVX2/FMA)
make DEBUG=1        # debug   -> bin/Debug/rspektrum     (-g, no optimization)
make run            # build + launch
make test           # build + run the DSP correctness tests
make bench          # FFT benchmark over mlnl_samples.wav
make clean
```

Useful overrides: `make CC=clang`, or `make ARCH=-march=native` to tune for your
own CPU (the default `-march=x86-64-v3` targets any ~2013+ x86-64 chip; drop it
with `make ARCH=` for an older CPU).

### System dependencies

The compiler needs the X11 and OpenGL **dev** headers (the runtime libs are
already present on any desktop; only the `-dev`/`-devel` packages are usually
missing). The X11 extension libraries (Xrandr, Xinerama, Xcursor, Xi) are opened
at runtime via `dlopen`, but their **headers** are still required to compile.

If `make` stops with an error like `fatal error: X11/Xlib.h: No such file` or
`GL/gl.h: No such file`, install the dev packages for your distribution:

| Distro | Command |
|--------|---------|
| **Debian / Ubuntu / Mint** | `sudo apt install build-essential libx11-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev libgl1-mesa-dev` |
| **Fedora / RHEL / Rocky** | `sudo dnf install gcc make libX11-devel libXrandr-devel libXinerama-devel libXcursor-devel libXi-devel mesa-libGL-devel` |
| **Arch / Manjaro** | `sudo pacman -S base-devel libx11 libxrandr libxinerama libxcursor libxi mesa` |
| **openSUSE** | `sudo zypper install gcc make libX11-devel libXrandr-devel libXinerama-devel libXcursor-devel libXi-devel Mesa-libGL-devel` |
| **Alpine** | `sudo apk add build-base libx11-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev mesa-dev` |

On Debian/Ubuntu the single metapackage `xorg-dev` pulls in all of the X11 `-dev`
packages above, if you'd rather not list them.

Run `make check-deps` to probe for the required headers before building — it
prints the install hint for your platform if anything is missing.

### Web (WebAssembly) build

```bash
source ~/emsdk/emsdk_env.sh          # emscripten on PATH
./build_web.sh                       # emits the bundle to bin/web/
cd bin/web && python3 -m http.server 8080
```

Then open <http://localhost:8080/rspektrum.html>.

The browser build's filesystem is an in-memory sandbox, so files come in by
**drag-and-drop** or the **Open file** button (which drives the host's native
file picker and copies the result in). The desktop file browser is bypassed
there — it could only ever list what the page itself had written.

The STFT is computed in one synchronous pass on load rather than progressively,
because the incremental fill depends on idle main-loop frames that the browser
doesn't give back the same way. Long captures therefore block until they finish.

---

## Usage (desktop GUI)

```bash
./bin/Release/rspektrum [input.wav]
```

Load a file by passing it on the command line, dragging a `.wav` onto the window,
or pressing **O** for the file browser. Try the bundled sample:

```bash
./bin/Release/rspektrum mlnl_samples.wav    # in-repo WAV with an embedded mLnL chunk
```

### Controls

**Navigating.** A left-drag pans by default and **Ctrl+drag** draws a selection
box; **Tab** (or the rail's pan/select icons) swaps which one is bare, and Ctrl
always means "the other one", so either mode does both without switching back.
Middle-drag always pans.

| Input | Action |
|-------|--------|
| **LMB drag** | Pan the view (**Ctrl+drag** to box-select) |
| **Tab** | Swap pan / select mode |
| **Middle-drag** / **Alt+drag** | Pan, regardless of mode |
| **Mouse wheel** | Zoom both axes (preserves aspect ratio) |
| **Shift+wheel** | Zoom the time axis only |
| **Ctrl+wheel** | Zoom the frequency axis only |
| **Wheel on a scrollbar** | Pan that axis (**Shift** to zoom it) |
| **Space** | Play / stop the selected region |
| **Drag the playhead** | (while stopped) set where the next play starts |
| **Hover an annotation** | Tooltip with that frame's mLnL detail; lists **every** overlapping frame under the cursor |
| **N** / **Shift+N** | Jump to the next / previous collision |
| **O** | Open file browser (the host's file picker on web) |
| **P** | Show / hide the waveform scope |
| **M** | Marker / ruler tool |
| **S** | Spectrum slice (PSD) |
| **E** | Export PNG |
| **W** | Export selection as WAV |
| **Home** | Reset view (fit all) |
| **End** | Zoom to start |
| **F11** | Toggle fullscreen |
| **F1** | About / help |
| **Esc** | Clear selection / close dialog |

### Layout

A menubar across the top holds one-shot actions (**File** — open, export
PNG/WAV; **View** — reset/zoom, hide the icon rail, fullscreen; **Annotations** —
jump to next collision; **Help**). Everything that toggles lives on the rail
instead, so no control has two homes. Menu items are defined by naming a keyboard shortcut, so an item and
its key can never drift apart, and items grey out under exactly the conditions
that make the shortcut a no-op.

Down the left is a narrow **icon rail** — one column of square buttons, sized so
it costs the spectrogram as little width as possible. Hover any icon for a
tooltip. Left to right in function: play/stop and clear selection; pan/select
mode; marker, spectrum slice, scope, grid, minimap; FFT size and colour/level
popouts; annotations, collisions, and the timeline lane. The three settings
popouts open beside the rail rather than widening it. `View → Hide icon rail`
hands its width back to the spectrogram.

The **minimap** (top-right, toggled from the rail) is a thumbnail of the whole
capture with the current view drawn on it — click or drag anywhere on it to
scrub. Annotation density runs along its bottom edge and collisions along its
top. The corner handle switches between two sizes. It is rendered once into a
texture and only rebuilt when its *content* changes (new file, colormap,
overlays toggled); panning and zooming just move the rectangle drawn on top, so
navigation costs nothing.

### Playback

**Space** plays the selected region, band-limited to the selected frequency box.
Selecting a region also brings up a small transport bar above it — rewind,
play/pause, stop, and loop. Pause resumes where it left off; loop repeats the
whole region.

While stopped, the playhead stays where it is and can be dragged: that sets
where the next play begins *within* the selection, so a subrange can be replayed
without redrawing the region. Clearing or redrawing the selection resets it.

### Inspecting overlapping transmissions

When several stations are on the air at once their annotation boxes stack, and
the one drawn last hides the rest. Two features address that:

- **Hover** any pile-up and the tooltip lists *every* frame under the cursor —
  one row per frame with its own colour swatch, led by the fields that actually
  tell them apart (node, frame name, position in the PTT, channel). Deep piles
  are capped with a `+N more` count.
- **Collisions** (sidebar toggle) highlights where transmissions genuinely
  overlap in **both** time and frequency. `N` / `Shift+N`, or the sidebar
  `< prev` / `next >` buttons, jump between them; each jump centres the region,
  keeps the current zoom unless the region needs more room, and reports its
  position (`Collision 7/54 — 3 frames at 1284.95s`).

A collision requires a real overlap in time *and* band, so two frames in
different channels at the same instant are not flagged, and neither are
zero-duration point markers (`control`, assertions), which annotate the run
rather than occupy the air. Markers are drawn only across the band the overlap
occupies, not the full frequency axis. Adjacent collisions merge into one
region, so a busy stretch reads as a single span rather than dozens of bars.

---

## Usage (headless render)

`--render` writes the spectrogram straight to a PNG **with no window, no GL
context, and no X server**. It computes the STFT, colorizes the bitmap, bakes the
annotation overlay onto it, and exports — all on the CPU — so it runs anywhere
(CI, a bare SSH session, a container with no display):

```bash
./bin/Debug/rspektrum --render OUT.png INPUT.wav [options]
```

The output is the **real spectrogram bitmap** at native STFT resolution (not a
screenshot of the UI), so it carries no sidebar/scope chrome — just the
time–frequency image with the annotation overlay.

| Flag | Effect |
|------|--------|
| `-r, --render OUT.png` | Render to `OUT.png` and exit (no window/GL/X) |
| `-a, --annotations` | Force the annotation overlay **on** |
| `--no-annotations` | Force the overlay off |
| `--annotation-opacity=V` | Overlay strength `0..1` (default `0.5`) |
| `--annotation-kinds=LIST` | Comma-separated kinds to draw (default: all) |
| `--width N` | Resize output to `N` px wide (default: native STFT size) |
| `-h, --help` | Usage |

Annotation boxes are drawn **outline + label only** (no translucent fill): mLnL
captures contain many overlapping full-band boxes whose fills would alpha-stack
to opaque and bury the signal, so the outline marks each region while the
spectrogram reads through.

```bash
# everything, brighter overlay
./bin/Debug/rspektrum --render /tmp/all.png mlnl_samples.wav --annotation-opacity=0.7

# only on-air frames and failed assertions
./bin/Debug/rspektrum --render /tmp/tx.png mlnl_samples.wav \
  --annotation-kinds=tx_frame,assertion_failed
```

Annotation kinds: `tx_frame`, `tx_burst`, `control`, `channel_up`,
`channel_down`, `assertion_passed`, `assertion_failed`, `impairment_fire`,
`gain_change`, `unknown`.

> The hover tooltip only appears with a live mouse over a box, so it cannot show
> up in a static `--render`. To verify tooltip behaviour you need a real (or
> virtual) display driving the GUI — see below.

---

## Driving the GUI headlessly (agents / CI)

The app runs, screenshots, and takes synthetic input on a virtual X display with
no monitor or GPU (Mesa software GL under Xvfb). The playbook is in
[`AGENTS.md`](AGENTS.md); [`shot_input.sh`](shot_input.sh) is the working
reference implementation.

---

## Technical notes

- **STFT** — Hann-windowed, 2048-point FFT with 50% overlap by default;
  frequency resolution `sampleRate / fftSize` Hz per bin. Amplitude in dB.
- **Axes** — X = time (s), Y = frequency (Hz, scaled to the file's Nyquist),
  colour = amplitude.
- **Loading** — the STFT overview is computed in one blocking pass behind the
  progress panel. It used to advance a fixed number of segments per frame, which
  made loading frame-paced rather than compute-bound: the frame limiter, not the
  FFT, set the speed, so a 478k-segment capture spent over a minute waiting
  between frames. The tell was that backgrounding the window — which skips
  presenting entirely — loaded the same file in seconds. Background work also
  continues while the window is unfocused, so a long capture can be left to
  finish behind another window.
- **Long files** — two things keep cost tied to what's on screen rather than to
  total duration. The spectrogram image is built for the *visible* segment range
  (capped at 8192 px wide), so a multi-hour capture renders at all — an
  unbounded full-file image exceeds the GPU texture limit and silently draws
  nothing — and zooming in genuinely re-renders at higher resolution instead of
  magnifying pixels. The scope draws from a precomputed min/max summary
  (1024-sample buckets) rather than rescanning every visible sample each frame,
  which on a 5.7-hour file is the difference between ~60 ms and ~0.07 ms per
  frame. Keeping both extremes per bucket means a single-sample transient still
  shows up when fully zoomed out.
- **Cursor dB readout** — averaged over a neighbourhood of STFT cells
  (±3 segments × ±6 bins, roughly 76 Hz × 300 ms at 12 kHz / 2048) rather than
  read from one bin. A single bin of an OFDM burst swings ~30 dB between
  adjacent subcarriers and symbols, so a one-bin readout reports where the
  cursor happened to land rather than the level of the signal under it. The mean
  is taken over power and converted to dB afterwards — averaging dB values is a
  geometric mean of power and reads a couple of dB low. Sized in STFT cells, not
  screen pixels: one pixel spans hundreds of segments zoomed out and a fraction
  of a bin zoomed in.
- **Time zoom limit** — the tightest visible window is derived from the STFT hop
  (`fftSize / HOP_RATIO` samples), not from a fixed fraction of the file, so time
  resolution does not degrade as files get longer: a 30-minute recording zooms in
  just as far as a 30-second one. At 48 kHz / 2048-point FFT the floor is ~85 ms
  across the viewport; a smaller FFT zooms correspondingly tighter. Past that
  point there are no further STFT segments to show, so the view would only
  interpolate.
- **Playback / WAV export** share one processing path: the selected time span,
  FFT-bandpassed to the selected frequency box, peak-normalised.
- **mLnL parsing** — walks the WAV's RIFF chunks for the four-CC `mLnL` chunk
  (UTF-8 JSON Lines); unknown chunks are skipped, so annotated files stay
  standards-compliant audio everywhere else.

---

## Source layout

```
src/
  spectrogram.c        # entry point, main loop, CLI args, headless render
  stft.c / fft.c       # STFT + FFT
  render.c             # spectrogram, annotations, tooltips, minimap, scope
  ui.c                 # menubar, icon rail + popouts, file browser
  primitives.c         # waveform scope + its min/max envelope summary
  audio.c              # WAV load (ffmpeg fallback), bandpass, playback, WAV export
  mlnl.c / mlnl.h      # mLnL annotation chunk parser
  platform_*.c         # per-OS shims (linux / win32 / web)
```

See [`raylib_for_desktop_applications.md`](raylib_for_desktop_applications.md)
for the performance / idle-CPU lessons behind the desktop build, and
[`AGENTS.md`](AGENTS.md) for the headless-testing playbook.

Known rough edges — behaviour that is unspecified or awkward rather than simply
broken — are tracked in [`known_bugs.md`](known_bugs.md).