feat: headless PNG render, mLnL annotations, and per-frame sched offset
Commit the accumulated working-tree changes as one snapshot. - Headless render: `--render OUT.png INPUT.wav` draws the spectrogram (full window, or `--pane` for the spectrogram pane only) to a PNG with no visible window. Options: `--annotations`/`--no-annotations`, `--annotation-opacity`, `--width`/`--height`. - mLnL annotations: parse the optional `mLnL` RIFF chunk (schema v2) and render tx_frame/assertion/control overlays, a timeline lane, and a waveform-scope echo, with hover tooltips on the spectrogram, timeline, and scope. - sched_offset_ms: parse the per-frame intent->air latency and surface it in the hover tooltips (boxes stay air-anchored upstream). - Supporting: build wiring (rspektrum.make), shared types/headers, web-build and capture-script tweaks, and removal of the old synchrosqueezing LaTeX doc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,276 @@
|
||||
# mLnL: mLink-annotated WAV chunk specification (v2)
|
||||
|
||||
A way to ship spectrograph annotations *inside* a regular WAV file so
|
||||
the file stays standards-compliant audio everywhere while a viewer that
|
||||
knows the format can render labelled regions (TX bursts, assertion
|
||||
outcomes, impairment fires, etc.) over the audio's spectrogram.
|
||||
|
||||
**Status**: v2, stable. Producer: `mlink_fake_ota` (see
|
||||
`src/tools/harness/fota_capture.c` + `fota_events.c`). Consumer: any
|
||||
tool that walks RIFF chunks and looks for the four-CC `mLnL`.
|
||||
|
||||
---
|
||||
|
||||
## 1. WAV / RIFF refresher
|
||||
|
||||
A standard WAV file is a RIFF container:
|
||||
|
||||
```
|
||||
RIFF<file_size - 8 (uint32 LE)>WAVE
|
||||
fmt <16 (uint32 LE)><PCM format fields>
|
||||
data<N (uint32 LE)><PCM samples>
|
||||
... more chunks may follow ...
|
||||
```
|
||||
|
||||
Each chunk = 4-byte four-CC ID + 4-byte unsigned little-endian size +
|
||||
payload of exactly that many bytes + 1 pad byte if the size is odd. Per
|
||||
the RIFF specification, a reader MUST skip any chunk whose ID it does
|
||||
not recognise.
|
||||
|
||||
## 2. The mLnL chunk
|
||||
|
||||
```
|
||||
mLnL<payload_size (uint32 LE)><JSONL bytes>(+ 1 pad if size odd)
|
||||
```
|
||||
|
||||
ID: ASCII `mLnL` (= bytes `0x6D 0x4C 0x6E 0x4C`).
|
||||
|
||||
Payload: UTF-8 JSON Lines. One JSON object per line, terminated by
|
||||
`\n`. Lines are emitted in chronological order of `t_start`.
|
||||
|
||||
A truncation marker may appear as the **last** line:
|
||||
|
||||
```
|
||||
{"truncated":true}
|
||||
```
|
||||
|
||||
This means the producer's in-memory buffer overflowed and some events
|
||||
were dropped.
|
||||
|
||||
## 3. Reading the chunk
|
||||
|
||||
```python
|
||||
def read_mlnl(path):
|
||||
with open(path, 'rb') as f:
|
||||
data = f.read()
|
||||
if data[:4] != b'RIFF' or data[8:12] != b'WAVE':
|
||||
return None
|
||||
i = 12
|
||||
while i + 8 <= len(data):
|
||||
cid = data[i:i+4]
|
||||
sz = int.from_bytes(data[i+4:i+8], 'little')
|
||||
if cid == b'mLnL':
|
||||
return data[i+8 : i+8+sz].decode('utf-8').splitlines()
|
||||
i += 8 + sz + (sz & 1)
|
||||
return None
|
||||
```
|
||||
|
||||
## 4. Event schema (v2)
|
||||
|
||||
**Every** event is a single JSON object with these REQUIRED fields:
|
||||
|
||||
| field | type | meaning |
|
||||
|----------|--------|---------|
|
||||
| `t_start`| number | start of the event's time range, seconds since channel start |
|
||||
| `t_end` | number | end of the event's time range, seconds; == t_start for instantaneous |
|
||||
| `kind` | string | event kind (see §4.2) |
|
||||
|
||||
### 4.1 Optional fields available on any kind
|
||||
|
||||
| field | type | meaning |
|
||||
|------------|--------|---------|
|
||||
| `f_lo` | number | low edge of frequency annotation (Hz) |
|
||||
| `f_hi` | number | high edge of frequency annotation (Hz) |
|
||||
| `f_center` | number | center frequency (Hz); alternative to `f_lo`/`f_hi` |
|
||||
| `f_bw` | number | bandwidth around `f_center` (Hz); paired with `f_center` |
|
||||
| `note` | string | free-form human-readable label (the "why this is here" line) |
|
||||
| `color` | string | suggested fill or outline colour, `#RRGGBB` |
|
||||
|
||||
A consumer should accept EITHER `f_lo`+`f_hi` OR `f_center`+`f_bw` (or
|
||||
both -- prefer `f_lo`/`f_hi` if both are present). Events with no
|
||||
frequency fields apply to the whole spectrum.
|
||||
|
||||
### 4.2 Kinds emitted by mlink_fake_ota today
|
||||
|
||||
**`tx_frame` is the primary annotation** — ONE per on-air air-frame,
|
||||
sourced from the *transmitting daemon's own per-frame log event* (its
|
||||
self-report, forwarded to the channel over the logging egress), NOT guessed
|
||||
from the mixed audio. A whole transmission renders as its real labelled
|
||||
sequence of boxes: e.g. PRESENCE + RTS in an announce channel, then N bulk
|
||||
OFDM frames in the bulk band, each tagged with its LDPC rate.
|
||||
|
||||
```
|
||||
tx_frame t_start = frame key-down, t_end = key-up, placed at the
|
||||
ACTUAL on-air anchor + the frame's sample offset (NOT the
|
||||
modem's render/intent time -- see note below)
|
||||
fields:
|
||||
node u32 transmitting node_id (attribution + color)
|
||||
seq int 0-based index of this frame within the PTT
|
||||
n int total frames in this PTT -> "seq of n"
|
||||
frame str mLink frame name (vocabulary below)
|
||||
ch str daemon channel: EMERGENCY / ANNOUNCE_0|1|2 /
|
||||
BULK (the region the frame occupies)
|
||||
rate str? LDPC rate of a bulk OFDM frame
|
||||
("1/2","2/3","3/4","5/6"); absent on announce
|
||||
f_lo num low edge of `ch`'s band (Hz; see §4.4)
|
||||
f_hi num high edge of `ch`'s band (Hz)
|
||||
sched_offset_ms num intent->air latency of this burst:
|
||||
(on-air rising edge) - (modem render time).
|
||||
A "modem wants vs actually does" metric;
|
||||
coarse (block-quantized, multi-stage).
|
||||
color str #RRGGBB, stable per node
|
||||
note str pre-formatted "node N <frame> [rate]"
|
||||
|
||||
control t_start == t_end ; one channel control verb dispatched
|
||||
fields: command, req_id?
|
||||
assertion_passed t_start = watch start, t_end = match observation
|
||||
fields: name, slack_s, note, color (green default
|
||||
#3CB371), f_lo?, f_hi?
|
||||
assertion_failed t_start = watch start, t_end = watch deadline
|
||||
fields: name, reason, note, color (red default
|
||||
#D62828), f_lo?, f_hi?
|
||||
```
|
||||
|
||||
**`frame` vocabulary** (announce family/subtype + bulk; render verbatim):
|
||||
`PRESENCE/AUTO`, `PRESENCE/OPERATOR`, `PRESENCE/GEOPOLL`,
|
||||
`PRESENCE/GEOPOLL_RESP`, `PRESENCE/SEED`, `PRESENCE/SEED_ACK`, `BULK/RTS`,
|
||||
`BULK/ACK`, `BULK/DNAK`, `BULK/SBSERIES`, `BULK/SBRTS`, `BULK/RETX_RTS`,
|
||||
`BULK/SB_RETX_RTS`, `BULK/INET_REQ`, `BULK` (one OFDM data frame),
|
||||
`SHORTBURST` (one strung-announce data frame), `RELAY`, `APRS`. New names
|
||||
appear as the protocol grows — treat `frame` as a free-form short string.
|
||||
|
||||
**REMOVED — `tx_burst`** (energy-detected): the channel's old *guess* at "a
|
||||
station keyed, roughly narrow or wide," from mixed-audio energy + an HF ratio +
|
||||
manual `tag_next_burst` labels. Replaced by `tx_frame` (the daemon knows exactly
|
||||
what it keyed) and no longer emitted; `tag_next_burst` is gone too. A pre-2026-
|
||||
05-28 wav may still contain `tx_burst`; treat it as opaque/legacy.
|
||||
|
||||
**On-air anchoring (why `sched_offset_ms` exists).** The daemon emits its
|
||||
`tx_frame` log when it *renders* a transmission (its intent), but the audio only
|
||||
reaches the air after the keying/buffering pipeline. The producer buffers each
|
||||
node's frames as the logs arrive (intent time) and places them only once the
|
||||
node's *actual* on-air energy rising edge is observed (air time) -- so the boxes
|
||||
land on the real signal. `sched_offset_ms = air_time - intent_time` is that gap,
|
||||
reported per burst. It's a useful "modem wants vs actually does" metric but
|
||||
coarse (block-quantized; conflates render/queue/buffer/transit), so read it as
|
||||
an indicator, not a precise stopwatch.
|
||||
|
||||
`channel_up` / `channel_down` are not emitted: the wav itself bounds the
|
||||
session (first/last sample == up/down edge).
|
||||
|
||||
Future kinds reserved (not all emitted yet):
|
||||
|
||||
```
|
||||
rx_event a receiver's decode/deliver outcome
|
||||
({ node, from, snr_db, frame|kind, ... })
|
||||
impairment_fire { name, ... }
|
||||
```
|
||||
|
||||
Consumers should ignore unknown kinds gracefully.
|
||||
|
||||
### 4.3 Field conventions
|
||||
|
||||
- `node` is a 32-bit unsigned integer (the registered node_id).
|
||||
- Frequencies are in Hz, durations in seconds, amplitudes in
|
||||
channel-internal units roughly [-1, +1] before auto-gain.
|
||||
- `command` matches the verbs in `tools/harness/fota_control.h`.
|
||||
- `name` (for assertions) should ideally read as a hypothesis like
|
||||
`IF SEED THEN B SEED_ACK WITHIN 4s` -- the renderer uses it as the
|
||||
region label.
|
||||
|
||||
### 4.4 Channel -> spectral band
|
||||
|
||||
`tx_frame` carries both the daemon channel name (`ch`) and the resolved
|
||||
band (`f_lo`/`f_hi`). The mapping (mLink's finalized spectrum allocation):
|
||||
|
||||
| `ch` | band (Hz) | mode / center |
|
||||
|--------------|---------------|----------------------|
|
||||
| `EMERGENCY` | 125 .. 175 | NB8FSK, 150 Hz |
|
||||
| `ANNOUNCE_0` | 265 .. 535 | QPSK, 400 Hz |
|
||||
| `ANNOUNCE_1` | 665 .. 935 | QPSK, 800 Hz |
|
||||
| `ANNOUNCE_2` | 1065 .. 1335 | QPSK, 1200 Hz |
|
||||
| `BULK` | 1594 .. 2906 | OFDM, 2250 Hz center |
|
||||
|
||||
The three announce channels are ~270 Hz wide and non-overlapping, so a
|
||||
PRESENCE/RTS box and the bulk blob of the same transmission land in clearly
|
||||
separate horizontal lanes. A renderer can trust `f_lo`/`f_hi` directly or
|
||||
re-derive from `ch` via this table.
|
||||
|
||||
## 5. Rendering tips for a spectrograph
|
||||
|
||||
- Time axis: `t_start` and `t_end` map directly to the wav's time axis
|
||||
(sample N = t * sample_rate). Draw the annotation as a horizontal
|
||||
span over [t_start, t_end].
|
||||
- Frequency axis: if the event has `f_lo`/`f_hi`, render as a rectangle
|
||||
spanning [t_start, t_end] x [f_lo, f_hi]. If only `f_center`/`f_bw`,
|
||||
use [f_center - f_bw/2, f_center + f_bw/2]. If neither, span the
|
||||
full frequency axis.
|
||||
- For `tx_frame` (the main case): render a filled / translucent box over
|
||||
[t_start, t_end] x [f_lo, f_hi] in the frame's `color`; label it with
|
||||
`frame` (or the pre-formatted `note`). Because each frame carries its own
|
||||
exact band, consecutive frames of one transmission draw as a readable
|
||||
sequence -- a narrow PRESENCE then RTS in an announce lane, then the bulk
|
||||
frames stacked in the bulk lane. Use `seq`/`n` to show position
|
||||
("3/6") and `rate` to annotate the LDPC ramp on bulk frames.
|
||||
- For `assertion_passed`: thin outlined rectangle in green (use `color`
|
||||
field if present, else `#3CB371`); label with `note`.
|
||||
- For `assertion_failed`: thin outlined rectangle in red (or `color`);
|
||||
label with `note`. This is the "we were expecting something here
|
||||
and it never arrived" overlay -- visually distinguishing it from
|
||||
passes is the point.
|
||||
- For `control`: vertical markers at `t_start` (zero-width).
|
||||
- Per-node colors are stable across the run (the producer cycles a
|
||||
built-in palette by node_id); consumers may honour them or override
|
||||
with their own per-node scheme.
|
||||
- Layer order: `tx_frame` (background fill) -> assertion outcomes (mid-
|
||||
layer outlines) -> `control` markers (top). Keep opacity moderate so
|
||||
overlapping annotations stay legible.
|
||||
- `tx_burst` is no longer emitted (removed). Only a legacy pre-2026-05-28 wav
|
||||
carries it; render `tx_frame` and ignore `tx_burst` if present.
|
||||
|
||||
## 6. Producer responsibilities
|
||||
|
||||
1. Write `fmt ` + `data` chunks first in standard form (file plays as
|
||||
plain audio for any tool).
|
||||
2. Append `mLnL` after `data` with the JSONL bytes.
|
||||
3. Patch the outer RIFF size at file offset 4 to `file_size - 8`.
|
||||
4. Pad to even byte boundary if needed (single `\0` byte; not counted
|
||||
in the chunk size field).
|
||||
5. Emit JSONL in monotonic `t_start` order.
|
||||
6. Each line is a single JSON object terminated by `\n` -- no
|
||||
multi-line objects, no trailing commas.
|
||||
|
||||
## 7. Consumer responsibilities
|
||||
|
||||
- Treat any unknown chunk ID as opaque -- skip it per RIFF rules.
|
||||
- Walk chunks from the start; don't assume `mLnL` is at a fixed offset.
|
||||
- Handle a `{"truncated":true}` last-line case as a status flag.
|
||||
- Tolerate unknown fields and unknown `kind` values.
|
||||
|
||||
## 8. Version + compatibility
|
||||
|
||||
The chunk ID `mLnL` corresponds to schema v2 (this document). v1 (now
|
||||
deprecated) used `{t_s, kind, ...}` with paired `*_start`/`*_stop`
|
||||
events; v2 collapses pairs into ranges with `t_start`/`t_end` and adds
|
||||
the frequency / note / color fields. Within v2: `tx_frame` (the
|
||||
daemon-sourced per-frame kind, with `node/seq/n/frame/ch/rate/f_lo/f_hi/
|
||||
sched_offset_ms/color/note`) was added 2026-05-28 and is the primary
|
||||
annotation, anchored on the actual on-air edge; the energy-detected
|
||||
`tx_burst` and the `tag_next_burst` verb were removed in its favour.
|
||||
Consumers MUST tolerate both presence and absence of all optional fields
|
||||
and unknown `kind`s.
|
||||
|
||||
If we need an incompatible schema change later, we'll mint a new
|
||||
four-CC (`mLn3` etc.) and keep `mLnL` reserved for v2.
|
||||
|
||||
## 9. References
|
||||
|
||||
- mLink producer source: `src/tools/harness/fota_capture.c`
|
||||
(`append_riff_chunk` + `fota_capture_write_wav_with_annotations`)
|
||||
- Event emitter: `src/tools/harness/fota_events.{c,h}`
|
||||
- `tx_frame` ingest (forwarded-log -> annotation): the FOTA_MSG_LOG handler
|
||||
+ `ingest_log_event` in `src/tools/harness/mlink_fake_ota.c`; the daemon
|
||||
side emits `tx_frame` in `src/protocol/tx_render.c`.
|
||||
- Assertion engine: `src/tools/harness/fota_assert.{c,h}`
|
||||
- RIFF specification: Microsoft Multimedia Programming Interface and
|
||||
Data Specifications 1.0 (1991).
|
||||
Reference in New Issue
Block a user