fix: check STFT allocations; reject files too large for the web build
Every malloc/calloc in stft.c was written through unchecked. On desktop that never bites — Linux overcommits and swaps — but WebAssembly has a hard 32-bit address-space ceiling, so on a long capture the allocations genuinely fail and the code wrote through NULL into low memory. That is why it surfaced as corrupted font glyphs rather than a crash, and only after loading a large file. The apparent hang had the same root: ComputeSegment returned void, so a failed allocation was invisible and the loop ground through all remaining segments failing identically. The cursor readout showing "-" for the level was the tell — the segments were there but their spectra were NULL. ComputeSegment now returns a bool, ComputeSTFTIncremental stops at the first failure instead of churning, and both halves of a segment's spectra are freed together (reassignment reads them in lockstep, so half a segment is worse than none). ComputeSTFTInit, CopySTFT and the scratch buffers are checked too. Segments left NULL are already skipped by every consumer — that is how the progressive fill renders partial results — so a file that nearly fits degrades to a truncated spectrogram rather than corrupting. The web load path treats exhaustion as fatal and says so: a multi-hour capture needs ~11 GB of spectra (478k segments x 1025 bins x two spectra x 12 bytes for the 5.7-hour case) against a 4 GB ceiling browsers cap below in practice, so there is nothing useful to fall back to. Better to explain that than present a spectrogram full of holes. Corrects the known_bugs.md entry, which blamed ALLOW_MEMORY_GROWTH invalidating cached pointers. That theory fit the symptom but was wrong; the allocations were simply failing. Also notes what making large files actually work would take — 16-bit magnitudes, dropping the derivative spectrum when synchrosqueezing is off, or streaming segments. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8ZWfr5XZyyDttvkhJUgHN
This commit is contained in:
+23
-27
@@ -6,38 +6,34 @@ need to decide.
|
||||
|
||||
---
|
||||
|
||||
## Large WAVs in the browser: text corruption and a long hang
|
||||
## Large WAVs in the browser: too large to analyse
|
||||
|
||||
**Status:** open. Reproduces on the web build only; desktop is unaffected.
|
||||
**Status:** diagnosed and handled. The file is rejected with a message instead
|
||||
of corrupting or hanging; it still cannot be opened in the browser.
|
||||
|
||||
Loading a large capture through the web UI (drag-drop or the Open file button)
|
||||
produces garbled glyphs in menu titles, tooltips and axis labels, and the page
|
||||
stops responding for a long stretch. Small files load and render correctly, and
|
||||
the corruption appears *after* the load rather than at startup — so it is
|
||||
triggered by the size of the work, not by the build being broken.
|
||||
A multi-hour capture needs far more memory than a 32-bit WebAssembly page can
|
||||
address. v23 (5.7 h at 12 kHz) works out to ~478k STFT segments x 1025 bins x
|
||||
two spectra x 12 bytes — roughly **11 GB of spectra alone**, against a hard
|
||||
wasm32 ceiling of 4 GB that browsers cap below in practice.
|
||||
|
||||
The hang is understood and partly by design: the web build computes its entire
|
||||
STFT in one synchronous pass (see the `__EMSCRIPTEN__` branch in the main loop),
|
||||
because the desktop's incremental fill depends on idle main-loop frames the
|
||||
browser doesn't hand back the same way. A DOM overlay now warns before it
|
||||
starts, but the page genuinely is frozen until it finishes. A real fix means
|
||||
chunking that work across frames or moving it to a Web Worker.
|
||||
The old symptom was not a memory-growth bug, as first assumed. The per-segment
|
||||
`malloc`s in `ComputeSegment` were simply failing and their results written
|
||||
through unchecked. On desktop that never bites (Linux overcommits and swaps),
|
||||
but in wasm the failure is real, so the code wrote through NULL into low memory
|
||||
— which is why it surfaced as *corrupted font glyphs* rather than a crash. The
|
||||
apparent "hang" was the loop grinding through every remaining segment, each
|
||||
failing the same way, since `ComputeSegment` returned `void` and nothing noticed.
|
||||
|
||||
The **corruption** is the unexplained part. The leading theory is heap growth:
|
||||
the build links with `ALLOW_MEMORY_GROWTH=1`, and a large file forces the wasm
|
||||
heap to grow mid-load. Growth reallocates the backing `ArrayBuffer`, which
|
||||
invalidates every cached view and raw pointer held across that moment — so
|
||||
anything retaining a `char*` or a texture-side pointer from before the growth
|
||||
would read garbage afterwards. Font glyph data reached through raylib's atlas is
|
||||
a plausible casualty, which fits the symptom being *text* specifically.
|
||||
Every allocation in `stft.c` is now checked. `ComputeSegment` reports failure,
|
||||
`ComputeSTFTIncremental` stops at the first one rather than churning, and the
|
||||
web load path frees the partial result and explains that the file is too large.
|
||||
A file that *nearly* fits should degrade to a truncated spectrogram — NULL
|
||||
segments are already skipped everywhere, which is how the progressive fill draws
|
||||
partial results — though that path is reasoned rather than tested.
|
||||
|
||||
Worth checking first:
|
||||
- Whether `INITIAL_MEMORY` large enough to avoid growth entirely makes it go
|
||||
away. That would confirm the theory cheaply, at the cost of a bigger initial
|
||||
allocation.
|
||||
- Whether the font atlas survives a deliberate `sbrk`-forced growth.
|
||||
- SAFE_HEAP + ASSERTIONS=2 (`build_web.sh` debug path) to catch the first bad
|
||||
access rather than the downstream symptom.
|
||||
Making large files actually work in the browser needs a different data layout:
|
||||
storing magnitudes as 16-bit, dropping the derivative spectrum unless
|
||||
synchrosqueezing is on, or streaming segments rather than holding them all.
|
||||
|
||||
**Testing note:** always serve the web build with `serve_web.py`, never
|
||||
`python3 -m http.server`. Browsers cache `.wasm` hard enough that a plain reload
|
||||
|
||||
Reference in New Issue
Block a user