diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..759bbd5 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,229 @@ +# Working on a raylib app as an agent: headless run, see, and drive it + +This is a portable playbook for an AI agent doing code-health / refactor work on a +**raylib** (or any GLFW/OpenGL) desktop app. The problem it solves: you can edit and +compile the code, but you can't *see* the window or click it — so how do you prove a +refactor didn't change behavior, and how do you exercise a UI path? + +The answer is a tight loop: **run under a virtual display → capture a frame → diff it +against a baseline → (optionally) inject keyboard/mouse → look at the PNG.** Everything +below is the accumulated know-how for doing that *efficiently*, plus the traps. + +> This repo's working reference implementation is [`shot_input.sh`](shot_input.sh). +> Read it alongside this doc; the "Porting" checklist at the end says what to change. + +--- + +## 0. The loop in one breath + +```bash +Xvfb :99 -screen 0 1280x800x24 & # 1. a fake screen +DISPLAY=:99 ./bin/Debug/game & # 2. run the app on it +sleep 2 # 3. let it reach a steady frame +DISPLAY=:99 import -window root /tmp/a.png # 4. screenshot the root +compare -metric AE /tmp/base.png /tmp/a.png /dev/null # 5. 0 == pixel-identical +``` + +If step 5 prints `0`, the render is byte-for-byte what it was before your change. That +single number is the workhorse of refactor verification. + +--- + +## 1. Prerequisites (Debian/Ubuntu) + +```bash +sudo apt-get install xvfb imagemagick xdotool +# If OpenGL fails under Xvfb (black frames / GLX errors), add software rendering: +# sudo apt-get install libgl1-mesa-dri +# and run the app with LIBGL_ALWAYS_SOFTWARE=1 +``` + +- **Xvfb** — an X server that renders to memory, no monitor. raylib's GL runs on it via + Mesa's software rasterizer; you do not need a GPU. +- **ImageMagick** — `import` (grab) + `compare` (diff). On ImageMagick v7 the commands + are `magick import` / `magick compare`; v6 uses `import` / `compare` directly. +- **xdotool** — synthesizes keyboard and mouse events into the X server. + +--- + +## 2. Run headless + +```bash +Xvfb :99 -screen 0 1280x800x24 >/tmp/xvfb.log 2>&1 & +DISPLAY=:99 ./bin/Debug/game [args] >/tmp/app.log 2>&1 & +``` + +- Match the Xvfb screen size to the app's window so a root grab == the app's frame. +- There is **no window manager**, so the window sits at `0,0` and fills the screen, and + X input focus is unmanaged (see §4). +- Capture stdout/stderr to a log — it's your only window into `TraceLog`/crashes. + +--- + +## 3. Capture a frame — two ways + +**A. Grab the X root (zero code change):** +```bash +DISPLAY=:99 import -window root /tmp/shot.png +``` +If your ImageMagick policy blocks `import` ("not authorized"), use option B. + +**B. raylib's own framebuffer dump (one line of code, more robust):** +```c +TakeScreenshot("/tmp/shot.png"); // writes exactly what GL rendered, no chrome/cursor +``` +Trigger it from a temp keybinding or a `--shoot-after N frames` flag. This bypasses +ImageMagick entirely and is the exact framebuffer — preferable when you can edit code +(which, in a refactor task, you already are). + +Then **look at it**: open the PNG with your image-reading tool. Vision catches "the HUD +vanished" or "text is now black-on-black" that a pixel count won't explain. + +--- + +## 4. Inject input (keyboard + mouse) + +```bash +WID=$(DISPLAY=:99 xdotool search --name "MyGameWindowTitle" | head -1) +DISPLAY=:99 xdotool windowactivate "$WID" # give it focus first; keys need this +DISPLAY=:99 xdotool key space # keys: by keysym (Return, Escape, F1, a, ...) +DISPLAY=:99 xdotool mousemove 640 360 +DISPLAY=:99 xdotool mousedown 1 ; sleep 0.2 ; xdotool mouseup 1 +``` + +**Both keys and mouse buttons reach GLFW under Xvfb.** (If a guide tells you synthetic +clicks are "dropped," it's wrong — see the next bullet for what actually bites you.) + +### The one mouse gotcha that wastes an hour +A button's **press and release must fall on different frames** of the app's loop. +raylib detects a click as `current && !previous` (a rising edge). A bare +`xdotool click 1` presses and releases inside ~1ms — if both land in the same +`PollInputEvents()` tick, the app sees the button go down *and* up in one frame and the +rising edge never registers. **Always** do `mousedown 1` → `sleep ≥0.15s` → `mouseup 1`. +Same for drags: `mousemove A` → `mousedown` → `mousemove B` → `sleep` → `mouseup`. + +### Lock/modifier keys +`Scroll_Lock`, `Num_Lock`, `Caps_Lock` get eaten by X as LED/lock toggles and do **not** +arrive as normal key presses. For temp test hooks use ordinary keys (letters, F-keys); +`F1`–`F12` are reliable. + +### Hit the right rectangle +A click only does something if the pointer is over a live hit-target. Buttons and +draggable regions are rectangles computed in code — read the layout code and aim at a +rectangle's center, don't eyeball the screenshot. Many "the click didn't work" dead-ends +are really "I clicked 10px into the sidebar / a dead margin / below a minimum-drag +threshold." (This app, for instance, ignores selection drags under 5px.) + +--- + +## 5. Determinism — the part that decides whether pixel-diff even works + +`compare -metric AE` is only a *proof* if the same inputs produce the same pixels. Two +cases: + +### Mostly-static apps (viewers, editors, tools) +Once loaded/settled the frame is constant, so two runs of the same binary give `AE 0`. +Beware **async/background work**: if the app keeps computing after the window appears +(streaming load, background workers, lazy caches), an early grab is nondeterministic. +Capture *after it settles* — find the quiet point and add margin (this app needs ~12s +for a background high-res pass; a 6s grab varied run-to-run and produced scary false +diffs). Confirm your settle time by grabbing the **same binary twice** and checking the +diff is 0. + +### Games and anything continuously animating +Frames differ every run even with zero code change, so `AE 0` is unattainable and the +naive gate is useless. Make the test deterministic instead, roughly in this order: + +1. **Pin the randomness and the clock.** `SetRandomSeed(N)` (raylib) / `srand(N)`; don't + drive simulation off wall-clock `GetTime()` in test builds. +2. **Fixed timestep.** Don't advance physics by `GetFrameTime()` (which jitters under + software GL). In a test mode, step a constant `dt` a fixed number of frames — or add + a "single-step frame" debug key and advance exactly N steps before the grab. Now + "frame N" is reproducible → its screenshot is a golden image. +3. **Freeze and pose.** Add a pause/step debug key. Drive the game with a scripted input + sequence to a known state, pause, then grab — a paused frame is static and comparable. +4. **Pick naturally-static screens.** Menus, the level-1 spawn, pause overlays, and + game-over screens are often deterministic even when gameplay isn't. Script the input + to land on one of those and diff *that*. +5. **When you can't fully pin it, measure the noise floor.** Run the same binary twice, + `compare` the two shots — that AE is the inherent run-to-run variance. A refactor is + "preserving" if pre-vs-post AE stays within that floor. Use `compare -metric AE -fuzz + 2%` to ignore sub-threshold per-pixel wobble, and/or crop to a stable region. + +Even when pixel-diff isn't reliable, the screenshot is still worth grabbing: **eyeball +it** to confirm the refactor didn't break rendering wholesale. + +--- + +## 6. Reaching states normal input can't + +To test a code path you can't drive from the keyboard/mouse (an internal reset, a state +deep in a menu, a value only mouse-wheel-settable), **add a temporary keybinding** that +forces the state, verify, then remove it. Example pattern: + +```c +// TEMP TEST KEYS (remove before commit) +if (IsKeyPressed(KEY_F2)) { /* force the app into the dirty/edge state */ } +if (IsKeyPressed(KEY_F3)) { TheFunctionUnderTest(); } +``` + +Drive F2 then F3 via xdotool, screenshot between, diff. This is how you exercise a +function end-to-end (e.g. "does load-new-file reset the view?") without the real trigger. +Build with the temp keys, verify, **delete them, rebuild, and re-confirm the baseline is +unchanged** before committing. + +--- + +## 7. A reusable harness + +Wrap §2–§4 in one script so a whole scenario is a single command. This repo's +[`shot_input.sh`](shot_input.sh) takes an output path plus a list of actions and runs +them in order, with `sleep N`, `key NAME`, `click X Y`, `drag X1 Y1 X2 Y2`, and +`rdrag` (modifier+drag) helpers that already bake in the frame-gap from §4: + +```bash +./shot_input.sh /tmp/out.png "sleep 12" "drag 450 120 1000 400" "click 150 200" +``` + +The win for an agent: one Bash call launches, drives, screenshots, and tears down — no +multi-turn babysitting of background processes, and the action list reads like a test +case. + +### Porting it to another raylib project +Change these and nothing else: +- `REPO` and the **run command** (binary path + how it takes/needs a data file). +- The **window title** in `xdotool search --name` (must match your `InitWindow` title). +- The **settle `sleep`** (§5) — or drop it to a small value if the app is static at once. +- The **coordinate cheat-sheet** in the header comment — recompute your own hit-rects + from the layout code at your window size, or just pass raw `mousemove`/`mousedown` + actions. + +--- + +## 8. Refactor verification philosophy + +- **Pure mechanical renames are compiler-proven.** When you rename a field/function and + the *old* name ceases to exist, a clean compile means every reference was updated — the + compiler found any you missed. For these, pixel-diff is a belt-and-suspenders check, + not the primary safety. +- **Behavior-changing edits need the visual gate.** Anything that alters control flow, + values, or layout: establish a baseline screenshot *before* the change, then prove the + intended pixels changed and nothing else did. +- **Keep a stable baseline** (`/tmp/base.png`) for the duration of a refactor arc so + every step diffs against the same known-good frame. +- **Commit at green points.** Build clean + diff clean = a safe checkpoint. + +--- + +## 9. Quick-reference: traps that cost time + +| Symptom | Cause | Fix | +|---|---|---| +| Black/empty frames | No GL under Xvfb | `LIBGL_ALWAYS_SOFTWARE=1`, install mesa-dri | +| `import` "not authorized" | ImageMagick policy | Use raylib `TakeScreenshot()` instead | +| Keys do nothing | Window not focused | `xdotool windowactivate $WID` first | +| Click does nothing | Press+release same frame | `mousedown` → `sleep` → `mouseup` | +| `Scroll_Lock`/`Caps` ignored | X swallows lock keys | Use letters / F-keys for temp hooks | +| Click "misses" | Wrong rectangle / dead margin / min-drag threshold | Aim at a hit-rect center from layout code | +| Diff nonzero with no code change | Async load or animation not settled | Settle longer; or determinize (§5) | +| Diff flaky run-to-run | RNG / wall-clock / variable timestep | Seed RNG, fixed `dt`, measure noise floor |