bench: the macOS warm startup leg kills its daemon, and legs ignore display sleep #408

Closed

opened 2026-09-25 02:18:28 +09:00 by natsukium · 0 comments

natsukium commented

2026-09-25 02:18:28 +09:00

Owner

Split from #398 and #402. Two faults in tools/bench found while reproducing the macOS run:

The macOS "felis (warm)" startup row never had a warm daemon. stop_launch (tools/bench/suites.py ~635) kills the launch's whole process tree. On macOS the auto-spawned daemon is a forked child of the client (crates/felis-client-core/src/spawn.rs ~62; the systemd hand-off is Linux-only), so each stop kills it, and clear_sessions (felis sessions list, ~525) does not spawn one. Replicating the procedure, the daemon was alive before 0 of 10 "warm" launches. Real warm (daemon started outside the tree) measured 116 ms against 128/134 ms for the two harness rows. Fix: start the warm daemon outside the launch tree (or exclude it from the kill), and assert it is alive before each warm launch. Check the Linux leg too: there the systemd unit survives, but assert anyway.
Legs do not record whether the display could present. With the display asleep or the screen locked, macOS never presents a frame, and throughput, CPU and memory figures change meaning (see the split client issue). Record screen lock and display sleep in envinfo.leg_env and refuse (or flag in the report) a leg taken in either state.

Split from #398 and #402. Two faults in `tools/bench` found while reproducing the macOS run: 1. **The macOS "felis (warm)" startup row never had a warm daemon.** `stop_launch` (`tools/bench/suites.py` ~635) kills the launch's whole process tree. On macOS the auto-spawned daemon is a forked child of the client (`crates/felis-client-core/src/spawn.rs` ~62; the systemd hand-off is Linux-only), so each stop kills it, and `clear_sessions` (`felis sessions list`, ~525) does not spawn one. Replicating the procedure, the daemon was alive before 0 of 10 "warm" launches. Real warm (daemon started outside the tree) measured 116 ms against 128/134 ms for the two harness rows. Fix: start the warm daemon outside the launch tree (or exclude it from the kill), and assert it is alive before each warm launch. Check the Linux leg too: there the systemd unit survives, but assert anyway. 2. **Legs do not record whether the display could present.** With the display asleep or the screen locked, macOS never presents a frame, and throughput, CPU and memory figures change meaning (see the split client issue). Record screen lock and display sleep in `envinfo.leg_env` and refuse (or flag in the report) a leg taken in either state.

natsukium added the

type/bug

label

2026-09-25 02:18:36 +09:00

natsukium referenced this issue

2026-09-25 02:18:36 +09:00

bench: the felis CPU split in the resource-use tables disagrees with #397's measurement #398

natsukium referenced this issue

2026-09-25 02:18:36 +09:00

bench: follow up the 9d1d842f cross-terminal run #405

natsukium referenced this issue

2026-09-25 06:45:18 +09:00