daemon: paint at the display rate under a sustained flood #380

Merged

natsukium merged 6 commits from daemon/one-lock-per-cycle into main 2026-09-24 18:37:25 +09:00

Conversation 0 Commits 6 Files changed 10 +580 -201

natsukium commented 2026-09-24 16:23:25 +09:00

Owner

Copy link

Closes #378.

A pull-paced window painted about 15 frames per second under a DOOM-fire flood. Three causes, each fixed and measured on this host (Linux, niri, 60 Hz, 188x82 grid, 16 s flood; GUI client pulls answered per second):

Change	pulls/s
baseline (27c5fb13)	12–21
5ae42481 daemon: take the parse lock once per grid cycle	22–32
3593a25a daemon: yield between flood drains so pulls are read off the socket	50–53
fa722c21 client: drop a redraw request the paint it waited for already satisfied	60–61

Every cycle is now about 16.7 ms (p99 20 ms). DOOM-fire's own fps is unchanged within noise; the 150 MB cat payloads are within noise or slightly faster (numbers in the commit bodies).

What each commit does:

• One lock acquisition per cycle. An answered pull made about six separate ParseCore acquisitions, each waiting out a parking_lot fairness slice against the parse thread. Every cycle entry point now takes the guard once, helpers compose into buffers, and pushes / eviction / meta / image materialization run after the guard drops in the same wire order. Also fixes a latent bug: the command arm did not reset the row cache, so a second mirror could receive a stale row after an in-place edit (regression test included).
• The run loop yields between drains. The coalescing window was stamped before the drain, which waits out the parse thread's hold, so the loop never returned Pending under a flood and the socket was not polled; the DECSET 2048 reconcile also relocked the grid every cycle. Regression test: a pull sent mid-flood is answered within a few chunks (fails 5/5 on the old stamping).
• No duplicate paint. The CycleEnd redraw request stayed pending after the paint that consumed it, and the redundant FIFO present cost a vsync every other cycle.

Docs: pipeline.md records the one-acquisition rule (with its two exceptions) and the window stamping; ipc.md the row-cache lifetime; benchmarks.md keeps the 75d08630 run and its non-comparability note unchanged (a new figure belongs to the next published run); CHANGELOG under Rendering.

Not verified: macOS, Windows, outputs above 60 Hz.

Closes #378. A pull-paced window painted about 15 frames per second under a DOOM-fire flood. Three causes, each fixed and measured on this host (Linux, niri, 60 Hz, 188x82 grid, 16 s flood; GUI client pulls answered per second): | Change | pulls/s | | --- | --- | | baseline (`27c5fb13`) | 12–21 | | `5ae42481` daemon: take the parse lock once per grid cycle | 22–32 | | `3593a25a` daemon: yield between flood drains so pulls are read off the socket | 50–53 | | `fa722c21` client: drop a redraw request the paint it waited for already satisfied | **60–61** | Every cycle is now about 16.7 ms (p99 20 ms). DOOM-fire's own fps is unchanged within noise; the 150 MB `cat` payloads are within noise or slightly faster (numbers in the commit bodies). What each commit does: - **One lock acquisition per cycle.** An answered pull made about six separate `ParseCore` acquisitions, each waiting out a parking_lot fairness slice against the parse thread. Every cycle entry point now takes the guard once, helpers compose into buffers, and pushes / eviction / meta / image materialization run after the guard drops in the same wire order. Also fixes a latent bug: the command arm did not reset the row cache, so a second mirror could receive a stale row after an in-place edit (regression test included). - **The run loop yields between drains.** The coalescing window was stamped before the drain, which waits out the parse thread's hold, so the loop never returned Pending under a flood and the socket was not polled; the DECSET 2048 reconcile also relocked the grid every cycle. Regression test: a pull sent mid-flood is answered within a few chunks (fails 5/5 on the old stamping). - **No duplicate paint.** The `CycleEnd` redraw request stayed pending after the paint that consumed it, and the redundant FIFO present cost a vsync every other cycle. Docs: `pipeline.md` records the one-acquisition rule (with its two exceptions) and the window stamping; `ipc.md` the row-cache lifetime; `benchmarks.md` keeps the `75d08630` run and its non-comparability note unchanged (a new figure belongs to the next published run); CHANGELOG under Rendering. Not verified: macOS, Windows, outputs above 60 Hz.

natsukium added 5 commits 2026-09-24 16:23:25 +09:00

daemon: take the parse lock once per grid cycle ... 5ae4248167

A pull-paced window painted only about 15 frames per second while a
program flooded the terminal (DOOM-fire), even with the ParseCore lock
handing off fairly. parking_lot's eventual fairness gives a parked
waiter the lock only about every 0.5 ms while the parse thread relocks
per swap buffer, and one answered pull made about six separate
acquisitions: the effect drain (plus one per APC and per scroll into
scrollback), two in the damage fan-out, notifications, facets, one
compose per subscriber, and the meta refresh. The command arm that
answers NextGridFrame made four. Each waited out its own fairness slice.

Every cycle entry point now takes the guard once: the PTY drain, the
command arm after each input, the EOF drain, subscribe, and resize. The
helpers take a borrowed ParseCore and compose into buffers. Outbox
pushes, eviction, the meta lock, image materialization and notification
publishing run after the guard drops, in the same wire order, and a dead
outbox is still removed as soon as a push fails. The lock is not
reentrant. remove_sub reaches refresh_meta and recompute_presentation,
which both relock, so no push may happen under the guard, and a compose
failure is recorded and evicted in the tail. A cycle with an eviction
therefore takes the lock a second time. The EOF arm still ships the
final frame when the effect drain fails, and the PTY arm reports the
drain error after the cycle has shipped.

The command arm now resets the row cache before composing. Without the
reset, a row edited in place (same geometry, same scroll sequence) after
a drain cycle cached it was shipped stale to a second mirror whose pull
the command arm answered, and that mirror's damage for the row was
cleared. The new test reproduces this against the old code.

The remaining lock_core calls are off the cycle path. Spawn setup runs
before the task does. The PTY reconciles (focus, color scheme, 2048
resize report) and recompute_presentation run from the run loop or from
remove_sub, which happens only after a guard drops. refresh_meta runs on
subscribe, eviction and push_to_scope. Per-input-kind reads (key and
mouse encoding, paste bracketing, a non-owner resize, viewport and
prompt jumps) are not taken for NextGridFrame. Search, row streams,
region replies and the size-ownership check in promote_active are
commands of their own.

Measured on Linux (i5-12400F, RTX 3080, niri, 60 Hz), before -> after:

  - measure_pull_cycles_under_flood (release, 10 s): 363, 285, 537 ->
    635, 1020, 854 cycles; frame-file passes (parse throughput) are
    unchanged at about 600-650
  - GUI client pulls per second over a 16 s DOOM-fire flood, bench-field
    font, 188x82 grid: 13.4, 11.9, 14.2 -> 26.4, 21.7, 25.7 (a 212x78
    grid with the user config: 16.1, 21.4, 16.9, 13.6 -> 27.1, 31.9,
    32.2)
  - DOOM-fire fps, same runs: 1,085-1,104 -> 1,062-1,095 (noise)
  - 150 MB cat, two runs of 3 each: ascii 1562/1572 -> 1563/1641 ms,
    unicode 880/985 -> 939/983 ms, csi 876/889 -> 908/903 ms (csi is
    2-3% slower in both pairs; the others overlap)

This is still short of the display rate. With one acquisition the daemon
answers a pull in 1.8 ms median (p90 5.6 ms). In the slow cycles (over
34 ms, 62% of the flood time), 93% of the time falls between the
daemon's ship and the client's next pull, which is client-side work
that this change does not touch.

Rejected: an unlock_fair in the sink only while a pull is pending. It
shortens each wait but not the number of waits per cycle, so it is worth
layering on only if a single acquisition turns out not to be enough.

Part of #378

Assisted-by: Claude Code

daemon: pin the deferred eviction of an outbox that overflows on its rows ... e3a6dbf11c

The eviction test for one-acquisition cycles dropped its receiver before
the cycle, so the title-facet broadcast evicted the subscriber, as it
would have with a lock per step. The composed rows never reached the
push that runs after the guard drops, and nothing checked what happens
to a buffer composed for a subscriber that then fails. The test now
fills the outbox gauge so that the facet still fits and the RowDelta
exceeds the cap. The failure therefore lands between compose and push,
and the test checks that the healthy peer still gets rows, the facet
and a closing CycleEnd.

The doc comments on MetaFacts, Cycle, TakenNotifications,
publish_notifications, grid_cycle_for and compose_at restated what the
code does, so they are dropped, and compose_cycle keeps only the
reentrancy invariant.

In benchmarks.md, the note under the Linux DOOM-fire table could be read
as a rerun of the published 20 s 141x61 leg. It now gives the conditions
of the check it cites (16 s, 188x82 grid, 22 to 26 pulls per second
against 12 to 14) and says the table is not updated.

Assisted-by: Claude Code

daemon: yield between flood drains so pulls are read off the socket ... 3593a25a95

Under a DOOM-fire flood a pull-paced window answered only 20 to 27
pulls per second on a 60 Hz output, and the tail was not in the client:
cycles longer than two frames spent most of their time between the
daemon's ship and its outbound pump writing the frames, and between the
client flushing NextGridFrame and the daemon reading it. The session
task never returned Pending while the parse thread held the grid lock
for a swap buffer at a time. Its runtime worker was the only one awake,
so the socket was polled only on the scheduler's periodic maintenance
and the connection task it had woken waited behind it; backtraces of
the idle workers showed all of them parked on condvars with nobody
parked on the I/O driver, and strace saw no epoll_wait for seconds.

Two things kept the task running. The coalescing window was stamped
before the drain, and a drain waits out the sink's hold on the lock
(about 5 ms here, longer than the 4 ms window), so the next parsed
notification always found the window elapsed and drained at once.
Stamping it after the drain leaves a Pending gap after every drain.
And every drain marked the DECSET 2048 reconcile dirty, which relocked
the grid at the bottom of the run loop: a second lock wait per cycle
for a mode DOOM-fire never sets. The drain now compares the report key
under the guard it already holds and marks the reconcile only when it
would write or forget a report; the key is one helper both sites share,
so the check cannot drift from the reconcile it gates.

Measured on Linux (niri, 60 Hz, 188x82 grid, 16 s DOOM-fire flood),
before -> after, GUI client pulls per second: 26.8, 20.0, 23.4 -> 50.2,
53.4, 50.5. DOOM-fire fps 1,135-1,140 -> 1,091-1,143 (noise). 150 MB
cat, two runs each: ascii 1661/1681 -> 1623/1532 ms, unicode 967/953 ->
861/879 ms, csi 913/904 -> 870/864 ms.

Assisted-by: Claude Code

client: drop a redraw request the paint it waited for already satisfied ... fa722c2131

With the daemon answering pulls within a few milliseconds, a flooding
window still alternated between one-vsync and two-vsync cycles, about
40 painted frames per second on a 60 Hz output: a third of the paints
redrew an unchanged grid. The frames of one cycle can reach the event
loop over two iterations. The first iteration's about_to_wait flushes a
request_redraw; the second delivers CycleEnd, whose request is left
pending, and then the RedrawRequested that paints it. The next
about_to_wait flushes that stale request, and the redundant paint
blocks in the FIFO present for a full vsync while the next cycle's
frames sit behind it in the queue.

A successful paint now clears the pending request, since everything
requested before it is on screen. Requests made during the paint (the
atlas-full repaint) are raised after the clear and survive, and a
failed paint leaves the request for the retry.

Measured on Linux (niri, 60 Hz, 188x82 grid, 16 s DOOM-fire flood), GUI
client pulls per second with the daemon's drain yielding: 50.2, 53.4,
50.5 -> 61.0, 59.9, 60.0. With the unfixed daemon the change alone
stays within that daemon's noise (28.4, 25.2 against 20.0 to 26.8),
because the daemon was then the bottleneck.

Assisted-by: Claude Code

daemon: pin that the run loop yields between flood drains ... 7f9f412653

The coalescing window is counted from the end of a drain because a
drain that waits out the parse thread's hold on the grid lock would
otherwise find the window already elapsed and drain again at once. The
loop then never returns Pending, and a pull that has to reach it
through the runtime (the connection socket, a timer) waits until the
flood ends. Nothing tested that. The new run-loop test floods from a
thread that holds the lock longer than the window per chunk and retakes
it as soon as the task drops it, then sends three pulls from a timer on
a single-threaded runtime, where the test body only runs when the loop
yields. Each pull must be answered within 20 flood chunks. It uses real
time, so the lag is counted in chunks rather than milliseconds, and a
pull that the flood outlives is bounded by a timeout instead of
hanging. With the window stamped before the drain, it failed 5 of 5
runs (the first pull waited about 800 chunks and the next was never
answered). With the current stamping it passed 5 of 5 with a lag of
6 chunks, the timer's own length.

pipeline.md said an eviction is the only reason a cycle takes the lock
twice. An owed DECSET 2048 report does too, because writing it reads
the grid again for the geometry it states.

Closes #378

Assisted-by: Claude Code

natsukium added 1 commit 2026-09-24 16:28:35 +09:00

docs: describe the published DOOM-fire run without narrating what changed after it ... 6e580309fc

The note under the Linux DOOM-fire table said which builds paint faster
than the published one and what a later check measured. A reference
page states the run it publishes; what changed after that run belongs
to git log and the CHANGELOG, and a new figure belongs to the next
published run. The paragraph keeps only why the published felis figure
is not comparable with the others.

Assisted-by: Claude Code

natsukium merged commit 220478356c into main 2026-09-24 18:37:25 +09:00

natsukium deleted branch daemon/one-lock-per-cycle 2026-09-24 18:37:25 +09:00

natsukium referenced this pull request from a commit 2026-09-24 18:37:26 +09:00

daemon: paint at the display rate under a sustained flood (#380)

natsukium referenced this pull request 2026-09-24 21:20:53 +09:00

docs: republish the macOS benchmarks with ghostty's release and resource use #382

natsukium referenced this pull request 2026-09-24 21:41:56 +09:00

docs: republish the Linux benchmarks with ghostty's release and felis's display-rate painting #383

natsukium referenced this pull request from a commit 2026-09-24 23:04:05 +09:00

docs: republish the macOS benchmarks with ghostty's release and resource use (#382)

natsukium referenced this pull request from a commit 2026-09-24 23:04:12 +09:00

docs: republish the Linux benchmarks with ghostty's release and felis's display-rate painting (#383)

natsukium referenced this pull request 2026-09-25 01:34:54 +09:00

daemon: CPU time under sustained output is 1.3-1.9x the leading peer's #399

natsukium referenced this pull request 2026-09-25 01:34:55 +09:00

daemon: Linux vtebench scrolling rows trail the peers and vary 25-50% between rounds #400

natsukium referenced this pull request 2026-09-25 02:32:44 +09:00

daemon: CPU time under sustained output is 1.3-1.9x the leading peer's #399

Commenting is not possible because the repository is archived.

Reviewers

No reviewers

Labels

Clear labels   

priority/P0

Must be resolved before the v0.1.0 tag

  

priority/P1

Must be settled before the first public contract freezes

  

priority/P2

Explicitly deferred until after v0.1.0

  

release/v0.1.0

Work tracked for the first tagged release

  

status/blocked

Cannot start now: waits on an upstream release, another issue, or a recorded revisit trigger

  

status/planned

Triaged with a source-grounded plan and a passed pi review; ready to implement

  

type/bug

Shipped behavior that contradicts the docs or the code's own intent; needs a product code change

  

type/design

An open scope or trade-off question; needs a recorded decision in the owning explanation doc before code

  

type/test-gap

Documented or reviewed behavior that no test pins; usually split from a review finding, needs no product code change

  

type/tracker

Parent issue that tracks child work

No labels

priority/P0

priority/P1

priority/P2

release/v0.1.0

status/blocked

status/planned

type/bug

type/design

type/test-gap

type/tracker

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

natsukium/felis!380

No description provided.