natsukium/felis

Archived

Watch

render: the window stops repainting under sustained output on Linux #375

Closed

opened 2026-09-24 01:16:53 +09:00 by natsukium · 2 comments

natsukium commented

2026-09-24 01:16:53 +09:00

Owner

Problem

On Linux the felis window stops repainting while a program writes output without pausing, and paints again only
once the output stops. Parsing keeps up; the frames are what go missing.

Observed during the cross-terminal benchmark run for PR #374:

DOOM-fire: no frame at all for the whole leg. Reproduced alone: screenshots of the window every 2 s over a 16 s
run all show one flat color, while DOOM-fire reported 26,885 frames (1,680 fps).
termbench-pro: the window stopped updating partway through the run.

kitty, alacritty, wezterm, ghostty tip and foot all painted continuously under the same workloads.

This matters beyond looks: a terminal that does not paint looks fast on the throughput suites. PR #374 drops felis's
Linux DOOM-fire result and marks its Linux throughput figures as upper bounds until this is fixed.

Environment

felis bc0e9a46 (v0.1.0-rc.6 code), nix build .#felis
NixOS 26.11, Linux 7.2.6-xanmod1, niri 26.04, Wayland
NVIDIA GeForce RTX 3080, driver 595.99.02; wgpu picks the Vulkan backend
3440x1440 @ 60 Hz, scale 1

Not checked on macOS or on other GPUs yet.

Reproduce

nix develop .#bench   # provides the patched DOOM-fire as $FELIS_BENCH_DOOM_FIRE
cat > /tmp/doom.sh <<EOS
#!/bin/sh
sleep 2
DOOM_BENCH_SECS=16 $FELIS_BENCH_DOOM_FIRE
sleep 3
EOS
chmod +x /tmp/doom.sh
SHELL=/tmp/doom.sh felis --socket /tmp/doom-test/d.sock

Watch the window (or screenshot it with niri msg action screenshot-window --id <id>): it stays blank for the 16 s
DOOM-fire runs. DOOM-fire does not use synchronized output (no DECSET 2026 in its stream), so a stuck BSU is not the
cause.

Expected

Under a continuous flood felis may drop frames, but it keeps presenting at some bounded interval, as the other
terminals in the field do.

Where to look

The client's redraw scheduling while daemon updates keep arriving: whether the event loop can stay busy draining
incoming updates so that RedrawRequested is never serviced, and whether any forced-frame deadline exists for a
flood that never goes quiet.

## Problem On Linux the felis window stops repainting while a program writes output without pausing, and paints again only once the output stops. Parsing keeps up; the frames are what go missing. Observed during the cross-terminal benchmark run for PR #374: - **DOOM-fire:** no frame at all for the whole leg. Reproduced alone: screenshots of the window every 2 s over a 16 s run all show one flat color, while DOOM-fire reported 26,885 frames (1,680 fps). - **termbench-pro:** the window stopped updating partway through the run. kitty, alacritty, wezterm, ghostty tip and foot all painted continuously under the same workloads. This matters beyond looks: a terminal that does not paint looks fast on the throughput suites. PR #374 drops felis's Linux DOOM-fire result and marks its Linux throughput figures as upper bounds until this is fixed. ## Environment - felis `bc0e9a46` (v0.1.0-rc.6 code), `nix build .#felis` - NixOS 26.11, Linux 7.2.6-xanmod1, niri 26.04, Wayland - NVIDIA GeForce RTX 3080, driver 595.99.02; wgpu picks the Vulkan backend - 3440x1440 @ 60 Hz, scale 1 Not checked on macOS or on other GPUs yet. ## Reproduce ```sh nix develop .#bench # provides the patched DOOM-fire as $FELIS_BENCH_DOOM_FIRE cat > /tmp/doom.sh <<EOS #!/bin/sh sleep 2 DOOM_BENCH_SECS=16 $FELIS_BENCH_DOOM_FIRE sleep 3 EOS chmod +x /tmp/doom.sh SHELL=/tmp/doom.sh felis --socket /tmp/doom-test/d.sock ``` Watch the window (or screenshot it with `niri msg action screenshot-window --id <id>`): it stays blank for the 16 s DOOM-fire runs. DOOM-fire does not use synchronized output (no DECSET 2026 in its stream), so a stuck BSU is not the cause. ## Expected Under a continuous flood felis may drop frames, but it keeps presenting at some bounded interval, as the other terminals in the field do. ## Where to look The client's redraw scheduling while daemon updates keep arriving: whether the event loop can stay busy draining incoming updates so that `RedrawRequested` is never serviced, and whether any forced-frame deadline exists for a flood that never goes quiet.

natsukium added the

type/bug

label

2026-09-24 01:16:59 +09:00

natsukium referenced this issue

2026-09-24 01:16:59 +09:00

docs: publish the Linux cross-terminal benchmark results #374

natsukium commented

2026-09-24 01:20:30 +09:00

Author

Owner

Code reading: where the redraw loop can stall

Not reproduced under a debugger yet; this is what the source shows, with candidates for the trigger.

How a frame is produced

The local GUI window is pull-paced (Hello.pull_paced, default for the GUI client):

The daemon parses on its own thread and marks the grid dirty. run_session coalesces the effect drain to
DRAIN_COALESCE (4 ms), but ship_sub_at ships nothing to a pull-paced subscriber until it holds a pull
(if sub.pull_pacing && !sub.pull_pending { return; }, crates/felis-daemon/src/serve/session_task.rs).
The client turns every wire message into its own winit user event (forward_frame,
crates/felis-client/src/main.rs); RedrawScheduler collapses them, and about_to_wait calls
window.request_redraw().
RedrawRequested runs on_redraw only when may_paint_grid(has_surface, rehydrating, cycle_open) holds
(crates/felis-client/src/event_handler.rs:326, :1272); otherwise the event is dropped with no fallback.
The next pull (InputMsg::NextGridFrame) is sent only from inside on_redraw, after a successful render
(self.pull.take_due(), event_handler.rs:1255).

So the loop is self-sustaining: no paint, no pull; no pull, nothing shipped; nothing shipped, nothing to paint.
It turns only while RedrawRequested keeps arriving and passes the gate.

No backstop

No client or daemon deadline forces a paint or a pull for grid content. The only daemon-side time slice in
docs/explanation/rendering/pipeline.md ("Push fallback") applies to eager-push (SSH) subscribers, not to the
pull-paced local window. Once one link in the loop above misses, nothing restarts it until the flood ends.

Candidates for the trigger

cycle_open gate. PullScheduler::cycle_open() is true from the first row of a compose cycle until its
GridMsg::CycleEnd. A full-screen cycle is split into several RowDelta messages
(OUTBOUND_BODY_BUDGET_BYTES in streaming.rs), so a RedrawRequested that lands mid-cycle is dropped. If the
next request_redraw is not raised after CycleEnd arrives, the pull is never re-armed.
pre_present_notify is never called (no hit in crates/). On Wayland, winit 0.30 aligns RedrawRequested
with wl_surface.frame callbacks only through Window::pre_present_notify; without it winit's frame-callback
throttle never engages. pipeline.md "Frame pacing" says felis mirrors the Wayland frame-callback model, which
the code does not do at the winit level.
Event-loop occupancy. winit's Wayland backend drains the EventLoopProxy calloop channel in an unbounded
try_recv loop and replays every queued user event through App::user_event before it reaches the
RedrawRequested / AboutToWait step of single_iteration. Under a continuous flood this is a plausible way for
an iteration to be spent on IPC instead of painting.

Surface configuration for reference: PresentMode::AutoVsync, desired_maximum_frame_latency: 2
(crates/felis-render-wgpu/src/lib.rs:396), rendered synchronously on the winit thread.

Next step

Instrument the loop during the DOOM-fire repro: count RedrawRequested received vs. passed by may_paint_grid,
CycleEnd received, and NextGridFrame sent per second. That separates "the gate drops redraws" from "redraws never
arrive".

## Code reading: where the redraw loop can stall Not reproduced under a debugger yet; this is what the source shows, with candidates for the trigger. ### How a frame is produced The local GUI window is pull-paced (`Hello.pull_paced`, default for the GUI client): 1. The daemon parses on its own thread and marks the grid dirty. `run_session` coalesces the effect drain to `DRAIN_COALESCE` (4 ms), but `ship_sub_at` ships nothing to a pull-paced subscriber until it holds a pull (`if sub.pull_pacing && !sub.pull_pending { return; }`, `crates/felis-daemon/src/serve/session_task.rs`). 2. The client turns every wire message into its own winit user event (`forward_frame`, `crates/felis-client/src/main.rs`); `RedrawScheduler` collapses them, and `about_to_wait` calls `window.request_redraw()`. 3. `RedrawRequested` runs `on_redraw` only when `may_paint_grid(has_surface, rehydrating, cycle_open)` holds (`crates/felis-client/src/event_handler.rs:326`, `:1272`); otherwise the event is dropped with no fallback. 4. The next pull (`InputMsg::NextGridFrame`) is sent only from inside `on_redraw`, after a successful render (`self.pull.take_due()`, `event_handler.rs:1255`). So the loop is self-sustaining: no paint, no pull; no pull, nothing shipped; nothing shipped, nothing to paint. It turns only while `RedrawRequested` keeps arriving and passes the gate. ### No backstop No client or daemon deadline forces a paint or a pull for grid content. The only daemon-side time slice in `docs/explanation/rendering/pipeline.md` ("Push fallback") applies to eager-push (SSH) subscribers, not to the pull-paced local window. Once one link in the loop above misses, nothing restarts it until the flood ends. ### Candidates for the trigger - **`cycle_open` gate.** `PullScheduler::cycle_open()` is true from the first row of a compose cycle until its `GridMsg::CycleEnd`. A full-screen cycle is split into several `RowDelta` messages (`OUTBOUND_BODY_BUDGET_BYTES` in `streaming.rs`), so a `RedrawRequested` that lands mid-cycle is dropped. If the next `request_redraw` is not raised after `CycleEnd` arrives, the pull is never re-armed. - **`pre_present_notify` is never called** (no hit in `crates/`). On Wayland, winit 0.30 aligns `RedrawRequested` with `wl_surface.frame` callbacks only through `Window::pre_present_notify`; without it winit's frame-callback throttle never engages. `pipeline.md` "Frame pacing" says felis mirrors the Wayland frame-callback model, which the code does not do at the winit level. - **Event-loop occupancy.** winit's Wayland backend drains the `EventLoopProxy` calloop channel in an unbounded `try_recv` loop and replays every queued user event through `App::user_event` before it reaches the `RedrawRequested` / `AboutToWait` step of `single_iteration`. Under a continuous flood this is a plausible way for an iteration to be spent on IPC instead of painting. Surface configuration for reference: `PresentMode::AutoVsync`, `desired_maximum_frame_latency: 2` (`crates/felis-render-wgpu/src/lib.rs:396`), rendered synchronously on the winit thread. ### Next step Instrument the loop during the DOOM-fire repro: count `RedrawRequested` received vs. passed by `may_paint_grid`, `CycleEnd` received, and `NextGridFrame` sent per second. That separates "the gate drops redraws" from "redraws never arrive".

natsukium commented

2026-09-24 02:24:19 +09:00

Author

Owner

Root cause: the daemon's drain loses the `ParseCore` lock to the parse thread

Reproduced and measured. The trigger is on the daemon side, not in the client's redraw loop.

Client: during a 10 s DOOM-fire flood it sent 3 pulls in total. Its event loop kept running (the 530 ms
blink-timer flushes kept firing), so it was idle while it waited for a cycle that never came.
Daemon (instrumented debug build): felis-pty-parse was runnable in 26 of 30 thread samples. The session
task's drain_effects took 40–70 ms per call and ran only a few times. The parse sink takes the ParseCore
lock once per swap buffer, and std::sync::Mutex on Linux does not hand the lock off, so the thread that just
unlocked takes it again before the woken session task runs. Under pull pacing a pull cannot be answered until
a drain lands, so the client never paints and never pulls again.

Swapping only that lock for parking_lot::Mutex, which hands the lock to a waiter at least every ~0.5 ms, with the same
client binary:

Daemon	Pulls during the 10 s flood	Different 1 s screenshots (of 10)	DOOM-fire fps
`std::sync::Mutex`	3	3	1,120
`parking_lot::Mutex`	161	9	1,095

Bulk cat of 150 MB (ascii, unicode and csi, 2×3 runs) shows no difference beyond run-to-run noise.

The client-side candidates in the previous comment are not the cause. A fix plan is under review.

## Root cause: the daemon's drain loses the `ParseCore` lock to the parse thread Reproduced and measured. The trigger is on the daemon side, not in the client's redraw loop. - **Client:** during a 10 s DOOM-fire flood it sent 3 pulls in total. Its event loop kept running (the 530 ms blink-timer flushes kept firing), so it was idle while it waited for a cycle that never came. - **Daemon (instrumented debug build):** `felis-pty-parse` was runnable in 26 of 30 thread samples. The session task's `drain_effects` took 40–70 ms per call and ran only a few times. The parse sink takes the `ParseCore` lock once per swap buffer, and `std::sync::Mutex` on Linux does not hand the lock off, so the thread that just unlocked takes it again before the woken session task runs. Under pull pacing a pull cannot be answered until a drain lands, so the client never paints and never pulls again. Swapping only that lock for `parking_lot::Mutex`, which hands the lock to a waiter at least every ~0.5 ms, with the same client binary: | Daemon | Pulls during the 10 s flood | Different 1 s screenshots (of 10) | DOOM-fire fps | |---|---|---|---| | `std::sync::Mutex` | 3 | 3 | 1,120 | | `parking_lot::Mutex` | 161 | 9 | 1,095 | Bulk `cat` of 150 MB (ascii, unicode and csi, 2×3 runs) shows no difference beyond run-to-run noise. The client-side candidates in the previous comment are not the cause. A fix plan is under review.

natsukium referenced this issue from a commit

2026-09-24 03:05:07 +09:00

daemon: hand the parse lock off so a flood cannot starve pull answers

natsukium referenced this issue from a pull request that will close it,

2026-09-24 03:05:20 +09:00

daemon: hand the parse lock off so a flood cannot starve pull answers #376

natsukium referenced this issue

2026-09-24 03:09:53 +09:00

ci: discard earlier Criterion results before the regression gate benches #377

natsukium referenced this issue from a commit

2026-09-24 03:29:53 +09:00

daemon: hand the parse lock off so a flood cannot starve pull answers