natsukium/felis

Archived

Watch

client: Linux startup trails alacritty by 70 ms, and a cold macOS launch beats a warm one #402

Closed

opened 2026-09-25 01:34:55 +09:00 by natsukium · 2 comments

natsukium commented

2026-09-25 01:34:55 +09:00

Owner

Time to the first window (docs/reference/benchmarks.md, felis 9d1d842f, ms):

	felis cold	felis warm	alacritty	foot	kitty
Linux	236	174	104	28.8	163
macOS	126	144	210	-	312

Two things to explain:

Linux warm is 70 ms behind alacritty. Break the client's path into its steps (GPU instance and adapter, device, font discovery, daemon connect and attach, first configure and present) and time each. Steps that do not depend on each other can overlap.
Cold adds 62 ms on Linux. The daemon hand-off itself (systemd-run with fork fallback) is a settled decision and stays; the question is whether the client can do its GPU and font work while the daemon starts instead of after.
On macOS cold is faster than warm in every round (122-135 against 144-149 ms). A warm launch has something on its path that a cold one skips, most likely a wait on the running daemon before the window is shown.

Done when each step's cost is known, the overlap that is available is taken, and the macOS inversion is explained.

Time to the first window (`docs/reference/benchmarks.md`, felis `9d1d842f`, ms): | | felis cold | felis warm | alacritty | foot | kitty | |---|---|---|---|---|---| | Linux | 236 | 174 | 104 | 28.8 | 163 | | macOS | 126 | 144 | 210 | - | 312 | Two things to explain: - **Linux warm is 70 ms behind alacritty.** Break the client's path into its steps (GPU instance and adapter, device, font discovery, daemon connect and attach, first configure and present) and time each. Steps that do not depend on each other can overlap. - **Cold adds 62 ms on Linux.** The daemon hand-off itself (systemd-run with fork fallback) is a settled decision and stays; the question is whether the client can do its GPU and font work while the daemon starts instead of after. - **On macOS cold is faster than warm in every round** (122-135 against 144-149 ms). A warm launch has something on its path that a cold one skips, most likely a wait on the running daemon before the window is shown. Done when each step's cost is known, the overlap that is available is taken, and the macOS inversion is explained.

natsukium referenced this issue

2026-09-25 01:35:05 +09:00

bench: follow up the 9d1d842f cross-terminal run #405

natsukium commented

2026-09-25 02:17:30 +09:00

Author

Owner

macOS: the "warm" row never had a warm daemon, and the window reaches the screen only after GPU and font init

The cold/warm inversion

On macOS the harness's warm leg times cold starts. Both published rows measure the same client path.

tools/bench/suites.py:635 (stop_launch) kills fieldmod.process_tree(proc.pid).
On macOS the auto-spawned daemon is a forked child of felis-client (crates/felis-client-core/src/spawn.rs:62-63). The systemd hand-off above it is Linux-only, so the daemon is in that tree, and stopping each launch also kills the daemon it spawned.
The warm leg's clear_sessions (suites.py:525) runs felis sessions list, which does not autospawn (Dial::Ops, RemoteSpawn::Refuse). The next "warm" launch therefore finds no daemon and spawns one.

I replicated the harness procedure: same felis_home, exec sleep 60 wrapper, MacWatcher, stop_launch, kill_daemon and clear_sessions. I added a third mode whose daemon starts outside any launch tree (felis-daemon serve with start_new_session). Setup: main a577fee0 with a local probe patch, 10 timed launches after 1 discarded, screen locked, display asleep, load 2.15 → 1.60.

Mode	first window, median (range) ms	daemon alive before launch	launch tree size
harness "cold"	128.4 (107.9-147.7; plus one 3,022 outlier)	0/10	3 (client, daemon, shell)
harness "warm"	134.2 (106.8-167.7)	0/10	3
real warm (persistent daemon)	116.1 (105.9-136.4)	10/10	1

An earlier batch of 5 timed runs (load about 1.8) gave "warm" 123.6 ms and real warm 117.6 ms. Four of its six cold runs took 8.5-9.3 s, with 6 s in font discovery and about 1 s each in event-loop build and resumed. That happened with the display asleep and I did not pursue it; the harness's 3 discarded launches would absorb it.

Tonight the two harness rows differ by 6 ms with overlapping ranges, so the published 18 ms gap (122-135 against 144-149 in all three rounds) did not reproduce. Whatever caused it, it sits outside the client: the client timeline of the two rows matches to 0.5 ms (below). One candidate is the extra felis sessions list process just before each "warm" launch; I have not verified it.

Where the warm path spends its time

Probe timestamps are wall time from spawning felis, medians of 10 runs:

Step	real warm	harness cold	step cost (warm)
exec `felis` → `felis-client` main	9.0	9.1	9.0
config loaded, tokio runtime	9.4	9.6	0.4
`dial_launch` (connect + handshake; cold: spawn + boot retry)	9.6	13.9	0.2 / 4.3
attach (`create_with`)	11.4	15.6	1.8
`EventLoop` built (NSApplication init)	28.8	33.0	17.4
`resumed` delivered	44.2	48.2	15.3
`create_window` returned	68.7	72.9	24.5
wgpu adapter / device / configure / pipelines	72.8	77.0	0.7 / 2.1 / 0.1 / 1.3
font discovery → renderer ready	85.1	89.3	12.3
first `render` returned	91.5	95.8	6.4
first GridFrame from the daemon	94.5	98.8	-
watcher sees the window on screen	116	128	-

The warm path does not wait on the daemon. Connect, handshake and attach cost 2 ms together.
The window is not listed on screen when create_window returns (event_handler.rs:44). In the medians it appears about 20-35 ms after the first render, once resumed has returned to AppKit. The GPU setup, font discovery and first render (about 23 ms) are therefore inside the macOS metric, although the docs describe the metric as "ordered on screen, not drawn into".
AppKit's own work is the bulk of the path: event-loop build, the wait for resumed, and create_window take about 57 ms.

Proposed fix

Bench: start the warm leg's daemon outside the launch tree (as the "real warm" mode above does), or leave the daemon out of stop_launch's kill set for felis. Assert that it is alive before each warm launch and record that in the leg JSON. The cold leg needs no change.
Client: move work that does not need the window off the path after it. Start FontStack::auto_discover (felis-render-wgpu/src/lib.rs:499) on a thread in main, so it overlaps dial/attach and NSApplication init. The wgpu instance, adapter and device can also be requested before the window exists. On macOS this saves at most about 15 ms of about 116 ms (fonts 12.3, GPU 3), because resumed gates the window's appearance.
Linux-relevant overlap (for the Linux lane): main.rs:793-795 (dial, including the daemon spawn/hand-off, and attach) runs strictly before main.rs:887 (event loop) and everything after it. Event loop, window, GPU and fonts do not depend on the session, so they could run while the daemon boots. Only the attach's dims and the first GridFrame are needed before the first paint.

## macOS: the "warm" row never had a warm daemon, and the window reaches the screen only after GPU and font init ### The cold/warm inversion On macOS the harness's warm leg times cold starts. Both published rows measure the same client path. - `tools/bench/suites.py:635` (`stop_launch`) kills `fieldmod.process_tree(proc.pid)`. - On macOS the auto-spawned daemon is a forked child of `felis-client` (`crates/felis-client-core/src/spawn.rs:62-63`). The systemd hand-off above it is Linux-only, so the daemon is in that tree, and stopping each launch also kills the daemon it spawned. - The warm leg's `clear_sessions` (`suites.py:525`) runs `felis sessions list`, which does not autospawn (`Dial::Ops`, `RemoteSpawn::Refuse`). The next "warm" launch therefore finds no daemon and spawns one. I replicated the harness procedure: same `felis_home`, `exec sleep 60` wrapper, `MacWatcher`, `stop_launch`, `kill_daemon` and `clear_sessions`. I added a third mode whose daemon starts outside any launch tree (`felis-daemon serve` with `start_new_session`). Setup: main a577fee0 with a local probe patch, 10 timed launches after 1 discarded, screen locked, display asleep, load 2.15 → 1.60. | Mode | first window, median (range) ms | daemon alive before launch | launch tree size | |---|---|---|---| | harness "cold" | 128.4 (107.9-147.7; plus one 3,022 outlier) | 0/10 | 3 (client, daemon, shell) | | harness "warm" | 134.2 (106.8-167.7) | **0/10** | 3 | | real warm (persistent daemon) | 116.1 (105.9-136.4) | 10/10 | 1 | An earlier batch of 5 timed runs (load about 1.8) gave "warm" 123.6 ms and real warm 117.6 ms. Four of its six cold runs took 8.5-9.3 s, with 6 s in font discovery and about 1 s each in event-loop build and `resumed`. That happened with the display asleep and I did not pursue it; the harness's 3 discarded launches would absorb it. Tonight the two harness rows differ by 6 ms with overlapping ranges, so the published 18 ms gap (122-135 against 144-149 in all three rounds) did not reproduce. Whatever caused it, it sits outside the client: the client timeline of the two rows matches to 0.5 ms (below). One candidate is the extra `felis sessions list` process just before each "warm" launch; I have not verified it. ### Where the warm path spends its time Probe timestamps are wall time from spawning `felis`, medians of 10 runs: | Step | real warm | harness cold | step cost (warm) | |---|---|---|---| | exec `felis` → `felis-client` main | 9.0 | 9.1 | 9.0 | | config loaded, tokio runtime | 9.4 | 9.6 | 0.4 | | `dial_launch` (connect + handshake; cold: spawn + boot retry) | 9.6 | 13.9 | 0.2 / 4.3 | | attach (`create_with`) | 11.4 | 15.6 | 1.8 | | `EventLoop` built (NSApplication init) | 28.8 | 33.0 | 17.4 | | `resumed` delivered | 44.2 | 48.2 | 15.3 | | `create_window` returned | 68.7 | 72.9 | 24.5 | | wgpu adapter / device / configure / pipelines | 72.8 | 77.0 | 0.7 / 2.1 / 0.1 / 1.3 | | font discovery → renderer ready | 85.1 | 89.3 | 12.3 | | first `render` returned | 91.5 | 95.8 | 6.4 | | first GridFrame from the daemon | 94.5 | 98.8 | - | | watcher sees the window on screen | 116 | 128 | - | - The warm path does not wait on the daemon. Connect, handshake and attach cost 2 ms together. - The window is not listed on screen when `create_window` returns (`event_handler.rs:44`). In the medians it appears about 20-35 ms after the first render, once `resumed` has returned to AppKit. The GPU setup, font discovery and first render (about 23 ms) are therefore inside the macOS metric, although the docs describe the metric as "ordered on screen, not drawn into". - AppKit's own work is the bulk of the path: event-loop build, the wait for `resumed`, and `create_window` take about 57 ms. ### Proposed fix 1. Bench: start the warm leg's daemon outside the launch tree (as the "real warm" mode above does), or leave the daemon out of `stop_launch`'s kill set for felis. Assert that it is alive before each warm launch and record that in the leg JSON. The cold leg needs no change. 2. Client: move work that does not need the window off the path after it. Start `FontStack::auto_discover` (`felis-render-wgpu/src/lib.rs:499`) on a thread in `main`, so it overlaps dial/attach and NSApplication init. The wgpu instance, adapter and device can also be requested before the window exists. On macOS this saves at most about 15 ms of about 116 ms (fonts 12.3, GPU 3), because `resumed` gates the window's appearance. 3. Linux-relevant overlap (for the Linux lane): `main.rs:793-795` (dial, including the daemon spawn/hand-off, and attach) runs strictly before `main.rs:887` (event loop) and everything after it. Event loop, window, GPU and fonts do not depend on the session, so they could run while the daemon boots. Only the attach's dims and the first GridFrame are needed before the first paint.

natsukium referenced this issue

2026-09-25 02:18:28 +09:00

bench: the macOS warm startup leg kills its daemon, and legs ignore display sleep #408

natsukium commented

2026-09-25 03:00:41 +09:00

Author

Owner

Linux part (kilimanjaro, i5-12400F, NVIDIA RTX 595.99 driver, niri, scale 1). I timed the client's path with a local build of main ef96bb46 that adds one info!("STARTUP-STEP …") log at each step boundary (not committed). Time zero is just before exec of felis, and "window mapped" is the moment niri's event stream reported the client's toplevel. Default config with an empty $HOME (a config naming DejaVu Sans Mono changes nothing: font discovery takes 2.8 ms either way). Load 3.5-4.7.

Where the time goes (ms from exec; three runs each)

step	warm	cold (systemd-run hand-off)	what runs
process start → logging initialized	2.5	2.4-2.5	exec `felis` → exec `felis-client`, dynamic loading
daemon dial + create/attach	2.2-2.5	29-38	`connect_or_spawn_daemon`, `create_with`
winit event loop (Wayland connect)	2.1-2.2	2.1-2.3	`EventLoop::build`
window creation	1.0-1.1	0.9-1.3	`create_window`
`Instance::new` + `create_surface`	34-65	34-46	Vulkan loader loads ICDs
`request_adapter`	2.3-3.7	2.3-2.6
`request_device`	72-94	62-94	NVIDIA Vulkan device
`surface.configure` + pipelines	5.5-6.0	5.5
font discovery + glyph/image atlases	3.2	3.3-3.4	`FontStack::auto_discover`, `GlyphCache::new`, `ImageAtlas::new`
first render → `present`	3.4-3.5	3.4-3.5
present → niri maps the window	1.6-2.2	1.6-1.7
total to window mapped	143-196	148-202

Those runs pinned the NVIDIA ICD alone (VK_DRIVER_FILES, needed because of the driver mismatch described in #398). The published run had every ICD the system ships (14 manifests, lavapipe among them). Pinning all 14 reproduces that condition:

warm, all 14 ICDs	instance + surface	device	total to window mapped
run 1 / 2 / 3	113 / 71 / 69 ms	91 / 78 / 78 ms	233 / 179 / 176 ms

That matches the published warm 174 ms. Loading every installed Vulkan driver at vkCreateInstance costs 35-50 ms on top of loading NVIDIA's alone.

Answers

Linux warm, 70 ms behind alacritty. GPU initialization is 110-200 ms of a 145-230 ms start: Vulkan instance creation 34-113 ms (depending on how many ICDs are installed), plus NVIDIA device creation 62-94 ms. Everything felis does itself (daemon attach, Wayland, window, fonts, atlases, pipelines, first frame) adds up to about 20 ms. The gap to alacritty is the Vulkan instance and device cost on this driver, not felis's own work.
Cold adds 27-36 ms, all in the dial. The systemd-run hand-off plus daemon boot takes 29-38 ms against 2.2-2.5 ms warm; every later step is the same.
Overlap available. Every step is serial today: run_client (crates/felis-client/src/main.rs:785-797) dials and attaches before drive builds the event loop, and Renderer::new_with_config (crates/felis-render-wgpu/src/lib.rs:326) starts Instance::new only inside resumed, after the window exists. None of the instance, adapter or device work needs the window or the daemon. Only create_surface and surface.configure need the window, and the adapter can be requested without compatible_surface and checked with adapter.is_surface_supported(&surface) afterwards. Starting instance + adapter + device on a thread at the top of main, in parallel with config load, the daemon dial, the Wayland connect and the window, would hide:
- cold: the 29-38 ms dial behind the GPU work, so cold ≈ warm (about −30 ms);
- warm: the about 10 ms of dial, event loop and window (about −8 to −10 ms).
  Font discovery (3 ms) and atlas creation can run beside the device request too (about −3 ms).
Things that are not worth attacking on this host: font discovery (2.8 ms), pipelines (2 ms), first frame (3.5 ms).

Proposed fix, ordered by expected gain: (1) start the GPU instance, adapter and device at process start on a helper thread, and join it in resumed: −30 ms cold, −10 ms warm. (2) Nothing felis should do about the ICD count; that cost belongs to the system's Vulkan loader. It is worth recording in the benchmark notes that the Linux startup figure moves by 35-50 ms with the installed driver set.

The macOS inversion is for the macOS lane.

Linux part (kilimanjaro, i5-12400F, NVIDIA RTX 595.99 driver, niri, scale 1). I timed the client's path with a local build of `main` ef96bb46 that adds one `info!("STARTUP-STEP …")` log at each step boundary (not committed). Time zero is just before `exec` of `felis`, and "window mapped" is the moment niri's event stream reported the client's toplevel. Default config with an empty `$HOME` (a config naming `DejaVu Sans Mono` changes nothing: font discovery takes 2.8 ms either way). Load 3.5-4.7. ## Where the time goes (ms from exec; three runs each) | step | warm | cold (systemd-run hand-off) | what runs | |---|---|---|---| | process start → logging initialized | 2.5 | 2.4-2.5 | exec `felis` → exec `felis-client`, dynamic loading | | daemon dial + create/attach | **2.2-2.5** | **29-38** | `connect_or_spawn_daemon`, `create_with` | | winit event loop (Wayland connect) | 2.1-2.2 | 2.1-2.3 | `EventLoop::build` | | window creation | 1.0-1.1 | 0.9-1.3 | `create_window` | | `Instance::new` + `create_surface` | **34-65** | 34-46 | Vulkan loader loads ICDs | | `request_adapter` | 2.3-3.7 | 2.3-2.6 | | | `request_device` | **72-94** | **62-94** | NVIDIA Vulkan device | | `surface.configure` + pipelines | 5.5-6.0 | 5.5 | | | font discovery + glyph/image atlases | 3.2 | 3.3-3.4 | `FontStack::auto_discover`, `GlyphCache::new`, `ImageAtlas::new` | | first render → `present` | 3.4-3.5 | 3.4-3.5 | | | present → niri maps the window | 1.6-2.2 | 1.6-1.7 | | | **total to window mapped** | **143-196** | **148-202** | | Those runs pinned the NVIDIA ICD alone (`VK_DRIVER_FILES`, needed because of the driver mismatch described in #398). The published run had every ICD the system ships (14 manifests, lavapipe among them). Pinning all 14 reproduces that condition: | warm, all 14 ICDs | instance + surface | device | total to window mapped | |---|---|---|---| | run 1 / 2 / 3 | 113 / 71 / 69 ms | 91 / 78 / 78 ms | 233 / 179 / 176 ms | That matches the published warm 174 ms. Loading every installed Vulkan driver at `vkCreateInstance` costs 35-50 ms on top of loading NVIDIA's alone. ## Answers - **Linux warm, 70 ms behind alacritty.** GPU initialization is 110-200 ms of a 145-230 ms start: Vulkan instance creation 34-113 ms (depending on how many ICDs are installed), plus NVIDIA device creation 62-94 ms. Everything felis does itself (daemon attach, Wayland, window, fonts, atlases, pipelines, first frame) adds up to about 20 ms. The gap to alacritty is the Vulkan instance and device cost on this driver, not felis's own work. - **Cold adds 27-36 ms, all in the dial.** The systemd-run hand-off plus daemon boot takes 29-38 ms against 2.2-2.5 ms warm; every later step is the same. - **Overlap available.** Every step is serial today: `run_client` (`crates/felis-client/src/main.rs:785-797`) dials and attaches before `drive` builds the event loop, and `Renderer::new_with_config` (`crates/felis-render-wgpu/src/lib.rs:326`) starts `Instance::new` only inside `resumed`, after the window exists. None of the instance, adapter or device work needs the window or the daemon. Only `create_surface` and `surface.configure` need the window, and the adapter can be requested without `compatible_surface` and checked with `adapter.is_surface_supported(&surface)` afterwards. Starting instance + adapter + device on a thread at the top of `main`, in parallel with config load, the daemon dial, the Wayland connect and the window, would hide: - cold: the 29-38 ms dial behind the GPU work, so cold ≈ warm (**about −30 ms**); - warm: the about 10 ms of dial, event loop and window (**about −8 to −10 ms**). Font discovery (3 ms) and atlas creation can run beside the device request too (about −3 ms). - Things that are not worth attacking on this host: font discovery (2.8 ms), pipelines (2 ms), first frame (3.5 ms). Proposed fix, ordered by expected gain: (1) start the GPU instance, adapter and device at process start on a helper thread, and join it in `resumed`: −30 ms cold, −10 ms warm. (2) Nothing felis should do about the ICD count; that cost belongs to the system's Vulkan loader. It is worth recording in the benchmark notes that the Linux startup figure moves by 35-50 ms with the installed driver set. The macOS inversion is for the macOS lane.

natsukium referenced this issue from a pull request that will close it,

2026-09-25 06:45:18 +09:00

client: start the GPU device and font stack before the first window #420

natsukium closed this issue

2026-09-25 07:46:27 +09:00

Commenting is not possible because the repository is archived.