client: announce new output to NVDA and VoiceOver #241

New issue

Open

opened 2026-10-24 14:29:13 +09:00 by natsukium · 0 comments

natsukium commented

2026-10-24 14:29:13 +09:00

Owner

Follow-up to #240.

#240 exposes the terminal as an accessibility tree with text-changed and caret events. That is enough for Orca, whose terminal script keys on the AT-SPI terminal role and speaks text-changed:insert. It is not enough on Windows and macOS:

NVDA applies its terminal behavior (live text diffing, typed-character handling) only to UIA elements whose ClassName is TermControl / WPFTermControl or to conhost's window class (source/NVDAObjects/UIA/__init__.py). AccessKit maps Role::Terminal to a UIA Document, so NVDA reads felis as a document.
VoiceOver is not expected to read new text in an AXTextArea on ValueChanged; iTerm2 posts NSAccessibilityAnnouncementRequestedNotification for that reason.

So with #240 alone, NVDA and VoiceOver users read new output through review commands.

First step

Observe NVDA, Narrator, and VoiceOver against the #240 tree before designing. The design should start from what they actually do with text-changed events, not from assumptions.

Constraints

Principle 4: no rule may depend on what the cells contain. Matching output against keystrokes (Windows Terminal, xterm.js) and classifying redraws are out.
No impersonation by default: setting the UIA ClassName to TermControl would give felis NVDA's terminal handling, but docs/explanation/architecture/terminal-identity.md keeps impersonation a user opt-in.

Candidates

A positional announcement (iTerm2 style): announce what the cursor has moved past, through an AccessKit live-region node added per announcement (the Windows adapter raises UIA_LiveRegionChangedEventId, the macOS adapter an announcement, on node add; the node must not be hidden). Two review rounds on a cursor-boundary state machine found holes each time: typing that wraps across a soft wrap, a prompt redrawn on the same row, the first output row after a slow Enter, rows scrolling off between flushes. The client shadow holds no scrollback, so output faster than a screenful per frame cannot be announced in full.
Upstream: ask NVDA to recognize a generic UIA terminal (for example by LocalizedControlType "terminal" or an ARIA role AccessKit sets), so every AccessKit terminal benefits.
A user opt-in ClassName override, if the identity doc's opt-in pattern is extended to the AT surface.

Follow-up to #240. #240 exposes the terminal as an accessibility tree with text-changed and caret events. That is enough for Orca, whose terminal script keys on the AT-SPI `terminal` role and speaks `text-changed:insert`. It is not enough on Windows and macOS: - NVDA applies its terminal behavior (live text diffing, typed-character handling) only to UIA elements whose ClassName is `TermControl` / `WPFTermControl` or to conhost's window class (`source/NVDAObjects/UIA/__init__.py`). AccessKit maps `Role::Terminal` to a UIA Document, so NVDA reads felis as a document. - VoiceOver is not expected to read new text in an `AXTextArea` on `ValueChanged`; iTerm2 posts `NSAccessibilityAnnouncementRequestedNotification` for that reason. So with #240 alone, NVDA and VoiceOver users read new output through review commands. ## First step Observe NVDA, Narrator, and VoiceOver against the #240 tree before designing. The design should start from what they actually do with text-changed events, not from assumptions. ## Constraints - Principle 4: no rule may depend on what the cells contain. Matching output against keystrokes (Windows Terminal, xterm.js) and classifying redraws are out. - No impersonation by default: setting the UIA ClassName to `TermControl` would give felis NVDA's terminal handling, but `docs/explanation/architecture/terminal-identity.md` keeps impersonation a user opt-in. ## Candidates - A positional announcement (iTerm2 style): announce what the cursor has moved past, through an AccessKit live-region node added per announcement (the Windows adapter raises `UIA_LiveRegionChangedEventId`, the macOS adapter an announcement, on node add; the node must not be `hidden`). Two review rounds on a cursor-boundary state machine found holes each time: typing that wraps across a soft wrap, a prompt redrawn on the same row, the first output row after a slow Enter, rows scrolling off between flushes. The client shadow holds no scrollback, so output faster than a screenful per frame cannot be announced in full. - Upstream: ask NVDA to recognize a generic UIA terminal (for example by `LocalizedControlType` "terminal" or an ARIA role AccessKit sets), so every AccessKit terminal benefits. - A user opt-in ClassName override, if the identity doc's opt-in pattern is extended to the AT surface.

natsukium added the

Area/client

Kind/Feature

labels

2026-10-24 14:29:22 +09:00

natsukium referenced this issue

2026-10-24 14:29:22 +09:00

client: expose the terminal to screen readers via AccessKit #240