I’ve been sitting on this project for a couple weeks, and it’s time to write about it.
Windows has had a Live Captions feature for a couple of years now. Turn it on and a small bar appears at the bottom of the screen, showing whatever it’s currently hearing — your microphone, a meeting, a video, anything the system is playing. For a sighted person who just wants a quick visual backup while they half-listen to something, it does the job. For someone using a screen reader, it barely exists. Narrator, NVDA, and JAWS have no reliable way to get at what’s in that bar. There’s no history — once a line scrolls off, it’s gone. You can’t review it in a meanaingful way, you can’t copy it, and there’s no way to arrow up and re-read something you missed. It’s a caption feature built entirely around glancing, and glancing isn’t an option for me.
So I vibe-coded my own.
It’s called Accessible Live Captions, and the pitch is simple: same idea as Windows Live Captions — real-time, on-device transcription of your microphone or your system audio — but built from the start around a screen reader, not around a glance. Everything lands in a real transcript: a scrollable, keyboard-navigable list where every finished line is its own focusable item. Arrow up, arrow down, Home, End — you review it exactly like you’d review any other list in Windows. Each new line gets announced through Narrator, NVDA, or JAWS using the UI Automation notification event, the same mechanism a lot of accessible apps use to speak things without stealing your focus. And because I know how fast a screen reader gets buried under a flood of interim, still-changing text, only the finished lines get announced. The “still deciding” partial text shows up in a small area labeled Now hearing, but it stays quiet.
If you actually want the compact, single-line presentation Windows’ version uses, you can have that too — F7 switches to a panel that shows one caption at a time in large text. The difference is what’s happening underneath it. The panel isn’t a separate, lesser mode; it’s a different view onto the same kept history, still announcing through the same notification event, still lets you step backward through what you missed. Switch back to the transcript and every line is still there. That’s really the core argument of the whole project: the compact view and the accessible view were never actually in tension. Keeping the text and making it reviewable costs nothing. Windows could do this today.
Under the hood it can caption your microphone or any audio playing on the PC — a specific app, or everything at once — and it gives you a choice of four on-device speech engines: an accurate Whisper model, a true word-by-word streaming recognizer, Windows’ own on-device NPU recognizer on Copilot+ PCs, and Windows’ built-in instant speech recognition. All four run locally. No cloud, no API key, nothing leaving the machine. On a Snapdragon X Copilot+ PC, the Whisper engine runs comfortably in real time on the CPU alone, which is honestly the point I most want to make with this project: accurate, private, accessible live captioning of anything happening on the PC is already achievable on this hardware, right now.
There’s one detail I’m particularly glad I got right. When you’re captioning system audio as a screen reader user, your screen reader’s own speech is part of that system audio too — so without doing anything about it, you’d end up with your captions transcribing screen reader speech too. The app can exclude your screen reader’s process from the captured audio, on by default, so what you get is the meeting or the video, not a feedback loop of your own screen reader.
None of this required exotic engineering. It’s WPF, System colors instead of hardcoded brushes so High Contrast just works, focus that starts and stays on the transcript because Tab doesn’t need to do anything when there’s only one real control, and every command reachable from an actual menu bar instead of a maze of toolbar buttons. It’s the same set of habits I’d want from any accessible Windows app, applied to a feature Microsoft already ships.
I want to be fair here: the recognizer Windows uses for Live Captions is genuinely good, and running it on the NPU on Copilot+ hardware is a real engineering achievement — my app’s Windows-on-device engine option actually calls into that same recognizer through Microsoft’s own API, because there was no reason to reinvent it. What’s missing isn’t the speech recognition. It’s the twenty additional lines of UI work to make the output of that recognition something you can actually read with a screen reader after the fact. That’s the gap this project is trying to make visible.
Right now this is a demonstration, not a polished product — it’s ARM64 Windows 11 only, built and tested on my own Copilot+ PC, and I’m calling it that on purpose. The code, the releases, and the full write-up of how it’s built are all on GitHub if you want to look under the hood or try it yourself: github.com/kellylford/LiveCaptionsWithAccessibility/releases. If you’re a screen reader user who deals with captions regularly, I’d genuinely like to hear whether the keyboard model and the announcements feel right to you, because that’s the part no amount of my own testing can fully substitute for.
Comments