Fully Local Mac Dictation with Ollama: Setup and Trade-offs

You want to talk to your Mac and have clean text appear at the cursor, and you want to be able to prove that no audio and no text left the machine. The do-it-yourself version is typically a Whisper build, a shell script and a local LLM for cleanup. ListenIn runs the same pipeline packaged, minus Whisper: Apple’s on-device recognizer, then Ollama on localhost for cleanup. In Offline mode, nothing leaves the Mac at either stage.

The pipeline: mic → on-device recognizer → text → localhost Ollama → cursor

ListenIn has two stages, and they are separate on purpose.

1. Recognition. Microphone audio goes to Apple’s speech engine with on-device recognition forced on, on both paths: the modern engine on macOS 26 and the classic fallback on macOS 14–15. The classic engine is not allowed to use Apple’s speech service. Audio never reaches ListenIn’s cleanup service in any configuration.
2. Cleanup. The recognized text goes to a language model that removes fillers and false starts, repairs sound-alike errors, and formats. The rule: reordered, never invented. It does not add content and it does not answer the question you dictated.

Stage two is where the cloud normally enters. By default the text goes to ListenIn’s built-in cleanup service. Point it at Ollama on localhost and switch Mode to Offline, and nothing in the flow leaves the Mac. The security page has the mode-by-mode table of what each path can send.

This guide covers Dictate only; Translate and Ask Anything are out of scope here.

Prerequisites

A downloaded dictation language, on macOS 14–15. Because recognition is forced on-device, a missing language pack does not quietly fall back to the cloud; recognition fails, and ListenIn shows “Dictation language unavailable.” Apple’s guide Dictate messages and documents on Mac (as of 2026-09-18) points to System Settings › Keyboard › Dictation and the Edit button next to Languages. On macOS 26, ListenIn asks the modern engine to install the language assets it needs. If dictation produces nothing at all, start with the language-pack post.

Ollama installed, with at least one model pulled. Ollama’s quickstart covers install; models come down with ollama pull <model>. ListenIn talks to Ollama through its OpenAI-compatible endpoint at http://localhost:11434/v1, the default address; no API key is needed.

Which model? Pick by what your Mac can hold in memory and how long you are willing to wait; this post does not rank them.

Point ListenIn’s cleanup at Ollama

Open Settings › Models. The tab has three parts.

• A Mode picker at the top: Hybrid or Offline.
• A Cloud (online) section with Built-in (ListenIn’s service) and any cloud models you added with Add cloud model.
• An On-device (offline) section that lists what Ollama has.

When the tab opens it shows “Looking for local models…” and queries Ollama. What you see next depends on your machine:

What you see	Meaning	Action
”Install Ollama to run models offline”	Ollama is not on this Mac	Get Ollama opens the download page; Recheck after installing
”Ollama isn’t running”	Installed, not started	Start Ollama launches it; the list fills in when it is up
Model names, each marked “On-device · offline-capable”	Ollama is running	Select one
Extra rows with a Download button	Models ListenIn can pull for you	Optional

Select a local model, then flip Mode to Offline. The footer under the picker states the contract: “Offline — cleanup runs only on your on-device model. Nothing leaves this Mac; cloud is off.” The cloud section greys out, and Check for Updates disappears from the menu bar while Offline is on, because an update check is an outbound request.

Two guards worth knowing. Offline cannot be switched on without a selected local model; the picker refuses. And “local” is defined narrowly: localhost, 127.0.0.1, ::1, or a .local host. In Offline mode a profile pointing anywhere else is treated as not configured, and you get the raw transcript rather than a request to a remote server.

Verify with Wi-Fi off

Do not take the footer’s word for it. Turn Wi-Fi off and run through three targets.

1. Notes. Tap your trigger key and say a sentence with a filler and a restart in it, then tap again. The default is fn; if macOS already used fn on first launch (emoji, input switching or dictation), ListenIn picked Right ⌘ instead — Settings › Dictation shows the current key if you are unsure. Cleaned text should land. If the raw sentence lands and the HUD says “Cleanup failed — raw transcript inserted (check network or key),” Ollama is not answering: check that it is running and that the selected model is still installed.
2. A terminal. Dictate a shell command, or a prompt into Claude Code: voice input for Claude Code runs on this same path, so in Offline mode it is fully local too. If Terminal has Secure Keyboard Entry on, ListenIn will not paste; it shows a copy card instead (Secure Keyboard Entry and dictation explains why), and the text is one click away.
3. A Quick Prompt. If you have bound a Quick Prompt key, press it and say only the prompt hint. Matching runs on the Mac, so the saved prompt pastes with the network down. This is a separate path from Ollama and works in either mode.

First requests after a cold start can be slow because Ollama has to load the model. ListenIn gives OpenAI-compatible endpoints, local ones included, 60 seconds before it gives up and inserts the raw transcript. A pause on the first dictation is expected; a raw transcript every time is not.

What changes with a small local model

The instructions ListenIn sends to Ollama mirror the ones the cloud path uses, including your settings. Three things carry over unchanged:

• Cleanup strength. Verbatim (punctuation only), Standard (remove fillers, fix slips), Refined (restructure & bullets). Each is an instruction in the prompt, so it applies to any model.
• Mixed Chinese and English. No mode switch; the script lock (Simplified stays Simplified, Traditional stays Traditional) is also a prompt instruction.
• Your Personal Dictionary and learned corrections. Both are folded into the same prompt as known terms and heard→meant pairs. They do not retrain Apple’s recognizer; they tell the cleanup model which spellings you meant. How that learning works is the same on every path.

What changes is how well the model follows all of that. Refined asks for restructuring; a small model may restructure less or drift. Long Chinese–English dictation is harder than a short English sentence. If Refined disappoints, drop to Standard, which asks less of the model. Quality now depends on your model and your machine.

The middle option: bring your own API key

Zero-cloud is one end of a spectrum. In the same Cloud (online) section, Add cloud model opens a sheet with a Provider picker, a Base URL, an API Key and a Model name, for any OpenAI-compatible provider or an Anthropic key. Text then goes straight from your Mac to that provider and never touches ListenIn’s server. Audio still never leaves the Mac; that boundary does not move with this choice.

This fits when you already have a provider account and a data agreement with it, or want a stronger model than your laptop can run. The trade: the text is now under that provider’s terms.

Hybrid mode makes the two ends cooperate. Leave Mode on Hybrid, keep a cloud model selected up top and a local model selected below. Cleanup goes to the cloud model; if that request fails, ListenIn retries the same text against the local model and labels the result “Cloud unreachable — cleaned by your local model.” Text does leave the Mac in Hybrid, so it is a reliability feature, not a privacy one.

What you give up, and what happens when Ollama isn’t running

Setup and upkeep are yours. Ollama has to be installed, running, and holding a model. ListenIn will start an installed Ollama for you and offer downloads, but it cannot keep the model loaded or manage your disk.

When Ollama is down in Offline mode, ListenIn does what it does whenever cleanup is unavailable: the raw Apple transcript still lands at your cursor and the HUD labels the failure. Nothing goes to the cloud instead; in Offline mode there is no cloud path. Delete the selected model from Ollama and each request fails the same way: a labeled raw transcript, until you select another model in Settings › Models.

For what this post did not cover, the FAQ is short, and the Mac dictation page covers the non-Ollama path. If the app behaves differently from this post, that is a bug; the security page says where to report it.