Make Siri Actually Smart — The Local LLM Bridge Trick

Photo: Unsplash

AI Power User

Make Siri Actually Smart — The Local LLM Bridge Trick

A single Shortcut turns Siri into a voice gateway to the LLM running on your Mac
By 5 min read
sirishortcutsollamatailscale

Siri can set timers and turn off lights. Ask it anything requiring actual thought and you get “Here’s what I found on the web.” Meanwhile, my Mac Studio sits at home running Llama 3.1 8B at 60 tokens per second, perfectly capable of answering the question Siri just punted on.

So I connected them. The result: I say “Hey Siri, Ask the Brain,” speak my question, and 2–4 seconds later Siri reads back an answer generated entirely on my own hardware — from anywhere, over Tailscale. No subscription, no data leaving machines I own. Here’s the complete build, including the exact Shortcut configuration, the honest limitations, and where Apple Intelligence still wins.

The Architecture in One Paragraph

A Shortcut named with a natural spoken phrase (“Ask the Brain”) captures dictated text, POSTs it to the Ollama API on your Mac, parses the JSON response, and hands the text back to Siri to speak. On your home network it talks to the Mac directly; with Tailscale installed on both devices, it works identically from a train in Brno. The whole thing is five Shortcut actions.

Step 1: Expose Ollama Properly

By default Ollama only listens on localhost. You need it listening on all interfaces so your phone can reach it:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
# restart the Ollama menu bar app afterwards
ollama pull llama3.1:8b

Verify from another machine on your network:

curl http://192.168.1.50:11434/api/tags

If you get JSON back, the server side is done. An 8B model is the sweet spot here — fast enough that voice interaction feels responsive, smart enough for real questions. On an M2 Ultra it generates at ~60 tok/s; even an M1 MacBook Air manages ~25 tok/s, which is still faster than Siri’s web-search shrug.

Step 2: Build the Shortcut

Create a new Shortcut and name it “Ask the Brain” — the name is the voice trigger, so pick something you can say naturally without Siri mishearing it. (“Ask the Brain” survives my Czech accent; “Query LLM” did not.)

The five actions:

  1. Dictate Text — this captures your spoken question when invoked via Siri.
  2. Get Contents of URL — the core. Configure it exactly like this:
    • URL: http://your-mac-tailscale-name:11434/api/generate
    • Method: POST
    • Headers: Content-Type: application/json
    • Request Body: JSON, with these fields:
      • model (Text): llama3.1:8b
      • prompt (Text): Answer concisely in 2-3 spoken sentences. Question: followed by the Dictated Text variable
      • stream (Boolean): false — critical, Shortcuts can’t handle streaming responses
  3. Get Dictionary from Input — parses the JSON.
  4. Get Dictionary Value — key: response.
  5. Speak Text — set rate to your preference; Siri reads the answer aloud.

The “answer concisely” prefix in the prompt matters enormously. Without it, an 8B model will happily generate four paragraphs, and listening to Siri narrate a wall of text is misery. Forcing 2–3 sentences keeps total round trips short.

Step 3: Tailscale Makes It Work Everywhere

On the home Wi-Fi you could use the Mac’s LAN IP, but the moment you leave the house it breaks. Tailscale fixes this with zero networking knowledge required: install it on the Mac and the iPhone, sign both into the same tailnet, and your Mac gets a stable hostname like mac-studio.tailnet-name.ts.net that resolves from anywhere.

Swap that hostname into the Shortcut’s URL and the bridge now works from cellular. I’ve queried my home Mac from a café in Vienna; the Tailscale relay added maybe 300ms. Crucially, nothing is exposed to the public internet — port 11434 is only reachable inside your private tailnet, which is the only responsible way to run an unauthenticated API like Ollama’s.

Realistic Latency: 2–4 Seconds, Here’s the Breakdown

Honest numbers from timing 30 invocations:

  • Dictation capture and processing: ~0.5–1s after you stop speaking
  • Network round trip (LAN: ~10ms; Tailscale direct: ~40ms; Tailscale relayed: ~300ms)
  • Model generation for a 3-sentence answer (~80 tokens at 60 tok/s): ~1.3s, plus ~200ms prompt evaluation
  • Speech synthesis start: ~0.3s

Total: 2–4 seconds from end-of-question to start-of-answer on the 8B model. That’s slower than asking a human sitting next to you, faster than unlocking your phone and typing into an app, and dramatically more useful than stock Siri’s “I found this on the web.” If you point it at a 14B model expect 4–6 seconds; for voice, I’d stay at 8B.

The Honest Limitation: This Is Single-Shot, Not a Conversation

You ask one question, you get one answer, the Shortcut ends. There is no “what about the second one?” follow-up, because the Shortcut holds no conversation state — each invocation is a fresh API call with no memory of the last.

Could you build multi-turn? Technically yes: store the conversation in a file, use Ollama’s /api/chat endpoint with message history, loop with Repeat actions. I built it. It’s fragile, Siri’s dictation timeout fights you on every turn, and the Shortcuts UI for it is a hedge maze. The single-shot version is the one that survives daily use, so that’s the one I’m teaching. Treat this as a voice oracle, not a voice chatbot.

Variants Worth Building

Once the core bridge works, the same five-action skeleton spawns genuinely useful tools:

Dictate-a-note-and-structure-it. Replace the prompt prefix with: Convert this rambling voice note into structured markdown with a title, bullet points, and action items:. Add a “Save File” action at the end pointing at your Obsidian vault’s inbox folder. I dump thoughts at my desk while my cat Mochi judges me from the windowsill, and they arrive as tidy notes.

Voice translation. Prefix: Translate to Czech, reply with only the translation:. Round trip is fast enough for slow-paced practical use — asking how to phrase something before a phone call, not live interpreting.

Summarize my clipboard. Swap Dictate Text for Get Clipboard, prefix with Summarize in 3 bullet points:, and keep Speak Text. Copy a long article on your phone, say “Hey Siri, Brief Me,” and listen to the summary while you make coffee.

Apple Intelligence Siri vs. the DIY Bridge

Fair comparison, because Apple Intelligence Siri has improved and each wins somewhere.

Apple’s Siri is better at: on-device actions (it can actually do things — send messages, manipulate apps, surface personal context from Mail and Calendar), zero-setup availability, and multi-turn context within its supported domains. The ChatGPT handoff also gives it frontier-level answers when it chooses to use it.

The DIY bridge is better at: answer quality under your control (pick any model, tune the system prompt), privacy (your questions never touch Apple’s or OpenAI’s servers — auditable on your own network), no rate limits or account requirements, and extensibility — Apple will never ship “dictate a note into my Obsidian vault via my own model.”

They’re complementary. I use stock Siri for actions and the Brain for knowledge. Total build time for the basic bridge: about 20 minutes, most of it tapping through the Shortcuts JSON editor. For making a $1,000+ Mac and a $700+ phone finally talk to each other intelligently, that’s the best 20 minutes of setup this series has offered yet.

Back to blog