Run Stable Diffusion on Your Mac — No Cloud, No Limits, No Cost

Every image in my last three blog post mockups, every thumbnail experiment, and a frankly embarrassing number of D&D character portraits came out of my Mac — not Midjourney, not DALL-E, not a credits-based web app. Local image generation on Apple Silicon crossed the “actually good” threshold a while ago, and most Mac owners still don’t know it’s sitting there for free. Let me break down the three promises in the title, honestly: no cloud (true), no limits (true, with one important nuance), no cost (true if you own the Mac).

The easy path: Draw Things

Draw Things is a free app on the Mac App Store (also iPhone/iPad), and it is shockingly good — a native, Metal-optimized Stable Diffusion studio that a non-technical person can use in ten minutes. Install it, open the model picker, and download a model directly inside the app: SDXL Base, Juggernaut XL (my recommendation for photorealism), or one of the Flux variants if you have the RAM.

Workflow for your first decent image: pick Juggernaut XL → set size to 1024×1024 → steps to 30 → DPM++ 2M Karras sampler → type a prompt with actual photographic language (“85mm portrait of an elderly fisherman, golden hour, shallow depth of field”) → Generate. That’s it. No Python, no virtual environments, no terminal.

Realistic SDXL generation times at 1024×1024, 30 steps, from my own machines and tests I’ve replicated on friends’ hardware:

• M1 / M2 MacBook Air (16 GB): 60–90 seconds per image
• M3 Pro / M4 Pro (24–36 GB): 25–40 seconds
• M4 Max (48 GB+): 12–20 seconds
• Mac Studio M3 Ultra: under 10 seconds, and Flux-class models become comfortable

Flux.1 [dev] — the current open-weights quality king — wants 24 GB+ to run pleasantly and takes roughly 2–4x SDXL times. On my M3 Pro, a Flux image is a 90-second coffee-sip wait; on a Studio it’s a beat. Slower than a cloud GPU? Yes. But there’s no queue, no “relax mode,” and you can leave a batch of 50 running overnight for the cost of a lightbulb.

Why your Mac beats a gaming GPU here

This is the part PC people don’t believe until they see it. A €600 RTX 4060 Ti has 16 GB of VRAM, and when a model doesn’t fit, it doesn’t run — or it crawls through system-RAM offloading. A Mac’s unified memory means the GPU can address nearly all of the machine’s RAM. A 36 GB MacBook Pro will load Flux.1 [dev] at full fp16 precision plus a refiner plus an upscale model simultaneously — a configuration that’s simply impossible on any consumer NVIDIA card short of a 4090/5090. Raw speed favors NVIDIA; capability per dollar at a given memory size favors the Mac, and in image generation, memory is destiny. This is exactly why the bigger open models increasingly get their community quantizations tested on Mac Studios first.

The power-user path: ComfyUI

When you outgrow Draw Things — you want img2img chains, ControlNet poses, regional prompting, or video models — ComfyUI is the node-based standard, and it runs natively on Metal:

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
python3 -m venv venv && source venv/bin/activate
pip install torch torchvision torchaudio
pip install -r requirements.txt
python main.py

Open http://127.0.0.1:8188, drop a checkpoint into models/checkpoints/, and you have a visual pipeline editor where every workflow is a shareable JSON. The community publishes thousands of them — drag someone’s workflow PNG onto the canvas and the entire node graph reconstructs itself. ComfyUI is also where Mac users get access to the bleeding edge (new samplers, Flux ControlNets, video models like LTX) months before any polished app wraps them.

LoRAs: the feature cloud services ration out

A LoRA is a small (50–300 MB) add-on that teaches a model a style, a character, or a product. Browse Civitai, download a .safetensors file, drop it in Draw Things’ LoRA import or ComfyUI’s models/loras/ folder, and reference it at a chosen strength. This is how you get consistency — the same illustrated style across every image in a blog series, the same mascot in twenty poses.

The deeper move: train your own. Draw Things has on-device LoRA training built in; give it 15–20 photos of a subject and a few hours of compute, and you have a personal model no cloud service will rent you. I trained one on my own product photography style — every e-shop mockup I generate now looks like my photos. Try doing that with a Midjourney subscription.

”No limits” — what that actually means

On your own hardware there are no content filters, no credit meters, no terms-of-service roulette, and no queue. For legitimate creative work this matters more than it sounds: Midjourney and DALL-E routinely refuse harmless prompts — anatomy studies for figure drawing, anything with a brand name in a product mockup, mildly dark fantasy art, even medical illustration. On a local model, the only editorial filter is your own judgment and your own legal responsibility. That’s the grown-up trade: full freedom, full accountability. (The nuance I promised: “no limits” is not “no ethics” — what’s illegal or harmful doesn’t become acceptable because it rendered on your GPU.)

The “no cost” claim, audited: the apps are free, the open-weights models are free, and a 30-second SDXL generation on an M3 Pro consumes roughly 0.0005 kWh — about a thousand images per cent of electricity. Against Midjourney’s $10–60/month, the math isn’t close if you generate regularly.

Honest comparison: can it match Midjourney?

Straight answer: out of the box, no — with effort, mostly yes, and in some dimensions local wins outright.

Midjourney v7’s house aesthetic is extraordinary; type six lazy words and get something gorgeous. Base SDXL with the same lazy prompt looks dated. But the gap closes fast with the right checkpoint (Juggernaut XL, RealVisXL) and proper prompting, and Flux.1 [dev] genuinely competes on photorealism and demolishes Midjourney on prompt adherence — ask for “three cats, the middle one wearing a red collar, text on the sign says OPEN” and watch which system actually delivers it. Where local wins decisively: consistency via LoRAs, ControlNet-level composition control, unlimited iteration, privacy of unreleased product designs, and editing pipelines (inpainting a client’s product into a scene without uploading it anywhere).

My honest recommendation by hardware: 8 GB Mac — use SD 1.5-class models in Draw Things, temper expectations. 16 GB — SDXL works well; this is the entry to “actually great.” 24 GB+ — Flux territory, and you stop missing the cloud. 48 GB+ — you’re running things tomorrow’s blog posts will be written about.

Download Draw Things tonight, grab Juggernaut XL, and generate the same prompt you last paid Midjourney for. The first time a better-than-expected image renders on your own silicon, with no meter running, something clicks — and you don’t go back.