How to Run an LLM Locally on a Mac

Apple Silicon turned the Mac into one of the simplest ways of running large language models at home. Memory is the main constraint, and you cannot add more later.

How to Run an LLM Locally on a Mac — Troiana insight cover

In short

On an Apple Silicon Mac, install LM Studio for a graphical app or Ollama for the command line, then download a model that fits your memory. LM Studio recommends 16 GB or more, with macOS 14 or newer; 8 GB works for small models. Unified memory lets Macs load larger models than many graphics cards, while speed depends on your chip's memory bandwidth, not simply its core count.

What you need

LM Studio's requirements are a useful baseline: Apple Silicon, macOS 14 or newer, and 16 GB of memory recommended. On an 8 GB Mac, it advises sticking to smaller models and modest context sizes. Intel Macs aren't currently supported by LM Studio.

Apple's unified memory is shared between the CPU and GPU, so the GPU can use most of it, unlike a typical PC where a graphics card has its own separate memory. That's why a high-memory Mac can load models that wouldn't fit on most consumer graphics cards.

Check your memory first

Open the Apple menu, then About This Mac, to see your chip and memory. As a rough guide using common sizing rules, at 4-bit quantisation:

Unified memoryComfortable model sizeRealistic use
8 GBSmall, under about 4BExperiments, short summaries; close other apps
16 GBAround 7–13BEveryday writing and chat, light coding help
24–36 GBAround 20–32BSolid coding help, longer context
48–64 GBAround 70BStrong general use, agent experiments
96 GB or moreLarger modelsHeavy use; speed may become the limit

macOS and your other apps need memory too, so keep some headroom, particularly on smaller machines.

Comfortable model size, in billions of parameters
8 GBUnder about 4B
16 GB7–13B
24–36 GB20–32B
48–64 GBAround 70B
1B10B100B
Rough guidance from the table above, on a log scale. Keep some headroom, as macOS and your other apps need memory too.

The quick start

Option 1: LM Studio, the easiest.

  1. Install it from lmstudio.ai.
  2. Search for a model, and choose a version that fits your memory. The app helps by indicating what's likely to fit.
  3. Prefer an MLX build, if available, on Apple Silicon.
  4. Load it and start chatting. To use it from your own code, see how to use LM Studio as a local API server.

Option 2: Ollama, for developers.

brew install ollama
ollama serve
ollama run gemma4

Swap gemma4 for whichever model suits you from its library. Ollama also serves an OpenAI-compatible API on port 11434, which allows you to connect apps and coding tools. There's a desktop app too if you'd rather not use Homebrew. See Ollama vs LM Studio for how to choose.

What MLX changes

MLX is Apple's machine learning framework, built specifically for Apple Silicon. GGUF is the file format used with llama.cpp, which also runs well on Metal. Both are good options, and MLX is often faster on a Mac, but by how much depends on the model.

Reports on the picture are still moving. Ollama announced an MLX backend as a preview in March 2026, with reports of near-doubled generation speed on some models, and a stated requirement of 32 GB or more of unified memory at that time. One optimisation guide suggests that the advantage shrinks on larger dense models, as memory bandwidth becomes the limit. Treat any multiplier as approximate, and test the model you actually plan to use.

Why your chip matters

For local inference, memory bandwidth has a major effect on speed. This is why a Max or Ultra chip can be much faster than a base chip with the same amount of memory, and why a base model with lots of RAM can still feel slow. If you're buying a Mac for this purpose, compare bandwidth along with GPU cores, and look at benchmarks for your preferred model on that exact chip.

Making it faster

  • Use a smaller quantisation or model. A smaller model that runs smoothly usually beats a large one that crawls.
  • Lower the context length. A long context uses more memory and slows everything down. See what a context window is.
  • Close memory-hungry apps. Browsers with many tabs can make a real difference on smaller machines.
  • Keep it plugged in. Laptops may reduce performance on battery or as they get hot.
  • Watch memory pressure. Open Activity Monitor; if memory pressure is high and swap is growing, your model is too big.

What to run it for

Local models are good at drafting, summarising, translating, extracting data from documents and answering questions about private files. Thanks to recent small models, a 16 GB laptop can do useful work offline. For hard coding or reasoning tasks, a hosted frontier model is still likely to be stronger, so many people use both.

Connecting a coding agent can work well on larger machines. Be realistic, though, as agents use a lot of context and send many requests, and a slow model makes for a slow agent.

Privacy

Running a model locally means your prompts stay on your machine, with the model itself not sending anything, as long as you haven't connected online tools. But check your setup: some apps have optional cloud features, and an agent with web access can send data elsewhere. Also, each model has its own licence, which matters if you plan to use it commercially.

Should you buy a Mac for this?

If you need a new computer anyway, then buying more memory than you otherwise would is reasonable, since you can't upgrade it later. Buying a high-end Mac purely to save on AI costs is harder to justify, as hosted APIs are cheap. The cheapest way to run an LLM has the maths. You can rent a GPU for a few dollars or use the hosted version of a model to find out whether a model is good enough before spending money.

If you're building something on local models, book a call.

Common questions

Can a MacBook Air run an LLM?

Yes. Newer Apple Silicon models can run smaller models well, and 16 GB or more is recommended. Fanless laptops may slow down under sustained load, so long sessions will be slower than on a Pro with active cooling.

Is 16 GB enough for local LLMs on a Mac?

It's enough for useful work with small to medium models, around 7–13B at 4-bit quantisation, such as writing help and summaries. Larger models and long contexts need more. Close other apps and keep an eye on memory pressure.

Should I use MLX or GGUF on a Mac?

Try MLX first if a build exists for your model, since it's designed for Apple Silicon and is often faster. GGUF has a wider selection. Differences vary by model, so test both with the model you intend to use.

Do local models on a Mac work offline?

Yes. Once downloaded, models run without an internet connection using tools such as LM Studio or Ollama. Any features that use web search or cloud models will still need a connection, so check your settings if privacy is your aim.

Have something worth building right?