Choosing a model
Which server to install, and how big a model your Mac can run.
Pallm doesn’t download or run models itself. It talks to a model server on your Mac, the way a browser talks to a website, and makes the most of whatever that server has. You choose and download models in the server.
Pick a server
LM Studio is the easiest start: an app with a model browser, a download button and a server switch. It runs MLX models, which are made for Apple silicon, and GGUF models.
Ollama is a good fit if you like the command line. ollama pull downloads a model and Pallm sees it straight away.
oMLX, llama.cpp and mlx-serve suit people who want more control. Pallm finds all of them, and you can run more than one: their models merge into one list, and each chat goes to the server that has its model.
How big a model
A model has to fit in your Mac’s memory, with room left for macOS and your apps. A rough guide for 4-bit models, the usual download:
| Your Mac’s memory | Comfortable model size |
|---|---|
| 16 GB | up to about 8 billion parameters |
| 32 GB | up to about 30 billion, or a mixture-of-experts model of that size |
| 64 GB | up to about 70 billion |
| 128 GB or more | the largest open models, or several mid-sized ones at once |
Mixture-of-experts models (often written like 30B-A3B) are large but fast, because only a small part of the model works on each word. They’re often the best choice on a Mac.
What to look for
- Tool use. Web search, the browser, memory and scheduled tasks all work through tools. Most recent instruction-tuned models handle them. Very small models often don’t.
- Vision. A model that can see images lets you send photos and screenshots. Pallm checks with the server and tells you in the composer when a model can’t take a file.
- Thinking. Reasoning models think before they answer. Pallm’s thinking level (Off, Low, Medium, High) is passed to the server, and a model that doesn’t reason ignores it.
- Context length. How much of a conversation the model can keep in mind. The meter under the message box shows how full it is.
More than one model
Pallm’s memory manager keeps several models loaded when they fit, and ejects the one used longest ago when a new one needs room. If something busy is in the way, it asks before stopping it. See Model servers.