Pallm

Model servers

The servers Pallm finds, adding your own, and how it fits models in memory.

Pallm works with any server that speaks the OpenAI API, which is nearly all of them. Every server’s models merge into one list, and each chat goes to the server that has its model.

Found by themselves

These are found when they run on the host, at startup and every 30 seconds after:

ServerWhere Pallm looks
LM Studio127.0.0.1:1234
Ollama127.0.0.1:11434
oMLXfrom oMLX’s own settings, including its API key
llama.cpp (llama-server)127.0.0.1:8080
mlx-serve127.0.0.1:11234

Add a server

For a server on another computer, a different port, or a hosted one you trust, open Settings › Advanced › Model servers and choose Add server…. Fill in its name, host and port, and if it needs them, HTTPS, a path (usually /v1), an API key and its context window.

An API key you enter is kept by the service on your Mac. It’s never stored on your phone.

Each server in the list has a status light, an on switch, Edit…, Remove and Restart.

pallm servers                                 # the list, with each one's state
pallm servers add studio 192.168.1.20:1234
pallm servers off studio

When a server stops answering

If a server crashes or hangs, its models show as offline in the pickers, and sends to them stop early with a clear message rather than waiting. Restart brings it back:

  • oMLX and LM Studio are restarted with their own commands.
  • Ollama is reopened.
  • Anything else is started again the way Pallm last saw it running.

Memory

Pallm’s memory manager decides which models stay loaded. It keeps as many as fit in your Mac’s memory, minus headroom for everything else (a quarter of the Mac’s memory, at most 16 GB). When a model needs room, the one used longest ago is ejected first.

Settings › Advanced › Loaded models shows what’s loaded and how much memory each takes:

  • Eject unloads a model now.
  • Pin keeps one loaded no matter what.
  • Load a model now… warms one up before you need it.

If the only way to make room would stop something running, such as another chat’s reply or an image being made, Pallm asks you first: Wait, Stop it and start, or Cancel.

To change the headroom, or turn the manager off:

[residency]
headroom_gb = 12      # memory left free for everything else
# manage = false      # leave memory to the servers themselves