AI models

LaunchPad can download open models, score them against this host, load them into a local runner, and expose an OpenAI-compatible API. The AI section is Models → Downloaded → Running models, plus Runner and Credentials (covered next).

Model catalog (Top picks)

Open AI → Models. Top picks shows curated cards with capability tags (chat, tools, reasoning, coding, multimodal) and a host-fit summary (for example perfect · mlx-8bit).

1. Optionally filter with Filter top picks….
2. Read size / fit on the card footer.
3. Click Download. Artifacts land under Downloaded.

Prefer a perfect or good fit for your first model. Large “marginal” fits will download but may thrash memory when loaded.

Advanced catalog

Switch to Advanced for the full llmfit table:

• Filter Runtime: Any / llama.cpp / MLX / vLLM
• Filter Fit: Any / Perfect / Good / Marginal
• Columns include params, score, tok/s, quant, disk, mem %, context, use-case tags
• Row actions: Download and Load

Use Advanced when Top picks does not list the checkpoint you need, or when you care about a specific runtime.

Downloaded

AI → Downloaded lists cached artifacts ready to load.

Load moves the model into Running models using the current Runner accelerator. Details shows formats (GGUF / MLX sizes). Delete frees disk (blocked or warned when In use).

Browse models returns you to the catalog.

Running models

AI → Running models shows what is loaded for inference.

1. Check host gauges before loading another model.
2. Click Load (top right) to pick from Downloaded, or load from the catalog / Downloaded card.
3. Note BACKEND (llamacpp-metal, cloud - OpenRouter, …), PORT, MEMORY, and CONTEXT.
4. Select rows and Unload when you need the memory back.
5. Click a model name to open logs and the try panel.

Local backends bind a port on this Mac (for example 61300). Cloud backends show zero local RAM and route through Credentials.

Try this model

Open a loaded model to see:

• Backend status pill (llamacpp-metal - loaded - 127.0.0.1:PORT)
• Unload
• Live server logs (prompt eval time, tokens/sec)
• Try this model chat: type a prompt, press Send

Use the try panel to sanity-check a load before you point Open WebUI or another client at the gateway.

Suggested first model path

1. Installers → confirm llama.cpp / MLX Ready.
2. Models → Top picks → download a small chat model (for example Gemma 2B-class).
3. Downloaded → Load.
4. Open the running model → send a short prompt.
5. Under Credentials, create an sk-elp- key and call http://127.0.0.1:11434/v1.

Next

• Credentials & runner — gateway URLs, LAN HTTPS, accelerators
• Services — Open WebUI and other packs that consume the gateway