Models
Where 3AM's models run. Your own endpoint, OpenRouter, or a GPU in the server, never the CPU. How to choose, and how to set each one up.
3AM uses two models. Neither one ever acts on your systems.
- The investigator digs into incidents that 3AM's checks don't settle, and writes the incident note. We test with Llama 3.3 70B Instruct.
- System 1 is a small model (tev1, 4B) that checks every cause, fix and sentence against the evidence before 3AM uses it.
They run on a GPU, or behind an endpoint you point 3AM at. Never on the server's CPU: there, every investigation would wait minutes on each model step.
Which to choose
| Your own endpoint | OpenRouter | A GPU in the server | |
|---|---|---|---|
| Data leaves your network | No | Yes, masked | No |
| You run | A model server | Nothing | A GPU server for 3AM |
| Models | Any, Llama 3.3 70B recommended | The ones 3AM is tested on | Qwen2.5 7B, smaller and weaker |
| Best for | Production | Trials, and teams without GPUs | Labs |
With no GPU in the server, the installer asks which endpoint to use. You can change it later by running the installer again with different options.
Your own endpoint
Any server that speaks the OpenAI chat completions API works: vLLM, Ollama on a GPU server, or an internal LLM gateway. The model needs a context of 32k tokens or more, and must be able to answer in JSON.
./install.sh --model-url https://llm.example.internal/v1 --model meta-llama/Llama-3.3-70B-Instruct \
--model-key-file ~/llm.key \
--s1-url https://llm.example.internal/v1/systemone--model is the name your endpoint lists at /v1/models. Leave out --model-key-file if the endpoint doesn't need a
key.
System 1 needs its own endpoint: tev1 served by Ollama (/v1/systemone) on a GPU, or OpenRouter's decisions
endpoint with an OpenRouter key:
--s1-url https://openrouter.ai/api/alpha/decisions --s1-key-file ~/openrouter.keyOpenRouter
Create an API key at openrouter.ai, save it to a file, and give it to the installer:
umask 077; printf '%s' 'sk-or-…' > ~/openrouter.key
./install.sh --openrouter-key-file ~/openrouter.keyBoth models then run on OpenRouter, with the same key. To use only providers that don't store or train on your data,
add --model-extra '{"provider":{"data_collection":"deny"}}'.
A GPU in the server
An NVIDIA or AMD GPU with 8 GB of memory or more. Containers must be able to reach it, so install the NVIDIA Container Toolkit (or AMD ROCm) first. The installer finds the GPU and runs the models on it, with no options needed. With 24 GB or more, both models run side by side with room to spare.
Common questions
The evidence for one investigation: alerts, check results, log lines and metrics. Secrets, tokens, passwords and card numbers are masked before anything is sent, and every call is in the audit log with the masked prompt. What a hosted model sees
No. Through OpenRouter, 3AM uses the models it's tested on. To use another model, serve it from your own endpoint. 3AM works best with Llama 3.3 70B; smaller or older models give weaker investigations.
Only in a lab. --profile none --allow-no-models keeps the checks working, but nothing is investigated or verified
by a model.
./install.sh --doctor checks that both models answer. The console's Doctor page shows the same.
The chart's externalModel.* and systemOne.* values. See Kubernetes → Models.
Air-gapped sites
Install 3AM where servers can't reach the internet. Download a signed release elsewhere, check it, carry it in, and install it with nothing fetched.
Sizing
How big the 3AM server should be, why it can be small, what limits how many incidents it works at once, and GPU sizing for running the models yourself.