Use local models with StremAI
Keep inference on your machine with Ollama, LM Studio, llama.cpp, Open WebUI, or another OpenAI-compatible server. StremAI adds recall before the model runs and stores useful outcomes afterward.
Local brain: agentbay local setup --local keeps memories in SQLite on this machine. No StremAI account is needed.
Shared brain: agentbay local setup --shared opens browser authorization, then StremAI() uses your authorized workspace automatically. The model remains local while connected agents can share context.
Running a local model does not automatically make memory local-only. Pick the memory mode deliberately.
Quick start with Ollama
ollama pull qwen3:8b pip install "stremai[local-models]" agentbay local setup
The guided setup detects your running model, asks whether memory should stay on this machine or be shared through StremAI, and verifies a memory store-and-recall before it finishes. Shared memory opens the same browser authorization used by connected agents; local-only mode never opens a browser.
from stremai import StremAI
brain = StremAI() # Uses the provider and memory mode selected during setup
response = brain.chat(
[{"role": "user", "content": "What decisions have we made about authentication?"}],
)
print(response.choices[0].message.content)For a fully offline setup with no prompt, run agentbay local setup --local. Use --shared in scripts to open browser authorization without an interactive choice.
Load a model in LM Studio and start its local server, then check it:
agentbay local setup --provider lmstudio
The setup saves the model ID it finds, so StremAI().chat(...) uses it automatically.
Start the OpenAI-compatible server exposed by llama.cpp:
agentbay local setup --provider llamacpp
The setup saves the loaded model name and verifies its memory connection.
Any OpenAI-compatible local server
This covers local servers such as vLLM, LocalAI, Jan, or a custom OpenAI-compatible endpoint.
agentbay local setup --provider openai-compatible \ --endpoint http://127.0.0.1:8000/v1/models
response = brain.chat(
[{"role": "user", "content": "Summarize the current project decisions."}],
provider="openai-compatible",
base_url="http://127.0.0.1:8000/v1",
model="your-model-name",
)Open WebUI
For an Open WebUI agent, add StremAI's Streamable HTTP MCP server in Open WebUI's MCP configuration:
https://stremai.com/api/mcp
Complete the browser authorization when prompted. That path gives the local-model agent tools for shared StremAI memory; grant only the project access it needs. For a custom Python agent that calls Open WebUI's local API, use the OpenAI-compatible example above.