Local AI for Home Assistant: Ollama and MCP

Local AI Stack for Home Assistant: Deployment and Optimization

Deploying AI agents for Home Assistant without cloud services is achieved by combining Ollama, Whisper, OpenClaw, and n8n. This stack processes voice commands in 3–6 seconds on a GPU: transcription, intent analysis, and HA service calls. The key workaround for Llama 3.1 8B limitations is injecting HA context into the prompt instead of using tool calling.

Ollama runs models locally: phi3:mini (4 GB VRAM), llama3.1:8b (6–8 GB). On CPU, delays can reach 30 seconds; with CUDA, responses take 2–5 seconds. For external access: set OLLAMA_HOST=0.0.0.0:11434 in a systemd override and OLLAMA_API_KEY=ollama-local in .env.

Whisper base handles Russian language. On a P106-100 (6 GB VRAM, CUDA) — 10 seconds of audio in 0.5 seconds (19× realtime). On CPU — 25–30 seconds (0.4× realtime). This difference determines the assistant's responsiveness.

MCP Protocol and OpenClaw Integration

Model Context Protocol (MCP) provides AI access to HA entities via /api/mcp. OpenClaw is an agent with MCP support, connecting to Ollama and HA.

MCP setup is required before onboarding OpenClaw: Long-Lived Token in HA, activate MCP Server in integrations, config in openclaw.json. Deployment options:

• Addon in HA for simplicity.
• Docker on a separate host (NUC) if HA hardware is weak (RPi/Odroid).

Network checks: docker compose exec openclaw ping -c 1 <HA_IP>. Web interface from version 2026.2.21 — HTTPS only (port 8443 if conflicting: 8443:443). Pairing: devices list → devices approve <requestId>.

OpenClaw config for stability:

• gateway.bind: "lan" or "auto".
• tools.profile: "full", deny: ["message", "sessions_send"].
• contextWindow: 16384, maxTokens: 163840.
• tools.deny: ["web_search", "web_fetch"] to avoid 402 errors.

Orchestration via n8n and Tool Calling Workaround

n8n links Telegram → Whisper → Ollama → HA. Two workflows:

• Main: Telegram Trigger → Whisper → Ollama → HA API → response.
• Sub-workflow as an agent tool for HA calls.

Llama 3.1 8B has poor tool calling support: hallucinations, fails to call real tools. Solution — update-ha-context.sh script:

• Fetches HA data via mcporter (thermostats, temperature).
• Writes to TOOLS.md.
• Cron: /5 * /path/to/update-ha-context.sh.

The model sees current context in the prompt: "Temperature in the bedroom is 19.5°C (target 25°C)." Disable mcp-hass for 8B models.

Automation Scripts and Deployment

Automation for deployment and updates:

| Script | Purpose |

|--------|---------|

| update-server.sh | Copies .env, docker-compose, openclaw.json, mcporter.json, TOOLS.md. Runs docker compose pull && up -d. |

| update-ha-context.sh | Updates TOOLS.md with HA data (cron every 5 min). |

| test-openclaw.sh | Tests Ollama API (90-second timeout). |

| approve-device.sh | Device pairing. |

| pre-commit | Blocks commits of secrets (.env, tokens). |

Deployment: SSHPASS=password ./scripts/update-server.sh to [email protected].

mcporter requires mcporter.json in data/workspace/config/ (copied by script). Skill: npx clawhub install mcp-hass.

Architecture Across Three Hosts

Optimal load distribution:

• Proxmox VM (GPU GTX 1060): Ollama (llama3.1:8b), Whisper base, n8n. Fits in 6 GB VRAM.
• NUC: OpenClaw Docker (CPU-only).
• Odroid: Home Assistant.

Request flow: Telegram/UI → OpenClaw → Ollama (TOOLS.md with HA data) → response. No tool calling, no cloud.

Key Points

• GPU is essential for Whisper: 50× speedup (0.5 sec vs 30 sec).
• MCP before onboarding: otherwise OpenClaw won't see HA.
• Context in prompt for Llama 8B: update-ha-context.sh instead of tool calling.
• HTTPS and pairing for OpenClaw Control UI.
• Two workflows in n8n: main + agent tool.

— Editorial Team