Hermes

Overview

Hermes is a multi-agent AI orchestration system, running in production on a VPS since 2025. Seven specialized agents — research, lead-gen, deep-research, growth, voice, orchestration, and CRM ops — each carry a distinct persona called a SOUL: hard behavioral rules, a specific toolset, a tool-call budget of roughly 35 calls per run, and review-gated output. Prompt engineering is treated as code — versioned and enforced, not loose text.

The distinction that matters: this is an AI system, not a product that uses AI. The core problems are agent coordination, semantic memory, and deciding when spending tokens is justified at all. It is private, in continuous personal production use — currently running an event-driven CRM and job-hunt automation — and it keeps evolving with its workloads.

Live walkthrough

Voice session to agent activity

live production control panel

Architecture

Agents communicate over the Agent Communication Protocol (ACP): persistent sessions, SSE streaming of deltas, tool calls, and plans, real-time cost tracking, and cold-start priming with the session transcript. Runs are long-horizon — 20 to 60 minutes — with tool loops, retry logic, and budget tracking, and every agent's output passes a review gate before it lands anywhere.

Cost is a first-class constraint. Deterministic gating in Python and shell checks state before any model is invoked: an empty inbox means exit with zero spend. When a run does fire, cliproxy routes it across providers — GPT models via ChatGPT Plus OAuth, DeepSeek, NVIDIA NIM, or local Ollama — swappable per agent without a restart, with per-session model overrides. Telemetry aggregates calls from agent logs through the cliproxy bridge plus ChatGPT Plus quota headers into a per-agent breakdown. Memory is PostgreSQL with pgvector and local embeddings (nomic-embed-text via Ollama); recall is shared across agents, with a kind/channel/source/derivation-layer taxonomy for provenance-aware retrieval.

Deployment is a two-server architecture that isolates the public surface:

• Main app — Node.js + TypeScript on native http, Postgres pool via pg, tailnet-only
• Webhook mini-server — exposed via Tailscale funnel, timing-safe HMAC signature validation
• Panel — Vite + React 18 + TypeScript, TanStack Query, SSE streaming, custom auth
• Voice slot — Piper TTS + Whisper STT, voice as a transport into the orchestrator
• Infra — Docker Compose, systemd timers; Linear GraphQL integration (issue creation + webhook receiver)

UNIQUE constraints and delivery-ID dedup make webhook processing idempotent — re-runs are safe by construction.

Screenshots

The panel's agent view, from the production deployment.

Stack & Decisions

Links

The repository is private — Hermes runs live workloads in production. The architecture, agent SOULs, and cost telemetry can be walked through in an interview. Get in touch.