The resulting host layout

I migrated log collection to Vector, split devstack compose files by host, and put Go defaults in one place across storage / desktop / compute. The resulting layout is below.

  desktop.home.arpa          compute.home.arpa          storage.home.arpa
─────────────────          ─────────────────          ─────────────────
agent-gateway :8080        vLLM :8000                 Vector
NATS :4222                 llama.cpp :8001             Loki :3100
reranker :8081             PostgreSQL :5432           Prometheus :9090
LM Studio :1234            Dagster :3300              MinIO :9000
                           MLflow :5050

desktop routes requests. compute handles inference, data processing, and experiment tracking. storage holds logs, metrics, and objects. The compose files follow these roles.

Background

The three hosts run agent-gateway. storage.home.arpa runs Prometheus, Loki, MinIO, and exporters 24/7. The macOS host desktop.home.arpa runs agent-gateway and NATS JetStream. compute.home.arpa starts on demand for GPU inference (vLLM, llama.cpp) and PostgreSQL, Dagster, and MLflow.

The use case is real-time Nikkei 225 prediction. It needs stable parallel event execution and memory-safe transforms, which fit Vector. I chose local infrastructure to control the machines and address concerns about cloud latency, data leakage, and resource changes. All components are containerized, so the setup can be reproduced with the hardware. For this AI system development environment, I addressed the following problems.

  1. Promtail outputs: it only tails logs for Loki, without NATS publishing or Prometheus metrics generation.
  2. One compose file: a single podman-compose.yml held every service, unlike the production host layout.
  3. Conflicting defaults: fixed hostnames and ports were also kept in .envrc, allowing the values to drift.

Phase 1: Promtail → Vector Migration

Motivation

Vector can send one input to Loki, NATS, and Prometheus metrics at the same time. A future option was to feed infrastructure logs into the Dagster data pipeline through NATS, which supported the move from Promtail.

Production Vector Setup

Vector was already running as a rootful systemd quadlet on storage.home.arpa. It used 19.3 MB of memory, peaking at 25.7 MB, and was a Prometheus scrape target.

The config has five sources:

  • journald—collecting only the current boot session with current_boot_only
  • file_security—alternatives.log, apport.log
  • file_apt—apt/dpkg logs
  • syslog_udp—UDP:1514, receiving syslog from MikroTik routers and others
  • internal_metrics—Vector’s own metrics

The route transform splits journald into kernel and systemd streams and adds job / host labels. Kernel logs match ._TRANSPORT == "kernel"; systemd logs also extract _SYSTEMD_UNIT and __UID__. Syslog supplies appname / severity / facility labels.

All transform outputs go to Loki as JSON. A Prometheus exporter exposes internal_metrics on :9598. External ports are limited to 9598 for scraping and 4222 for NATS clients.

Removing devstack Vector

A second Vector in devstack subscribed to NATS JetStream telemetry.> subjects, parsed JSON, and routed it by domain. storage collected logs while devstack consumed telemetry.

I considered three ways to connect them:

  • Option A: storage → NATS publish, flowing telemetry.infra.* to desktop
  • Option B: devstack → Loki sink, aggregating NATS telemetry into Loki as well
  • Option C: flow both directions so Loki and NATS both have complete data

I started with Option C, then moved aggregation out of the development stack. I removed devstack Vector and restored the storage config to journald + file + syslog → Loki + Prometheus exporter.

agent-gateway publishes to NATS in Go. Its goroutines send telemetry.* and pipeline.* events during request processing, and JetStream stores them. Vector stays focused on logs; CorrelationID connects the records.

Phase 2: Splitting devstack by Host

Motivation

The root podman-compose.yml held 8 services, including NATS, PostgreSQL, Dagster (3 containers), MLflow, Reranker, and minio-init. Production distributes them across three hosts, so development used different connection targets.

Split Result

I split compose into two files matching production:

  • devstack/desktop/ — NATS + nats-init + Reranker
  • devstack/compute/ — PostgreSQL + Dagster (x3) + MLflow + minio-init (for development)

desktop runs NATS with JetStream and a health check, nats-init to create PIPELINE / TELEMETRY streams, and Reranker.

On compute, dagster-user-code and dagster-daemon use NATS_URL: nats://desktop.home.arpa:4222. This replaces the old internal address nats://nats:4222. Cross-compose dependencies cannot enforce startup order, so I manage that operationally.

Dagster / MLflow Placement Decision

I considered putting only Dagster UI on desktop. Its webserver can be separated from user-code / daemon using remote gRPC endpoints in workspace.yaml.

The mlflow server command used here runs UI and tracking together. Dagster assets link to MLflow experiments, so I placed both services on compute. From desktop, I open http://compute.home.arpa:3300 (Dagster UI) and :5050 (MLflow UI).

minio-init Handling

I restored minio-init after deleting it because MinIO on storage.home.arpa still lacked the agw-mlflow / agw-iceberg buckets. This initialization job creates them. MLflow depends_on uses service_completed_successfully to wait for it.

Phase 3: Removing .envrc and Consolidating into defaults.go

Motivation

direnv supplied about 25 lines from .envrc at go run time, including COMPUTE_HOST, STORAGE_HOST, VLLM_BASE_URL, NATS_URL, and POSTGRES_DSN.

Hostnames, ports, and development secrets are fixed in this environment. A stale .envrc could conflict with config.go defaults, so I moved the normal values into Go.

Design

internal/config/defaults.go now holds the three hostnames, service ports, DB details, and log defaults as const values. In config.go, Load() uses four helpers—envOr / boolEnvOr / intEnvOr / optionalBoolEnv—and one return statement to build AppConfig.

Service addresses are built from the three hostnames:

  COMPUTE_HOST → vLLM, llama.cpp, PostgreSQL, Dagster, MLflow
STORAGE_HOST → Loki, MinIO, Prometheus
DESKTOP_HOST → NATS, Reranker, LM Studio

Individual URLs need no override in normal use. Environment overrides remain available, such as VLLM_BASE_URL=http://localhost:8000 go run ./cmd/server during development.

I mixed up ports twice during implementation. llama.cpp was set to 8081 and Reranker to 8001; the correct values are llama.cpp:8001 on compute and Reranker:8081 on desktop. I also changed DagsterBaseURL from desktop to compute. defaults.go comments identify the host for each port.

Overall Event Flow

There are two data paths:

  • Path A: agent-gateway goroutine → NATS publish → Dagster sensor. A non-real-time path used for request lineage tracking and data synthesis
  • Path B: storage Vector → Loki. A real-time infrastructure log aggregation path

Path A already runs through agent-gateway, so Vector does not need to publish to NATS. CorrelationID links the records from both paths.

Development now uses a layout close to production. Dagster on compute uses NATS_URL with nats://desktop.home.arpa:4222. Environment overrides allow the whole stack to run on localhost.