Infrastructure: Environments and Topology
Olly runs the same service code in three environments. Each environment has different backing services and operational characteristics. This page is the entry point; follow the per-environment pages for details.
Environments at a glance
| Environment | Purpose | Topology | Doc |
|---|---|---|---|
| Local dev | Single-developer hacking on services | Docker Compose + Tilt + Go binaries from go.work, all on one workstation | Local Development |
| Shared dev (Hetzner) | Team-shared dev cluster | Four Hetzner VMs (dev-1, dev-2, ci, uat) on a VPC at 10.0.1.0/24; mosh/ssh access | (see below - Shared dev) |
| Production (GCP) | Customer-facing | GKE Standard + Cloud SQL + Memorystore + Strimzi + GCLB in olly-prod project, europe-west2 | Production (GCP) |
Local dev - summary
Docker Compose brings up the infrastructure stack (Postgres, Kafka via Redpanda, Keycloak, OpenSearch, Temporal, OpenBao, OTel). Go services run from the go.work workspace via make run-all, or hot-reloaded by Tilt. Every domain points to *.localhost aliases routed through APISIX. See Local Development for prerequisites and a full step-by-step.
Shared dev (Hetzner) - summary
Four Hetzner VMs in a single VPC (10.0.1.0/24). dev-1 hosts services currently under active development; dev-2 hosts shared infra. CLAUDE.md in the repo root is the authoritative description; this page summarises.
| Host | Public IP | VPC IP | Role |
|---|---|---|---|
| dev-1 | 135.181.39.23 | 10.0.1.1 | Active-dev workloads: triage, open-webui, nhs111, nginx (TLS termination for *.dev.hiolly.com) |
| dev-2 | 204.168.197.175 | 10.0.1.2 | Shared infra: postgres, valkey, kafka, opensearch, keycloak, temporal, openbao, otel, langfuse, mirth |
| ci | 204.168.206.223 | 10.0.1.3 | CI runners + scrape pool |
| uat | 204.168.192.32 | 10.0.1.4 | Novu (notifications) + UAT-clone of Go stack |
Cross-host wiring uses VPC IPs (10.0.1.2:<port>), never container hostnames. New shared-infra services bind to 10.0.1.2:<port>:<port> not 127.0.0.1:.
Production (GCP) - summary
GCP migration is in flight. The production data plane is live (Cloud SQL, Memorystore, GKE, Strimzi Kafka). Application workloads still run on shared dev and migrate to GKE per the sub-project roadmap. See Production (GCP) for the GCP target architecture and /gcp/MIGRATION_STATUS.md for current cutover state.
| Layer | GCP service | Status (2026-06) |
|---|---|---|
| Compute | GKE Standard private cluster olly-prod-ew2 (europe-west2) | live |
| Postgres | Cloud SQL prod-pg-16 (PG16, regional HA, CMEK) | 22/22 DBs migrated |
| Cache | Memorystore Valkey prod-valkey (BASIC, TLS+AUTH) | live, empty (source had 0 keys) |
| Kafka | Strimzi 4.0 KRaft on GKE | live, 8 topics |
| Search | OpenSearch helm on GKE | up, indexes regenerating |
| Secrets | GCP Secret Manager | replacing OpenBao |
| Gateway | GCLB + Cloud Armor | replacing APISIX (sub-project #5) |
| DNS / TLS | Cloud DNS + Google-managed certs | replacing Let's Encrypt (sub-project #5) |
| Observability | Cloud Logging + Managed Prometheus (interim); Tempo/Loki/Grafana planned for sub-project #6 | partial |
Backing-services parity
What's the same across environments and what differs:
| Concept | Local dev | Shared dev (Hetzner) | Production (GCP) |
|---|---|---|---|
| Postgres | docker-compose postgres:16 | dev-2 olly-postgres @ 10.0.1.2:5432 | Cloud SQL prod-pg-16 @ 10.10.49.2 |
| Cache | docker-compose valkey:7 | dev-2 olly-valkey @ 10.0.1.2:16379 | Memorystore Valkey @ 10.10.48.3:6378 |
| Kafka | docker-compose Redpanda | dev-2 olly-kafka @ 10.0.1.2:9092 | Strimzi on GKE |
| Keycloak | docker-compose keycloak:24 | dev-2 olly-keycloak :8093 | GKE deploy in sub-project #4 |
| Search | docker-compose OpenSearch | dev-2 olly-opensearch @ 10.0.1.2:9200 | OpenSearch helm on GKE |
| Temporal | docker-compose | dev-2 olly-temporal :7233 | GKE deploy in sub-project #4 |
| Secrets | docker-compose OpenBao | dev-2 olly-openbao :8200 | GCP Secret Manager |
| Gateway | docker-compose APISIX | dev-1 nginx + APISIX | GCLB + Cloud Armor |
| docker-compose Mailpit | dev-2 olly-mailpit :8025 | SES via Novu integration | |
| Observability | docker-compose OTel + Jaeger | dev-2 OTel + Tempo + Loki + Grafana | Cloud Logging + Managed Prometheus (interim) |
| Notifications | direct Novu (uat) | dev-1 nginx proxies to uat Novu | GKE Novu (sub-project #4) |
What changes per environment
- Auth: local dev uses a permissive Keycloak realm seeded by
make local-up; shared dev + prod use the canonicalolly-members/olly-providers/olly-internalrealms. - TLS: local dev is HTTP-only (
.localhost); shared dev uses Let's Encrypt via nginx; prod uses Google-managed certs via GCLB. - Network: local dev is loopback; shared dev uses Hetzner VPC; prod uses Google VPC
prod-vpc-ew2+ Cloud NAT for egress. - Observability: local dev logs to stdout + local Jaeger; shared dev to dev-2 Grafana stack; prod to Cloud Logging + Managed Prometheus.
When NOT to use which environment
- Local dev: solo iteration, no shared dependencies on others' work.
- Shared dev: cross-service integration, peer review, demo to non-engineers.
- Production: customer-facing only - never demo from prod.