POC #3 - FINDINGS
Verdict: GO, with enormous margin. OpenBao Transit is nowhere near a bottleneck on the vault detok path. Sustained ~24k single unwraps/s and ~400k batched unwraps/s on dev-class hardware - vs a target of ≥300/s. Transit is not the thing to worry about; the vault's own DB lookup + audit write will dominate (sized elsewhere).
Setup: OpenBao 2.0.2 (same as prod olly-openbao), dev mode, transit KEK pii. Unwrap = POST transit/decrypt/pii of a wrapped 32-byte DEK. Harness: Go, stdlib HTTP.
1. Throughput & latency
| Shape | conc | calls/s | unwrap ops/s | p50 | p95 | p99 |
|---|---|---|---|---|---|---|
| single | 64 | 24,306 | 24,306 | 2.31 | 5.37 | 7.38 |
| single | 128 | 26,242 | 26,242 | 3.88 | 11.7 | 17.0 |
| batch-20 | 64 | 19,445 | 388,894 | 2.82 | 7.01 | 9.86 |
| batch-20 (sustained, 8M unwraps / 20s) | 64 | 20,226 | 404,511 | 2.75 | 6.80 | 9.27 |
Unloaded (c=16): p50 0.82ms, p99 3.4ms. 0 errors throughout.
- Target ≥300 unwrap/s @ p99<10ms: ✅ by 80×-1300×.
- Batch is the right detok shape: one HTTP call unwraps 20 ciphertexts at ~the same per-call latency, so a detail-view detok of ~20 fields ≈ a single ~3ms p50 call.
2. CPU / mem (saturation)
At ~400k unwrap/s the container used ~920% CPU (≈9 cores), ~80 MiB RAM. That is the ceiling. Our operating point (~60-300 unwrap/s) is <0.1% of it → CPU at real load is a rounding error. We would need ~1000× projected load before Transit is CPU-bound.
3. Scorecard
| Threshold | Result |
|---|---|
| sustained ≥300 unwrap/s, p99 < 10ms | ✅ 24k/s single (p99 7.4ms), 404k/s batched (p99 9.3ms) |
| Bao CPU headroom | ✅ negligible at load; saturates ~9 cores only at 400k/s |
| bulk-of-20 path | ✅ batch_input, 1 call = 20 unwraps, same latency |
4. Validity / caveats
transit/decryptis pure crypto + token check - no storage write - so dev-modeinmemstorage is representative for the unwrap path. (Storage matters for key rotation/encryptwith convergent, not for decrypt.)- Used the root token; production AppRole token validation is cached, similar cost.
- This is Transit only, not the whole
pii-vault. The vault adds, per detok: a Postgres lookup of{ct, wDEK}, the audit-row write, and the OPAΦcheck. Transit shows ~3ms; the audit write (~1ms sync) and the DB round-trip will dominate - that composed cost is POC #4 / the vault service build, not here. - Single-unwrap p99 grows (7→17ms) from c=64→128 as the dev container saturates; at the real operating point latency stays at the unloaded ~0.8ms p50 / ~3ms p99.
Implication for the design: keep the per-record-DEK
- Transit-wrap envelope (C1) as specified; batch unwraps per detail view (already mandated by I4). No caching of DEKs is needed for throughput - which is good, since DEK caching would reintroduce key exposure.
Reproduce / source files
cd /root/transit-poc && docker compose up -d # then run the harness (see README)Source (raw):
