POC #5 - FINDINGS

Verdict: GO, with a hard design constraint. Blind indexes work and are sargable, but leakage scales with field cardinality, and partial-name search cannot coexist with "no single admin can read names" on the operational store. The design must route low-cardinality / partial-match indexes into the vault trust domain, gated by Φ - not the operational DB an insider reads.

Setup: 100,000 synthetic parties, 30-surname UK pool, keyed-HMAC blind indexes, in the rls-poc postgres. Plaintext columns stand in for vault-held values.

---

1. What works (sargable, zero plaintext in the query)

Search	Mechanism	Plan
exact NHS number	WHERE nhs_bidx = hmac(input)	Index Scan on idx_party_nhs_bidx
surname starts-with "SMI"	prefix-token table, WHERE ptok = hmac('smi') → 1,687 hits	Bitmap Index Scan on idx_prefix_ptok

Both return the right rows without the plaintext being present or sent.

2. Leakage (the constraint)

Equality index - 100,000 rows → 30 distinct surname_bidx. The index reveals the surname frequency histogram 1:1 (top buckets 3597 / 3588 / 3540 …). An attacker with only the index + public UK surname frequencies re-identifies the common surnames.

Prefix index - far worse:

prefix len	distinct tokens	biggest bucket
1	15	20,873
2	24	7,029
3	27	7,029

A 1-char prefix token covers ~21k people (reveals first-letter distribution); the per-row prefix chain leaks surname length and shared prefixes → reconstructable via a trie + frequency.

The governing rule: leakage ∝ 1/cardinality.

• High-cardinality unique fields (NHS#, email): every bucket ≈ 1 → equality bidx leaks ~nothing. Safe.
• Low-cardinality fields (surname, sex, postcode-area) and any prefix/n-gram scheme: the index is effectively a partial plaintext disclosure. Not safe on the open store.

This is intrinsic - any deterministic searchable index leaks equality/structure. No hash choice fixes it; only access control + placement do.

3. Product reality (from the apps)

The member/employer UIs need partial name search ("Search by name or email…", "Search members…") - i.e. exactly the leaky case. Email is high-cardinality (OK via equality bidx); name partial-match is the conflict.

4. Recommendation (the decision this POC informs)

1. Exact high-cardinality identifier search (NHS#, email, exact DOB) → equality blind index in the operational store. Free, sargable, safe.

• (a) keep it, but store it in the vault trust domain and gate queries behind the same OPA Φ that authorizes reading names - leakage is then contained to roles that could already see names. (recommended for staff-facing search)
• (b) restrict that UI to exact-identifier search (change UX).
• (c) search a non-PII surrogate (member reference) instead of name.

2. Partial-name search → the prefix/n-gram index is sensitive-equivalent to the name itself. Options, decide per surface:
3. Never place a low-cardinality or prefix index in the same store an unprivileged insider (T3) can read - it defeats the at-rest confidentiality goal.

Tension to state plainly: cheap partial-name search and strong name-confidentiality are mutually exclusive on one store. Pick per surface; default to (a).

5. Scorecard

Threshold	Result
enumerate search UIs needing partial match	✅ "Search by name or email", "Search members"
equality blind index proven	✅ Index Scan, exact NHS/email
leakage of prefix scheme quantified	✅ 1-char prefix: 15 tokens, 20,873-row bucket; equality: 30 buckets reveal full histogram
recommendation written	✅ §4

---

Reproduce / source files

cd /root/blindindex-poc && docker compose up -d   # then run the harness (see README)

Source (raw):

• demo.sql