Sentinel Gateway - LLM Observability Firewall
An LLM gateway that sits between internal applications and AI providers — blocking prompt-injection attacks, redacting PII in flight, scoring completions for hallucination risk, and metering every call in PostgreSQL.
- Fastify
- transformers.js (ONNX)
- PostgreSQL 16
- React
- Recharts
SYSTEM BUILD
What went into it
- 01
Runs semantic guardrails fully in-process: a zero-false-positive regex PII engine plus local ONNX NER, with a seven-signal prompt-injection scorer enforcing allow/redact/block policy.
- 02
Proxies OpenAI, Anthropic, Gemini, and Groq dialects with byte-for-byte SSE streaming while still reconstructing messages to capture usage for per-model cost accounting.
- 03
Scores every completion asynchronously for hallucination risk using weighted embedding-cosine, lexical, and numeric-consistency signals — never adding client latency.
- 04
Stores only redacted, shape-preserving previews so raw PII never reaches the database, and keeps provider credentials server-side behind per-client gateway keys.
MEASURED OUTPUT
What changed
Centralized provider authentication and rotation so client applications swap provider secrets for revocable gateway keys, with per-key token-bucket rate limiting.
Verified by 46 gateway tests (unit, socket-level E2E with a mock upstream, Postgres integration) and 9 Playwright browser tests green against production builds.