Vivek Keshava

LLM inference gateway that admits requests on estimated token cost rather than request rate — a 200-token classification and a 50k-token summarization are one request each to an RPS limiter, but differ ~100x in cost. Deficit Round Robin fair queueing across tenants with the deficit counted in tokens, and queue depth derived from a latency SLO via Little's Law. Under ~1.6x overload, p99 time-to-first-token measured 2,730 ms against 14,256 ms for a request-rate limiter.

// Java 21, Spring WebFlux, Reactor, Redis (Lua), Resilience4j, Micrometer, Gatling

Durable execution runtime for LLM agents. Journals every model and tool call to Postgres so a recovering worker replays completed steps instead of re-running them, and puts side effects behind a content-keyed idempotency barrier. Lease-based distribution via SKIP LOCKED with fencing tokens that reject zombie-worker writes, plus suspend/resume that holds zero process state. A chaos harness SIGKILLing workers at random offsets — 100 runs, 60 kills — recorded zero duplicate side effects.

// Python, PostgreSQL, asyncio, asyncpg, pytest, Hypothesis

Production-ready MCP server built with the TypeScript MCP SDK, Zod validation, and the GitHub REST API, exposing repository intelligence as structured tools for AI agents and LLMs. Implements repo health scoring, PR risk classification, and workflow bottleneck detection — deployed as a live integration in Claude's ecosystem.

// MCP SDK, GitHub REST API, Zod, TypeScript, Python

LLM-powered document parser and authenticator: extracts structured fields from document images (licenses, passports) as JSON with per-field confidence scores, validates authenticity, and flags anomalies with over 90% accuracy — cutting manual verification by 60%. Ships a Gradio UI and a one-shot CLI with defensive JSON parsing and env-based key management.

// Gemini API, Llama, DeepSeek, Python, Gradio