The short answer
● What it is — Kinference, an inference control plane for vLLM
● Who — Buoyant, the company behind Linkerd
● Licence — Apache 2.0
● Vendor claim — up to 95% lower median time to first token
● Status — alpha, with real gaps
Buoyant open-sourced Kinference on 4 November, a control plane that sits in front of vLLM on Kubernetes. It watches KV-cache events and engine metrics, then routes requests across endpoints by policy rather than round-robining them like a generic load balancer.
That distinction is the whole point. Standard Kubernetes serving has no idea which replica already holds the prefix you are about to send. Kinference does, and routes accordingly. Buoyant claims up to 95% lower median time to first token against plain Kubernetes serving — a vendor benchmark with no independent replication yet.
It speaks OpenAI chat-completions format over gRPC, configures through Kubernetes CRDs and ships a kinfctl CLI for validating policies. It is alpha, and the gaps are stated: no Anthropic-style /v1/messages support, no rate limiting, no semantic classifiers and no multi-replica operation.
Sources: vLLM · AIwire.
Published 8 November 2026. Every figure above was checked against the reporting linked here on the same day.