Technical notesAI Infrastructure
Overview
LLM Faucet gives developers one OpenAI-compatible API surface for model capacity that is otherwise fragmented across providers. It focuses on honest availability: route requests to usable capacity, stream responses, fall back when a provider is unavailable, and make limits visible.
What I built
- OpenAI-compatible
/v1chat-completions surface - Provider-capacity aggregation and model selection
- Smart routing with fallback behavior
- Streaming responses for interactive clients
- Transparent availability and usage limits for coding tools
Stack
OpenAI-compatible APIs, TypeScript, Cloudflare Workers, edge routing, streaming, and provider adapters.
What I learned
Infrastructure products earn trust through predictable failure behavior. Compatibility is only the beginning; routing decisions, limits, and fallbacks must be understandable to the people building on top of them.
