Cost

The LLM Gateway: How We Cut AI Running Costs by 80% and Survived Multi-Hour Provider Outages

One layer in front of every AI request, choosing which model answers it. It kept 700+ employees working through outages at two providers and cut AI running costs by up to 80%. The architecture, the trade-offs and the lessons.