DoorDash engineers describe their transition from vendor-dependent setups to an internal GenAI platform utilizing open-weights models. The architecture addresses the complexities of LLM and agent gateways while optimizing for accuracy, latency, and cost. This infrastructure now supports over 5,000 internal users with a focus on balanced performance metrics.
- Shift from vendor-first to open-weights models for greater control and cost management.
- Architectural focus on balancing latency, accuracy, and cost for large user bases.
- Implementation of dedicated LLM and agent gateways to manage internal traffic.
- Platform scaled to support over 5,000 internal employees effectively.