The HuggingFace speech-to-speech project provides a low-latency, fully modular pipeline for building local voice agents. It chains VAD, STT, LLM, and TTS components while exposing an OpenAI Realtime-compatible WebSocket API. The architecture allows swapping any component, supporting hosted providers or fully local inference via vLLM and llama.cpp.
- Exposes OpenAI Realtime-compatible WebSocket API for easy client integration
- Supports fully local stacks using vLLM or llama.cpp for inference
- Modular design allows swapping VAD, STT, LLM, and TTS components
- Production-ready, used as backend for Reachy Mini robots