Ben O'Mahony presents a method for building custom AI-powered Language Server Protocols that leverage OpenTelemetry to capture implicit user feedback. By tracking concrete actions like accepting or dismissing code fixes, the system creates a continuous data flywheel. This approach allows organizations to distill capabilities from expensive frontier models into cheaper, local Small Language Models (SLMs).
- Instrument AI agents with OpenTelemetry to capture implicit labels from user actions.
- Track specific interactions like accepting, dismissing, or regenerating code fixes.
- Use the resulting data flywheel to distill frontier model behavior into local SLMs.
- Reduce inference costs by replacing expensive models with distilled, cheaper alternatives.