LLMs

A new study reveals that large language model weights can be extracted from cloud inference endpoints using timing side channels. The attack exploits subtle latency variations in the model's response generation to reconstruct the underlying parameter data. This method bypasses traditional API access controls by leveraging the physical execution characteristics of the hardware.

• Timing side channels can leak proprietary model weights even with secure APIs.
• Physical execution latency correlates with specific model parameter access patterns.
• Current API rate limiting and access controls do not mitigate this extraction vector.
• Model providers should evaluate hardware-level noise and jitter as defense mechanisms.
• High-value models may need obfuscation or noise injection to prevent reconstruction.

An engineer shares results from building non-autoregressive decision models using reinforcement learning a year ago. The approach deviates from standard token-by-token generation by producing decisions in parallel. The post highlights experimental outcomes and technical hurdles encountered during development.

• Non-autoregressive architectures can be viable for decision-making tasks.
• Reinforcement learning provides a training signal for parallel decision models.
• Parallel generation may offer latency benefits over sequential token prediction.
• Experimental results suggest potential but also highlight significant challenges.

Moonshot AI's Kimi K3 is now generally available on Amazon Bedrock, offering a 2.8 trillion parameter open-weight model with native vision and a 1-million-token context window. The model delivers approximately 2.5x better scaling efficiency than its predecessor and supports long-running coding sessions and multi-document analysis. It operates within the same security boundaries as other Bedrock models, enabling safe integration into enterprise agent workflows.

• Kimi K3 features a 1M token context window, ideal for large codebases and document analysis.
• Native vision capabilities allow processing of scanned pages and screenshots alongside text.
• Runs within standard Bedrock security boundaries for consistent governance and compliance.
• Reports 2.5x scaling efficiency improvement over Kimi K2 for better cost-performance ratio.
• Supports extended agent workflows requiring long-horizon reasoning and memory.

Percona evaluated how well large language models handle specialized database administration workloads by building a custom evaluation harness. The study focused on the ability of these models to execute actual DBA tasks on remote systems rather than just generating code. This provides a practical benchmark for the current state of LLM capabilities in systems administration.

• Percona built a custom harness to test LLMs on real remote DBA tasks
• Focus is on practical execution ability rather than theoretical code generation
• Results offer a baseline for LLM readiness in specialized database roles