AI / ML

This paper characterizes the serving behavior of masked diffusion language models (dLLMs) under concurrent load, using LLaDA-8B with a D2F adapter on an NVIDIA H200. It highlights that dLLM serving cannot simply replicate autoregressive assumptions. The study reveals that request difficulty manifests as discrete denoising steps rather than a continuous variable, challenging existing infrastructure designs.

• dLLMs denoise tokens in parallel, offering potential speedups over autoregressive models.
• Request difficulty is discrete (step count), not continuous, affecting scheduling.
• Existing AR serving infrastructure assumptions do not hold for dLLMs.
• Evaluation used LLaDA-8B with D2F LoRA on a single H200 GPU.
• Real-world load characterization is critical for designing dLLM serving systems.

Research shows that local coding LLMs can hallucinate non-existent Python package names, allowing adversaries to register these names on PyPI and execute supply chain compromises, a threat termed 'slopsquatting'. To mitigate this, the authors propose a two-layer detection system combining deterministic PyPI existence checks with a Random Forest classifier. This solution includes an import name reconciler to handle discrepancies between import statements and package names, integrated into a LangGraph state machine for robust retry logic.

• Local LLM hallucinations create real supply chain risks via pre-registered PyPI names.
• A two-layer detector uses deterministic checks and Random Forest classification.
• Import reconciler bridges gaps like 'import cv2' vs 'pip install opencv-python'.
• LangGraph integration enables escalating temperature retries for robust validation.

Researchers introduce the Actionable Policy schema, a YAML-based format for specifying content-based constraints in generative AI applications. This approach enables exception-based governance, allowing organizations to track policy violations rather than relying on rigid access control models. The work also includes a synthetic data generation pipeline to support proactive policy enforcement.

• YAML schema defines precise content constraints for GenAI model responses.
• Enables exception-based governance to track and manage policy violations.
• Addresses limitations of traditional access control in GenAI contexts.
• Includes synthetic data pipeline for proactive policy enforcement.

Giga-Embeddings introduces a 10B-parameter sparse Mixture-of-Experts encoder that activates only 1.8B parameters per token, balancing retrieval quality with high throughput. Benchmarks show it processes 114.5k tokens per second on 1024-token inputs, outperforming dense 3B models by 25% and external systems by up to 2.65x. The model family also includes a 3B dense and 480M distilled variant for lower compute budgets, achieving top aggregate scores across English, multilingual, and code MTEB suites.

• 10B MoE encoder uses 1.8B active params/token, enabling high throughput without full model load.
• Processes 114.5k tokens/sec on 1024-token inputs, exceeding dense 3B models by 25%.
• Outperforms evaluated external systems by 1.56-2.65x in vLLM benchmarks.
• Achieves strongest aggregate performance on English, Russian, multilingual, and code MTEB suites.
• Family includes 3B dense and 480M distilled models for tighter compute and memory constraints.

A new arXiv paper identifies that standard Reinforcement Learning recipes for LLM instruction following suffer from exploration bias when prompts contain multiple instructions. The policy model fails to explore hard instructions because its initial capability is too low to trigger successful exploration, and cumulative reward structures incentivize satisfying easy instructions first. This results in suboptimal training where the model optimizes for low-hanging fruit rather than overall capability.

• Multi-instruction RL training currently favors easy tasks due to cumulative reward structures.
• Initial model weakness on hard instructions prevents successful exploration during RL.
• Standard RL recipes may need rebalancing to prevent bias toward simple prompts.
• Researchers should audit reward signals when training on complex, multi-step instructions.
• This bias limits LLM robustness in scenarios requiring adherence to all prompt constraints.