AI / ML

CONDUIT addresses KV cache reuse failures in vision-language models when visual prefixes change, using a training-free residual-stream restoration approach. It improves upon raw-attention selection by avoiding wasted recomputation on irrelevant or low-value tokens through norm-weighted ranking. The framework unifies single- and multi-image reuse scenarios to maintain quality with a minimal visual-token budget.

• Eliminates need for retraining by using a training-free refresh policy.
• Unifies single- and multi-image KV cache reuse strategies.
• Prevents wasted compute on high-attention but low-value tokens.
• Maintains output quality under strict visual-token budgets.
• Restores residual streams to handle changed visual prefixes.

Researchers frame structured pruning of vision and language transformers as a multi-armed bandit problem to identify functional units with minimal performance impact. The method masks attention heads and MLP channel groups on calibration batches, calculating damage as the difference between masked and base loss to reduce variance. A smooth reward function guides a UCB or Thompson Sampling policy to sequentially select and zero out units in the dense checkpoint.

• Frames pruning as a damage-aware bandit problem for structured unit selection.
• Uses paired masked loss to neutralize batch-to-batch variance during evaluation.
• Applies UCB or fractional-Beta Thompson Sampling for sequential unit masking.
• Zeros out selected attention heads and MLP channels in the original dense model.

This study evaluates whether rewriting user queries into multiple variants improves Retrieval-Augmented Generation when the underlying search infrastructure is already robust. Using a fixed pipeline with BGE dense retrieval, cross-encoder reranking, and MMR diversification, the authors compared four rewriting strategies against HyDE and Query2Doc baselines across HotpotQA, AmbigNQ, and EnterpriseRAG-Bench. The results show that while individual rewriting methods are merely competitive with strong baselines, combining them delivers significant performance gains due to complementary failure modes.

• Single query-rewriting strategies offer limited gains over strong search baselines.
• Combining multiple rewriting methods yields outsized improvements in retrieval quality.
• Different strategies fail on different queries, making them complementary rather than redundant.
• Validation used paired-bootstrap significance tests across three datasets and seeds.

The RAPID method addresses the quadratic complexity of computing all example pairs in relational distillation by introducing a reliability-gated target and an adaptive pair proposal mechanism. It leverages calibrated teacher entropy and detached student-teacher residuals to determine which relations are evaluated, ensuring that the limited relation budget is used efficiently. The approach employs exact inverse proposal correction to maintain conditionally unbiased loss and gradient estimators.

• Reduces computational overhead by avoiding full all-pairs computation in mini-batches.
• Uses teacher entropy and residuals to gate which relational targets are emphasized.
• Ensures unbiased gradients through exact inverse proposal correction.
• Improves relation budget efficiency compared to uniform subsampling strategies.

CUSP addresses the reliability gap in aggregating heterogeneous vision-language models by mapping individual responses to a shared semantic space. It computes system-level collective uncertainty via the dispersion of a pooled semantic opinion and measures model conflict using Jensen-Shannon divergence. This training-free framework provides precise uncertainty signals without requiring additional model fine-tuning.

• Maps diverse VLM outputs to a unified semantic space for consistent comparison
• Quantifies system reliability via collective entropy and model-level conflict
• Uses Jensen-Shannon divergence to detect disagreement among agent opinions
• Requires no training or fine-tuning, enabling immediate deployment integration

Researchers introduce UniRRM, a unified reasoning reward model designed to address reliability challenges in open-ended reinforcement learning tasks. The model leverages MixReward, a large-scale multilingual dataset covering six domains and 103 languages, to support both pairwise and listwise evaluation paradigms. This approach aims to replace costly proprietary LLM-as-a-Judge systems and opaque scalar models with an interpretable, multilingual alternative.

• Replaces opaque scalar rewards with interpretable, generative reasoning models.
• Supports 103 languages across six domains via the new MixReward dataset.
• Handles both pairwise and listwise evaluation paradigms in a single framework.
• Reduces dependency on expensive proprietary LLM-as-a-Judge systems.

Amazon SageMaker AI now supports EC2 G6e instances for offline batch inference jobs. These instances feature up to eight NVIDIA L40S GPUs with 48 GB of memory each and third-generation AMD EPYC processors. This update targets GPU-intensive workloads such as large language models and generative AI for images, video, and audio. Users can enable this capability by selecting ml.g6e instance types via the AWS SDK or CLI.

• Use G6e instances for high-throughput, offline inference on large S3 datasets.
• L40S GPUs with 48 GB VRAM improve performance for LLMs and diffusion models.
• Select ml.g6e instance types in CreateTransformJob to enable this support.
• Optimized for batch workloads that do not require persistent endpoints.