LLMs

Sparse Mixture-of-Experts models face a routing bottleneck when offloading inactive experts to host memory, as transfers can only begin after top-K routing completes. SpecPrefetch introduces a lightweight shared adapter to asynchronously predict next-layer expert candidates, decoupling prediction from execution. This approach allows expert data to be fetched in parallel with routing, mitigating the serialization delay inherent in current offloading strategies.

• Decouples expert loading from routing to eliminate serialization bottlenecks.
• Uses a parameter-efficient shared adapter for asynchronous prefetching.
• Enables faster inference on memory-constrained accelerators.
• Reduces latency by overlapping data transfer with computation.

GLIDE addresses the KV cache bottleneck in long-context LLM inference by combining sliding-window softmax attention with linear recurrent aggregation. The method leverages layer-wise heterogeneity, using full softmax in early sensitive layers and efficient linear recurrence in deeper redundant layers. This adaptive balance reduces memory I/O and computational costs during decoding without significant accuracy loss.

• Hybrid architecture mixes softmax and linear attention per layer to optimize throughput.
• Early layers retain high-sensitivity softmax while deeper layers use low-cost recurrence.
• Reduces KV cache memory footprint and decoding latency for long-context generation.
• Adaptive layer-wise balancing offers a practical path to efficient large-scale inference.

LiquidAI has introduced LFM2.5-Encoders, designed to accelerate long-context inference directly on CPU hardware. This release targets practitioners needing efficient processing without relying on GPU resources. The models aim to reduce latency and improve throughput for large context windows in standard server environments.

• Enables fast long-context inference on CPU, reducing GPU dependency
• Optimized for efficiency in standard server environments
• Targets latency reduction for large context window processing
• Available via Hugging Face for immediate integration

LivingArena is an automated evaluation framework that addresses static benchmark contamination and saturation by having LLMs generate questions specifically designed to exploit the weaknesses of other models. In this adversarial setup, one model acts as a questioner seeking to stump an opponent, while the other must answer correctly to earn rewards. This dynamic approach aims to distinguish top-tier models by revealing specific failure modes that human preference or static datasets might miss.

• Moves beyond static benchmarks to combat data contamination in frontier LLM evaluation.
• Uses adversarial peer-probing where models actively exploit each other's knowledge gaps.
• Provides a scalable, automated method to identify specific model failure modes.
• Rewards questioners for stumping opponents and answerers for correct responses.

ReMem addresses the context window limits of Multimodal LLMs in long video understanding by replacing uniform keyframe sampling with a temporal granularity-adaptive framework. It operates without training, using a dual-level memory system to parse question intent and extract relevant semantic entities. This approach dynamically adjusts which frames are selected based on the specific temporal scope required by the query, improving accuracy over static methods.

• Eliminates training overhead while improving long video QA performance via adaptive selection.
• Uses LLM long-term memory to decode the temporal granularity of user questions.
• Outperforms uniform or static query-guided keyframe sampling techniques.
• Focuses extraction on semantic entities relevant to the specific query context.

A new prompt-level method called the Cognitive Kernel Model (CKM) addresses LLM instability by forcing models to categorize inputs into Fact, Heuristic, and Emotion states before generating a response. This structured state tracking aims to reduce inconsistent answers and decision reversals without requiring model weight changes. The approach treats behavioral consistency as a measurable property by explicitly separating verifiable data from inferred assumptions and evaluative signals.

• CKM operates at the prompt level, requiring no model retraining or weight updates.
• Models must explicitly tag inputs as Fact, Heuristic, or Emotion prior to decision-making.
• This state enforcement reduces answer variance and prevents context-induced decision flips.
• Consistency is treated as a structural constraint rather than a generative capability.
• Evaluates whether explicit epistemic role separation improves behavioral stability.