LLMs

DeLS-Spec improves speculative decoding by pairing a fixed DFlash block-parallel drafter with a lightweight local head. This decoupled approach maintains intra-block causal conditioning without requiring the draft model to be trained from scratch, addressing limitations in methods like Domino and DSpark. The method treats the existing model as a long-context expert while the new head handles short-context specifics.

• Avoids costly full retraining of draft models by keeping DFlash fixed as the long-context expert.
• Introduces a lightweight local head to enforce intra-block causal conditioning in block-parallel drafting.
• Reduces training costs while improving flexibility compared to scratch-trained methods like Domino.
• Enables parallel verification of token blocks with better local context awareness.

PALS introduces percentile-aware layerwise sparsity for large language model pruning, deviating from uniform approaches by adjusting sparsity per layer based on the 99th percentile of activation magnitudes. The method bounds adjustments to ±5% around the target ratio, resulting in a 0.96 perplexity improvement on WikiText-2 for LLaMA-2-7B compared to Wanda. However, benefits are architecture-specific, with LLaMA-3-8B showing marginal gains and Mistral-7B showing none. The authors also note that gradient-based allocation strategies performed worse than random, challenging their assumed efficacy.

• PALS uses 99th percentile activation magnitudes to set per-layer sparsity, improving LLaMA-2-7B perplexity by ~0.96 over uniform Wanda.
• Gains are architecture-dependent: LLaMA-3-8B sees marginal improvement, while Mistral-7B shows no benefit.
• Gradient-based sparsity allocation performed worse than random, suggesting magnitude may be a more reliable proxy than gradients.
• Adjustments are tightly bounded (±5%), limiting deviation from the global target sparsity ratio.

TF-Engram addresses the collision issues of GPU-resident engram memory by storing phrase-specific semantic tables across a GPU-DRAM-SSD hierarchy. The system operates without training, constructing offline memory from external corpora to avoid parameter-heavy updates. It employs Early-Exit Guided Predictive Prefetching to mitigate the latency penalties associated with accessing external storage.

• Eliminates training costs by using offline corpus construction for semantic memory.
• Prevents phrase collision by storing specific semantic memories rather than hashed slots.
• Extends memory capacity beyond VRAM limits using SSD-backed hierarchical storage.
• Hides I/O latency through Early-Exit Guided Predictive Prefetching mechanisms.

A new study compares pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence metrics in large language models. The research finds that confidence estimates generated after the model completes its reasoning process are consistently better calibrated and more discriminative than those made beforehand. This temporal distinction highlights the limitations of treating confidence as a static property of the final output alone.

• Post-solution JOL metrics outperform pre-solution FOK for reliability estimation.
• Temporal evolution of confidence matters for accurate system decision-making.
• Better calibration enables safer retrieval and tool-use triggers.
• Frontier and open-source models both show this post-solution advantage.