LLMs

Researchers introduce interventional grounding audits to test whether LLM chain-of-thought steps genuinely depend on stated premises. The black-box technique substitutes a target predicate in a single premise and checks if the model's normalized conclusion changes. Evaluated on ProntoQA with GPT-4o, the method achieved an F1 score of 0.806 in detecting proof-tree dependencies.

• Proves CoT reasoning may appear logical without true premise dependency.
• Uses black-box predicate substitution to isolate step-level causal links.
• Validated on ProntoQA with GPT-4o showing 0.806 F1 for dependency detection.
• Offers a concrete metric for auditing LLM logical grounding capabilities.

Research into attention-based KV cache eviction reveals that non-stationary filtering disproportionately retains structural noise over content. In schema-dense inputs like nested JSON, delimiters and whitespace carry significantly more attention energy than actual data values. This structural-role bias causes exact-match accuracy to collapse from 88% to 0% at a 5% compression budget due to signal degradation.

• Attention mass is a flawed metric for content retention in structured data contexts.
• Structural tokens like delimiters dominate KV cache, starving content tokens of space.
• Aggressive KV eviction (5% budget) can cause total accuracy failure on JSON schemas.
• Current eviction algorithms need structural-aware weighting to preserve signal integrity.

OriginBlame introduces a system that tracks author identity through data processing pipelines at the record and token level, addressing the gap in locating specific training data for removal requests. Unlike existing tools that operate at the file or dataset level and force over-deletion, this approach resolves revocation requests into precise forget sets via deterministic queries. Evaluation on nearly 220,000 Wikipedia pages shows it reduces dataset-level over-deletion from 101x down to 1.3x, with integration overheads ranging from 1.3% to 19% depending on the pipeline.

• Eliminates catastrophic over-deletion by pinpointing exact records for unlearning requests.
• Propagates author identity through pipelines to create deterministic forget sets.
• Adds 1.3-4.0% throughput overhead on HuggingFace and up to 19% on Datatrove.
• Addresses practical gaps in data contributor removal compliance for model trainers.

This paper addresses the engineering instability of scaling Generative Flow Networks to large language models, where learned prompt-conditional partition functions often cause gradient issues. The authors demonstrate that this partition function, previously considered essential for normalization, can be safely removed. This simplification reduces overhead and improves stability in post-training pipelines for reasoning models.

• Learned partition functions in GFlowNets cause gradient instability at scale, adding unnecessary engineering overhead.
• Replacing the partition function with a simpler alternative stabilizes training for large reasoning models.
• Distribution-matching RL remains viable for LLMs if the normalization component is decoupled from prompt conditioning.
• Simplifying the GFlowNet architecture enables better scaling for math and code generation tasks.

This paper analyzes on-policy distillation (OPD) in LLM post-training, identifying it primarily as an exploration catalyst that steers students toward correct reasoning paths via dense token-level guidance. The study reveals that prompt diversity outweighs per-problem sampling counts and that OPD effectiveness depends entirely on guiding signal quality. A critical Student-Teacher Mismatch pathology emerges when large distributional gaps between teacher and student models derail this exploration process.

• OPD acts as an exploration catalyst, not a capability ceiling expander.
• Prompt diversity is more critical than high per-problem sampling rates.
• Guiding signal quality is the single most important factor for success.
• Large teacher-student distribution gaps cause Student-Teacher Mismatch pathologies.
• Monitor distributional gaps to prevent exploration derailment during training.