LLMs

New research introduces Chopthin-Consensus Power Sampling (CCPS), an inference-time decoding method that enhances LLM reasoning without post-training. Unlike standard Sequential Monte Carlo approaches that use equal-weight resampling and aggressively prune low-probability paths, CCPS uses the Chopthin resampler to cap the weight ratio between particles. This technique carries unequal weights forward, maintaining genealogical diversity and preventing the loss of potentially correct reasoning trajectories.

• CCPS improves LLM reasoning at inference time without requiring model retraining or fine-tuning.
• It replaces aggressive equal-weight resampling with a bounded weight ratio strategy to preserve diversity.
• Unequal weights are carried forward, reducing the risk of discarding valid reasoning paths too early.
• The method mitigates the pruning problem inherent in standard Sequential Monte Carlo decoding.

This paper introduces EAR, an approach that replaces fixed-size text chunks with entity-aware partitioning for Retrieval-Augmented Generation. By extracting normalized surface anchors from questions, answers, and the corpus, EAR retrieves local windows around matching terms and optionally attaches a parent passage via extractive summary. Evaluated on a cleaned MMLU-style subset, the method aims to resolve the ambiguity often found in standard chunking strategies.

• Replaces fixed-size chunks to improve retrieval precision for multiple-choice QA.
• Uses normalized surface anchors from queries and corpus to find relevant content.
• Attaches larger parent passages via extractive summary for broader context.
• Validates approach on a curated 153-question MMLU-style test set.

R2VC addresses the diagnostic opacity of end-to-end LLM fact-checking by decoupling retrieval, reasoning, and confidence estimation. The architecture combines hybrid sparse and dense Wikipedia retrieval with a supervised, DPO-aligned generator to produce structured verdict candidates. An external NLI cross-encoder selects the best candidate, while a lightweight calibrator estimates confidence to enable selective abstention when uncertainty is high.

• Modular design isolates retrieval, reasoning, and uncertainty for easier debugging and improvement.
• Hybrid sparse+dense retrieval over Wikipedia provides robust evidence grounding.
• DPO-aligned generator produces diverse, structured verdict candidates for better selection.
• External NLI cross-encoder ensures evidence-based candidate filtering.
• Lightweight calibrator enables selective abstention to prevent hallucinated confidence.

This study compares scaling trends between distilled byte and token models by introducing two methods to convert token logits to byte logits: an approximate approach called Marginalize-It and an exact method named End-Of-Token. The authors conduct a large-scale analysis of decoder-only dense transformers, varying tokenization schemes and training objectives across models ranging from 1 billion parameters to 1 trillion bytes of data. The work aims to determine if distilled byte models exhibit similar scaling behaviors to their tokenized counterparts as compute and dataset size increase.

• Introduces Marginalize-It for approximate token-to-byte logit conversion.
• Presents End-Of-Token as an exact alternative for logit conversion.
• Compares scaling trends of distilled byte vs. token models at scale.
• Analyzes decoder-only transformers up to 1 trillion bytes of data.
• Evaluates impact of distillation vs cross-entropy on byte-level training.

Researchers introduce SynthSentry, a method to identify synthetic data in training corpora without needing access to the generating model or its history. The tool calculates a contamination score by measuring distributional divergence across lexical diversity, n-gram tail truncation, and perplexity variance. This approach aims to prevent model collapse by screening data provenance before the training phase begins.

• SynthSentry flags synthetic data without requiring generating model access or labels.
• It measures lexical diversity collapse and n-gram tail truncation as key signals.
• Perplexity variance across reference models helps quantify contamination levels.
• Pre-training screening addresses model collapse before distributional tails deteriorate.