AI / ML

Production RAG systems face split-knowledge attacks where individually benign documents create false associations when combined. Existing per-document filters like LlamaGuard cannot detect these structurally invisible threats. TopoGuard addresses this by building a semantic similarity graph of retrieved documents to identify and mitigate these cross-document attacks.

• Split-knowledge attacks bypass per-document safety filters by combining benign inputs.
• TopoGuard uses graph theory to detect false associations across retrieved documents.
• Semantic similarity graphs are built to expose structural attack patterns.
• Current LLM safety tools are insufficient for multi-document context threats.

This paper analyzes preference-based post-training by decomposing parameter updates into spectral components, revealing a consistent head-tail organization. The dominant endpoint shift is carried by a compact spectral head that emerges early, while a heterogeneous residual tail remains. This framework allows preference-induced updates to be isolated, recomposed, and intervened on directly.

• Decomposes LoRA updates spectrally to reveal structural patterns in preference tuning.
• A compact spectral head drives the main behavioral shift and appears early in training.
• Heterogeneous residual tails remain distinct from the dominant head components.
• Updates become modular objects that can be isolated, recomposed, or intervened on.

This paper addresses the performance degradation that occurs when increasing compression ratios in Large Language Models. It proposes combining low-rank decomposition and quantization, challenging the assumption that these methods are orthogonal. The study investigates their interaction to achieve higher compression without significant accuracy loss.

• Existing compression methods hit a bottleneck where higher ratios cause sharp accuracy drops.
• Low-rank decomposition and quantization can be combined to overcome this limit.
• Researchers must understand the interaction between these two techniques, not treat them as separate.
• This approach aims to reduce memory and compute overhead while maintaining model fidelity.