Agentic AI

VEPA addresses the inconsistency of LLM-based safety checks by enforcing policies through deterministic formal verification. This approach ensures inspectable and reproducible allow-or-refuse decisions before an agent executes potentially irreversible actions on sensitive resources. The system is designed to adapt dynamically, updating its formal policy definitions as new security risks emerge during deployment rather than relying solely on model inference.

• Replaces probabilistic LLM safety inference with deterministic formal verification for consistent decision-making.
• Provides inspectable and reproducible audit trails for agent actions on mutable system states.
• Supports evolving policy updates to handle emerging risks without full retraining.
• Reduces ambiguity in safety interpretations by decoupling logic enforcement from model generation.

Researchers propose SWE-Duel, a dynamic evaluation arena where coding agents generate their own tasks at runtime to prevent benchmark contamination. The system pits agents against each other in mirrored turns, with one implementing features and the other attempting to find or validate subtle bugs. By creating content only during evaluation, the benchmark aims to stay ahead of model training data and scale difficulty based on agent performance.

• Eliminates data leakage risks by generating tasks and oracles at evaluation time rather than pre-releasing them.
• Uses a Red-Blue adversarial format where agents create features and embed bugs for each other to solve.
• Difficulty scales automatically with agent capability, preventing saturation common in static benchmarks.
• Validates real-world coding skills by operating within actual repositories across isolated sessions.

This research paper examines whether reflective prompt evolution (GEPA) yields genuine generalization or mere memorization when adapting multi-agent deep-research systems. By using a topic-disjoint split where training, validation, and test topics never overlap, the authors isolate transferable behavior from in-distribution copying. The study finds that while held-out citation reward improved from 0.702 to 0.781, the methodology highlights a critical failure mode where optimizers memorize answers instead of learning transferable reasoning patterns.

• Prompt optimization risks memorizing training answers rather than learning generalizable reasoning strategies.
• Topic-disjoint evaluation splits are essential to detect overfitting in prompt-level adaptation.
• GEPA showed a 0.079 lift in held-out citation reward, proving some transferable gain is possible.
• Human-readable prompts offer a cheap alternative to RL but require stricter validation protocols.

The MADS pipeline generates synthetic, structured dialogue trajectories to address the fragility of task-oriented conversational agents under non-cooperative user behavior. By injecting adversarial conversation paths, it creates validated unhappy paths that existing benchmarks often underrepresent. The resulting MADS-Bench dataset spans six domains and 55 personas, with roughly two-thirds of its 1,095 conversations containing failure-prone trajectories.

• MADS generates synthetic data targeting non-cooperative user behavior often missing in standard benchmarks.
• MADS-Bench includes 1,095 conversations across six domains and 55 distinct personas.
• Approximately 66% of the benchmark conversations contain failure-prone, adversarial paths.
• Provides a robust training resource for developing agents resistant to adversarial dialogues.

Researchers introduce MisVisAgentBench, a paired benchmark designed to evaluate how misleading visualizations impact web agents that act on behalf of users. The dataset contains 280 task instances featuring matched clean and misleading chart pairs, isolating visual evidence as the sole variable. This approach highlights a reliability risk where misinterpreted charts propagate into concrete downstream actions, a gap distinct from standard chart question-answering tasks.

• Web agents treat charts as intermediate evidence, making misinterpretation a critical reliability risk.
• MisVisAgentBench isolates visual deception by using matched clean versus misleading task pairs.
• Current benchmarks overlook how bad visualizations drive incorrect automated actions.
• 280 instances provide a concrete testbed for measuring agent robustness to deceptive data.

SEEDS addresses the flaw in search agents that treat retrieval and consumption as a single step, often admitting irrelevant or misleading high-ranked documents. It introduces evidence admission, a mechanism that selectively allows retrieved candidates into the reasoning context based on the agent's current state. This approach ensures that lower-ranked but critical evidence is not discarded simply due to initial ranking scores.

• Moves beyond fixed rank-based policies for incorporating external knowledge into agent context.
• Dynamically evaluates evidence utility against the agent's current reasoning state.
• Prevents redundancy and misinformation from high-ranked but irrelevant documents.
• Enables selective admission of lower-ranked candidates that are actually critical for the step.

Current RLVR methods for software engineering agents rely on binary rewards from executable tests, which fail to differentiate between trajectories of varying debugging quality. This paper introduces Behavioral Self-Rewarding (BSR), a method that reuses the policy to compare observable debugging behaviors across trajectories sharing the same verifier outcome. BSR aggregates pairwise preferences into task-local scores to generate bounded auxiliary rewards, enabling the model to judge and improve its own trajectories without a separate reward model.

• BSR eliminates the need for a separately trained reward model by leveraging the existing policy for self-evaluation.
• Binary verifier outcomes are augmented with continuous auxiliary rewards based on debugging behavior quality.
• Pairwise preferences among same-task trajectories are aggregated to create task-local performance scores.
• The method specifically targets the limitation where binary rewards cannot distinguish debugging trajectory nuances.