Agentic AI

Researchers introduce RealtimeBench, a benchmark with 180 tasks across six game types that evaluate LLM agents while environments remain active during inference. The study reveals that models optimized for accuracy often fall behind faster models when latency causes actions to arrive too late in dynamic settings. To address this, the paper proposes TORSI, a two-timescale approach where a slow teacher corrects errors in paused simulations to generate reusable instructions for a fast student agent.

• RealtimeBench tests agent performance in continuously running environments, exposing latency-sensitive failures.
• High-accuracy models may underperform faster alternatives when real-time constraints are enforced.
• TORSI uses a teacher-student framework to decouple error correction from real-time execution.
• Slow offline self-improvement generates instructions that enable fast online inference without teacher calls.
• This approach balances reasoning depth with the speed requirements of live interactive tasks.

Research identifies a fragility in self-improving agents where the policy model and skill bank co-evolve but drift apart post-deployment. Current evaluation metrics only test against the final static skill bank, ignoring the reality that skills may be updated more frequently than the policy is retrained. The study demonstrates that this misalignment causes performance degradation due to cooperative acceptance rules that fail to account for skill drift.

• Standard evaluations are blind to deployment-time skill drift in self-improving agents.
• Policy models become fragile when skills are rewritten or removed without retraining.
• Cooperative acceptance rules exacerbate failures under skill-policy misalignment.
• New evaluation protocols must account for dynamic skill bank changes.
• Practitioners should monitor drift between static policy weights and dynamic skill sets.

Long-horizon LLM agents often fail because context compaction mixes environmental facts with compressor reasoning, making them indistinguishable. Analysis shows 83% of failures stem from actions based on unsupported summary entries. The authors propose Source-Judged Compaction, a training-free method that scores summary entries to filter out unsupported claims.

• Compaction prompts inject reasoning that agents mistakenly treat as environmental facts.
• 83% of agent failures trace back to executing actions without environment support.
• Source-Judged Compaction is a training-free post-processing step for better accuracy.
• This approach prevents hallucinated actions by scoring entry validity before execution.

Researchers analyzed 1,514 long-horizon tasks across five benchmarks to evaluate three context-engineering techniques for data agents: semantic layers, orchestration, and compaction. The study examined seven LLMs to determine how these techniques impact accuracy, cost, and agent behavior. Results indicate that without these techniques, agents fall into three distinct behavioral groups, including investigators and premature responders.

• Context-engineering techniques significantly alter agent accuracy and cost efficiency.
• Semantic layers, orchestration, and compaction are most effective when combined.
• Agents without these techniques exhibit unstable behaviors like premature answering.
• Behavioral analysis reveals distinct agent archetypes in long-horizon tasks.

Amazon MWAA now integrates with the Agent Toolkit to support AI coding assistants via four specific skills: authoring, debugging, testing, and upgrading workflows. These tools allow engineers to use natural language to generate DAGs with dependencies and error handling, or interact directly with the environment for maintenance. The feature is compatible with both Provisioned and Serverless MWAA environments.

• AI assistants can now generate working DAGs from plain language descriptions.
• Four distinct skills cover authoring, debugging, testing, and upgrading.
• Reduces manual effort in building and maintaining data pipelines.
• Compatible with both MWAA Provisioned and Serverless environments.