Agentic AI

A new study identifies 'token maxing' as a key driver of rising AI spend, where organizations increase reasoning depth and context size faster than task value. The research isolates the orchestration layer as the decisive lever for control, testing six foundation models while swapping only the harness design. Results suggest that better context assembly, tool exposure, and turn sequencing can significantly reduce token consumption without compromising capability.

• Token efficiency depends more on orchestration logic than the underlying foundation model chosen.
• Stop scaling token limits blindly; optimize context assembly and tool delegation first.
• Implementing stricter governance and observability in the harness curbs runaway token usage.
• Benchmarking should fix the harness to isolate model performance from orchestration waste.

This arXiv paper introduces Agentic Data Environments as an execution substrate designed to balance the efficiency gains of autonomous agents with strict safety guarantees. It argues that while databases are central, agents operate across a broader ecosystem of files, APIs, and system states. The proposed approach reframes data systems from passive storage into active components that enforce reliability and bound the consequences of agent failures.

• Agents require execution substrates beyond traditional databases to manage files, APIs, and state.
• New frameworks aim to amplify agent capabilities while strictly bounding failure costs.
• Data systems are shifting from passive storage to active safety enforcement layers.
• Focus is on preventing abrupt, irreversible costs in agentic automation workflows.

Training agentic models via multi-teacher on-policy distillation can induce subtle behavior shifts not visible in aggregate loss metrics. In tool-use scenarios, vanilla generalized knowledge distillation improves tool-call recall but simultaneously drives the model toward over-calling tools on direct-response examples. This imbalance occurs despite equal token exposure across sample types, indicating that standard aggregate explanations are insufficient for diagnosing these distributional changes.

• Multi-teacher distillation improves recall but risks over-calling tools
• Aggregate loss metrics fail to detect invisible behavior shifts
• Token exposure balance does not prevent distributional drift
• Direct-response examples may be negatively impacted by specialization
• Monitor per-behavior metrics, not just overall loss, during training

AgentLens introduces a benchmark for interactive code agents that assesses the entire execution trajectory rather than reducing results to a binary pass or fail. It combines formal verification with LLM-generated reviews to explain why a specific score was assigned. This approach allows engineers to diagnose model behavior and compare successive versions based on how agents handle instructions, tools, and error recovery.

• Moves beyond binary metrics to evaluate the full agent interaction trajectory
• Uses LLM-written reviews to provide readable explanations for scoring decisions
• Enables diagnosis of specific model behaviors like tool usage and error recovery
• Facilitates comparison of successive model versions through side-by-side analysis

Current LLM agents rely on static, granular toolsets, forcing them to reinvent low-level logic for recurring workflows and increasing failure rates. This paper introduces EvoSOP, a framework where agents analyze execution trajectories to extract and synthesize atomic actions into reusable Standard Operating Procedures (SOPs). These SOPs act as higher-order tools that encapsulate multi-step logic, enabling agents to self-evolve and reduce reasoning overhead.

• Shifts agent design from static atomic tools to dynamic, self-evolving SOPs.
• Reduces reasoning overhead by encapsulating recurring multi-step logic.
• Lowers failure rates by eliminating redundant low-level logic reinvention.
• EvoSOP extracts SOPs directly from agent execution trajectories.

Tool-using LLM agents can violate operational policies while appearing to succeed, resulting in silent wrong-state updates like unauthorized cancellations. Research on the tau2-bench airline domain shows 78% of failures are this type, with no tool errors or self-reporting flags to alert the system. The study evaluates lightweight inter-process deterministic gates as a mechanism to catch these violations before they persist.

• Agents may execute forbidden state transitions without raising errors, creating silent data corruption.
• 78% of observed failures in the tau2-bench airline domain are silent wrong-state events.
• Failure rates are reproducible across seeds, indicating a systematic policy-enforcement gap.
• Standard tool outputs and agent self-reports do not expose these specific policy violations.
• Deterministic gates offer a lightweight verification layer to prevent silent policy breaches.

This study evaluates whether reusable, LLM-generated skill files improve performance over direct prompting for common data science tasks. Testing across data preparation, extraction, statistical analysis, and reporting stages, the findings show no reliable performance gain from using these generated skills. The results suggest that manually curating expert-written skills may still be necessary to avoid bottlenecks while ensuring quality.

• LLM-generated skills do not reliably beat raw prompts in data science tasks.
• No performance gain found across four key lifecycle stages.
• Manual curation remains critical for high-quality, reusable guidance.
• Avoid assuming auto-generated skills simplify workflow maintenance.