Agentic AI

A new arXiv paper introduces HybridDeepResearch, the first benchmark evaluating agents that combine open web search with structured SQL queries. It specifically targets the critical 'handoff' phase where constraints must be preserved while moving evidence between unstructured text and relational databases. This approach addresses the gap in existing benchmarks that test these modalities in isolation.

• Evaluates agent ability to weave web search results with precise SQL data for complex tasks.
• Focuses on the 'handoff' mechanic to ensure constraint preservation between systems.
• Moves beyond isolated benchmarks to test real-world hybrid analytical workflows.
• Aims to verify complete, verifiable answers derived from mixed evidence sources.

This arXiv paper investigates how language model agents can better leverage reusable knowledge for long-horizon tasks. It contrasts the standard approach of loading skill packages into the context window with invoking them as separate subagents. The study highlights that as context grows, reasoning quality degrades, making the subagent approach a more robust alternative for maintaining performance.

• Standard agent skills load instructions into context, causing brittleness as horizons grow.
• Reasoning quality degrades significantly when context windows accumulate too much information.
• Invoking skill packages as subagents offers a more stable execution model for complex tasks.
• This approach addresses the scalability limits of context-based skill loading.

This paper addresses the challenge of selecting tools from massive libraries by introducing a tool menu that restricts agents to a short, ordered subset. Instead of ranking tools solely by request relevance, the method constructs a state path that maps observable inputs to desired outputs in the correct execution sequence. The framework treats this menu as an execution prior, ensuring prerequisite tools are available before the final action is called.

• Replaces relevance-based ranking with ordered execution priors to prevent missing prerequisite tools.
• Constructs state paths linking initial request state to final outcome for reliable multi-step tasks.
• Restricts agent access to a curated menu, reducing search space and hallucination risks.
• Encoder learns optimal tool sequences rather than static relevance scores.

This paper investigates whether internal model representations can accurately predict success in multi-turn agentic workflows, addressing the complex failure modes of planning and tool invocation. It introduces Latent Trajectory Dynamics (LTD) to summarize residual-stream changes across interaction trajectories and Action Representation Probe (ARP) to predict outcomes from action-specific representations. The study aims to provide a reliable confidence signal for agents operating in safety-critical environments.

• LTD analyzes residual-stream changes across the full interaction trajectory to gauge confidence.
• ARP predicts task success directly from representations formed at specific action steps.
• Internal representations offer stronger signals than traditional metrics for agentic workflows.
• Enables better risk management for agents in safety-critical, multi-turn scenarios.

ContractEval addresses the gap where LLM agents produce correct-looking outputs while skipping critical procedural steps like checks or invariants. It maps active obligations from instructions to response evidence, exposing omissions and ordering errors that standard output-only or trace-aware methods miss. This framework makes hidden non-conformance visible by explicitly matching query-specific requirements against agent actions.

• Standard evaluation misses hidden skips in agent workflows despite correct final outputs.
• ContractEval makes active procedural obligations explicit for precise diagnostic matching.
• Identifies omissions, wrong branches, and invariant breaches often invisible to trace analysis.
• Enables targeted debugging of LLM agent conformance to complex procedural instructions.