Agentic AI

This paper argues that standard multi-agent forecasting fails because identical evidence causes LLMs to herd rather than revise beliefs. The authors propose designing information asymmetry by partitioning data into shared public and disjoint private subsets. This forces agents to rely on deliberation to share exclusive knowledge, theoretically improving calibration and reasoning.

• Identical evidence causes multi-agent LLMs to herd, negating deliberation benefits.
• Partitioning evidence into public and private subsets breaks symmetry.
• Private subsets force agents to deliberate to share unique insights.
• This design theoretically improves forecasting calibration over single-agent models.

Hugging Face and IBM Research introduced ScarfBench, a new benchmark designed to evaluate how well AI agents handle the complex process of migrating enterprise Java frameworks. The tool focuses on measuring agent performance in understanding legacy codebases and executing modernization tasks accurately. This release provides a standardized way to assess the readiness of AI-driven development tools for large-scale corporate refactoring.

• ScarfBench targets the specific challenges of enterprise Java framework migration
• Provides a standardized metric for comparing AI agent modernization capabilities
• Helps teams assess risk before deploying AI agents for large-scale refactoring
• Collaborative effort between Hugging Face and IBM Research
• Focuses on practical code transformation rather than just generation

This paper addresses the operational risks of autonomous agents executing backend writes like refunds and cancellations by introducing a difficulty-routed control architecture. A lightweight router evaluates request complexity to decide whether an agent should proceed on a low-cost baseline path or reconsider before acting. The system aims to maintain speed for routine tasks while preventing errors on operationally coupled requests involving complex policy or record interactions.

• Agents now perform operational writes, not just conversation, increasing error stakes.
• A router decides when to 'reconsider' based on request difficulty and coupling.
• Routine sessions stay on low-cost paths to preserve speed and efficiency.
• Complex interactions trigger reconsideration to prevent operational errors.
• Balances friction reduction with safety for backend service executions.

Researchers introduce Janus, a system designed to explore how users participate in managing permissions for autonomous AI agents. It comprises Janus-Core, a modular framework supporting various permission designs, and Janus-Harness, an automated evaluation tool. The work addresses the under-explored role of users in agentic permission management by implementing six distinct permission assistants across key design axes.

• Janus provides a structured playground to test diverse user-involved permission models for AI agents.
• Janus-Core allows implementation of multiple agentic permission management designs.
• Janus-Harness offers automated evaluation for comparing these permission strategies.
• Six permission assistants are implemented to span the conceptual design space of user involvement.
• The system targets the gap in understanding user roles in autonomous agent permissioning.