Agentic AI

AWS Agent Registry now allows custom metadata fields on registry records to support governance at scale. Publishers can define a schema once per registry and extend it with type-specific attributes like team ownership or data classification. This enables both human discovery and automated system enforcement through structured data.

• Define a single schema per registry to standardize metadata across all resources.
• Add type-specific fields, such as model family, to distinguish record types.
• Enable automated governance by structuring fields for system consumption.
• Track critical attributes like data classification directly on registry records.

AWS has released the AWS AI/ML skill as a core component of the Agent Toolkit for AWS, enabling coding agents to manage SageMaker AI tasks via natural language. This skill replaces previous Labs plugins and integrates with other toolkit skills like EC2 and Bedrock for unified workflows. It is pre-installed in SageMaker Studio JupyterLab to support agents like Kiro and now includes extended inference optimization capabilities.

• Coding agents can now select, train, optimize, and deploy SageMaker models using natural language.
• The new skill is the single supported integration point, replacing earlier AWS Labs plugins.
• Pre-installed in SageMaker Studio JupyterLab for immediate use with compatible coding agents.
• Extends supported SageMaker AI areas, including new inference optimization features.

This study evaluates CTI-REALM, a benchmark where language-model agents convert threat-intelligence reports into detection rules by querying live telemetry. Comparing twenty-six models under a unified harness, the research finds that open-weight agents can rival the performance of the strongest closed models. However, this parity requires allowing the agent significantly more interaction turns than typical benchmarks permit, suggesting previous cost-performance comparisons may have been skewed by message limits.

• Open-weight models can match top closed models in threat-intelligence detection tasks.
• Cost savings exceed 90% compared to frontier closed-source alternatives.
• Current benchmark message limits may artificially suppress open-model performance.
• Allowing more agent turns is critical for realizing open-model parity.
• Self-hosting open models enables full auditability for security-sensitive workflows.

A new study introduces CIMEMORIES DIALOGUE, a benchmark of 4,410 two-agent interactions grounded in Contextual Integrity theory, to evaluate how LLM agents handle sensitive information. The research tests seven open-weight models from the Qwen3 and Llama 3.1 families across varying conversation lengths and prompting conditions. Results indicate a fundamental coupling between task capability and privacy leakage, where more capable models systematically leak more data.

• CIMEMORIES DIALOGUE offers a 4,410-instance benchmark for auditing privacy in agentic multi-turn conversations.
• Privacy leakage scales with model capability; high-performing Qwen3/Llama 3.1 models leak more sensitive info.
• Contextual Integrity theory provides the framework for evaluating whether information flows respect origin norms.
• Enterprise deployments must account for the trade-off between agent capability and data confidentiality.

This paper introduces Agentic Data Engineering (ADE), a framework where AI agents autonomously design and refine data strategies for LLM training. By leveraging experimental feedback, agents select, revise, or combine research directions to improve data selection, rewards, and curricula. The study demonstrates that agents can progressively enhance their strategies, leading to better model performance on held-out benchmarks across math and code tasks.

• Agents autonomously iterate on data engineering strategies using in-loop experimental feedback.
• ADE framework combines cumulative findings to optimize data selection and reward mechanisms.
• Validated improvements on held-out benchmarks for math and code generation tasks.
• Demonstrates viability of AI-for-AI workflows for complex data curation pipelines.

A new study argues that current evaluations of agentic data science agents focus too heavily on predictive accuracy and code correctness, ignoring fundamental methodological principles. This oversight allows critical failures like data leakage and inappropriate validation to go undetected. The authors contend that without assessing domain knowledge and rigorous methodology, these systems cannot be considered robust for production use.

• Current benchmarks for agentic DS are insufficient as they prioritize output accuracy over methodological integrity.
• Data leakage and flawed validation strategies can exist in agents that still produce high-performing models.
• Evaluating domain knowledge application is essential for building trustworthy automated data science workflows.
• Code correctness does not guarantee statistical validity in agentic data science pipelines.