AI / ML

A new project named Mesh LLM enables large language model inference across a decentralized peer-to-peer network built on iroh. This approach distributes compute load among participants rather than relying on centralized cloud infrastructure. The system leverages iroh's networking capabilities to manage connectivity between nodes running the model.

• Distributed inference reduces reliance on centralized GPU clouds.
• Leverages iroh for decentralized connectivity and NAT traversal.
• Shares compute load across a peer-to-peer mesh network.
• Offers a federated alternative for scaling LLM workloads.

AWS has enabled the Apache Spark troubleshooting agent for Amazon EMR on EKS, allowing data engineers to diagnose job failures using natural language queries. The agent automatically analyzes Spark History Server data, distributed executor logs, and cluster configurations to provide root cause analysis and PySpark code recommendations. This eliminates the need for manual log navigation and extends AI-driven troubleshooting support to all EMR deployment options.

• Diagnose EMR on EKS job failures via natural language in the console.
• Agent analyzes history server data, logs, and config for root cause analysis.
• Provides automated PySpark code recommendations to fix identified issues.
• Troubleshooting coverage now spans EC2, Serverless, and EKS deployments.

Amazon SageMaker Feature Store now supports BatchWriteRecord to ingest multiple records across feature groups in a single API call, reducing latency and call volume. It also introduces ListRecords for discovering stored items without pre-known identifiers and allows creating databases and tables with custom names in the offline store. These updates target high-throughput ingestion and easier data discovery for AI model training pipelines.

• Use BatchWriteRecord to reduce API overhead and latency when ingesting large feature datasets.
• Leverage ListRecords to discover available data in feature groups without tracking specific IDs.
• Gain flexibility in offline storage by naming databases and tables explicitly in S3-backed stores.
• Optimize training data pipelines by batching writes to the online store for faster feature retrieval.

Amazon SageMaker Unified Studio Workflows now includes 19 new operators covering Amazon Bedrock, S3 Tables, S3 Vectors, AWS Glue Data Catalog, and MWAA Serverless. These additions enable visual orchestration of tasks like managing Bedrock guardrails, provisioning S3 resources, and triggering workflow runs without custom code. The feature is available in all AWS Regions.

• No custom integration code needed for Bedrock, S3 Tables, S3 Vectors, Glue, or MWAA Serverless
• Visual workflow creator supports managing Bedrock guardrails and S3 resource lifecycle
• Reduces context switching between consoles by centralizing orchestration in SageMaker
• Glue Data Catalog operations for tables and databases are now directly supported
• Feature availability extends to all AWS Regions

AWS Neuron 2.31.0 introduces NKI 0.5.0 with MX FP8 scale support, tensor indirection for optimized indexed access, and zero-cost view APIs for layout transformations. The release includes the Neuron UltraServer Operator for Amazon EKS in public beta to automate Trainium UltraServer workload management. Additionally, the Neuron Compiler features a redesigned code generation backend now enabled by default on Trn2 and Trn3, while the Runtime adds contiguous shared scratchpad support to simplify device configuration.

• NKI 0.5.0 adds MX FP8 scale dtype support for improved precision efficiency.
• Tensor indirection reduces instruction count in indexed access patterns.
• UltraServer Operator for EKS automates discovery and resource claims for Trainium.
• Redesigned compiler backend is now default on Trn2 and Trn3 for better performance.
• Contiguous shared scratchpad removes manual page size configuration in Runtime.

Amazon SageMaker Unified Studio now enables OpenLineage-compatible data lineage tracking within IAM-based domains, a feature previously limited to IAM Identity Center environments. The service captures lineage events from Apache Spark jobs on EMR, AWS Glue, and SageMaker Visual ETL, offering interactive graphs with configurable depth and column-level detail. Practitioners can programmatically manage, query, and delete lineage events via the new DeleteLineageEvent API.

• Lineage tracking is now available in IAM-based domains, matching IAM Identity Center capabilities.
• Supports event ingestion from EMR, AWS Glue, SageMaker Visual ETL, and notebooks.
• Interactive graphs allow configurable depth and column-level lineage views.
• New DeleteLineageEvent API enables programmatic removal of published events.