AI / ML

Amazon GuardDuty now includes AI Protection to monitor AWS AI services like Bedrock and SageMaker. It analyzes CloudTrail events to identify AI-specific risks such as prompt injections and anomalous model usage. This feature helps security teams detect cost harvesting attacks without requiring custom tooling or manual configuration.

• Monitors Amazon Bedrock and SageMaker for AI-specific threats automatically.
• Detects prompt injection attempts and unusual model invocation patterns.
• Identifies cost harvesting attacks that force excessive GPU time and token usage.
• Leverages CloudTrail management and data events for continuous visibility.
• Reduces need for custom security tooling to secure AI workloads.

AWS Security Hub now includes an AI inventory feature to give security teams a centralized, continuously updated view of AI assets across the organization. This addresses the visibility gap caused by rapid deployment of AI agents, models, and pipelines. The feature automatically discovers and catalogs AI workloads using three discovery methods, starting with managed services like Amazon Bedrock and SageMaker.

• Centralized visibility is now available for AI agents, models, and pipelines across the organization.
• Security teams can connect AI assets to active threats and misconfigurations in one place.
• Managed services like Bedrock and SageMaker are inventoried automatically via AWS Config.
• No additional configuration is required to start discovering workloads in supported services.

IBM Research highlights that while model routing appears straightforward, real-world deployment introduces significant complexity. The analysis suggests that simple strategies often fail under dynamic workloads, requiring more robust architectural approaches. This insight warns practitioners against underestimating the operational overhead of dynamic model selection.

• Simple routing logic often fails in production environments
• Dynamic workloads introduce hidden complexity to model selection
• Robust routing requires advanced strategies beyond basic heuristics
• Operational overhead of dynamic selection is frequently underestimated

Amazon SageMaker HyperPod now allows network topology configuration at the partition level for Slurm-orchestrated clusters. Users can mix tree and block topologies within a single cluster, assigning the optimal setting based on the instance types in each partition. This alignment reduces GPU-to-GPU latency and improves NCNC collective operation efficiency, boosting distributed training throughput.

• Split clusters into partitions with distinct topologies (tree vs block) based on hardware.
• Optimize NCCL performance by matching topology to instance interconnect capabilities.
• HyperPod auto-selects topology per partition using compute instance group specs.
• Block topology suits high-bandwidth instances like ml.p6e-gb200.36xlarge.
• Tree topology remains viable for hierarchical interconnect instance types.