AWS

Amazon SageMaker AI now supports G7e instances in Asia Pacific (Seoul, Tokyo) and Europe (London). These instances utilize up to 8 NVIDIA RTX PRO 6000 Blackwell GPUs and 5th Gen Intel Xeon processors, offering 2.3x better performance than G6e. The expansion allows practitioners to deploy inference endpoints closer to users in Asia and Europe to reduce latency for generative AI workloads.

• G7e instances are now available in Seoul, London, and Tokyo for lower-latency AI inference.
• Hardware includes up to 8 NVIDIA RTX PRO 6000 Blackwell GPUs with 96 GB VRAM each.
• Performance is up to 2.3x faster than previous-generation G6e instances.
• Single instances provide up to 768 GB GPU memory to serve LLMs up to 70B parameters.
• Elastic Fabric Adapter networking delivers up to 1,600 Gbps bandwidth for distributed workloads.

Amazon SageMaker AI inference now supports G7 instances equipped with NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. These instances deliver up to 4.6x AI inference performance over G6, leveraging 32 GB of GPU memory per card and 5th Generation Tensor Cores. Networking throughput increases to 700 Gbps via EFA, while local NVMe storage scales up to 7.6 TB to keep large models close to compute.

• G7 instances offer 4.6x inference performance over G6 for generative AI workloads.
• 32 GB GPU memory per card reduces the need for model quantization or over-provisioning.
• 700 Gbps EFA networking provides 7x bandwidth improvement for distributed serving.
• Up to 7.6 TB local NVMe allows large models to reside near compute for lower latency.

Amazon EC2 I8ge instances are now generally available in Europe (London) and Canada (Central). Powered by Graviton4 processors and third-generation Nitro SSDs, these storage-optimized instances offer up to 120TB of local NVMe storage. They deliver up to 60% better compute performance than Graviton2-based instances and significantly lower storage I/O latency compared to the Im4gn generation.

• I8ge instances are now live in AWS Europe (London) and Canada (Central).
• Graviton4 processors provide up to 60% better compute performance than Graviton2.
• Storage I/O latency is up to 60% lower with improved variability vs Im4gn.
• Ideal for workloads requiring high throughput and low latency on local NVMe.

Amazon EKS Auto Mode and Karpenter node pools can now configure Elastic Fabric Adapter (EFA) networks and EC2 placement groups. This enables fine-grained control over instance distribution and network interfaces for high-performance distributed training and inference workloads. EFA-only interfaces are supported, which avoids consuming VPC IP addresses while optimizing network throughput.

• Enable EFA and placement groups in EKS Auto Mode and Karpenter node pools.
• Supports EFA-only interfaces to save VPC IP addresses on EFA-capable instances.
• Optimize instance placement for low-latency distributed AI training and inference.
• Configures both dynamic and static capacity node pools for EFA workloads.