OffNet Newsroom

Daily topic roundup

AI / ML

Sunday, September 06, 2026 · 2 stories, curated & summarized — click any story for the source.

Share this roundup on LinkedIn Full edition
AWS What's New awsdatabase ↺ since 09-05

SageMaker Batch Transform adds G6e instances for LLM inference

Amazon SageMaker AI now supports EC2 G6e instances for offline batch transform jobs, enabling GPU-intensive predictions on S3 datasets without persistent endpoints. These instances feature up to eight NVIDIA L40S GPUs with 48 GB memory each and third-generation AMD EPYC processors. This addition targets large language models and generative AI workloads like image, video, and audio diffusion models.

  • G6e instances offer up to 8x NVIDIA L40S GPUs, ideal for heavy GPU workloads.
  • Supports offline inference for LLMs and diffusion models via Batch Transform.
  • Use ml.g6e instance types in AWS SDK, CLI, or CreateTransformJob API.
  • Suitable for large datasets where persistent endpoints are unnecessary.
BY THE NUMBERSG6e GPU Power8xNVIDIA L40S GPUs per instanceIdeal for heavy LLM and diffusion workloads
Read at source →
InfoQ generaldevops ↺ since 09-05

Vortex Columnar Format Streams S3 Data Directly to GPUs at 60 Gbps

Onur Satici presents Vortex, an open-source columnar file format that eliminates CPU and NVMe bottlenecks during ML training data loading. By leveraging cascading lightweight encodings, layout-based segment pruning, and zero-copy memory pipelines, the system streams data directly from S3 to GPUs. This approach achieves throughput speeds up to 60 Gbps without requiring upfront data reprocessing.

  • Vortex is an open-source columnar format under the Linux Foundation.
  • Zero-copy memory pipelines remove CPU/NVMe bottlenecks in data loading.
  • Layout-based segment pruning enables efficient data access patterns.
  • Direct S3-to-GPU streaming reaches up to 60 Gbps throughput.
  • No upfront data reprocessing is required for immediate usage.
Read at source →
↑