AI / ML

Amazon SageMaker Inference now supports automatic container image caching to accelerate scale-out events for generative AI models. By pre-caching large images (often 10 GB or more) from Amazon ECR, the service eliminates the cold-start latency previously caused by pulling full images for each new instance. This optimization allows new instances to start serving traffic significantly faster, reducing end-to-end scaling time by up to 50%.

• Generative AI endpoints scale up to 2x faster during scale-out events due to reduced cold-start latency.
• Large container images are pre-pulled from ECR, preventing new instances from waiting for full downloads.
• End-to-end scaling time is cut by up to half, improving responsiveness for traffic spikes.
• No code changes required; the caching feature is enabled automatically for supported SageMaker Inference endpoints.

Cloudflare has introduced Town Lake, an internal unified data platform leveraging a lakehouse architecture built on Trino, Iceberg, R2, and DataHub. The system integrates operational, billing, security, and business data, enabling governed cross-system analytics. Notably, billing workloads dominate usage, accounting for approximately 53% of the ~91K queries processed.

• Billing queries constitute 53% of total volume, highlighting their critical role in data infrastructure planning.
• Town Lake unifies disparate data sources (ops, billing, security, business) into a single governed lakehouse.
• The stack relies on open standards: Trino for query execution, Iceberg for table format, and R2 for storage.
• Skipper acts as an AI analytics agent, providing natural language access to the unified data layer.