LLMs

OpenAI's GPT-5.6 Sol, Terra, and Luna models now support 1 million token context windows on Amazon Bedrock. This update allows processing of full codebases, lengthy documents, and multi-turn agent histories in a single request without chunking. Prompt caching with explicit breakpoints applies to these long context requests, offering billing discounts for repeated context.

• Eliminate context chunking for full codebase or document analysis in one pass.
• Maintain complete conversation history in multi-step agentic workflows.
• Prompt caching discounts apply to long context requests with explicit breakpoints.
• GPT-5.6 Sol is available in US East (N. Virginia) and other regions.

The Swiftlet project demonstrates running an 80-billion parameter Qwen model on macOS with just 4.3 GB of RAM, alongside a 35B variant for iOS devices. This achievement highlights significant advances in on-device inference efficiency, allowing large language models to operate on consumer hardware without cloud dependency. The work showcases practical applications of quantization and memory optimization for edge computing scenarios.

• Large models like 80B Qwen can fit in minimal RAM via aggressive optimization
• iOS devices can now run 35B parameter models locally for offline inference
• Swiftlet provides a practical framework for on-device LLM deployment
• Edge AI capabilities are expanding beyond typical mobile constraints
• Memory-efficient inference enables privacy-preserving local model execution

Batched LLM serving complicates energy accounting because GPU telemetry is aggregate, not per-request. JouleShare addresses this by using an offline harness to establish ground truth energy costs through reproducible vLLM replays. This framework enables request-level attribution for sustainability reporting and chargeback, moving beyond model- or token-level estimates.

• Aggregate GPU power data is insufficient for precise per-request energy billing or carbon accounting.
• JouleShare uses offline vLLM replays to create ground truth for request-level energy attribution.
• Shapley fairness concepts guide the allocation, though this work provides measured data rather than just theory.
• Enables accurate chargeback and sustainability reporting for batched inference workloads.

DiffusionGemma is an experimental open-weight model that generates text using discrete diffusion rather than autoregressive decoding. It refines blocks of 256 tokens in parallel, bypassing the sequential bottleneck of standard LLMs. The model is derived from Gemma 4 via fine-tuning, utilizing only 10% of the original training token budget.

• Parallel block decoding of 256 tokens eliminates autoregressive sequential bottlenecks.
• Open-weight model built on Gemma 4 MoE with 25.2B total parameters.
• Training uses 10% of the source model's token budget via supervised fine-tuning.
• Bidirectional denoising replaces next-token prediction for faster generation.