AI / ML

Mixture-of-Experts models often trigger frequent expert switching between consecutive tokens, causing costly weight swaps between storage and fast memory. This work introduces StickyMoE, a differentiable routing consistency loss that penalizes abrupt switches, encouraging the router to maintain assignments across semantically coherent spans. Unlike post-hoc fine-tuning, this method integrates directly into pretraining without architectural changes.

• StickyMoE adds a single hyperparameter lambda to penalize abrupt expert switches during training.
• Encourages router consistency across adjacent tokens to reduce memory bandwidth pressure.
• No architectural modifications required; integrates directly into the pretraining loop.
• Addresses the root cause of weight swapping rather than relying on caching heuristics.
• Enables more memory-efficient inference on edge devices by stabilizing expert activation.

Amazon SageMaker Feature Store now supports BatchWriteRecord, enabling high-throughput ingestion of multiple records across feature groups in a single API call. The update also introduces ListRecords for discovering stored data without predefined identifiers and allows custom naming for offline store tables and databases. These enhancements reduce API overhead and latency while improving offline store cataloging flexibility.

• Use BatchWriteRecord to ingest features at scale with fewer API calls and lower latency.
• Discover stored records dynamically via ListRecords without needing specific record identifiers.
• Define custom names for databases and tables within the offline store for better organization.
• Optimize training data pipelines by reducing the overhead of single-record writes.