OffNet Newsroom

Daily topic roundup

AI / ML

Thursday, August 20, 2026 · 5 stories, curated & summarized — click any story for the source.

Share this roundup on LinkedIn Full edition
arXiv cs.AI researchai

MoE Router Locality Training Fails: 235B Model Bottlenecked by Edge Memory Bandwidth

A pre-registered study measures the severe memory bandwidth wall for serving 235B-parameter Mixture-of-Experts models on consumer hardware, where decode speeds drop to 0.44 tok/s due to SSD-bound expert streaming. Researchers developed llama-moe-trace to analyze routing on Qwen3-30B, revealing that while adjacent-token expert reuse offers a 2.0x chance and 95% of traffic hits only 52.5% of experts, this locality is insufficient to overcome paging thrash at scale. The work presents a negative result for training routers specifically for cache locality, as batching schemes collapse under disk I/O pressure.

  • 235B MoE decode on 8GB GPU is I/O bound, not compute bound, hitting 0.44 tok/s due to SSD latency.
  • llama-moe-trace tool enables zero-surgery telemetry to quantify expert routing patterns and cache misses.
  • 95% of Qwen3-30B traffic accesses only 52.5% of experts, showing inherent but insufficient locality.
  • Batching schemes designed to amortize disk sweeps collapse at batch 32 due to paging thrash.
  • Training routers for cache locality against the edge bandwidth wall proved ineffective in this study.
TRADE-OFFMoE Locality vs RealityTheoretical Locality95% traffic hits 52.5% experts2.0x reuse chance for tokensEdge Reality235B model hits 8GB GPUBatch 32 causes paging thrashvs
Read at source →
The Register general

Feds warn AI-generated malware is actively targeting critical infrastructure controllers

US federal agencies are issuing warnings that attackers are now deploying AI-generated code to compromise critical infrastructure controllers. Officials emphasize that this is an active, not theoretical, threat vector currently being exploited in the wild. The use of AI allows adversaries to rapidly produce tailored malware variants for industrial systems.

  • AI-generated code is already being used in active attacks against critical infrastructure
  • Federal agencies classify this as a current, active threat rather than a future risk
  • Attackers leverage AI to quickly create tailored malware for industrial control systems
  • Organizations must update defenses to detect AI-crafted code patterns
  • This represents a shift from theoretical risks to practical, ongoing exploitation
Read at source →
AWS What's New awsdatabase

SageMaker Unified Studio adds Glue-powered data profiling and anomaly detection

Amazon SageMaker Unified Studio now integrates AWS Glue Data Quality to enable statistical profiling and anomaly detection for datasets. Users can generate on-demand or scheduled profiles to assess data shape and completeness across catalog tables and Visual ETL jobs. The system automatically detects drift from historical patterns without requiring manual threshold configuration.

  • Leverages AWS Glue Data Quality under the hood for profiling engine.
  • Supports both data at rest in catalogs and data in transit via Visual ETL.
  • Tracks dataset and column-level statistics with historical baselines.
  • Automates anomaly detection by identifying drift from expected behavior.
  • No custom rules or predefined thresholds needed for drift detection.
CHECKLISTWhat matters hereLeverages AWS Glue Data Quality under the hood for profiling engine.Supports both data at rest in catalogs and data in transit via Visual…Tracks dataset and column-level statistics with historical baselines.Automates anomaly detection by identifying drift from expected…No custom rules or predefined thresholds needed for drift detection.
Read at source →
Hugging Face Blog llmaiml

Hugging Face introduces multi-vector late interaction embeddings in Sentence Transformers

Hugging Face has integrated multi-vector late interaction embedding models into the Sentence Transformers library. This approach allows for more precise retrieval by comparing individual token vectors rather than relying on single dense vectors. The update expands the toolkit available to engineers building advanced semantic search systems.

  • Late interaction models capture finer-grained semantic details than traditional dense embeddings.
  • Sentence Transformers now supports these multi-vector architectures out of the box.
  • Enables more accurate retrieval for complex queries without sacrificing implementation simplicity.
Read at source →
InfoQ generaldevops ↺ since 08-19

WhatsApp Tests On-Device ML for Scam Detection with Privacy Preserving Analytics

WhatsApp is beta-testing Scam Alert, an on-device machine learning system designed to flag scam messages from unknown contacts. The architecture retains message content locally while leveraging confidential computing, Oblivious HTTP, and differential privacy to evaluate model performance securely. This approach ensures model delivery transparency without exposing sensitive user data to the network or central servers.

  • On-device ML processes scam detection locally, keeping message content off-network.
  • Confidential computing and Oblivious HTTP secure the evaluation pipeline.
  • Differential privacy protects aggregate metrics while measuring model accuracy.
  • Model transparency features allow users to understand detection logic.
  • Architecture prioritizes privacy by design for sensitive communication data.
HOW IT WORKSPrivacy-First Scam Detection Pipeline1Message arrives on device2On-device ML analyzes content3Confidential computing evaluates model4Oblivious HTTP sends metrics5Differential privacy aggregates data
Read at source →
↑