TileMix introduces a kernel that routes numerical precision spatially over hardware-aligned score tiles within fused dense attention. By partitioning the attention matrix and packing routing decisions into compact bitmasks, it dispatches tile groups to either FP16 or INT8 computation paths. This approach addresses the high memory traffic and computation costs of long-context prefill without relying on uniform low-precision or token interaction selection methods.
- Routes precision spatially over score tiles instead of uniformly across all tokens.
- Uses compact bitmasks to dispatch tile groups to FP16 or INT8 paths efficiently.
- Reduces memory traffic and computation costs during long-context LLM prefill.
- Integrates precision decisions directly into fused dense attention kernels.