#Training Methods

Models · RSS

This week in Training Methods · updated Thu Nov 12 2026

This week in model research, advancements in training and inference efficiency took center stage. Meta FAIR introduced e-means, a faster k-means variant that achieves comparable clustering performance in just 1.4 to 2.6 epochs—dramatically reducing training time—and is now integrated into FAISS 1.16.0 with up to 45x GPU speedups. Simultaneously, deeper insights emerged into Mixture of Experts (MoE) routing mechanisms, exploring how models like Mixtral and DeepSeek-V4 assign tokens via Top-K softmax, sigmoid gating, and hash-based strategies, each balancing performance and computational cost. On the hardware optimization front, Zyphra released open-source FP8 attention kernels for AMD MI355X, delivering up to 1.76x faster inference than native implementations while reducing error rates by 62–78%. These developments highlight progress in algorithmic efficiency, routing intelligence, and hardware-specific acceleration.

AI-written weekly briefing drawn from this topic's recent stories.

Latest Training Methods stories

AI Labs Turn to Dead Startups' Data for Training Models

As AI training faces limitations with public and synthetic data, labs are increasingly purchasing digital archives from defunct startups, turning discarded code and workspace records into valuable training material. This…

· via MIT Technology Review

RealityHacker