GEMトレーニング：MetaがLLMスケールの広告基盤モデルの効率を2倍にした方法GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model

Meta has published a technical account of how it doubled the training efficiency of GEM, the Generative Ads Recommendation Model that underpins ad recommendations across Instagram and Facebook. The work matters because recommendation systems are central to how Meta matches ads to users, and extracting more useful computation from expensive GPU clusters directly affects both model quality and infrastructure cost.

In a post on its Engineering at Meta blog, the company says GEM now trains at "LLM scale" on several thousand of the latest-generation GPUs. The headline claim is a doubling of end-to-end (E2E) training efficiency to 20–25% Model FLOPs Utilization (MFU), achieved while the total training FLOPs were scaled up fourfold.

Model FLOPs Utilization is a widely used measure of training efficiency. It expresses the fraction of a processor's theoretical peak floating-point throughput that is actually spent on useful model computation, rather than lost to idle time, data movement, or synchronization overhead. A higher MFU means a given cluster delivers more effective training per hour and per dollar. Large language model runs commonly report MFU in the 30–50% range, so a recommendation model reaching 20–25% is notable given the different workload characteristics involved.

Recommendation models have traditionally been harder to run at high utilization than dense transformer language models. They typically rely on very large embedding tables and sparse features—representations of users, items, and interactions—that generate heavy memory access and network communication relative to the amount of arithmetic performed. As a result, these systems are often memory-bound or communication-bound rather than compute-bound, which tends to cap MFU well below what LLM training achieves. Doubling efficiency while quadrupling compute suggests Meta addressed several of these bottlenecks at once, likely spanning parallelism strategy, kernel optimization, and the balance between sparse and dense computation, though the blog post is the authoritative source on the exact techniques.

The framing of GEM as a "foundation model" reflects a broader industry shift. Recommendation systems were long built from many task-specific models, but companies including Meta have moved toward larger, more general models trained on vast interaction data, echoing the scaling approach that produced modern LLMs. Meta's earlier work in this area includes its Deep Learning Recommendation Model (DLRM) and its generative recommendation research, which reframed ranking and retrieval as sequence modeling problems. Applying scaling laws—the observation that model quality tends to improve predictably with more data, parameters, and compute—to advertising is a significant bet, because even small accuracy gains can translate into meaningful revenue at Meta's scale.

The reported figures also connect to Meta's substantial hardware investment. The company has been expanding its fleet of GPU-based training clusters and developing its own MTIA accelerators, and its executives have repeatedly cited AI infrastructure as a major capital expense. When a single model consumes several thousand top-tier GPUs, efficiency improvements are not incremental niceties; they determine how much can be trained within a fixed power and hardware budget. Doubling E2E efficiency effectively means a comparable amount of training work can be done with roughly half the resources, or that far larger models can be trained for the same footprint.

For readers outside Meta, the announcement is a useful window into how large advertising platforms are converging with the LLM training playbook. Techniques developed for language models—advanced parallelism, mixed precision, optimized attention and matrix kernels, and careful overlap of computation with communication—are increasingly being adapted to recommendation workloads, and vice versa. The 4x increase in training FLOPs implies Meta expects the additional compute to pay off in improved recommendation quality, though the post focuses on the eng