From the source
At Meta, we’ve built a system that enhances ads recommendations by leveraging idle inference capacity in our fleet.
Two elasticity levers in the system allow ads recommendation work to expand across opportunistic capacity, without compromising baseline performance.
This post describes how we designed and now operate the system at Meta’s scale and how it drives significant advertiser value and capacity efficiency.
Inference fleets face a capacity paradox: They can be globally constrained while meaningful pockets of capacity remain locally idle.
At Meta, we built a system that aggregates these pockets into a shared opportunistic capacity pool and turns them into better ads recommendations.
By putting otherwise idle capacity to work, the system helps people see more relevant ads and improves advertiser outcomes without requiring additional capacity orders.
Using this opportunistic capacity safely is difficult.
It’s distributed across team-owned capacity allocations, regions, and accelerator types, and may be reclaimed with little notice.
…
