F3 Lightning: How Meta Accelerates Ads ML Features to Production at Scale

From the source

Facebook Feature Framework (F3) is Meta’s framework for authoring, compiling, and serving the machine learning features underlying ads ranking models, where a single request can require more than 20,000 features. We first presented F3 at AI @ Scale 2020 .

F3 reduced execution computational costs by compiling all features into a single merged graph, but that graph also served as the release unit, so a single failure or invalid definition blocked the entire release pipeline. This post introduces F3 Lightning , an infrastructure layer built on top of F3 to resolve this challenge by decoupling feature releases from shared execution, so each feature is compiled, validated, deployed, and rolled back independently.

The hardest part to solve was preserving F3’s efficiency improvements. Early per-feature release prototypes increased CPU usage because independent features no longer shared computation. We aimed to close this gap through four optimizations: stable computation identifiers, a single shared execution plan, one shared memory buffer across all features, and dense index mapping for direct array access. …

engineering.fb.com