From the source
Databricks Feature Store brings real-time freshness to ML features: streaming aggregations from Kafka can now reach the online feature store with 200ms p99 latency, collapsing feature lag from minutes or hours to milliseconds.
Spark Real-Time Mode (RTM) makes millisecond feature computation possible: RTM processes rows continuously instead of waiting for microbatches, updates rolling-window aggregates per event, and amortizes checkpointing to keep stateful streaming latency low.
Lakebase enables high-throughput online feature writes: the separation of compute and storage layers reduces write amplification for frequent small upserts, making fresh feature values quickly available for low-latency model inference.
Machine learning models are only as good as the signals they receive.
A fraud detection use case must decide in milliseconds within a user pressing purchase whether to allow the transaction.
Making the right call depends on seeing a suspicious transaction happening only seconds ago.
Combining a user’s average transactions for the last 30 days along with the total transaction amount from the last 10 minutes highlights the potential fraud.
…
