Nexus Feature Store powering Disney Magic
Dustin will describe his team's journey towards building Nexus, an in-house feature store and how it accelerates feature engineering.
Dustin will describe his team's journey towards building Nexus, an in-house feature store and how it accelerates feature engineering.
Disney Streaming is a diverse streaming service comprising video-on-demand, live sports and tv, and pay per view events. The personalized and seamless experience for this vast user base is powered by many machine learning based solutions running on a robust ML platform comprising a feature store, workflow orchestration and management system, and a model development and deployment ecosystem.
In this talk, we will describe our journey towards building Nexus, an in-house feature store. Nexus enables different models to consume curated and easily discoverable features in various computing environments through a standardized set of APIs. In particular, it serves a dual purpose: a) row-oriented data store for serving features at low latency to nearline and online models; and b) column-oriented storage for large sets of features to run batch training and inference workflows. Furthermore, Nexus decouples feature engineering - the process of computing features from one or more heterogeneous sources (e.g. event streams, message queues, and data lake) - from model development, enabling time-travel and lifecycle management for features.
