Building A Feature Store For Hyper Growth
Brian will go over Doordash's feature store architecture, learnings from supporting CRDB and what it takes to build a feature store.
Brian will go over Doordash's feature store architecture, learnings from supporting CRDB and what it takes to build a feature store.
In the last year Doordash has seen tremendous increases in the number of use cases on its machine learning platform, causing a dramatic growth in the number of models and features being generated. This in turn has exposed the challenges and limitations of using Redis at scale. The ML Platform team at DoorDash ultimately opted to explore a complementary disk based storage backend as a supplement to our existing tech, ultimately landing on using CRDB. There have been many learnings on upload throughput, query performance, and general design patterns that we’d like to share with the broader ML community. In this presentation we’ll go over our feature store architecture and some of the learnings in supporting CRDB in addition to Redis and our thoughts on what it takes to build a feature store that can scale up rapidly on demand.
