The Coming Wave of Self-Supervised Embedding Ecosystems

There is a paradigm shift in industrial machine learning pipelines where customized architectures and hand-curated features are being replaced by self-supervised embedding ecosystems. In these embedding ecosystems, models are trained in a self-supervised fashion without manual labels on massive corpora and adapted to numerous of downstream tasks. Managing these embeddings and the downstream systems that use them introduces new challenges with respect to managing embedding training data, measuring embedding quality, monitoring downstream models, and correcting for errors in an end-to-end fashion. In this talk, we'll introduce the embedding ecosystem and then discuss the future challenges and opportunities with supporting these embedding pipelines in feature stores.