Organisations are increasingly putting machine learning models into production at scale. The increasing popularity of serverless scale-to-zero paradigms presents an opportunity for deploying machine learning models to help mitigate infrastructure costs when many models may not be in continuous use. We will discuss the KFServing project which builds on the KNative serverless paradigm to provide a serverless machine learning inference solution that allows a consistent and simple interface for data scientists to deploy their models. We will show how it solves the challenges of autoscaling GPU based inference and discuss some of the lessons learnt from using it in production.
Clive Cox (Seldon)
Clive is CTO of Seldon. Seldon helps enterprises put machine learning into production. Clive developed Seldon's open source Kubernetes based machine learning deployment platform Seldon Core. He is also a core contributor to the Kubeflow and KFServing projects.
More from the Same Authors
2020 : Poster session »
Janis Klaise · Lang Liu · Begum Taskazan · Lasse F. Wolff Anthony · Clive Cox · Omid Aramoon · Ting-wu Chin · Alexander Lavin