Cortex Labs helps data scientists deploy machine learning models in the cloud

NelsonG · January 23, 2020

It’s one thing to develop a working machine learning model, it’s another to put it to work in an application. Cortex Labs is an early-stage startup with some open-source tooling designed to help data scientists take that last step.

The company’s founders were students at Berkeley when they observed that one of the problems around creating machine learning models was finding a way to deploy them. While there was a lot of open-source tooling available, data scientists are not experts in infrastructure.

CEO Omer Spillinger says that infrastructure was something the four members of the founding team — himself, CTO David Eliahu, head of engineering Vishal Bollu and head of growth Caleb Kaiser — understood well.

What the four founders did was take a set of open-source tools and combine them with AWS services to provide a way to deploy models more easily. “We take open-source tools like TensorFlow, Kubernetes and Docker and we combine them with AWS services like CloudWatch, EKS (Amazon’s flavor of Kubernetes) and S3 to basically give one API for developers to deploy their models,” Spillinger explained.

He says that a data scientist starts by uploading an exported model file to S3 cloud storage. “Then we pull it, containerize it and deploy it on Kubernetes behind the scenes. We automatically scale the workload, and let you switch to GPUs if it’s compute intensive. We stream logs and expose [the model] to the web. We help you manage security around that, stuff like that,” he said.

While he acknowledges this is not unlike Amazon SageMaker, the company’s long-term goal is to support all of the major cloud platforms. SageMaker, of course, only works on the Amazon cloud, while Cortex will eventually work on any cloud. In fact, Spillinger says the biggest feature request they’ve gotten to this point is to support Google Cloud. He says that and support for Microsoft Azure are on the road map.

The Cortex founders have been keeping their head above water while they wait for a commercial product with the help of an $888,888 seed round from Engineering Capital in 2018. If you’re wondering about that oddly specific number, it’s partly an inside joke — Spillinger’s birthday is August 8th — and partly a number arrived at to make the valuation work, he said.

For now, the company is offering the open-source tools, and building a community of developers and data scientists. Eventually, it wants to monetize by building a cloud service for companies that don’t want to manage clusters — but that is down the road, Spillinger said.