Tools for Serving

Serving of ML models in Kubeflow

Overview

Model serving overview

KFServing

Model serving using KFServing

Seldon Serving

Model serving using Seldon

NVIDIA TensorRT Inference Server

Model serving with TRT Inference Server

TensorFlow Serving

Serving TensorFlow models

TensorFlow Batch Prediction

See Kubeflow v0.6 docs for batch prediction with TensorFlow models

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.

Last modified 12.02.2020: rename training and serving section titles (#1668) (32e4df31)