KServe

Beginners who are new to the world of LLM inferencing and serving can learn about why it's a complicated thing to do and gain a clearer idea of how to get started using two open source tools: vLLM and KServe. Learn the 'why' and 'how' of LLM inferencing and serving.

Learn how open source impacts watsonx through some of the key open source projects that IBM has invested in.

Use Caikit to serve Hugging Face AI models that are consumed by your application.

A standard cloud-agnostic model inference platform that provides pluggable production serving.

A mature, general-purpose model serving management and routing layer. Optimized for high volume, high density, and frequently changing model use cases, ModelMesh intelligently loads and unloads models to and from memory to strike a balance between responsiveness and compute.

In this tutorial, learn how to serve your custom models by using ModelMesh Serving.

Using KServe and Ray Serve together is a flexible, scalable, and efficient approach to serving machine learning models in production

In this tutorial, learn how to configure ModelMesh Serving to use Kubernetes "built-in" storage via persistent volumes.

Create, deploy, and manage machine learning workflows on Kubernetes using Kubeflow Pipelines

Learn about some of ModelMesh's features and core resources like the ServingRuntime and the InferenceService, all while deploying and inferencing your first model deployed on your own ModelMesh Serving instance.