Data Prep Kit (DPK)

Introducing Data Prep Kit (DPK) as a streamlined data preparation tool for LLMs

In this tutorial, learn how to build a Ray runtime, create unit tests, integrate with CI/CD, and add KFP extensions to existing transforms.

The Data Prep Kit (DPK) offers a diverse set of pre-built transforms to streamline data preparation for AI applications. These transforms cater to various data formats, including text, code, and structured data. These built-in transforms provide a strong foundation for handling common data processing tasks. This rich collection of built-in transforms empowers one to efficiently prepare the data for various AI and machine learning applications.

In this learning path, learn how to use Data Prep Kit (DPK) to prepare data for large language model (LLM) applications.

In this tutorial, you learn how to use the Data Prep Kit (DPK) to prepare data for fine-tuning and to achieve the business use case of contract analysis.

In this tutorial, we explore two experimental notebooks that demonstrate how agentic workflows can support the creation of data pipelines using the Data Prep Kit based on user input provided in natural language.

In this tutorial, you learn how to use the Data Prep Kit (DPK) to prepare PDFs for a RAG pipeline.

While Data Prep Kit includes several built-in transforms, you can also build custom data preparation transforms to meet your needs. In this tutorial, learn how to implement a Processing transform that calculates a signature value for a document and stores the signature as part of the metadata associated with the document.