Create Knowledge Base

Description

Creates an EGP knowledge base.

Details

A knowledge base is a storage device for all data that needs to be accessible to EGP models. Users can upload data from a variety of data sources into a knowledge base, and then query the knowledge base for chunks that are semantically relevant to the query.

Every knowledge base must be associated with a fixed embedding model. This embedding model will be used to embed all data that is stored in the knowledge base. The embedding model cannot be changed once the knowledge base is created. Only the embedding models in the dropdown menu below are supported.

Differences from V1

• V1 data ingestion consisted of knowledge bases, vector stores, and data connectors. V1 Knowledge bases interacted with natural language, V1 vector stores interacted with chunks and embeddings, and V1 data connectors set up automatic ingestion pipelines with third party data sources.
• In V2, all data ingestion is done through knowledge bases. Low level configuration such as chunking strategies and data sources are now handled by this unified knowledge base v2 upload API.
• The way data is stores in V2 allows for better observability on the ingestion progress and content of the knowledge base.
• Reliability and scalability is also improved via distributed temporal workflows.

Backwards Compatibility

V2 and V1 Knowledge Bases are entirely separate and not backwards compatible. Users who have existing V1 knowledge bases will need to migrate their data to V2 knowledge bases.