top_k
chunks of text most semantically
similar to the query.
top_k
to a large number) and then use a cross-encoder model to re-rank the chunks before
choosing the top-scoring chunks to append to the user query for the final LLM prompt.
What is retrieval and why is it important for enterprise?
How do we think Company X will perform this quarter?a RAG application would search various data sources (articles, databases, etc.) for information about Company X and create a prompt such as the following:
Instructions: Please answer the following user query using the additional context appended below. User Query: How do we think Company X will perform this quarter? Additional Context Retrieved from internal sources: Expected 2023 stock growth: +10%, Actual 2023 stock growth: +20% Source: internal_database Restrictions on international exports of Product Y are expected to slow down Company X’s growth. Source: external_news/international_exports_restricted_on_product_y.pdf Publication Timestamp: 1 week ago Author Z recommends that investors hold current assets, but temper expectations and slow down stock purchases. Source: internal_article_source/recommendations/author_z.pdf Publication Timestamp: 3 minutes agoFor the original user prompt, an off-the-shelf LLM would produce the following response:
I’m sorry I could not find any information about how Company X will perform this quarter. As an AI model I am unable to make future predictions.For the Retrieval Augmented prompt, the same off-the-shelf LLM would now be able to generated a reasonable response:
According to data retrieved from internal sources, Company X has double stock growth estimates so far in 2023, however, is expected to slow down due to restrictions imposed last week on exports of Product Y. Author Z recommends that investors hold current assets and not execute additional stock purchases.LLMs have a unique capability to do what is called “in-context learning”. This means that even if they weren’t trained on a specific piece of data, if it is given that data live in the prompt, it is generally capable of interpreting that data and using it to answer a user query.