As chat histories get longer, passing the entire history on every call is not a good practice. More so, user expects information from several messages ago to be available as context.
Goal: Improve size of the chat history context window to allow users to reference messages that fall outside existing window.
Solution: Leverage semantic search to index the entire chat history of a conversation and pull messages that are related to the latest message from the user.
Implementation:
- Create a pseudo-
Pipeline object that uses a custom source connector that simply bypasses messages written to it into a vector database.
- Pipeline is declared with an Embed Connector and Sink Connector to be used as part of the operation.
- At
search we would run a normal search against the sink with filters to only pull messages from the given conversation.
- Then the user would add the retrieved messages as context into the conversation alongside the last 3-4 messages
Prototyped: https://github.com/NeumTry/Pensieve
Other ideas:
- Any chat systems that are worth integrating? (Twilio?)
As chat histories get longer, passing the entire history on every call is not a good practice. More so, user expects information from several messages ago to be available as context.
Goal: Improve size of the chat history context window to allow users to reference messages that fall outside existing window.
Solution: Leverage semantic search to index the entire chat history of a conversation and pull messages that are related to the latest message from the user.
Implementation:
Pipelineobject that uses a custom source connector that simply bypasses messages written to it into a vector database.searchwe would run a normal search against the sink with filters to only pull messages from the given conversation.Prototyped: https://github.com/NeumTry/Pensieve
Other ideas: