We have a number of agents that are successfully completing the programmed tasks, but if users ask follow-up questions in the chat, it starts to reference any organizational document it thinks is relevant. This is fine in some use cases, but in others, we would want to constrain the documents referenced in follow-up answers to just the ones that were:
- explicitly read into the agent
- compatible with the filtered sources of any company search actions
A couple of examples:
- We have a payroll Q&A agent that references a large bank of carefully curated and answered questions. It answers questions from the spreadsheet well, but we don't want it to reference any other sources for follow-up questions.
- We have a contract analysis agent where you tag a specific contract to get a summary. If you ask follow up questions, it performs a search instead of staying constrained to the supplied contract, which can sometimes pull in completely unrelated contracts into the context.
In some sensitive use cases, it's arguably worse to allow unconstrained follow-ups than it would be to prevent any type of follow up.
