Glean Assistant Context - New Issues

amilazzo

I've noticed this week that there are issues with Glean going off topic and maintaining context throughout a conversation. For example:

User: Tell me about the security implications for Project X

Glean: The security requirements for Project X include A, B, C

User: Is data encrypted?

Glean: Let me tell you about how our Employee Experience portal is the new home for employees

Glean has always suffered from issues like this, but I feel like it is happening in nearly every conversation. I noticed it doesn't maintain the same sources in follow-up answers. I would at least expect it to give me some irrelevant information about Project X or repeat its first answer, rather than completely go off topic and forget about any sources it used in the answer we discussed previously.

I've downvoted a handful of answers where this happened.

EmptySea

I havent seen something blatant but I have seen longer conversations fail as things fall out of the context window and there isnt any indication to me as the user that what it is now working with is missing some of the things from the earlier part of the conversation. Its as if the conversation gets better… as it cranks up the context and then falls off a cliff. I have seen this in the regulars as well (Gemini, ChatGPT, Perplexity, Claude) so I am not shocked but I cant say that Ive seen a wholesale shift off of the subject.

Maybe the new DEBUG in Glean Agents could be added (for admins at least) to the Assistant so we could see more about what is going on under the hood.

Have you tried to ask "Why did you respond about the employee experience portal and how it is it relevant to our conversation?" I would be curious to see what it says, maybe it could indicate an issue or gap.