Data Source Indexing Best Practices

faxelbp

Curious to hear if other Glean customers have recommendations for setting the indexing configuration with connected data sources. For example, we currently index the full history of connected data sources and wonder if that contributes to too much out of data information being considered in Search and Chat. We're considering reducing the length of historical indexing to something like created, updated, or accessed in the last 24 months and wondering how others have approached this.

Trevor

In my experience as a customer, it was useful to have the historical context, even prior to 24 months, for certain data sources. For example, data sources that contain content that is intended to be long-living (e.g., design documents, research reports) were valuable even when older. Other data sources that are more ephemeral in nature (e.g., Slack) did not benefit as much from older content.

A couple other things to keep in mind:

• Glean's relevancy model includes recency of content, so resources that have been worked on more recently tend to surface near the top automatically. There are many other factors included in the relevancy model, but recency is a really important one.
• The Glean Verification feature can be used to verify the official versions of documents, so that they surface to the top.
• Search filters can be used to restrict results to more recent content (e.g., after:2024-06-01) when needed. This can be beneficial with Glean Search, Glean Assistant, and Glean Agents.

I love your question. It will be great to hear from some other customer experiences.