Search Indexer Delays
Incident Report for Close
Postmortem

Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring.

Impact

Changes to Lead data in Close were not reflected on search-based pages, including Leads, Inbox, and Reporting pages from 13:40 UTC to 15:05 UTC on Nov 18th 2024.

Root Cause and Resolution

Due to a change in a recent release, we sent bad data to the system that powers our search tools. As a result, this system failed to process any new incoming information and search results became outdated.

Once we determined the root cause, we updated the search system to ignore such bad data in the future. Search results are now up-to-date.

Timeline

  • 13:40 UTC - bad data gets sent to the system that powers our search
  • 14:15 UTC - engineers identify the issue
  • 15:05 UTC - rollout of the fix is complete
Posted Nov 18, 2024 - 09:26 PST

Resolved
This incident has been resolved.
Posted Nov 18, 2024 - 07:48 PST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 18, 2024 - 07:07 PST
Identified
We are investigating an issue where our Search Indexer's ingestion of new events is delayed. This may result in the latest changes not being reflected on many pages across the application – Leads Search, Contacts Search, Inbox, Opportunities, Reporting, etc.
Posted Nov 18, 2024 - 06:57 PST
This incident affected: Search (Indexing).