Search Indexing Delays

Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring.

Impact

On February 14th, 2025 for two and a half hours (between 19:50 UTC and 22:00 UTC) some users saw outdated data in search results and smart views. No data was lost during the outage.

Root Cause and Resolution

A software update introduced malformed data to our Search platform that disrupted updates to our Search database. Close Engineering reverted the change and deleted the malformed data, which allowed the platform to resume processing updates to our Search database. The processing backlog was cleared by 22:00 UTC when full functionality was restored.

We are implementing a new set of metrics and alerts that will help us identify this kind of failure more quickly. We are also enhancing the Search platform to better identify and discard malformed data to prevent similar incidents from occurring.

Timeline

All times are in UTC and events take place on 2025-02-14.

• 19:50: A software update is deployed that sends malformed data to our search platform.
• 20:33: Close Engineering is notified of an elevated error rate on the search platform and start investigating.
• 21:04: Root cause is identified and work starts on a fix.
• 21:40: The issue is resolved and all data is up-to-date.