Elevated errors in Silverfin
Incident Report for Silverfin
Postmortem

On 2019-09-03 we had scheduled database maintenance that included a change in collation (which specifies how data is sorted and compared). Unfortunately, this resulted in various failures such as syncs or logins not working. This caused us to have to rebuild various indexes on our (very big) database, which took quite some time.

We take these types of maintenances very seriously and always thoroughly test things, but this one slipped through the cracks. We will revise and review our upgrade procedure to try to avoid this sort of thing from happening again.

We apologise for the inconvenience. We’d also like to stress that customer data was safe at all times. If you have any questions, do not hesitate to contact us.

Posted 14 days ago. Sep 04, 2019 - 18:32 CEST

Resolved
This incident has been resolved.
Posted 14 days ago. Sep 04, 2019 - 18:31 CEST
Monitoring
We mitigated most of the impact. We still might have some degraded performance on particular syncs, but we've fully identified the issues and are putting the necessary fixes in place. We will update with a full disclosure when it's fully resolved. We apologize for the inconvenience and thank you for bearing with us.
Posted 14 days ago. Sep 04, 2019 - 15:59 CEST
Identified
We are currently facing some instability issues within the Silverfin platform. These can result in behavior such as not being able to log in, or syncs not working. Our team is aware of this and is working on resolving this with utmost priority. We apologize for the inconvenience this is causing.
Posted 15 days ago. Sep 04, 2019 - 10:33 CEST
This incident affected: Silverfin (Silverfin, Syncing, Exports).