Incident lesson learned: Do not depends on system dependencies!
April 14, 2024•551 words
Last month, my company had an incident that causes a certain part of our system to goes down. We're running everything on-premise since it's regulated by the law of financial institution of Indonesia, so we can't really go with cloud. It was caused by a sudden Virtual Machine failure that causes the VM to be corrupted. There were no backups or snapshot for that VM. The incident itself last for the entire week, and we're struggling to get it working again.
The case of tightly-coupled systems
Th...
Read post