Time to Recover

In this episode of Beneficial Intelligence, I discuss time to recover. The entire network of the justice ministry of South Africa has been disabled by ransomware, and they don't know when they'll be back. Do you know how long it would take you to recover each system your organization is running? 

When you have an IT outage, what the business wants most is a realistic timeline for when services will be back. If IT can confidently tell them that it will take 72 hours to restore services, the business knows what they are dealing with. They can inform their stakeholders and make informed decisions about in which areas manual procedures or alternative workflows should be implemented. The worst thing IT can do in such a case is to keep promising "a few hours" for days in a row. 

In the 1980s, I was working for Hewlett-Packard. They had a large LED scrolling display mounted over their open-plan office. The only time it was ever used was when their main email and calendar system was unexpectedly down, telling everyone when it would be back up.  

In the 1990s, I was doing military service in the Royal Danish Air Force as a Damage Control Officer. After an attack, I had to tell the base commander how much runway we had available. I had planned our reconnaissance and could confidently say that I would know in less than 28 minutes af the all-clear.  

In the early 2000s, I was working with database professionals. These people spent much of their time preparing to recover their databases. They had practiced recovery many times and knew exactly how long recovery would take. 

As the CIO, take a look at the list of your system. It needs to list the expected time to recover for every system. The technical person for the system should verify that this time has been tested recently, and the business responsible should verify that this time is acceptable. If you don't have a documented time to recover per system, you need to put your people to work to create it.
 

Beneficial Intelligence is a bi-weekly podcast with stories and pragmatic advice for CIOs, CTOs, and other IT leaders. To get in touch, please contact me at sten@vesterli.com