DISASTER RECOVERY FROM A COUP D'ETAT

Business continuity in action!

Four years ago this month, a disaster recovery solution we created proved its worth. It saved our client, an international property consulting firm, from tanking after an attempted coup d’etat in 2003.

The Philippines has been wracked by four or five coup attempts in the last ten years. That averages to one every 24 months!

The renegades, 300 heavily armed soldiers and their leaders, barricaded themselves in several buildings in Makati’s business district. The standoff lasted for 19 hours before they surrendered. During the crisis, authorities turned off the power grid that served the contested area. My client’s office was within that grid. After it was over, her staff returned to a ransacked office.

Other companies in her building were not so fortunate. In fact, my client was the only one who was able to restore her data and resume operations as if nothing had happened.

About half of her neighbors—branch offices of large companies as well as individual businesses—did not back up at all. As for the other half, the IT manager kept their backup data offsite by bringing the media home. I learned that many of the ones who backed up discovered that their backups were too old or could not be restored.

The latter didn’t surprise me. In smaller shops, many IT administrators diligently back up their data but neglect to regularly test the media’s integrity by doing test restores. Back in the days when I was a network engineer, I was installing Citrix in a 25-desktop network. They had two Windows NT servers and had always been using Windows NT’s built-in backup utility. I knew about that utility’s notorious reputation so I challenged them to restore the data (prior to my continuing my work). Their office manager, who doubled as the IT administrator, pulled out seven tape cartridges. One by one, she tried to restore the contents of each tape. And one by one, she discovered they were empty. In fact, if I recall correctly, the MS-DOS directory listing revealed one empty folder in each tape. That’s how we sold a lot of ARCserve software back then!

As for my client, months earlier, we added additional storage. I persuaded them to configure it to do double-duty as a disaster recovery system. The hardware was kept in a cabinet closet (literally) down the hall. The system consisted of a NetApp NAS (Network Attached Storage) appliance and Symantec’s Backup Exec (System Recovery version).

I was back in the U.S. when this happened and I was able to talk them through the procedure. They were up and running by the end of the day!