In early March 2021, OVH experienced a fire at their data centre located in Strasbourg, France. This resulted in numerous websites going down and if you had a VPS in that data centre, all your data was gone unless you had your own backups.

Not all backups are created equal and they’re not enough to prevent you from losing data. Backups need to work in conjunction with a disaster recovery plan for maximum effect.

The 3 – 2 – 1 Backup Solution

A well-known rule exists for a good backup system following a 3-2-1 solution: 3 copies of data, on 2 different media, 1 off-site. Let’s break it down:

Three copies

That’s pretty easy. You want three copies of the file. 

Two different media

This is a little more complicated. Having two copies of the data on the same hard drive (same media) is NOT a backup. If the single drive fails, you’ve lost both copies. Likewise, having two copies in the same email account (same media) is NOT a backup. If that email service stops working, you’ve lost both copies. You don’t want a single failure to wipe 2 or more copies of your data. RAID is also not a backup – if something corrupts or gets encrypted by ransomware, that copy is gone.  To qualify as different media, your backup should be on separate storage – not on the same hard drive if you’re storing your data there. 

At least one offsite

This is the most important! If you have 1000s of critical copies stored in the same location, a flood or fire, like the OVH fire, will wipe out all of your data. At least one copy should be stored offsite. It could be cloud storage, in a remote server, or even on your local computer.  

Keep in mind that real-time sync without versioning control is NOT a backup. If the file is deleted, it’ll be deleted on remote backup. 

Disaster Recovery Plan 

A Disaster Recovery Plan (DRP) is a plan of action that businesses implement in case a disaster hits. Disruptions can lead to lost revenue, brand damage and dissatisfied customers. And, the longer the recovery time, the greater the adverse business impact. Therefore, a good disaster recovery plan should enable rapid recovery from disruptions, regardless of the source of the disruption. This plan should be detailed and tested before the actual disaster. So, if actual disaster occurs, everyone in your business will know how to react and cause less panic. Three elements should be addressed: 

  • Emergency response procedures to document the appropriate emergency response to a fire, natural disaster, or any other activities in order to protect lives and limit damages. 
  • Backup operations procedures to ensure that essential data processing operational tasks can be conducted after the disruption. 
  • Recovery actions procedures to facilitate the rapid restoration of a data processing system following a disaster. 

Incident Response Plan

DRP should also include an Incident Response Plan, a plan in case of data breach. There are 5 phases:

1. Preparation 

This phase will be the workhorse of your incident response planning, and in the end, the most crucial phase to protect your business.  Ensure that your employees are properly trained for their roles regarding the incident. Develop incident response drill scenarios and run through them with the employees.

2. Identification 

This is the process where you determine whether you’ve been breached. A breach, or incident, could originate from many different areas.  

3. Containment 

When a breach is first discovered, you might want to securely delete everything so you can just get rid of the breach. However, doing this will likely hurt you in the long run as you will be destroying valuable evidence that you need to determine where the breach started and devise a plan to prevent it from happening again. 

Instead, contain the breach so it doesn’t spread and cause further damage to your business. If you can, disconnect affected devices from the Internet and have a redundant system back-up to help restore business operations. That way, any compromised data isn’t lost forever. 

4. Eradication 

Once you’ve contained the issue, you need to find and eliminate the root cause of the breach. This means all malware should be securely removed, systems should again be hardened and patched, and updates should be applied. 

5. Recovery 

This is the process of restoring and returning affected systems and devices back into your business environment. During this time, it’s important to get your systems and business operations up and running again without the fear of another breach. You should also learn lessons from this breach to avoid it in the future. 

DRP in Action 

A great example of a disaster recovery plan in action would be CDPR’s recent ransomware attack. Back in February, this gaming company got hit by a ransomware with all of their servers getting all of their data encrypted. But because they had proper backups in place, they were back in business within 2 weeks. Although it sounds like a long time, a proper audit was needed to find and patch any vulnerabilities. If no audit is needed, you can have your servers back online almost immediately. Just spin up a new server and transfer all of your needed data back to the server. 

DRP Solutions 

Canadian Web Hosting offers solutions for DRP, but you can create a backup plan in case of disaster for yourself. Like the image below, all you really need is 2 servers, each located in different datacenters: the main server and your backup server with all of your data.

For a backup server, I would recommend installing FreeNAS. FreeNAS uses ZFS for snapshot control, so even if your data gets deleted or encrypted by ransomware you can roll back your snapshots and recover your data. 

Testing Your Backups

Before you go congratulating yourself for having good backups, verify that your backups actually work. Spin up a new server or delete data from your current server, and try to recover from your current backups. Many people forget this important step thinking that their backups are good but when disaster hits find out that their backup solution was badly implemented.