Over the last few months, the recent spate of natural disasters has come up in many businesses discussions. One of Dataweave’s team, and several of our customers were caught up in the major floods in Brisbane, so for many it has been a deeply personal experience.
In my experience, good planning and preparation are the best ways to manage the risks involved with natural disasters. In addition to incorporating high availability Oracle database technologies into your IT systems, you should have an up-to-date disaster recovery plan. In essence your “plan B” – covering an alternate set of procedures and infrastructure to be used in case of a major outage of key services, or when building evacuation is required.
- which business operations will continue
- which IT services are offered on the alternate infrastructure
- what system and device reconfigurations which might be needed for your plan B
- how data migration and service continuity will occur between production environment applications and Oracle databases, and the systems used in the disaster scenario.
While most larger organisations have basic plans in place, it is still easy for things to fall apart when a disaster occurs.
Printed documentation might seem unnecessary, but it can make or break your team’s ability to respond quickly. As seen in the recent Brisbane floods, power and communication outages can be lengthy, and occur with relatively short notice. Printed documentation might be the only technical info you can access if mains electricity is cut for more than a few hours.
When a problem occurs, deciding when to cutover to a disaster scenario might not be clear. Key team members or executives may be unreachable and unable to make decisions, so documenting clear cut criteria for invoking a disaster scenario can greatly reduce delays and outages.
Technologies which automatically replicate and allow for automated failovers can clearly speed up transitions and improve overall IT department efficiency. Technologies such as Oracle Data Guard make it possible to automatically replicate your Oracle database data with automated seamless failover. Oracle Data Guard is included with your Oracle Database Enterprise Edition licence. For an additional fee, Oracle Active Data Guard will allow you utilise your DR server for online reporting, improving your production database performance.
When a cutover to the disaster plan is commenced, it is critical to understand how long it might actually take to carry out any reconfigurations needed to enable business operations. Dataweave works closely with clients to carry out regular “drills” and scenario rehearsals, so that they become part of ongoing staff training. Drills can be very helpful in ironing out glitches in the plan, as well as reducing potential downtime.
An often overlooked area is the transition back from disaster mode of operations.
For complex environments, the effort and downtime required to transition back to normal production systems might be considerable. In these situations, it is worth considering the cost benefit tradeoff for short downtime scenarios – evaluating whether full cutover to DR is necessary, versus a shutdown and restore.