Blackout blues ...and what you can do to avoid them

By on

The recent power outage was a powerful reminder of the value of business continuity provisions. Ron Mobed reports.

Over the past two years, businesses, government departments and regulatory authorities around the world have been working steadily to increase the scope and level of their business continuity provisions

The massive power outage in North America this summer has highlighted the need for such preparedness, providing particularly graphic justification for the efforts made in the New York area, but also pointing out that more needs to be done as a whole.

The blackout affected people and organizations across some 950 square miles in the U.S. and Canada, revealing the network complexities and interdependencies that now exist. Like the events of 9/11, the blackout raised the bar on what is considered adequate provision.

Still, a core of first-class provision exists, but industry analysts and government audits consistently call for general levels to improve. The good news is that the subject is high on the agenda, and recovery options available are growing.

Out with the old and in with the new

Not all new business continuity thinking began with the terrorist attacks of September 11. However, those attacks did prompt businesses and governments to consider how well equipped they were to react to such events.

One fact remains clear: national and international commerce can be exposed to major disruption from many sources. The most extreme threat now conceived has increased in scale and the impact under consideration has moved from a local to a global stage.

In the months following the September 11 attacks, business continuity decisions were put on hold while organizations reconsidered all aspects of their continuity principles. Even extreme views were examined objectively as new solutions were sought. After a period of deliberation, which typically lasted until early 2002, a few common conclusions were identified: a collaborative approach is more effective than the prescriptive method dictated by regulations; all provisions have to be reviewed and, in most cases, upgraded; and the linkages between risk, security and business continuity would, in future, become stronger and more intricate.

The continuous testing and re-testing of business continuity plans is the only way to ensure recovery with minimum delay or disruption. Three factors of success necessary for any business continuity program are the right people, processes and technology.

People

People are the backbone of any business operation. Account must be taken of the reactions of staff and business partners to an event, be it traumatic or not. Any outage is stressful in itself, especially when a particularly intense or important business activity is involved. Training, testing and the provision of support mechanisms are key to ensuring that the necessary activities are performed.

Make sure the critical people have been through a realistic test. Make it clear where they should go and what they should do. Some companies even play out different scenarios and assign roles, all to reduce errors from lack of familiarity. And if people are having trouble coping during an event, counseling and support are essential, even if it means reducing a person's duties temporarily.

Planning

An organization's business continuity process is its formula for success. Plans need to be both detailed and flexible. Too much vagueness will lead to confusion and delay, but a rigid plan will fail at the slightest deviation from the expected scenario. Rapid communication of tactical directions to key employees at the outset of an event is a key factor in successful process management.

Recent outages have included both cell phone, landline, as well as internet communications failures. Organizations are challenged with the task of establishing both simple and multiple means of communicating with employees, partners, and vendors on an immediate basis. As a result, an investment must be made toward tightening up processes, as well as building in flexibility of response.

Having a network of aligned partners is vital in this respect. Going at it alone is not an option. A good plan is robust, flexible, up-to-date and tested. Don't ignore indirect impact - a fire in a nearby building may block your premises from escape or aide.

Technology

Technology is enabling a wider range of continuity options. For example, the cost of replicating large volumes of data has been falling to the point where complete datasets can be copied to a disaster recovery center in real time.

The two facilities can then be used together in day-to-day operations, with one taking over if the other fails. Thus the dividing line between what constitutes a center used for production, and one used for disaster recovery, becomes less clear.

Redirection of internet traffic to an alternative site is becoming more common as organizations recognize the importance of the internet for email and web services. Employees operating remotely can handle some processes. VPN services are attracting more attention. With the increased choice that these technologies offer comes a growth in complexity and potentially greater exposure to security issues. As a result, companies are searching for more complete business continuity solutions - widening the scope to cover networks, operational facilities, managed services and security. But remember the basics too. Are your backups in the same building as your data center? Have you recently tried to restart using those backups?

Where is the money going?

Financial institutions have traditionally been the heaviest investors in business continuity planning. While largely due to the high impact of even a short outage, it is also based on the experience of severe events in the past, such as the bombing attacks in London in the 1990s.

The financial services industry, especially banking, has been scrutinized closely by regulatory authorities and detailed consultations have taken place, notably in the U.S. and the U.K. A number of documents, mostly guidelines, have been published, and more are expected. Many larger companies feel confident that their current provisions meet those guidelines. Attention is now turning to smaller institutions and insurance companies.

The public sector has also invested heavily in business continuity. Initiatives that support Homeland security compliance for local, state and federal government agencies have spawned business continuity activities. In health care, public services, and utilities, regulations are being introduced which include the need for adequate business continuity.

Steps are being taken in other countries too. The U.K. has always been a leading exponent of business continuity, and private and public sector investment in it continues to grow. Singapore has recently introduced a government-sponsored education and assessment program for businesses. In Australia, government bodies are reviewing their own continuity plans and are working with companies providing critical infrastructure.

The result is a general increase in spending on business continuity. However, given today's business climate, that spending is closely monitored for effectiveness. Because not all activities in an organization need to be available in the same timeframe, and because the cost of providing availability rises dramatically with the speed of recovery, it is vital to segment and prioritize the critical processes in an organization.

To outsource or not to outsource

The main benefit of in-house provisions is they allow complete control of the continuity process. However, in-house provisions can isolate a company from the network of alternative provision, including facilities, equipment and suppliers, that the larger business continuity providers are able to draw upon.

Some vendor-managed, dedicated solutions deliver the necessary control, while at the same time, through economies of scale, provide a mechanism to spread costs across a broader infrastructure - including savings in phone systems, communications networks, power infrastructure, support staffing and the ability to avoid "over buying" real estate.

Additionally, third-party providers can present a shared option at a fraction of the dedicated price. With sharing options it is important to analyze the terms of the sharing agreement and the procedures upon invocation. The first-come, first-served model may not be appropriate where many subscribers are located in close proximity to each other. An equitable-share, guaranteed-minimum model, seems better suited.

A look to the future

While many practical steps are being taken, there is also recognition that in the future business continuity, risk and security will be more closely linked.

Understanding those links helps ensure that necessary and practical business continuity decisions integrate with and take advantage of the wider security canvas. Rather than treat business continuity as a standalone issue, many regulations actually cover operational risk, of which business continuity is a fundamental part.

The Basel II accord on Operational Risk has sections on business continuity, as do the relevant documents from the U.K. Financial Services Authority (FSA). Those responsible for business continuity arrangements often report by solid or dotted-line relationships to risk management and CSOs.

Security could be viewed as a continuum. At one end there are basic point solutions such as IDS, firewalls and patches to cover specific intrusions and failures. As the size, scope and scale of an incident increases, the opposite pole is reached, with business continuity sites being invoked due to a complete failure of a facility. This view helps determine the key decision points in the changing landscape of possible risks, threats, events and correlated responses and provides a working model for integrated, scalable management of incidents of any nature.

On a practical level again, business continuity facilities and services must be secured at levels that correspond to those in force at primary business locations, or go to even higher levels. There must be a "security fit" between the processes, equipment and standards of data center security, so that the business continuity facility operates as an extension of the normal working environment, neither compromising security nor creating unnecessary barriers.

With that said, business continuity is evolving on both tactical and strategic levels - focusing on the requirement that "this has to be done right." The tactical steps are seen in steady improvements to existing coverage, increasing coverage to previously neglected processes and a greater willingness to use technology to find new solutions to old issues.

Strategic steps, on the other hand, include more communication and cooperation between organizations, and a far stronger connection between the treatment of business continuity, security and risk. And as part of both of these, integrating security throughout the business continuum is the new model for adjusted security policy.

However you look at it, better preparedness for more detailed processes at the right cost is the future of the industry.

Ron Mobed is business continuity director, SchlumbergerSema (www.slb.com)

Copyright © SC Magazine, US edition
Tags:

Most Read Articles

Log In

Username:
Password:
|  Forgot your password?