The importance of incident response downtime

This year I changed teams at the dayjob and and I've had some time to compare and contrast a few incident-response models that I've seen or participated in over the years.

The big change for me was that I went from four hours a day in meetings down to two hours a week.

Not only has my productivity gone up, but I feel much less stressed and harried, even when an incident at-hand is severe.

Common incident-response team models

I've seen incident-response teams organised into one of three ways. The choice of model seems to be dependent upon the size of the environment and the level of perceived threat.

They can be described as:

• One-man-band
• Volunteer fire department
• Police department

One-man-band

The one-man-band invariably occurs in small organisations where they have only one full or part-time IT person.

I've been in the shoes of these jacks-of-all-trades and I don't envy them: They are either already master jugglers or facing a nervous breakdown.

The only advice I have for them is to try and track time spent on:

• Building the infrastructure (design, or new installs)
• Maintaining the infrastructure (upgrades, patching, what passes for trouble tickets in your environment.)
• Defending the infrastructure (maintaining security tools, training users)
• Responding when the defense fails (dealing with a compromise, or infected system)

This may help you organise your budget or lobby for more help.

Volunteer fire department

An organisation that is large enough to have a real IT staff, but not big enough or under enough (perceived) threat to justify full-time security IT staff falls into this category.

Individuals with the appropriate skills or desire may be tapped from time to time to help respond to the periodic infection, or intrusion.

As the threat or exposure grows, it seems that some people will be always responding to incidents. At this point it is time to move to the next model.

Police department

An organisation with a dedicated security staff can also become overwhelmed with the challenge of balancing the constant flow of events, and ongoing improvement of environment security.

Staff can either be pigeon-holed into tasks, or expected to know everything. Managing the personnel and their time can be very challenging.

Stand-by time

Stand-by or on-call time is set aside in the daily schedule that is devoted to incident-response.

It should focus on the first stage of incident-response, or preparation.

It's time spent keeping up to date on security news and events, updating documentation, and building tools and response processes.

Stand-by time is an interruptable time should an incident arise, but it's not interruptable for other meetings or projects.

It is important because incident response members need time to step out of event-stream and gather thoughts so they can produce the documentation the organisation needs.

Similarly, if an incident-responder is tasked with organising and managing a long-term project, but does not have time to organise and manage that project, it's going to suffer.

Scheduling stand-by time

Ideally, someone on the dedicated incident-response staff should be on stand-by during hours of operation.

This is not on-call time, but time spent in the office doing the job of keeping up to date and preparing.

Overlapping this time with other members will help in collaborating both tool-building and updating processes.

Daily chunks of stand-by time can give the incidence response team time to improve documentation.

Via the Internet Storm Center

Copyright © SC Magazine, Australia