During the last decade and a half, we have built disaster-recovery/business-continuity plans for hundreds of diverse organizations. In each of these situations, we have found that all planners, no matter how experienced and systematic they may be, tend to overlook certain items. Some of these are "small but crucial" items which simply add insult to injury when disaster occurs, but other can threaten the survival of the organization.
These overlooked items fall into a number of general areas:
- missing things "too close to see"
- failure to track out-of-the-ordinary situations
- intuitively assuming how other departments function
- forgetting "unforgettable" events
- ignoring "external" factors
- not keeping “outside” emergency organizations up-to-date
- missing things "too close to see"
An example of this oversight, which we have seen in almost 100% of the facilities we have examined, is the vulnerability of network and telephone "panels" to falling water. Most facilities have a panel on a wall where the telephone wires enter the building. Typically this is not covered and in the event of water coming down the wall, they "burn out" and are destroyed.
Similarly, most rack-mounted computer "networking" hubs are also not covered. This is in order to reduce heat buildup. However, they are typically not shielded from nearby sprinkler heads or simply from water coming from above the ceiling and following the cables down to the rack. When they are hit by water, they also burn out.
Another overlooked item we have frequently seen, particularly in older buildings, is the need for a key to exit a building via a locked door. Often government institutions which are also open to the public, lock their doors but remain open for many hours after their public hours have ended. Every exit door needs to be operable from the inside without a key.
Another frequent condition is the use of non-fireproof safes to protect key documents. All safes are burglar-proof, however, most are not insulated, and in a fire, the contents are incinerated.
- failure to track out-of-the-ordinary situations
Many disaster plans become fixated only on full-blown disaster events, rather than trying to also track "non-normal" situations which can make the disaster worse. One of the most frequent out-of-the-ordinary situations to occur is that of temporarily disabled employees (usually temporarily on crutches) - requiring special assistance to exit the facilities, and perhaps unable to carry key materials with them. Planners need to have a system in place whereby these employees are identified and have someone tasked with ensuring their safety and their ability to carry out their assignments during a disaster.
Another situation occurs when key employees go on maternity, military or other extended absence and are replaced by their backups. This can leave their functions without emergency backups, and can also affect the areas for which they themselves were the emergency backups.
- intuitively assuming how other departments function
All organizations depend on a series of support functions. The most typical of these are: janitorial, mail delivery, check printing, voicemail, and personnel. Because of their familiarity, planners often don't spend sufficient time going through the details of their operations. Some of the resultant oversights we have found include:
- lack of a procedure to track locations and proper protection of hazardous materials used by janitors and exterminators, which could, in a flood or fire, be spread throughout the facility,
- a mail room having no telephone numbers off-site to enable notification of couriers and
overnight delivery services where to deliver when the facility is not in operation,
- personnel files stored non-fireproof file cabinets,
- an accounting department with a customized check-printer, using blank check stock - requiring over a week to replace the printer,
- lack of an ability to access employees' voicemail if they are incapacitated.
- forgetting "unforgettable" events
Almost all organizations we've worked with have had major disasters which have entered into their institutional lore. However, typically they fail to document the details of what went wrong, what went right, what they've learned, what they need to change for the future, and a tracking of follow-up on the recommended changes. Planners need to maintain a separate logbook documenting these events, and tracking the implementation of the recommendations.
- ignoring "external" factors
Many organizations don't take proper cognizance of the fact that they are located nearby to or within areas which can be a focus of demonstrations or targets of violence. An example we encountered after September 11, 2001, was a non-military organization with a second office and backup records-storage site within that office, located on an Air Force Base, hundreds of miles away from the terrorist attacks. As a result of the attacks, the base was closed to non-military personnel and the organization was unable to operate or gain access to that facility for an extended period of time.
Beyond simply looking around their own buildings, planners need to be involved in and incorporate thinking from outside region-wide disaster-planning organizations in order to acquire this wider perspective.
- not keeping “outside” emergency organizations up-to-date
Particularly in this age of governmental consolidation and reorganizations, many departments have taken on new responsibilities covering new locations. We have often found that the local fire and police departments covering these locations often do not have up-to-date contact information in the event of an emergency in the local facility. For more complicated facilities, we have almost never found that a set of building plans has been filed with the local fire department. Planners need to obtain a confirmation from each local site as part of their periodic disaster-plan review, that all permissible and relevant information has been communicated to the local authorities.
These overlooked items fall into a number of general areas:
- missing things "too close to see"
- failure to track out-of-the-ordinary situations
- intuitively assuming how other departments function
- forgetting "unforgettable" events
- ignoring "external" factors
- not keeping “outside” emergency organizations up-to-date
- missing things "too close to see"
An example of this oversight, which we have seen in almost 100% of the facilities we have examined, is the vulnerability of network and telephone "panels" to falling water. Most facilities have a panel on a wall where the telephone wires enter the building. Typically this is not covered and in the event of water coming down the wall, they "burn out" and are destroyed.
Similarly, most rack-mounted computer "networking" hubs are also not covered. This is in order to reduce heat buildup. However, they are typically not shielded from nearby sprinkler heads or simply from water coming from above the ceiling and following the cables down to the rack. When they are hit by water, they also burn out.
Another overlooked item we have frequently seen, particularly in older buildings, is the need for a key to exit a building via a locked door. Often government institutions which are also open to the public, lock their doors but remain open for many hours after their public hours have ended. Every exit door needs to be operable from the inside without a key.
Another frequent condition is the use of non-fireproof safes to protect key documents. All safes are burglar-proof, however, most are not insulated, and in a fire, the contents are incinerated.
- failure to track out-of-the-ordinary situations
Many disaster plans become fixated only on full-blown disaster events, rather than trying to also track "non-normal" situations which can make the disaster worse. One of the most frequent out-of-the-ordinary situations to occur is that of temporarily disabled employees (usually temporarily on crutches) - requiring special assistance to exit the facilities, and perhaps unable to carry key materials with them. Planners need to have a system in place whereby these employees are identified and have someone tasked with ensuring their safety and their ability to carry out their assignments during a disaster.
Another situation occurs when key employees go on maternity, military or other extended absence and are replaced by their backups. This can leave their functions without emergency backups, and can also affect the areas for which they themselves were the emergency backups.
- intuitively assuming how other departments function
All organizations depend on a series of support functions. The most typical of these are: janitorial, mail delivery, check printing, voicemail, and personnel. Because of their familiarity, planners often don't spend sufficient time going through the details of their operations. Some of the resultant oversights we have found include:
- lack of a procedure to track locations and proper protection of hazardous materials used by janitors and exterminators, which could, in a flood or fire, be spread throughout the facility,
- a mail room having no telephone numbers off-site to enable notification of couriers and
overnight delivery services where to deliver when the facility is not in operation,
- personnel files stored non-fireproof file cabinets,
- an accounting department with a customized check-printer, using blank check stock - requiring over a week to replace the printer,
- lack of an ability to access employees' voicemail if they are incapacitated.
- forgetting "unforgettable" events
Almost all organizations we've worked with have had major disasters which have entered into their institutional lore. However, typically they fail to document the details of what went wrong, what went right, what they've learned, what they need to change for the future, and a tracking of follow-up on the recommended changes. Planners need to maintain a separate logbook documenting these events, and tracking the implementation of the recommendations.
- ignoring "external" factors
Many organizations don't take proper cognizance of the fact that they are located nearby to or within areas which can be a focus of demonstrations or targets of violence. An example we encountered after September 11, 2001, was a non-military organization with a second office and backup records-storage site within that office, located on an Air Force Base, hundreds of miles away from the terrorist attacks. As a result of the attacks, the base was closed to non-military personnel and the organization was unable to operate or gain access to that facility for an extended period of time.
Beyond simply looking around their own buildings, planners need to be involved in and incorporate thinking from outside region-wide disaster-planning organizations in order to acquire this wider perspective.
- not keeping “outside” emergency organizations up-to-date
Particularly in this age of governmental consolidation and reorganizations, many departments have taken on new responsibilities covering new locations. We have often found that the local fire and police departments covering these locations often do not have up-to-date contact information in the event of an emergency in the local facility. For more complicated facilities, we have almost never found that a set of building plans has been filed with the local fire department. Planners need to obtain a confirmation from each local site as part of their periodic disaster-plan review, that all permissible and relevant information has been communicated to the local authorities.
====================
Steven Lewis, Ph.D. is the president of The Systems Audit Group, Inc, in Newton, MA. He is a Certified Information Systems Auditor (CISA) with a PhD in Systems from the Univ. of Pennsylvania, and a Masters and Bachelors in Engineering from Cornell University.
During the last fifteen years, he has developed over 120 comprehensive disaster recovery/business continuity plans for networked based organizations. Many of these also included "Year/2000" risk analyses, evaluation and testing.
Steven Lewis, Ph.D. is the president of The Systems Audit Group, Inc, in Newton, MA. He is a Certified Information Systems Auditor (CISA) with a PhD in Systems from the Univ. of Pennsylvania, and a Masters and Bachelors in Engineering from Cornell University.
During the last fifteen years, he has developed over 120 comprehensive disaster recovery/business continuity plans for networked based organizations. Many of these also included "Year/2000" risk analyses, evaluation and testing.
Dr. Lewis has also authored numerous articles, including “Plan for a Disaster Without Destroying Your Budget,” which appeared in PUBLIC RISK magazine, and “Disaster Recovery Planning: A HIPAA Requirement,” which appeared in HEALTH FACILITIES MANAGEMENT magazine.
No comments:
Post a Comment