In my previous life, I played the role of a mildly successful branch manager for the nation's 4th
largest bank and during this time, I participated in many a campaign and many initiatives to drive business into my branch, bolster my monthly sales results and single handedly feel as if I was positively affecting the price for a share of my company's stock. Why does this matter? It doesn't, other than to provide a base to present an idea that stuck with me over the years and to provide a nice seguay on how to practically apply this concept in the world of IT Support
Preparing for the unexpected. As part of our internal profiling strategy at the bank, this was a question we would ask our clients as a way of probing deeper into their financial ‘hey nows' in the hopes of attracting them with additional services that the bank provided. Preparing for the unexpected is not a foreign concept to most, though I am sure that when you are with your friends, it is an idea that NEVER comes up. No one likes to talk about the unexpected, let alone prepare for the unexpected. We are all going to live until we're 100. . .maybe longer. . . right?
If I asked you to associate the concept of ‘Preparing for the unexpected' to the IT Services world, I am confident that our conversation would take an express elevator to the floor where we would get off and begin discussing your company's back up strategy. I am certain this is what the phrase conjures up, and rightly so. In the world of IT, the ‘unexpected' would in no doubt take the form of some sort of data or information loss and ‘Preparing for this' would in no doubt be identifying the best (albeit ‘best' may = most cost effective) type of back up strategy and disaster recovery plan to put in place. But I ask you, and challenge you to consider that this concept can be and should be applied to all spectrums of your IT environment. Whether it be a large arena like implementing a sound back up strategy that will affect everyone or a one on one interaction, perhaps with a principal of your company, where you thought that you had done your due diligence, but in reality may have been blindly walking through the paces, we can all benefit from the idea that we need to prepare for the unexpected. Emphasis is clearly placed on those ideas that will most certainly negatively impact a large audience, but many of us forget, including myself, that it is often the small one on one interactions that, if done with true thought and purpose can have resoundingly great effects but, if overlooked and neglected, can have an unsightly ripple effect that may seem to never end. I offer that preparing for the unexpected does not always need to be an overly involved, multi-tiered process involving input from many different resources. In reflecting on incidents and interactions from my past (I say that tongue in cheek), planning for the unexpected can be something as simple as a 10 minute meeting with an associate to review, plan and confirm or even checklist or work order sign off document.
This concept is not new, ‘Preparing for the unexpected', but in writing this, I am overwhelmed at the idea that I (I will only speak for myself on this point!)) continue to overlook the easy things in life, the things that should be absolute ‘gimme's', and still cannot understand why. It is dawning on me that perhaps rather than ‘Prepare for the Unexpected', I should begin ‘Planning for the Expected' - I may just surprise myself with the results.
Sign up for your free IT Consultation and find out how Trigon can help you with your Disaster Recovery Plan!
Everyone backs up the system state in their nightly backups of their critical systems. It is a best practice that anytime you make changes to the hardware or the software on a Windows 2003 Server or an XP workstation, you should be backing up the system state before you make the changes.
The system state includes the following files: the boot files (boot.ini, NDTLDR, NTDetect.com); the registry (including the COM settings); the SYSVOL (group policy and logon scripts); the active directory NTDS.DIT (on domain controllers) and if the certificate service is installed on the server, the certificate store.
As you know these are the critical files you will need to restore the operating system configuration to its previous working state if something goes wrong during your enhancement of the server. Having these backed up will give you a good fall back position.
To many times I have seen people make major changes to their servers or workstations without taking the simple precaution of backing up the system state. Having good backups from the night before is an adequate precaution, but to restore from the backup media might take time and it may not be the most recent system state. This is time in which the server is down and costing the company money because it is down. Having a recent copy of the system state will save you time, money, and give you a piece of mind when enacting changes on the server.
I like to use the ntbackup that is resident on the Windows 2003 Server OS and the XP workstation OS to backup the system state locally. The job will take between 3-5 minutes on average. Backing up the system state across a network can greatly increase this time to unacceptable ranges. With an investment of about 5 minutes you have taken an additional step to guarantee a successful rollback.
I have found that some companies even backup the system state during the day to catch any changes between backup jobs that run at night. Again, the system state is backed up locally, but some even transferred the backup file to a designated web share using a third party software such as Robocopy.
For those who are using Window Server 2008 variants or Vista variants, you will not find the ntbackup. The methods you will use will be the command line wbadmin.exe utility. An example of the command to use would be:
Wbadmin start systemstatebackup -backupTarget:D:
In this command example, the system state is being backed up to the D: drive.
By taking these simple extra steps you can enhance your recovery success from a bad install. A couple of minutes before you make changes to a system can make you a hero in the eyes of the users and clients!
Most IT professionals are often satisfied knowing that their backups complete successfully daily. They may even be satisfied with the occasional exception but are they prepared to restore in the event of failure? I like to use the following three questions as my model:
- What am I backing up?
- Do my backups work?
- What is my data retention policy?
I see all too often that an important folder is not being backed up. If you are responsible for your company's backups, you should routinely examine selection lists for all of your backup jobs. Verify if the backups contain everything that needs to be backed up. I prefer to backup everything as often as possible. Of course, this is dependant on media size and time limitations. You can never backup too much.
Periodically, I like to restore a few files or a few E-mails to test my backups. Routinely restore files from backups to verify that the back job are indeed working as expected. Exchange and Active Directory should also be tested in disaster recovery scenarios. If time and resources permit, build a recovery lab and verify that you could restore your entire network in the event of a failure.
Lastly, what is your data retention policy? How far back do you want to be able to restore from? And how available do you want it to be? There are many options here and it will vary company to company. Ideally, you will keep as much data as possible for as long as you can. Easier said than done. A model I like to use is a disk to disk to tape implementation with 5 days worth of backups online and 2 weeks on tape. The previous week set of tapes will be taken offsite to a safe location such as a safety deposit box. It is also a good idea to maintain monthly and even yearly tapes. Depending on the company type, there may be a mandatory data retention policy in place.