As IT managers and engineers, we can sometimes get so deep in the details of what we do that we struggle to answer the simple questions for our user base and the higher ups. Sure, we can write scripts to automate builds and we can train users on the tools to implement configuration management, but we can also freeze when asked why organizations should have configuration management teams, processes and tools. If this has ever happened to you, remember that you’re not alone.
1. Why Is Configuration Management Important?
Configuration management (CM) ensures that the current design and build state of the system is known, good & trusted; and doesn’t rely on the tacit knowledge of the development team. It is sometimes referred to as IT automation. Being able to access an accurate historical record of system state is very useful – not only for project management and audit purposes, but for development activities such as debugging (for example, knowing what has changed between one set of tests and the next to help identify what could possibly be causing a fault).
Some of the key benefits of configuration management include:
Increased efficiencies, stability and control by improving visibility and tracking.
Cost reduction by having detailed knowledge of all the elements of the configuration which allows for unnecessary duplication to be avoided.
Enhanced system reliability through more rapid detection and correction of improper configurations that could negatively impact performance.
The ability to define and enforce formal policies and procedures that govern asset identification, status monitoring, and auditing.
Greater agility and faster problem resolution, thus giving better quality of service.
Decreased risk and greater levels of security.
More efficient change management by knowing what the prior structure is in order to design changes that do not produce new incompatibilities and/or problems.
Another way to answer this question is to consider where configuration management gets applied to the service lifecycle in enterprise IT. With DevOps taking the IT world by storm, it is important to articulate that configuration state is critical during the entire execution phase and even more important as handoffs occur throughout each stage. As you can see from the diagram below, it is a critical component throughout the plan, build, run and govern processes and handoffs that occur daily within an typical IT enterprise - whether you’re ‘doing DevOps’ or not.
2. What Happens If We Don’t Do Configuration Management?
There’s a commercial that you may have seen. An auto mechanic talks about a costly engine repair that could have been avoided if the car’s owner had replaced his oil filter. The mechanic says, “You can pay me now, or you can pay me later.” The quote is just as valid with regard to configuration management.
You can avoid the costs associated with configuration management by not bothering to employ it in your enterprise. If you do, however, you’ll probably pay instead in costs for:
Figuring out which system components to change when requirements change.
Re-doing an implementation because you implemented to meet requirements that had changed and you didn’t communicate that to all parties.
Losing productivity when you replace a component with a flawed new version and can’t quickly revert to a working state.
Replacing the wrong component because you couldn’t accurately determine which component needed replacing.
The reason that configuration management is included as a key systems engineering practice is simple. It works! It keeps you from incurring costs preventatively and helps IT stop fire fighting. And, good systems engineers have learned, through practical experience, that it pays for itself many times over.
The lesson to learn is simple: Don’t pay the price later! Use configuration management to focus on fire prevention, not fire fighting.
3. Can You Give Me A Real-World Example of Configuration Management?
One of the best examples I know of is what the New York Stock Exchange (NYSE) dealt with back over 10 years ago. A software “glitch” prevented the NYSE from trading stocks for almost 90 minutes. The financial markets felt the impact even beyond the NYSE trading floor. Since investors couldn’t calculate market indexes without NYSE data, trading also stopped at the American Stock Exchange and some futures and options markets. It also slowed trading on the NASDAQ Stock Market, due to investor reluctance to do business without information on NYSE trading. A new software installation caused the problem. The NYSE had installed the software on 8 of its 20 trading terminals and the system tested out the night before. However, the morning of June 8th, it failed to operate properly on the 8 installations. The NYSE tried to switch back to its old software, but was unable to do so before the opening of the trading session. Although you might see this as a failure of the NYSE’s configuration management process, in reality, it was a success. Although the problem didn’t arise until right before the opening of trading, the NYSE recovered from the problem relatively quickly. The computer system problems caused some red faces at the NYSE, but they minimized the damage. They were back up and operating within 90 minutes. Had the problem continued longer, the repercussions would have been more severe.
4. How Much Does Configuration Management Cost
Configuration management doesn’t come free. One cost associated with it is the cost of the configuration management system itself. The configuration management systems market place changes frequently and there are many different solutions out there that range from open source projects to commercial off the shelf technology sold by the likes of Microsoft & IBM. Typically the costs for these configuration management tools are ‘by node’ with varying price points depending on the type of system (servers, databases, network devices, mobile devices, storage, virtual instances, et. al). It is important to remember, however, that you don’t always get what you pay for and it shouldn’t have to be as expensive as many of the larger vendors make it.
Another important cost is that of administering the configuration management system itself. The contractor on a CM project may handle the project’s configuration management responsibilities from a set-up and/or ongoing administrative perspective, so be sure to plan for this and incorporate these costs into the initial budget submission. Configuration management is an ongoing need as the system evolves and requires maintenance over time. You should integrate these ongoing configuration management needs into your organizational structure and make sure that you’re planning for these costs over time.
5. What is the ROI for Using Configuration Management?
Some of the financial benefits contributing to positive ROI results found in leveraging configuration management include:
IT staff productivity increase. Optimizing IT staff activities through automation reduced IT staff time spent "keeping the lights on", freeing up valuable staff resources for business-related initiatives.
User productivity increase. User downtime caused by system outages, cyber attacks, security intrusions, and change and configuration activities is reduced.
IT cost reduction. Optimizing IT operations reduces costs in multiple areas, including infrastructure, outsourced services and management software.
There is plenty of ROI calculations that you can apply for your business that consider things like number of devices/nodes you want to apply configuration management to, the hourly cost (try $35/hour conservatively) it takes a system/network engineer to handle manually and then think about the cost when (not if) a system outage occurs with no backup configuration, when a bulk configuration update/new rollout needs to happen, or when a configuration audit request has to be responded to. These activities take time and time is money.
Over time, IT systems and their configuration items (CIs) invariably move towards a state of disorder. Left unchecked, these continuous changes to the environment’s software and hardware result in performance degradation, unanticipated downtime, data loss, non-compliant systems, cybersecurity events, and data breaches.