Misconfiguration - The Hidden IT Project Killer

Posted by Alan Sharp-Paul

There's a hidden killer lurking below the surface of every Enterprise IT project. No, it's not Kevin, that sysadmin who spends a disturbing amount of time in the bathroom each day. It's not even that 400 page requirements document, although from a conservationist's point of view the PM's insistence on reprinting it every few days can't be doing the world too much good. So what is it? Well, let me give you a clue:

Environment waterfall and misconfiguration during handoffs




Environment Waterfall

Clearer now? If you've not worked in a big corporate this may seem like an extreme number of environments to have to negotiate in getting a project through to completion but this is pretty common. I've certainly seen a lot worse. What does it mean in practice though? Well, while the adoption of agile or devops methodologies may have improved the collaboration between your IT teams and reduced your reliance on the waterfall approach to project delivery the fact is that from an environment perspective you're still forced to navigate the above linear road to production.

The Impact

It's a stretch to say this in most cases but let's assume that your Enterprise has successfully adopted agile development practices. From a functional perspective this is great. Issues with both requirements and implementation are discovered earlier in the process, saving valuable project time. What about configuration issues though? Who is tracking them?

It worked in DEV/UAT/STAGING

Ah yes, this old chestnut. It worked fine in my environment. If it's not working over the fence it must be your problem. That doesn't help anyone though. We accept that some things have to change between environments. We move from developer workstations to test VMs. In some instances (legacy systems) we may even move to dedicated hardware in production. The first thing we need is a way to identify what is different, not just between infrastructure we know is different but also between that which we think is the same.

Differencing is a highly valuable analysis tool but what about being proactive? If you have configuration standards why not capture them and enforce them throughout the environment lifecycle. If you try this in Word though good luck making it work. The docs will quickly become stale. They'll be worked around and eventually ignored altogether.

Automated Environment Validation

A key driver for us building UpGuard was our realization that when documentation is executable it remains relevant as it constantly validates, and is constantly validated by, the systems it defines. Imagine the difference in the migration process of a system between environments when automated validation is in place, not just at the migration point but right the way through the development process.

Environment Issues Kill Projects

Issues and misconfigurations between environments are truly project killers. One incorrectly promoted configuration file. One undetected version update on a critical binary. We're not talking about an easily debugged error on a web page. These are hidden nightmares that can take environments out of action for days, burning thousands of dollars in project delays.

Get it under control. Be proactive. Documentation is a good start but you need visibility of the state of your environments and the differences between them. You also need to automate environment validation to guarantee quality and insure yourselves against serious project delays.

Topics: configuration, devops

UpGuard Customers