TeamCity vs Jenkins for Continuous Integration

Posted by UpGuard

TeamCity vs. Jenkins

Continuous Integration (CI) is one of the formative concepts behind DevOps, driven by a need to regularly integrate new and changed code back into the master repository, and is often combined with Continuous Delivery (CD) to achieve faster and more stable builds with automation. Teams compile software, and run it through a series of tests in a production-identical development environment to ensure the success of the build. The logic behind this is wonderfully simple, though it only came about in response to the problems of the traditional deployment cycle: the more often you build and test during development, the less you have to worry about each time. Instead of having a D-Day, where the software will finally be compiled and run in production for the first time, continuous building and testing makes the go-live date just another routine deployment.

Jenkins and TeamCity serve as CI tools that allow developers to integrate code branches during the development process and run a series of automated tests against them. Although they cover much of the same ground, they do so in very different ways, given that Jenkins is an open source project, while TeamCity is a proprietary offering by JetBrains. In our comparison, we'll see how they stack up against each other, discuss some of their similarities and differences and take a look at what makes each a good fit for certain environments. 

Continuous Integration and Upguard

TeamCity

TeamCity's slogan "Powerful Continuous Integration out of the box" speaks to what they offer in their solution over and above a free option like Jenkins: out of the box usability. Its slick graphical interface and ease of use features make it more palatable for some new to Continuous Integration. TeamCity runs in a Java environment, usually on an Apache Tomcat server, though it can be installed on both Windows and Linux servers. TeamCity offers equal support for .NET and open stack projects, integrating into both Visual Studio and IDEs like Eclipse. 

Example of the TeamCity UI

Team City Interface (Source: jetbrains.com)

TeamCity has converted many Jenkins faithful with its interface and secure default configuration. JetBrains offers a free edition as well, with up to 20 build configurations and 3 build agents. Scoping beyond that requires their commercial "enterprise" edition. 

Jenkins

Jenkins, formerly Hudson, is an open source Continuous Integration and Continuous Delivery tool used widely and considered by many to be the de facto standard for CI. Jenkins also runs in Java, so you can install it on Windows or Linux. Highly extensible and with a gallery of plug-ins readily at hand, Jenkins offers granular customization for any size CI operation. But as you can tell by comparing the two screenshots, Jenkins' interface is more reminiscent of the last generation of web applications.

Example of the Jenkins UIExample of the Jenkins 1.x web interface. (Source: wiki.jenkins-ci.org)

If you're considering CI, you're considering Jenkins. But like many feature rich, open source projects, Jenkins gets a reputation for having a difficult setup and configuration process, which is the gap TeamCity tries to fill with its commercial offering.

Update, Friday May 13th: Jenkins 2.0 has improved the user interface significantly. See here for details on what's new.

Example of Jenkins 2.0 Stage View interface(Example of the Jenkins 2.0 interface, Stage View. Source: jenkins.io)

Side-by-Side Scoring: TeamCity vs. Jenkins

1. Capability Set

Really, both of these are quite capable products. Jenkins leads the way for CI, as it has for some time, so its feature set is quite rich, offering many different ways to achieve both CI and Continuous Delivery. TeamCity may not be exactly matched with Jenkins when it comes to the capability set, but the differences are small and TeamCity will serve most CI operations just as well. Good news for TeamCity, who wouldn't be able to compete if they didn't offer most of the same capabilities as their open source counterpart.

Capability Set

TeamCity score_5.png
Jenkins score_4.png

2. Ease of Use

TeamCity's interface, simple setup and out of the box security give it a leg up over Jenkins when it comes to ease of use. The selling point for any commercial alternative to an open source product is that it reduces the administrative overhead by visualizing or otherwise simplfying access to the functionality set. This isn't to say that Jenkins is not usable, because it is, and has been used well by many people, but for those who can afford it and don't have the time or inclination to learn Jenkins in and out, TeamCity offers a more usable interface for the same basic technology. 

Ease of Use

TeamCity score_5.png
Jenkins score_3.png

3. Community Support

Both TeamCity and Jenkins have community support available from their website, allowing users of their software to share information and troubleshoot problems collectively, as well as providing a knowledge base to trawl through when looking for specific answers. Jenkins has a slight advantage here, if only because they have been around longer and as an open source project, Jenkins users rely more on their community for troubleshooting. 

TeamCity score_4.png
Jenkins score_5.png

4. Release Rate

TeamCity is currently on release 9.1.7, which came out on May 4th, 2016. Their releases have been regular and well documented, and their last major version release (9.0) came out December of 2014. Jenkins' release history is quite impressive, summarized on their change log, with detailed information about each release. Version 2.2 went live on May 8th, 2016, with major version 2.0 debuting April 20th of this same year. As they make CI products, these companies know the proof is in the pudding, so it should come as no surprise that their release processes are high quality.

Release Rate

TeamCity score_760.png
Jenkins score_570-2-1.png

5. Pricing and Support

As of May 2016, TeamCity offers two versions of its CI product. TeamCity Professional is free for up to 20 build configurations and 3 build agents and additional licenses can be purchased for $299 that include 1 build agent and 10 build configurations. TeamCity Enterprise allows for an unlimited number of build configurations and licenses start at $1999 for 3 build agents, scaling up to 100 agents for $21,999. As we've mentioned, Jenkins is open source, under the MIT license, and as such can be downloaded and used for free. For this reason, smaller companies may want to approach CI from a "why wouldn't we use Jenkins" perspective. One potential answer to that question might be because your organization lacks the manpower, knowledge or capability to organize and manage a Jenkins solution. TeamCity offers professional support services, whereas support for Jenkins mostly comes from the community and online resources, though third party companies offer Jenkins-specific professional services.

 

Pricing and Support

TeamCity score_2.png
Jenkins

score_5.png

6. API and Extensibility

Both TeamCity and Jenkins offer a RESTful API. Jenkins' API comes in three flavors: XML, JSON and Python. Jenkins has greater extensibility as nearly every part can be customized as needed. Their extensibility page has excellent resources for almost every kind of possible customization. TeamCity has far less information available on their site, another difference between the commercial and open source offerings; however, TeamCity users seem pleased with the API overall.

API and Extensibility

TeamCity score_570-2-1.png
Jenkins score_570-2-1.png

7. 3rd Party Integrations

Plug-ins are the name of the game for Jenkins, who offer hundreds of free plug-ins, from source code management, to build tools, to language-specific development tools. This pre-existing database of useful plug-ins makes fitting Jenkins into your environment much easier, hopefully sparing the need for costly in-house or contracted customization. TeamCity also has over 100 free, crowd developed plug-ins available, which they offer on an as-is basis, outside of their supported commercial product.

3rd Party Integrations

TeamCity score_570.png
Jenkins score_570.png

8. Companies that Use It

TeamCity boasts a wide range of top companies as customers, including internet powerhouses Twitter, eBay and Wikipedia. Jenkins has no lack of big names on its list of users either, Dell, Tumblr, and Netflix among them. Clearly both of these products are capable of operating at a high level at even the largest scale.

Companies that Use It

TeamCity score_570.png
Jenkins score_570.png

9. Learning Curve

As with ease of use, the learning curve on TeamCity is shorter than Jenkins as part of the value they offer for their commercial product. People new to CI itself might have an easier time grasping the concepts and procedures in TeamCity's superior interface, instead of adapting to Jenkins' old-style UI. That said, documentation abounds for Jenkins, and salty sysadmins might prefer its Linux-like essence to the proprietary blackbox of TeamCity.

Learning Curve

TeamCity score_570.png
Jenkins score_3.png

10. CSTAR SCORE

Both JetBrains and Jenkins have external CSTAR scores in the "good" range, with TeamCity having the higher score thanks to an enabled DMARC configuration on their email in DNS.

CSTAR Score

TeamCity 884 (good)
Jenkins 789 (good)

 

Scoreboard and Summary

  TeamCity Jenkins
Capability Set score_5.png score_5.png
Ease of Use score_5.png score_3.png
Community Support score_4.png score_570.png
Release Rate score_570.png score_570.png
Pricing and Support score_2.png score_5.png
API and Extensibility score_570.png score_570.png
3rd Party Integrations score_570.png score_570.png
Companies that Use It score_570.png score_570.png
Learning Curve score_570.png score_3.png
CSTAR Score 884 789
Total  4.3 out of 5  4.5 out of 5

It's a close outcome, but the big differences between the two products should guide you: do you need an endlessly extensible open source solution that you're willing to learn, or would you rather pay up front for a commercial solution geared to increase usability? Both solutions can run your CI/CD environment well and look to only be improving on their offerings into the future. Keep in mind that CI and CD are great, but stand to be complemented by a visibility solution, such as UpGuard. Ensuring your configurations match your expectations is the important first step toward CI, CD and automation. 

Free DevOps Toolchain eBook

More Articles

Cisco vs. FireEye for Continuous Security

Who provides better continuous security: the world's largest maker of networking equipment or the first cybersecurity firm certified by the U.S. Department of Homeland Security?
Read Article >

ITIL vs. DevOps

People seem to have a hard time deciding what DevOps even is, much less how (or whether) it compares to a highly structured methodology like ITIL.
Read Article >

Tripwire Vs. Puppet

We’re going to compare Tripwire to Puppet here, not necessarily as identical tools, but how they fit into an IT environment.
Read Article 

 
 

Topics: devops, teamcity, jenkins, continuous integration

UpGuard customers