Penetration testing (pen testing) is crucial for developing and maintaining hardened, attack-resilient systems—these can be applications, nodes, or entire networks/environments. Specialized tools are readily available for discovering vulnerabilities and security gaps in these systems; in this comparison, we'll compare Arachni and OWASP Zed Attack Proxy (ZAP), two popular security suites for application-level pen testing.
Unlike well-known pen testing tools like Kali Linux and Backbox that combine network, host, and software/web application testing capabilities, Arachni and OWASP ZAP are specifically designed to scan web applications for flaws. These tools (and others like them) alert testers of weaknesses that are readily exploitable by cyber attackers (e.g. a SQL Injection flaw or cross-site scripting issue).
Arachni and OWASP ZAP are two of the most popular web application pen testing tools on the market; fortunately, they are also both free and open source. ZAP is maintained by the Open Web Application Security Project (OWASP), a venerable online community and non-profit dedicated to improving software security, while Arachni is supported by Sarosys, the project's corporate arm that provides commercial services around the tool.
OWASP has been developing cutting-edge tools and resources for the general public since 2001, with the goal of improving software application security and overall online security. Its ZAP web application pen testing suite is one the world's most popular solutions for automatically finding vulnerabilities in web applications during development/testing. The project started as a fork of the popular Paros proxy, a Java-based tool for discovering application vulnerabilities and assessing web security fitness.
The OWASP ZAP UI. Source: sourceforge.net/projects/zaproxy.
ZAP is written in Java (alas, Java 7 is required) and is available for Windows, Linux, and MacOS platforms. The suite includes a range of security tools: an intercepting proxy, spider, scanner collection (automated/passive, brute force, port, web sockets), and REST API for custom integrations. ZAP is commonly used with CI/CD tools like Jenkins or Bamboo to add automated pen testing to a firm's continuous delivery lifecycle and CI/CD pipeline.
Arachni is a long-standing favorite among software pen testers, especially those partial to the Ruby programming language—the open source pen testing framework was written in Ruby and is highly extensible in this regard. In fact, one of Arachni's most lauded attributes is its scalability and modularity; for example, the tool can be used as a simple command line scanner utility or configured in a high performance scanner grid to support large-scale application security testing routines.
The Arachni interface. Source: arachni-scanner.com.
Side-by-Side Scoring: OWASP ZAP vs. Arachni
1. Capability Set
OWASP ZAP and Arachni are comprehensive and highly capable security testing suites in their own rights—impressive, considering their price tag. That said, the two open source tools have their limitations; firms tend to extract more value by integrating them into their CI/CD pipelines for automated security testing.
2. Ease of Use
Like it or hate it, both OWASP ZAP and Arachni's rudimentary and somewhat outdated UIs make for straightforward usability. Both solutions are easy to operate, but the experience won't be a feast for the eyes.
3. Community Support
Powered by the reputation and reach of OWASP, ZAP commands a larger community of followers and subsequent support resources. The tool came out with top honors in the 2015 Top Security Tools survey held by ToolsWatch.org, beating out tools like Burp Suite and Nmap (Arachni didn't place). Arachni's community resources are not as extensive as ZAP's, but it does offer a support portal with a sizable database of resources.
4. Release Rate
As open source projects, both pen testing suites have seen regular, albeit slow coming releases over the years. A fork of the popular Paros proxy, OWASP ZAP is currently on version 2.5; the Arachni framework is on version 1.5, and its WebUI (0.5.11) has yet to reach a full release.
5. Pricing and Support
Both solutions are free, open source solutions. OWASP ZAP is supported by a community of volunteer developers, online donations, and t-shirt sales.
Arachni derives some revenue from commercial services and support provided through Sarosys, its so-called "corporate branch" of the project.
6. API and Extensibility
Arachi comes with a well-documented REST API that enables the remote management of scans over a simple web service. Similarly, OWASP ZAP's REST API allows for interacting with the suite programmatically. And of course, both of their open source codebases are available via GitHub.
7. 3rd Party Integrations
Both offerings can be readily integrated with third party applications, but OWASP has a more comprehensive selection of pre-built integrations at its disposal. For example, its ZAP Jenkins plugin makes it easy to extend the functionality of the ZAP scanner into a CI Environment.
8. Companies That Use It
OWASP ZAP is used by countless organizations across the globe for validating their web application security postures, from governments agencies and educational institutions to large enterprises. Some of these include Mozilla, Microsoft, Ernst & Young, Accenture, and Google. Again, a fairly common ZAP implementation sees the framework integrated with Jenkins to automate security tests in a CI/CD pipeline. Arachni also finds itself integrated with Jenkins quite often (i.e, security tests automated/triggered by a Jenkins build) and boasts a similarly expansive footprint worldwide; some marquee users include Infobyte Security, eBay, Bentley Systems, Manwin, and Katana Security, among others.
9. Learning Curve
An intermediate proficiency with cybersecurity concepts and terminology is assumed with OWASP ZAP and Arachni; that said, it's unlikely that serious pen testing efforts would be left in the hands of infosec neophytes. Other than that, both tools are trivial to get up to speed with.
OWASP ZAP's web presence scores a CSTAR score of 696—respectable, but less-than-ideal due to security flaws like missing HTTP strict transport security, disabled DMARC, open server administration ports, and disabled DNSSEC. Arachni's is poor CSTAR score of 570.
Scoreboard and Summary
|Ease of Use|
|Pricing and Support|
|API and Extensibility|
|3rd Party Integrations|
|Companies that Use It|
|Total||4.6 out of 5||4.3 out of 5|
For world-class web application pen testing on a budget, either of these leading security tools will suffice. OWASP ZAP is more common in enterprise environments and with SaaS providers, especially as part of an integrated CI/CD pipeline with automated security testing in place. But if you're a Ruby software shop, Arachni's modular, high-performance Ruby framework is likely to be a better fit.