Information technology has changed the way people do business. For better, it has brought speed, scale, and functionality to all aspects of commerce and communication. For worse, it has brought the risks of data exposure, breach, and outage. The damage that can be done to a business through its technology is known as cyber risk, and with the increasing consequences of such incidents, managing cyber risk, especially among third parties, is fast becoming a critical aspect of any organization. The specialized nature of cyber risk requires the translation of technical details into business terms. Security ratings and cyber risk assessments serve this purpose, much like a credit score does for assessing the risk of a loan. But the methodologies employed by solutions in this space vary greatly, as do their results.
BitSight and SecurityScorecard focus on external cyber risk management. UpGuard handles internal risk with our behind-the-firewall Core product, and external risk with the cloud-hosted CyberRisk. In this comparison, we’ll look at what matters in a solution and see how BitSight, SecurityScorecard and UpGuard stack up for managing internet-facing primary risk and third party vendors.
Cambridge-based BitSight offers a platform for quantifying the external cybersecurity posture of organizations using publicly accessible data. The resulting FICO-like cybersecurity ratings can then be used in various use cases: shaping/pricing cyber risk insurance policies, due diligence research for private equity and M&A activities, and more.
The BitSight UI. Source: bitsighttech.com.
Additionally, the BitSight Discover solution identifies and monitors third/fourth parties to highlight security risks and single points of failure in an organization’s supply chain.
Like BitSight, NYC-based SecurityScorecard uses data gleaned from traffic to/from an organization as well as other publicly accessible data to build security ratings for evaluating vendors and partners, pricing cyber risk insurance policies, among other use cases. The platform also monitors so-called "hacker chatter", social networks, and public data breach feeds for indicators of compromise. Additionally, the company offers its ThreatMarket database on a standalone basis and its Malware Grader tool for free.
The SecurityScorecard UI. Source: securityscorecard.com.
UpGuard was founded in 2012 by technologists from some of Australia’s largest banks. Using their first-hand experience, they built a platform to fill an important need in the nascent DevOps market: reducing the risk of incidents through proactive documentation and testing of desired system state. With proprietary, patented data visualization and risk analysis algorithms, UpGuard gave Operations and Security departments the ability to discover and understand their digital surfaces within the data center and the cloud. Now, those capabilities have been turned outward to the public internet to capture the digital surfaces beyond the boundaries of the enterprise. UpGuard is headquartered in Mountain View, California with offices in Sydney, Australia.
Side-by-Side Scoring: BitSight vs. SecurityScorecard vs. UpGuard
1. Capability Set
All three solutions provide security ratings, aggregated summaries of risk for immediate understanding and comparison. Each scoring methodology provides a standardized way to look at organizations in the context of cyber risk.
- BitSight - a credit score-like rating between 250-900.
- SecurityScorecard - a rating from A to F, like a report card.
- UpGuard - a rating called CSTAR, that scores companies from 0-950, with individual ratings for every site under a company’s umbrella.
The way these security ratings are derived differ among the solutions to some degree. IP reputation relies on attributing malware traffic to an organization based on IP address. Scanning for misconfigurations means looking at an organization’s actual internet footprint and determining how it compares to best practices and what vulnerabilities that lead to data breaches are exposed.
- BitSight - Primarily IP reputation.
- SecurityScorecard - Primarily IP reputation.
- UpGuard - Misconfiguration focused assessment.
Not every solution supports every vendor. Many organizations employ small specialists vendors that are not covered by solutions with a narrow scope of focus. Every vendor that handles data is a potential point of breach and should be covered for the risk they bring to the company.
- BitSight - ~100,000 supported organizations
- SecurityScorecard - ~120,000 supported organizations
- UpGuard - ~2,000,000 organizations scanned every day
2. Usability and Learning Curve
Minimizing the amount of overhead in acquiring and using a new solution is key to making that solution deliver on its capabilities. All three solutions are cloud-based services with minimal installation criteria. Similarly, all three use web interfaces to navigate graphical representations of cyber risk.
- BitSight - Provides high level summation of vendor risk, can easily compare vendors.
- SecurityScorecard - Simple interface for quick grade reports and charts.
- UpGuard - High level summation of risk with the ability to drill down into precise technical detail. Provides remediation and protection suggestions for all risks. Comes with built in questionnaires that can be sent and managed from within the console.
4. Community Support
Making information publicly available to help customers and prospects reduces administrative overhead and provides users with resources for troubleshooting and decision making. How each company ties into the community speaks to its mission and focus.
- BitSight - Company and product blog.
- SecurityScorecard - Company and product blog.
- UpGuard - Company and product blog, and a breach research blog that has broken stories on some of the largest data breaches of 2017.
4. Release rate
Technology is always changing. The rate at which a company updates their software determines how agile it is in response to customer requests and new features, as well as how consistently it is being patched and hardened. The full release histories for these products are not publicly available. However, like most new software, the releases are relatively fast and regular.
UpGuard has always adopted DevOps principles internally to develop, test, and release software, ensuring fast and consistent releases that have been tested for quality.
5. Pricing and Support
Vendor risk solutions are typically priced per vendor per year, except in some cases where one-off reports are generated for a set price. Some of the more expensive solutions price out small and medium businesses, while relegating even large companies to managing only their top few vendors. All three provide professional services to assist with setup and maintenance.
- BitSight - Public pricing information is not directly available, but prices are reported to be around $2000-$2500 per vendor per year.
- SecurityScorecard - Public pricing is not directly available. Prices reported to be between $1500-$2000 per vendor per year.
- UpGuard - With a transparent pricing model, UpGuard offers vendor risk management for nearly a tenth of other solutions at $250 per vendor per year.
|Pricing and Support|
6. API and Extensibility
Accessing the information in a cyber risk product outside of its graphical interface is important for integrated business strategies and consolidating data to a preferred system. All three solutions offer standard APIs to pull data into other enterprise applications.
|API and Extensibility|
7. Third-Party Integrations
While the API is useful for building custom integrations, standard integrations can cut the overhead of tying the vendor risk product to the rest of the data center.
- Bitsight - Offers limited integrations with select partners such as GRC solutions provider Rsam and information services provider Markit.
- SecurityScorecard - Offers integrations with GRC platforms such as Archer.
- UpGuard - Integrates with GRC platforms, ticketing systems like ServiceNow, and more.
|Third Party Integrations|
8. Companies that use the product
Sometimes the proof is in the pudding, but all three companies have impressive client lists, with none more distinguished than the other.
- BitSight - Customers include Lowe's, Target, Ferrari, and The Hartford, and T-Mobile
- SecurityScorecard - Customers include Netflix, McDonald's, Allstate, Symantec, and Pepsi
- UpGuard - Customers include ADP, the New York Stock Exchange (ICE), and NASA
|Companies That Use It|
9. Predict Capabilities
The whole point of managing vendor risk is to head off incidents down the road. This makes the ability of the solution to predict data breaches and other incidents a key consideration. What differentiates predictive capability is how well the methodology that determines the security ratings applies to actual breach vectors.
- BitSight/SecurityScorecard - The IP reputation methodology helps catch active malware installations, but that’s only one possible way a data breach can occur. Both companies include additional data, but lack the transparency to prove the efficacy of their scores.
- UpGuard - Because UpGuard checks for misconfigurations across the internet footprint, many important breach vectors are covered, including phishing, ransomware susceptibility, man-in-the-middle attacks, and DNS issues. The UpGuard methodology has been refined based on the actual data breaches we have discovered and reported to the world.
For an example of vendor risk scoring, let’s take a look at all three companies through UpGuard’s CSTAR security rating and see how they compare as of January 2nd, 2018.
Scoreboard and Summary
|Pricing and Support|
|API and Extensibility|
|3rd Party Integrations|
|Companies that Use It|
Deciding which solution best suits your needs depends on the objectives of your cyber risk initiative. IP reputation can sometimes detect malware signals that are attributed to the address space owned by a company, but UpGuard’s cyber resilience strategy looks at each company’s internet footprint and examines all of the vectors by which data exposure and service outage occur, including misconfigurations, a leading cause of successful attacks, and one undetected by IP reputation tactics. With vendor questionnaire integration, only UpGuard manages the entire third party cyber risk process, end-to-end. UpGuard is also the only company to offer an internal cyber risk management solution, Core, allowing organizations to completely manage primary risk as well.
Interested to know how UpGuard compares?