When it comes to protecting sensitive data, preventing data breaches, and detecting cyber attacks, a checklist should be followed to track your efforts. Key performance indicators (KPIs) are an effective way to measure the success of any program (including cybersecurity) and aid in decision-making.

According to PwC, just 22 percent of Chief Executive Officers believe their risk exposure data is comprehensive enough to form decisions. A figure that - alarmingly - hasn't changed in 10 years. The EY Global Information Security Survey supports this with only 15% of organizations saying their information security (InfoSec) reporting fully meets their expectations.

In this post, we outline 14 actionable cybersecurity metrics to help you take ownership of your vendor risk identification and remediation efforts.

upguard demo request cta

Learn how UpGuard simplifies Vendor Risk Management >

Why are cybersecurity metrics Important for managing vendor risk?

As Peter Drucker said, what gets measured, gets managed - and cybersecurity is no different. If you can't measure your security efforts, you won't know how you're tracking.

Cybersecurity is not a one-time affair. Cyber threats and vendor risks are constantly evolving and the processes and technology needed to prevent them are constantly changing. You need to have measures in place to frequently assess the effectiveness of the safeguards you have invested in.

This is important for two reasons:

  1. Analysis of KPIs, key risk indicators (KRIs), and security postures provides a snapshot of how your security team is functioning over time. Helping you better understand what is working and what is worsening, improving decision-making about future projects.
  2. Metrics provide quantitative information that you can use to show management and board members you take the protection and integrity of sensitive information and information technology assets seriously.

Reporting and providing context on cybersecurity metrics is being an important part of the job for many Chief Information Security Officers (CISOs) and Chief Information Officers (CIOs), driven by increasing interest in reporting at the shareholder, regulatory, and board levels.

For many board members in sectors like financial services, they have a fiduciary or regulatory duty to manage cybersecurity risk and protect personally identifiable information (PII).

This has been driven by new regulations like the Gramm-Leach-Bliley Act, NYDFS Cybersecurity Regulation, PIPEDA, and CPS 234. Pair this with extraterritorial data protection laws like GDPR, CCPA, and LGPD and security management becomes a key focus for every organization.  

The best IT security professionals use metrics to tell a story, especially when giving a report to non-technical colleagues.

Learn how UpGuard simplifies attack surface management >

14 Cybersecurity KPIs to track in Vendor Risk Management

Below are examples of clear KPIs and metrics you can track and present to your stakeholders to demonstrate your Vendor Risk Management efforts. To serve as a guide for improving performance across all 14 primary cybersecurity metrics, each checklist item is presented in question form.

Download this list as an editable checklist >

1. Level of preparedness

Your organization’s level of cyberattack and third-party breach preparedness is a major metric determining your security posture and the overall value of your Vendor Risk Management program. The effectiveness of your cybersecurity efforts when a vendor suffers a cyber incident can be measured with the following set of metrics.

  • The number of security incidents detected and resolved within a specific period (e.g., month, quarter, or year).
  • The percentage of incidents prevented due to proactive security measures, such as endpoint protection, intrusion detection systems, and threat intelligence.
  • The number of false positives and false negatives generated by security monitoring tools, and how these numbers are being reduced through continuous refinement of the monitoring process.
  • The level of employee security awareness and the frequency of cybersecurity awareness training programs.
  • The frequency of simulated phishing attacks to test phishing attack susceptibility.
  • How many devices on your corporate network have the latest security patches installed?
  • How many high-risk vulnerabilities have been identified?
  • How many systems have failed vulnerability scans, and what is the plan to remediate those issues?
  • How frequently are backups taken, and how are they tested for completeness and accuracy?
  • How often are disaster recovery, incident response, and business continuity plans tested, and when was the last successful test?
  • How is your organization managing data classification and data retention policies, and how are those policies enforced?
  • What is the frequency of security awareness training for employees, and what metrics are used to measure its effectiveness?
  • How are security policies and procedures updated and communicated to employees, and how is compliance monitored?
  • How many devices on your corporate network are running outdated operating systems or software?
  • How many devices on your network are running end-of-life (EOL) software no longer receiving security updates?
  • How often are internal and vendor risk assessments conducted, and what actions are taken as a result of those assessments?
  • How are security controls tested for effectiveness and assurance?
  • How often are security policies and procedures reviewed and updated to reflect changes in the threat landscape?

2. Unidentified devices on internal networks

  • What is the inventory of authorized devices on your network, and how is it maintained and kept up-to-date?
  • How many assets are there in your network?
  • How many of those assets store sensitive data?
  • What is the process for responding to unauthorized devices on the network, and how are these devices quarantined and monitored?
  • How are IoT devices secured, and what is the process for monitoring and patching their vulnerabilities?
  • How is network segmentation implemented, and how are different types of devices segregated on the network?
  • How are access controls implemented for devices on your network, and what is the process for granting and revoking access permissions?
  • How are devices authenticated and authorized before being allowed to connect to the network?
  • What is the policy for employees bringing their own devices (BYOD) to work, and how are these devices managed and secured?
  • What measures are in place to detect and respond to rogue access points or other unauthorized network infrastructure?
  • What is the process for tracking the lifecycle of devices on your network, including acquisition, deployment, maintenance, and retirement?
  • How are third-party devices and services securely integrated into your network, and what is the process for managing their access and permissions?
  • What is the policy for remote access to your network, and what measures are in place to secure and monitor remote connections?

UpGuard’s attack surface monitoring solution can help you quickly map your attack surface by identifying all IP addresses in your digital inventory. This scanner can help you discover unmaintained assets expanding your attack surface and increasing your risk of suffering a data breach.

To learn some tricks for quickly reducing your attack surface with UpGuard, watch the video below:

Take a tour of UpGuard's attack surface management features >

3. Intrusion attempts

  • How many intrusion attempts have been detected and blocked by your intrusion detection system?
  • What is the average time it takes to investigate and respond to detected intrusion attempts?
  • What is the process for reporting intrusion attempts to relevant stakeholders, including management, legal, and law enforcement?
  • How many unauthorized access attempts have been detected and blocked by your firewall?
  • What is the process for investigating and responding to detected intrusion attempts, and how are those findings communicated?
  • How are logs and other security event data collected and analyzed, and what tools and processes are used for this purpose?
  • How are security incidents classified and prioritized, and what response procedures are in place for each classification?
  • How frequently are security logs reviewed, and what is the process for reviewing them?
  • How are security events and incidents correlated and analyzed to identify potential threats and attacks?
  • What measures are in place to prevent false positives and false negatives in intrusion detection systems?
  • How are network traffic patterns and anomalies monitored to detect potential intrusions?
  • How are incident response plans updated and tested in response to new intrusion attempts and attack trends?
  • How are security controls adjusted and fine-tuned based on the results of intrusion detection and response efforts?

4. Security incidents

  • How many security incidents have been detected and resolved in the past month/quarter/year?
  • How many successful cyber attacks have occurred in the past month/quarter/year?
  • What types of incidents have occurred, and what was the impact of each incident?
  • What metrics are used to track incident response and resolution times, and how are these metrics used to improve the incident response process?
  • How is data recovery managed in the event of a security incident, and how are backups tested and validated?
  • What is the root cause analysis of each incident, and what corrective actions were taken to prevent similar incidents from occurring in the future?
  • What is the average downtime experienced during a security incident, and what is the impact on the organization's operations?
  • What is the average cost associated with a security incident, including costs for incident response, remediation, and reputational damage?
  • How is user behavior monitored to identify potential security incidents or insider threats?
  • How is threat intelligence gathered and used to proactively detect and prevent security incidents?
  • What is the process for reporting security incidents to regulatory authorities, customers, and other stakeholders?
  • How is the organization's incident response plan updated and tested to ensure it remains effective and relevant?

UpGuard’s vulnerability detection module ranks discovered internal and vendor security risks by criticality, helping security teams address threats most likely to result in a data breach. By making it easier to prioritize critical risks, UpGuard keeps your security posture optimized to resilient levels at all times.

UpGuard’s Vulnerability Management Module.
UpGuard’s Vulnerability Management Module.

Request a free trial of UpGuard >

5. Mean Time to Detect (MTTD)

MTTD is a crucial metric for determining the efficiency of your organization's threat detection and response capabilities should a third-party vendor become compromised. To improve MTTD, consider the following:

  • Utilizing threat intelligence feeds and other sources of security information to enhance your vendor risk detection capabilities.
  • Tuning security controls and monitoring tools to improve detection and response times, reducing the potential impact of third-party breaches..
  • Implementing a robust incident classification and prioritization system to ensure that high-priority threats are addressed promptly.
  • How long does it take for your team to become aware of security threats and incidents?
  • What is the average MTTD for your organization?
  • What is the process for detecting and responding to security threats and incidents, and how is this process tested and validated?
  • How are threat intelligence feeds and other sources of security information used to improve MTTD?
  • How are security controls and monitoring tools tuned to improve detection and response times?
  • How are alerts and events from security monitoring tools triaged and prioritized, and what criteria are used to determine severity?
  • How often are security monitoring tools and sensors updated, and how is their updated performance monitored?
  • What is the process for investigating and resolving security alerts and incidents, and how are those findings communicated?
  • How are false positives and false negatives addressed in the security monitoring process, and how is this process continually refined?
  • How are security incidents classified and prioritized, and what response procedures are in place for each classification?
  • What training and education programs are in place for security analysts and incident responders, and how is their performance monitored and evaluated?
  • How are key metrics and KPIs related to MTTD?

6. Mean Time to Resolve (MTTR)

  • What is your mean response time following immediate awareness of a cyber attack involving a vendor?
  • What is the average MTTR for your organization?
  • How is incident response coordinated and managed, and what resources and personnel are involved in the response process?
  • How is the incident response process continually evaluated and improved, and what metrics are used to track this process?
  • How are security incidents categorized and prioritized, and what response procedures are in place for each category?
  • What are the key steps involved in the incident response process, and how are they tracked and measured?
  • What is the average time it takes to identify the root cause of security incidents, and what measures are in place to ensure a thorough investigation?
  • How are incident response teams trained and prepared for different types of security incidents, and how is their performance assessed during incident response exercises?
  • What is the process for restoring systems and data following a security incident, and how is the effectiveness of this process validated?
  • How are lessons learned from security incidents incorporated into incident response plans and procedures to prevent similar incidents in the future?
  • What is the role of external resources, such as incident response vendors and law enforcement agencies, in the incident response process, and how are they coordinated and managed?
  • How are stakeholders, such as customers and business partners, informed and kept up-to-date during the incident response process?

7. Mean Time to Contain (MTTC)

  • How long does it take to contain identified internal and third-party attack vectors across all endpoints and systems from the time of initial detection?
  • What is the average MTTC for each type of security incident or attack, such as malware infections, data breaches, and DDoS attacks?
  • How effective are your containment measures in preventing further damage or data loss, as measured by the scope and severity of each incident?
  • How well do your incident response team and processes work in coordinating containment efforts across different departments such as IT, legal, and public relations?
  • How do you prioritize and allocate resources to different types of incidents based on their severity, impact, and risk to your business operations and reputation?
  • How will you prevent similar incidents in the future across each of the following threat mitigation categories - security controls, awareness training, policy and procedure updates?
  • How do you evaluate the success of your containment efforts, such as by measuring the reduction in incident frequency, cost, and time-to-remediation, as well as the improvement in security awareness and compliance?
  • How do you measure the reduction in incident frequency?
  • How do you measure the reduction in time-to-remediation?
  • How do you measure improvement in the cybersecurity habits of your staff?

8. First-party security ratings

First-party security ratings are an essential metric for evaluating your organization's cybersecurity posture. By utilizing a security rating system, such as the one provided by UpGuard, you can quickly assess your organization's security performance based on various criteria, including network security, phishing risk, DNSSEC, email spoofing, social engineering risk, DMARC, risk of man-in-the-middle attacks, data leaks, and vulnerabilities.

Security ratings are often the easiest way to communicate metrics to non-technical colleagues through an easy-to-understand score.

Security ratings can feed into your cybersecurity risk assessment process and help inform which information security metrics need attention.

To maintain or improve your security rating, consider the following:

  • Regularly reviewing and updating your security controls and practices to stay aligned with industry best practices
  • Leveraging communication channels to share your security rating with stakeholders, building trust with customers and partners
  • Implementing a continuous improvement process to track and evaluate the effectiveness of your security measures
  • What is your organization's current security rating, and how is it calculated?
  • How has your security rating changed over time, and what factors have contributed to these changes?
  • What security controls and practices are evaluated as part of the security rating assessment?
  • How does your organization compare to industry benchmarks and best practices in terms of security rating?
  • How is the security rating used to identify areas of weakness and prioritize security investments?
  • What communication channels are used to share the security rating with stakeholders, and how is this information used to build trust with customers and partners?
  • What actions are taken to maintain or improve the security rating over time, and how are these actions tracked and evaluated?

9. Average vendor security rating

The threat landscape for your organization extends beyond your borders and your security performance metrics must do the same.

This is why vendor risk management and a robust third-party risk management framework is an essential requirement for security operations. UpGuard's Executive Summary Report provide you with instant access to your average vendor rating over the last twelve months, as well as your distribution of vendor ratings. Traditional vendor management practices were limited to a snapshot of your vendor security ratings at a single point in time. By continuously monitoring vendor risks, you can greatly reduce your third-party and fourth-party risk.

  • How many vendors are in your organization's supply chain, and what percentage of those vendors are considered high-risk?
  • What criteria are used to evaluate vendor security, and how are those criteria weighted?
  • How frequently are vendor security assessments conducted, and what is the process for conducting those assessments?
  • What types of security ratings or scoring systems are used to evaluate vendor security, and how are those ratings incorporated into the vendor selection process?
  • How are vendor security ratings monitored and updated over time, and what is the process for reevaluating vendor security when new vulnerabilities or threats emerge?
  • What is the process for addressing vendor security issues, and how are those issues communicated to the vendor?
  • How is vendor security performance evaluated and reported to senior management or the board, and what metrics are used to measure vendor security performance?

UpGuard’s security ratings features allow you to track the security postures of all vendors in real-time.  With security ratings quantified using an objective and reliable calculation mechanism, a drop in security ratings is a likely indication of a new security exposure that could result in a security incident if exploited by hackers.

Security Ratings by UpGuard.
Security Ratings by UpGuard.

Learn how UpGuard calculates security ratings >

10. Patching cadence

  • How frequently are security patches and updates released by software vendors, and how quickly are they implemented?
  • How are high-risk vulnerabilities prioritized for patching, and what is the process for testing and validating patches before implementation?
  • How are legacy systems and software that are no longer supported by vendors patched, and what measures are in place to mitigate their security risks?
  • How are patches and updates distributed and installed across different devices and systems, and how is this process managed and monitored?
  • What is the average time it takes to apply patches once they are released, and what is the maximum acceptable patching window for high-risk vulnerabilities?
  • What metrics are used to track patching effectiveness and compliance, and how are these metrics used to drive improvements in the patching process?
  • How are patches validated to ensure they do not cause any conflicts or disruptions in the systems they are being applied to?
  • How are legacy systems and applications that are no longer supported with security patches being handled? Is there a plan in place for dealing with these systems?
  • Are there any exceptions to the patching process, such as certain systems or applications that cannot be patched for operational or other reasons? How are these exceptions managed and mitigated?

11. Access management

  • How is access to sensitive data and systems controlled and monitored, and how is privilege escalation prevented?
  • What are the different types of user roles and access levels, and how are they defined and documented?
  • How often are user accounts reviewed and audited for compliance with access policies and procedures?
  • Are all accounts secured with Muli-Factor Authentication (MFA)?
  • Have you created password policies addressing common malpractices, such as password recycling and weak passwords?
  • What is the process for monitoring user activity and access logs, and how are suspicious or anomalous behaviors detected and investigated?
  • What controls are in place to protect privileged accounts
  • What are the procedures for granting temporary or emergency access to users, and how are these situations documented and reviewed?
  • How is access to third-party applications and services managed, and what additional controls are in place to prevent unauthorized access or data leakage?
  • How are access policies and procedures communicated to users, and what training or awareness programs are in place to promote secure access practices?
  • How is access granted to new employees, and what is the process for removing access when an employee leaves the company?
  • What is the process for managing access requests and approvals, and how are these requests documented and tracked?
  • How is access control regularly audited and reviewed, and how often are access policies and procedures updated?
  • What are the consequences for non-compliance with access policies, and how is compliance with access policies monitored?
  • How is access to sensitive data and systems restricted, and how are those restrictions enforced?
  • How is the principle of least privilege applied to limit user access and reduce the risk of privilege escalation attacks?
  • What tools and processes are used to monitor user activity and detect potential insider threats?

12. Company vs peer performance

Benchmarking your organization's security performance and cybersecurity strategy against industry peers can provide valuable insights into areas for improvement. To effectively compare your security posture with that of your peers, consider the following:

  • Utilizing key performance indicators (KPIs) to measure your organization's security performance against industry standards and best practices
  • Analyzing specific security controls and policies implemented by peer organizations to identify potential gaps in your own security program
  • Leveraging competitive intelligence and industry insights to inform your security strategy and decision-making
  • What key performance indicators are used to measure your organization's security posture compared to industry peers?
  • What specific security controls and policies do peer organizations have in place that your organization does not?
  • How is your organization using benchmarking data to identify areas for improvement in your security program?
  • What strategies are your peers using to stay ahead of emerging threats, and how can your organization adopt those strategies to better protect against cyber attacks?
  • How has your organization's security performance compared to your peers over time, and what trends or patterns have emerged?
  • How is your organization using competitive intelligence and industry insights to inform your security strategy and decision-making?

An executive summary report is one of the best methods of communicating your security performance with stakeholders. UpGuard offers a library of cybersecurity report designs to help you reflect your cybersecurity efforts in a style that meets the unique communication requirements of your stakeholders.

UpGuard's security report library.
UpGuard's security report library.

Learn more about UpGuard’s reporting capabilities >

13. Vendor patching cadence

  • How frequently are your third-party vendors' systems scanned for vulnerabilities, and how are these scans conducted?
  • How many risks have been identified in your third-party vendor's systems, and what is the plan to remediate these risks?
  • How many critical vulnerabilities are yet to be remediated in your vendor's systems?
  • What is the process for validating vendors have implemented security patches?
  • What is the process for terminating vendor relationships in the event of poor security performance or failure to comply with security standards?
  • How is your organization monitoring fourth-party vendor risk (the vendors used by your vendors)?
  • How is your organization prioritizing patching for third-party vendors based on risk level?
  • What is the process for communicating patching requirements and deadlines to third-party vendors?
  • How is your organization tracking compliance with vendor patching requirements and deadlines?

UpGuard’s Vendor Tiering feature allows third-party vendors to be tiered based on security criticality. This allows vendors with the highest potential impact on your secuity posture to be prioritized in monitoring and remediation processes, reducing the liklekyhood and impact of third-party breaches.

Vendor Tiering by UpGuard
Vendor Tiering by UpGuard

Request a free trial of UpGuard >

14. Mean time for vendor incident response

The efficiency of your vendors' incident response is crucial for minimizing the risk of data breaches. The longer it takes vendors to respond to incidents, the higher the chance you will suffer from a third-party data breach.

A complete guide to data breaches by UpGuard

Get this free guide to learn how to prevent data breaches with a proven stratergy.

Download the data breach prevention guide >

To ensure prompt and effective incident response from your vendors, consider the following:

  • Establishing clear communication channels and coordination processes for reporting and addressing security incidents and vulnerabilities
  • Monitoring vendor response times and performance, and holding them accountable for meeting established service level agreements (SLAs)
  • Integrating vendor incident response procedures into your overall incident response plan and ensuring that relevant personnel are trained on these processes
  • How long does it take for a vendor to respond to security incidents and vulnerabilities?
  • What is the average MTTR for your vendor's incident response?
  • How is incident response coordination managed between your organization and your vendors?
  • How are security incidents and vulnerabilities communicated to vendors, and how is response progress tracked?
  • How are vendor response times and incident response performance evaluated and monitored?
  • How are vendor incident response procedures continually evaluated and improved, and what metrics are used to track this process?
  • How are incident response procedures for third-party vendors integrated into your overall incident response plan, and how are they updated and communicated to relevant personnel?
  • How are incident response responsibilities and expectations outlined in service level agreements (SLAs) with third-party vendors, and how are these SLAs monitored and enforced?

How to choose the right cybersecurity metrics for your VRM program

There’s no objective standard for choosing the right set of cybersecurity KPIs and KRIs in the context of Vendor Risk Management. Your choice of metrics depends on your industry, security needs, regulations, guidelines, best practices, and ultimately, you and your customer's appetite for risk. Outside of the metrics outlined above, the CIS Controls also provides a cost-effective, prioritized list of security controls for improving cybersecurity performance internally and across the vendor threat landscape

That said, you will want to choose metrics that are clear to anyone, even non-technical stakeholders. A good rule of thumb is if your non-technical stakeholders can't understand them, you need to either pick new metrics or do a better job of explaining them. Benchmarks and industry comparisons are an easy way to make even complex metrics understandable.

When referencing cybersecurity metrics in an executive meeting, remember the most important metric to focus on is cost. The objective of these meetings is to demonstrate how cybersecurity is saving the organization money. For best results, it's highly recommended to support your presentation with a cybersecurity executive report.

Ready to see
UpGuard in action?

Ready to save time and streamline your trust management process?