5 KPIs for assessing and monitoring IT assets security

What are the threats to your organization: intrusion, espionage, data theft, ransomware…? What is the level of risk? Are teams able to react, and how quickly?  

Measuring the security of an organization’s IT assets over time is essential for effective protection, and for reacting appropriately when an attack occurs. 

Cybersecurity: who keeps track of KPIs? 

This data can be used to identify strengths (tools, expertise, processes, etc.) and areas for improvement (vulnerabilities, processes to be optimized, expertise to be reinforced, etc.).  

KPIs relating to the security of an IT infrastructure can be monitored internally by CISOs, SOC Managers or CIOs, depending on how technical teams or staffed; and they can also be challenged with MSSPs for organizations that call on a partner to manage their cybersecurity solutions. 

How do you measure security-related KPIs? 

Before getting to the heart of the matter, if we mentioned earlier the need to measure these indicators over time, it’s for a good reason! In fact, these KPIs need to be monitored on a regular basis, not only to see improvements, but also to identify and remedy any deterioration in performance.   

Depending on the type of KPI, you need to define a suitable frequency for observing them, so that their use is relevant. For example, KPIs relating to training need to be monitored over long periods of time, whereas data relating to security events needs to be measured on a daily basis.  

The cyber strategy and roadmap can thus be adjusted, while justifying the investments made, whether in training, solutions, or time devoted to improving internal processes…  

In order to be understood and taken into account by all stakeholders, these KPIs must be shared on a regular basis, and serve as a basis for proposing security optimizations.  

In addition, certain KPIs can help to set targets – thresholds to be reached or not exceeded – in order to guide and prioritize the actions to be taken. In all cases, objectives must remain realistic and achievable (SMART), and they can also be adjusted over time according to the context and the organization’s priorities.

Cybersecurity: key points to follow and share

Training level of security teams 

The expertise of human experts in cybersecurity is irreplaceable, even with the most powerful tools. Thus, members of technical teams, whether in-house or external, need to continually update their knowledge about threats, and about tools for detecting and responding to them.  
 

Indeed, tools evolve to remain state-of-the-art and often anticipate emerging threats, and their use also requires regular training to make the most of them.  

To this end, cybersecurity solution publishers are planning training programs for their users. In this way, an organization can define objectives in terms of the number of employees trained, on which subjects or functionalities, with which certifications…   

Number of audits and evaluations planned 

Attacks are inevitable – Thales Data Threat 2024 report indicated a 27% rise in the number of companies falling victim to ransomware attacks in 2023. In order to react as quickly as possible, it is essential to keep IT infrastructure clean. This notion of cleanliness implies being able to list assets with a view to correctly segmenting the Information System, identifying vulnerabilities and detecting Shadow IT…  

These operations require regular audits and penetration tests to assess the level of security. Due to lack of time, these procedures are often de-prioritized, even though they enable to optimize security incident management by taking the right decisions – and in this situation, every minute counts.   

It is therefore essential to set objectives for the audits and penetration tests to be carried out, because identifying vulnerabilities enables them to be corrected, and visibility over Information System enables an appropriate response to be made in the event of an attack. These assessments can also highlight the need for additional tools, resources or support to reinforce security.  

Volume of security incidents and contextualization 

This is obviously a figure that all CISOs and SOC Managers would like to see as low as possible… Nonetheless, sharing the number of security incidents reported internally using the tools put in place gives an indication of the relevance of the budgets committed to Information System protection.  

Ransomware is an attack with an immediately visible impact (explicit ransom demand, Information System shutdown…), but it’s also important to be able to detect attacks with no visible impact, such as intrusion attempts (for data theft, espionage…). As this second type of attack is much more difficult to detect, you need to be equipped with tools capable of identifying suspicious behavior.   

The ability to demonstrate the value of the tools deployed for this purpose is also crucial to ensuring that Information System users fully understand the stakes.   

In addition, educational simulation tests, for example, help to “propagate a healthy paranoia”.  

For security teams, daily monitoring, as well as assessments to gain perspective, also help to better identify, and therefore anticipate, the types of risk weighing on the organization. In this way, cyber strategy can be aligned with evolving threats.  

Indeed, knowing how to assess the criticality of incidents detected by security tools (EDR, XDR, SOAR…) enables to effectively measure the level of risk and the potential impact on the organization’s business. This is essential if to be ready to deploy the appropriate resources, and to consider equipping with solutions or updating procedures to harden protection. 

Relevance of alerts and false positives   

Cybersecurity tools are designed so that experts can be alerted in the event of an incident, but without being drowned out. Alert fatigue is an unfortunately common phenomenon, a source of stress but also of error.  

A study carried out in 2023 among 2,000 SOC analysts (in French) revealed that for 90% of them, threat detection tools are effective, but 97% are afraid of missing an important security event. They spend almost 3 hours manually sorting alerts on a daily basis (4500 alerts on average), and 67% of these alerts cannot be processed for lack of time, knowing that 83% are false positives.   

In the light of these figures, it’s easy to see why the relevance of alerts is a fundamental issue for effective protection.  

Tools must therefore offer fine-tuned configuration and whitelisting options, and these parameters must be adjusted right from the start, and then on an ongoing basis. Monitoring false positives ensures that the tool is always correctly configured to the state of the threat and to the needs of security teams.    

Note that the more a security tool provides access to detection rules, the easier it is to adjust them. This is an important factor in the efficiency and working conditions of analysts.  

Incident response time 

Reactivity and velocity are essential to identify and remediate attacks, and in this sense, security tools have a crucial role to play. Just as they help to correctly identify security incidents with a minimal false positive rate, they must also enable optimal contextualization and visualization of the data collected.  

Using these tools, in real-life conditions or as part of simulation exercises, it becomes possible to assess various aspects of the time required to respond to a security incident, including:   

• Mean Time to Detect – MTTD 
  This is the time it takes for a tool to detect a threat. In some cases, there may be a latency due to the detection mode, if it is operated via a Sandbox or using a correlation engine.  
  This indicator can be discussed with the solution’s provider or the MSSP operating it.

• Mean Time to Respond/Remediate – MTTR 
  This is the time required for a security solution to contain and eradicate a threat, and it can be optimized by setting up the automation of certain actions. 

• Mean Time to Acknowledge – MTTA 
  This is the average time between when an alert is triggered and when the teams are working on the incident. This measure tracks the responsiveness of teams, and highlights any need to adjust tools, train or reinforce teams.

• Mean Time to Investigate – MTTI 
  Unlike the previous two metrics, which are tool-related, MTTI reflects the time required by analysts, whether in-house or at a partner, to investigate an alert: gathering information, analyzing data, assessing criticality… The more open a security solution is to interfacing with other monitoring and incident management solutions (ITSM, SIEM…), the shorter this time will be.  
  MTTI can be evaluated on the basis of the data available and the time required to process it (the more transparent a tool, the more the data will help analysts in the investigation phase), the resources available, the SOC performance… 

If security tools are managed by a MSSP, a Service Level Agreement (SLA) can be included in the service contract to guarantee response and remediation times. 

Team training, audits and penetration tests, and the effectiveness of tools and resources are all crucial aspects in guaranteeing optimum security for Information System assets.   

They need to be monitored over the long term to make them genuine decision-making tools for CISOs and SOC Managers, but also to validate a cyber strategy with an organization’s board, and identify areas for improvement.

Speaking of board, are you wondering how to break down misconceptions about cybersecurity and convince stakeholders of the relevance of your roadmap?