Thread by @jason_trost

Recently I contributed to a paper on "Reporting Cyber Risk to Boards". It summarizes findings and best practices from a CISO Working Group, and is a must read for CISOs, CIOs, and other Cybersecurity executives that report to the Board.
Highlights🧵

Boards often fail to see the continuous importance of cybersecurity and have knee jerk reactions to breaking cyber stories in the media then quickly forget about it until the next big cyber incident. Typically, cybersecurity only becomes an issue when it is already too late.

In cases where cybersecurity reporting to the Board is taking place, there is a wide variety of methods, tools, and processes in use. Organizations struggle with what to report and how to obtain effective feedback from the Board.

We believe we should tackle the cybersecurity problem as a risk management problem and to use informed risk management and mitigation to prioritize action continuously.

Cyber must be integrated in the overall management system, it should not be considered as something special/isolated but as an integral part of the organizational activities and processes including the risk management process.

We can distinguish an upper, senior executive part and a lower implementation/operations part with the CISO in the central overlapping zone, connecting the cybersecurity operational level with the strategic level.

Cyber security frameworks are a tool to manage cybersecurity risks in a coherent manner and to implement a corporate cyber security strategy. These often include monitoring of key controls.

It makes sense to combine the selection of a framework profile with the definition of quantitative metrics (KPIs, KRIs, KCIs, OKR12) with goals/outcomes and link these to the relevant processes/systems and process owners

Monitoring by self-assessment has fundamental drawbacks to provide the status of the cyber risk mitigation measures and their effectiveness. These include:

Machine-generated data can provide a very useful complement to self-assessment or even replace it to a large extent. They can make reporting on cybersecurity risk objective, repeatable, and automated << When done at scale, this is game changing.

The number one danger of metrics for cyber risk is that they begin to reflect work done or effort applied, instead of risk reduction. A Board or executive team must rigorously push back against the inclusion of such metrics

We propose a Metrics Model with the following three steps:
1. Gather relevant cyber evidence
2. Transform the evidence into business risk13
3. Report to the Board, provide reasonable assurance, and highlight gaps.

A number of keywords come to mind when thinking of what makes good metrics:
• Objective
• Immutable
• Repeatable
• Continuous
• Relevant
• Effective
• Informed
• Agreed
• Actionable

Gathering inputs – measure what matters most

We can distinguish different families of operational key metrics which we group by nature (control-centric, threat-centric, tool-centric, and event-centric).

Control-centric metrics could include:
• Coverage of a control - for all assets or a selected group of (key) assets;
• Effectiveness of a control;
• Data source and update frequency;
• Threshold level.

A (more granular) variation of this control-centric approach decomposes a control’s coverage into three components: deployed, operational, and effective.

Threat centric metrics - metrics in which the organization identifies its most important adversaries and tracks the TTPs (Techniques, Tactics, Procedures) that they are known to deploy by using the MITRE ATT&CK® Framework.

Tool-centric metrics - metrics in which the organization focuses on the deployment of specific cybersecurity tools (EDR, perimeter defenses, MFA, etc.) to achieve risk mitigation

Event centric metrics - Many organizations collect data on cybersecurity events (#alerts, #incidents, #false positives, #vulnerabilities, etc.). Such statistics can provide valuable input into the management of cybersecurity risk, but they need to be interpreted

Supply chain metrics - Mapping dependencies on suppliers, gaining insight into their cyber security posture, and implementing appropriate controls is becoming an integral part of cyber risk management and should therefore also be included in the metrics.

In order for cyber risk metrics to resonate at the Board level, they need to be transformed to meaningful business reporting (money, safety, brand value, etc.) and compared with the risk appetite

In any org there might be a large num of metrics to describe the state of controls & org perf. This can cause a negative effect where stakeholders become overwhelmed by detail. To overcome this challenge, sets of metrics can be normalized or harmonized to provide a holistic view

Reporting cyber to the Board should serve the purpose of (re)assuring the Board that the cyber risk is w/in the risk appetite today & tomorrow:
• Are we good enough?
• Are the resources allocated to cyber appropriate & effective?
• How do we compare w/ our peers & our sector?

To learn more, read the full paper, "Reporting Cyber Risk to Boards, CISO Edition: Control, Measure, Report, Repeat". researchgate.net/publication/35…
@FDezeure @blackswanburst @joaosobral @CyberRiskInst @EATMCERT

Some exemplar cybersecurity metrics & reports from the paper.