Monitoring, Evaluation and Learning Frameworks for Cyber Security Strategies
Cyber security is an increasingly critical component of Australia’s national security as technology evolves at a rapid pace and opportunities grow for malicious cyber actors who have become better resourced.
Figure 1: Objectives and benefits of MEL frameworks
MEL frameworks for cyber security strategies have had limited success, both in Australia and internationally, at checking and adjusting the effectiveness of actions and tasks against planned outcomes. While progress indicators are used to track the progression of action tasks, to achieve success it is critical that MEL approaches are expanded to measure the intended outcomes of these action tasks.Only by measuring our current state against these outcomes can governments be confident that they are truly confronting these threats.
Noetic employs a tried and trusted methodology for developing MEL frameworks that break down and simplify what has traditionally been a complex and confusing process. Noetic’s process provides clarity to those responsible for implementing the MEL framework as to what data to collect and what it will be used to measure. For the MEL framework to be most effective, it is critical that the following design and implementation stages are completed concurrently with the design and implementation stages of the strategy.
Figure 2: How to design and implement a MEL framework
Building Effective MEL in a Cyber Security Context
Due to the complex nature of cyber security, designing an effective MEL framework for cyber security strategies can be challenging and requires broad consultation with MEL experts.
The dynamic nature of cyber security threats and responses means that cyber security strategies need to be reviewed and updated regularly to remain relevant. MEL frameworks therefore need to be reviewed and refreshed more frequently than these strategies, while still providing continuity of measurements. Achieving this requires a flexible and adaptable approach to be used during design to meet the challenges of the fast-paced technological environment.
There is currently limited experience and expertise in managing cyber threats from a business perspective with small numbers of tried, tested and proven metrics and no standard MEL methodology that can be applied in this emerging area. Broad and deep collaboration under governmental direction is critical to develop relevant metrics in this area to build resilience. Monitoring, evaluation and learning must be recognised as a strategic priority that requires appropriate resourcing and investment in both design and implementation.
MEL frameworks need to be tailored to be effective in their environments – there is no ‘one-size-fits-all’ solution. The unique cyber security environment would benefit from a principles-based model that is flexible and adaptable to the diversity of cyber threats, as well as the varied levels of vulnerabilities across sectors and across both governmental and non-governmental organisations. This approach was exemplified through “A call for views” on Australia’s 2020 Cyber Strategy.
Opportunities in MEL for Cyber Security
The cyber security domain also presents unique opportunities that are advantageous for designing and implementing effective MEL frameworks.
The technological nature of cyber security typically allows for high quality data sources to be generated that can be used for measurement. Costs for data collection are also low due to the relative ease of automating data collection and analysis.
Developing a MEL framework for cyber security will provide widespread benefits as the framework can be leveraged and applied to other industries that relate to national security more broadly that lack a tried and tested MEL methodology.
Cyber security incidents are growing in frequency and severity. It is critical that governments respond by designing and implementing strategies and MEL frameworks to support the community in combatting this problem. Whilst developing and implementing effective MEL frameworks in the cyber security domain has unique challenges, the potential benefits in better protecting Australia cannot be overstated.