Getting management buy-in by telling “data stories” and articulating outcomes
You’re here because you want to reduce human cyber risk.
When you started out in this industry, you probably expected something like: reduce risk, enjoy success and glory.
By now you know the reality:
It’s not just about reducing risk. It’s about how we articulate and demonstrate value. Two wildly different things.
We can’t demonstrate value without metrics. But let’s face it: throwing around basic metrics—like training completion rates and simulated phishing clicks—is unlikely to get the C-suite on the edge of their seats.
Chances are you’re yearning to track meaningful measures of success, like security behaviors. But that takes resources, which means getting the top table on side.
One snag: senior stakeholders are a notoriously tough crowd. It can feel like you’ve walked into an episode of Shark Tank.
You need better buy-in to get better metrics, but you need better metrics to get better buy-in. Talk about Catch-22, right?
We can’t lay the blame solely on busy, skeptical leadership, though. Because the human risk community has a tunnel vision problem.
For quite a long time, many practitioners have been focusing on just three things: training engagement, simulated phishing click rates, and security incident report rates.
It’s maybe not surprising. As security awareness expert Ava Woods-Fleegal, Global Security Awareness Lead, RTX Corporation, points out, these metrics are “the easiest we can get our hands on as awareness professionals…we haven’t really gotten into how to convince people to give us more data.”
True. And it’s not easy to ask for data, nor to provide it. The above three things are easier, however, because they’re things often done by us.
But here’s the good news: even with the “good ol'” basic metrics you can still illustrate the value of your program (and maybe even evolve it in terms of effectiveness and maturity—but more on that later).
More good news: This blog explores how to make the most of what you have, so you can make a lasting impression on the senior management.
Whether you want to raise your program’s profile or raise the game completely, you need to give stakeholders something that resonates.
“Demonstrating security awareness value to senior stakeholders” is our smash-hit webinar that helps you build on the approaches we’ll discuss in this blog.
Join…
-
- Ava Woods-Fleegal, Global Security Awareness Lead, RTX Corporation
- Ira Winkler, CISO and Vice President, CYE and author of Security Awareness for Dummies
- Oz Alashe, CEO & Founder, CybSafe
…for an expert walk-through of turning basic security awareness and training (SA&T) metrics into senior stakeholder buy-in.
But first, where are you?
You are here: Security awareness & training (SA&T) program awareness levels
The SANS Security Awareness Maturity Model® is the industry standard for two key purposes:
- A benchmarking tool to pinpoint the maturity of your organization’s program, and,
- A strategic roadmap to demonstrate your program’s impact and to plot your next steps.
The model is an easy way to gauge why a program may be falling short of what you’d hoped it to achieve. And it provides a way to communicate the value of said program.
It’s been used by hundreds of organizations as a framework for building their program. And ultimately it’s enabled them to more effectively manage their human risk.
The five program maturity levels are:
-
- Non-existent: A security awareness program does not exist in any capacity. This means on the whole people are unaware, therefore vulnerable. Non-compliance is a dead cert.
- Compliance Focused: The program is designed around meeting audit and compliance requirements. Training is ad-hoc or annual, doing little to drive engagement and impact. Therefore, people aren’t clear on their role in protecting the organization or themselves.
- Promoting Awareness: The program targets key topics and groups that will have the most impact in reducing risk and ultimately supporting business objectives. Training and awareness are reinforced throughout the year. Content is engaging and positive, encouraging behavior change. People understand and follow policies and can spot, prevent and report incidents.
- Long-term Sustainment: The program is all set for a long-term life cycle thanks to robust resources, leadership, and processes being in place. There is usually a review of the program at least every year, with necessary updates being made. The program is current and engaging, meaning that it starts changing behaviors and perceptions around security.
- Metrics Framework: The program continuously improves and demonstrates value through data-driven insights. It has a robust metrics framework that’s in step with the organization’s mission, and this tracks progress and measures impact. This is a program that’s continuously improving and able to demonstrate return on investment.
The key thing here?
Metrics play a role at every maturity level. But only a fully mature program can demonstrate value through data-driven insights.
The stories hiding in your data (and how to find them)
So, how do you get started on the metrics-stretching journey? How do you spin tiny security data points into compelling business-value arguments?
A great first step is to play around with your people. (Disclaimer: not literally!!)
By segmenting data based on different user groups, you can tell compelling “data stories” about how each group is being impacted by your SA&T activities.
Let’s look at a few examples.
New hires: How are they doing with their training completion rates, phishing click rates and number of incidents reported? How they compare to the average person in your organization will reveal potential opportunities to improve the onboarding process for future hires.
Privileged users: Are there any strange access request patterns? How is their password hygiene, and how about their involvement in security incidents? What does the information tell you about your existing access controls? Do you need to implement some extra training, or better monitoring?
Phishing repeat clickers: Is there a correlation in which department(s) they are from? Are there other common factors like training completion status or phishing email types? This story can tell you where and how to target further interventions.
The fun doesn’t stop there. Other popular groups include developers/engineers, IT admins, and the C-suite.
You can divide segments into even smaller groups for deeper insights, or to test interventions. For instance, in a way that answers the question:
“Does one group respond better (meaning higher completion, lower clicks, higher reports) to particular communications, training and/or other interventions/controls?”
Using security metrics to demonstrate business value
So, you know how to unravel the stories ingrained in the numbers. But if the story is the “what” here, we need to be able to answer the “so what?”
How can you use the metrics you already have to demonstrate genuine business value? How do you show that how you’re changing click rates is foundational to the whole organization’s success?
There is a through-line, which takes us through four different levels of metrics/outcomes. Before we navigate them, let’s meet them:
1. Security Operational Metrics
These are the fundamental building blocks: click rates, report rates, and training completion rates. These individual actions, while seemingly minor, provide crucial data points for further analysis.
2. Security Outcome-Driven Metrics
Operational metrics feed into broader security outcomes like uptime, RTO (return to operations), MTTD (mean time to detect), and MTTR (mean time to recover). Tracking these metrics reveals the effectiveness of the security posture in minimizing disruptions and ensuring business continuity.
3. Business Operational Outcomes
Security outcomes naturally impact broader business operations. Improved security translates to enhanced productivity and service delivery, ultimately contributing to the smooth functioning of the organization.
4. Organizational Outcomes
At the pinnacle are the ultimate organizational goals: cost management and—in many cases—profit. By demonstrably improving security posture and reducing operational disruptions, security efforts directly contribute to the organization’s success and survival.
The further through the framework you can travel, the more clearly you’ll be able demonstrate the cumulative value of your security efforts.
And if you’re in a position to connect basic security operations metrics right through to achieving those overarching organizational objectives, fair play to you.
But—and important but—not everyone will have the resources to complete the full four-stage journey.
And…that’s OK! Really.
In many cases just being able to link Security Operational Metrics to Security Outcome-Driven Metrics will be a huge improvement.
So, what does it look like to convert one set of metrics to the next?
How password interventions translate into business success
So, what does the journey from password hygiene to bottom-line impact look like?
Security Operational Metrics: At the base, we track foundational password practices. This includes identifying weak passwords (below a set strength threshold) and reused passwords (appearing in past breaches).
Security Outcome-Driven Metrics: Here, we see the direct results of improved password habits. This translates to fewer compromised accounts, reduced training time and an increase in strong passwords and a decrease in reused ones.
Business Outcome-Driven Metrics: Now we connect security improvements to tangible business benefits. Stronger passwords lead to reductions in insurance premiums, improved risk audit scores, minimized business disruptions, and fewer regulatory fines.
Business Outcomes: At the top, we reach the sweet spot for executives — the impact on the organization’s core objectives. Effective password management translates to cost reduction, risk reduction, increased revenue generation, and ultimately, increased productivity.
How phishing interventions translate into business outcomes
Ava has this important point about phishing and business outcomes: “One of the things that I’ve noticed more and more are insurance companies looking for phishing programs and looking to see that we are measuring… So I think even being aware of what those companies or those partners are looking for can help prioritize and even maybe dig deeper and some of those more basic metrics.”
But just how do we demonstrate business value when it comes to phishing interventions?
Just like strong passwords, effective anti-phishing measures contribute to your organization’s success – but how do we show the through line?
Security Operational Metrics: Here, at the bottom of the stack, we measure those foundational phishing defense activities. This includes email click rates, training completion and attendance rates, user engagement with training materials, confidence levels in identifying phishing attempts, eLearning scores, and reporting rates of suspected phishing emails.
Security Outcome-Driven Metrics: Moving up, we’ve got metrics like a decrease in phishing incidents, reduced time spent on phishing-related issues (training, reporting, remediation), faster response times when phishing attempts are identified, and a reduction in overall user-related security incidents.
Business Outcome-Driven Metrics: Now we connect the dots between security improvements and tangible business benefits. Effective phishing defense translates to lower insurance premiums, improved risk audit scores, minimized business disruptions caused by phishing attacks, and fewer regulatory fines for non-compliance with data security regulations.
Business Outcomes: At the top, we reach the ultimate goal – demonstrating the impact on the organization’s core objectives. A strong anti-phishing program translates to cost reduction, risk reduction, increased revenue generation, and ultimately, increased productivity.
Notice how both stacks end up at the same place?
That’s kind of the point. That top level is where we want to be arriving because that is what leadership cares about the most.
Making a data wishlist
As we touched on earlier, the “good ol’” basic metrics (training engagement, phish clicks and incident reporting) can demonstrate value.
But what can you do to expand your access to data?
Ira Winkler, CISO and Vice President, CYE and author of Security Awareness for Dummies, has some great suggestions:
-
- The number of malware incidents on the network, because malware typically gets onto the network through a phishing incident, unsafe web browsing, or uploading accidentally through USBs.
- How many times the DLP system stopped data from getting out.
- Web content filters – how many times were people blocked from going to a given website?
These are all “root metrics”. They measure actual behaviors in practice.
“The more you can show that with the rollout of the awareness program those actual behaviors – which could create a major incident – have gone down, that’s showing the success of your awareness program,” Ira adds.
Getting your hands on the good stuff (data)
Busy, busy, busy. Everyone is always so very busy.
So how can you make sure they’ll even entertain your requests for data?
Ava is keen to share her experience here:
“Two things: One is to embed yourself where you can…in those meetings so you’re actually hearing what’s going on. A lot of those times metrics are being shared and they’re discussing trends, and you can start picking up and saying, ‘Oh, I want to see that,’ and, ‘You realize that these are all about people, right? These are all things that we can impact.’
“And I would say also try to just listen, keep your ear on the ground because you can hear things like ‘I’m so tired of these people who won’t patch their systems.’ There is very clear data that you can tap into and leverage for different types of campaigns or activities… And you can start saying, ‘I did this thing, and here is how that number is reducing.’ Or even creating visibility and trying to layer in some accountability.”
What keeps them up at night?
Senior leaders are tasked with keeping the organization afloat, keeping it successful, in good financial shape.
What’s most important to them won’t be what’s most important to you.
“In terms of getting into that next tier of business-outcome-driven metrics,” says Ava, “I think you have to have a better understanding of what actually matters to the executive leaders because you’re going to find yourself stuck in the security outcome driven metrics if you don’t start asking the right questions.”
We need to get into their heads. What keeps them up at night? For decades, people have been using that question to prepare for any conversation with any executive.
Or, to refine it a little?
What are the things that drive decisions for those senior leaders, and how are your actions tying back to it?
Let’s push things forward
By now, you’ve picked up on the loud and clear message that successfully demonstrating value means more than just doing training and giving leadership the completion rates.
Demonstrating the business value of human risk management requires us to step out of our traditional roles, and maybe our comfort zone.
Where can you get started?
“It’s almost impossible to speak in the context of value if you don’t understand what is valued,” says CybSafe’s CEO and Founder, Oz Alashe.
“We have to work to understand what is important to our organizations, which is not necessarily ‘did you run a security awareness campaign?’ / ‘did you manage to drive down the click rates?’ What was the point in doing that? What were you doing it for, and does the business actually care about that, and how do you articulate it in that context?”
“People in human risk management have to be able to understand the mathematics and the economics of what they do,” Ira expands. “For example, know how to take basic math… to say, ‘I reduced simulated phishing clicks from 20% down to 5%… and by doing that I reduced the number of clicks in theory by 75%, and therefore an incident cost us X amount and I therefore save the company X times whatever a 75% decrease is.’”
It’s also worth moving towards a “generalist” skill set. Being an expert programmer is impressive, but it’s important to understand how that programming fits within the network.
“I’d make sure people understand statistical design, how they can work with interface designers and reduce the number of accidents people have and know how to conduct repeatable studies,” Ira reflects.
But won’t leadership see this as getting distracted from the “core” work?
“If you’re in a decent organization, they would welcome you to expand your definition of human risk management,” Ira says. “A good CISO would welcome [it],” Ira remarks.
“Ultimately,” Oz concludes, “the extent to which we go is only determined – and curtailed – by the extent to which we’re willing to broaden ourselves.”
Secure the buy-in you deserve
“This is part of the evolution. Articulating what impact we’re having on the organizations that we actually exist to support, serve, and help – that’s really important, and the human aspect is fundamental to all of that.” — Oz Alashe
The world isn’t always fair—this we know. And it applies when it comes to allocating budgets, resources, and the spotlight.
“The problem with cyber security is that essentially security practitioners get the budgets that they deserve, not the budgets they need,” says Ira. “They need to learn to deserve what they need.”
Human cyber risk is a poor relation of sorts, says Ira, “because we fail to demonstrate value.” People (rightly) believe that no matter what you do a user will always click on a phishing message. “No counter measure is 100% perfect and we fail to push that concept, because the goal of any cyber security program is risk reduction and that is also the goal of any department or function within cyber security. We need to do risk reduction and show how we do it.”
Nothing turns the heads of stakeholders like an engaging narrative about how an existing element of their organization is delivering demonstrable value.
And it doesn’t take fancy metrics to get noticed. By keeping storytelling and outcomes in mind, you can show your program’s impact and gain that all-important support from senior management.
Above all, start where you are. Don’t give up just because you can’t implement all four levels of metrics (yet).
It’s a journey, after all. Feel free to send us a postcard.
Ready for the next level in metrics mastery?
This blog has, we hope, got you contemplating getting started on demonstrating business value.
But when it comes to knocking the C-suite’s socks clean off…what if you could ask the experts to weigh in?
“Demonstrating security awareness value to senior stakeholders” is our smash-hit webinar that helps you build on the approaches we’ve touched on here.
It’s packed with tips, insights and wisdom from Ava Woods-Fleegal and Ira Winkler, and hosted by Oz Alashe. And all three of them have oodles of experience in working metrics magic.
And…it’s FREE to watch, and available right now.
What you’ll learn:
-
- How to connect with senior stakeholders by talking their language and resonating with their goals.
- How to use your existing metrics (like training completion, phishing click rates, security incident report rates) to craft powerful, persuasive narratives.
- Senior buy-in: Ways to transform basic metrics into a watertight argument for greater program support.
- How to identify and gather new datasets to build on your value-proving evidence base.
- How to demonstrate tangible business value of a robust, human-centric program by converting “security operational metrics” into “business outcome metrics”
- How to use the SANS Security Awareness Maturity Model™ to assess and evolve your own program (and understand the benefits of doing so).
