How do I measure success as a CISO?

How miserable can I make an attacker's life?

So you are blowing all this money on security. Is it a black hole? How do you know if it's actually doing any good? You've got KPIs for all your other teams but the security people just wave their hands and say "hard to measure, sorry". If you're a CEO, how do you make sense of this?

How do you measure security?

Well, measuring security risk is indeed an unsolved problem in computer science. The greatest precision available today to measure risk involves labels like "LOW", "MEDIUM" and "HIGH", with the occasional "CRITICAL" thrown in as a scare factor.

But there are pragmatic ways to measure the effectiveness of your security spend. This ties in with the core concept of Security ROI I've discussed previously. How can I spend the least amount of money to mitigate the most amount of risk?

It goes beyond that, though. As a security-driven CISO (and not a compliance-driven one), my threat model includes malicious adversaries large and small. How much money can I force them to spend to attack me?

Because criminals also measure their ROI. Why do you think Windows has more malware than other operating system? Because more people use Windows than any other operating system. Malware, like all software, scales. And a smart criminal will maximize their ROI by looking for advantages of scale.

If it only costs a financially-motivated attacker the time to send phishing emails, with an expected very large payout, then that's a great ROI. If I can raise the bar and force them to spend years and millions of dollars, it's likely I can shake them off my tail and encourage them to eat another camper instead.

Even when the bear decides he wants you, specifically, you buy yourself time to stay ahead of their claws by making their job as difficult as possible. Force your adversaries to spend as much money and time as possible in order to attack you.

This is why you'll hear security people talking constantly about "low-hanging fruit" and doing the "blocking and tackling". We are not fruit-obsessed short people or enamoured by the sportsball (well, I'm not, anyway lol).

An attacker will always try the cheapest and easiest ways to attack you first. They aren't going to try that crazy cryptographic side-channel attack that involves eavesdropping on your power consumption like something out of a movie. They will try phishing, and social engineering, and metasploit.

Making an attacker's job harder is how you measure defensive security.

That being said, if an attacker is prepared to spend a billion dollars to steal a billion dollars, then there's nothing I can do about that. And this is where nation states motivated by geopolitical concerns are especially dangerous, because the intelligence agency of a major or medium power is not motivated by money, and they will happily print as much fiat as needed to achieve their international foreign policy goals.

But for most of us, most of the time, defending against most attackers, an excellent measure of security spend is, "How can I asymmetrically spend as little time and money as possible to force attackers to spend the most time and money as possible?"

Other Resources:

A Tale of Two Attackers

The parable of a bear and two campers

During a Bear Market, Focus on Security ROI

How much bang are you getting for your buck?

Pragmatic Risk Management

How to be a CISO