Benchmarking is Bad ☠️
Benchmarking is Bad ☠️
Why is benchmarking is bad? Also, some of my favorite benchmark reports that most folks shouldn't look at :)
👋Welcome to the OnlyCFO newsletter where 25k+ readers learn about finance and operations.
💡Finance Webinars: We are organizing free webinars on important topics for software leaders. Sign up and tell me what topics you want to hear about.
Sponsored by Brex. Make every dollar count with Brex.
Every software leader is trying to maximize efficient growth, so check out the CFO’s guide to efficient growth from Brex. Learn how top finance leaders optimize their precious resources and their decision frameworks for investing in growth.
Is Benchmarking Bad?
I have a love/hate relationship with benchmark reports:
❤️Love: The data is super interesting and helps me understand how my company stacks up against others.
🤬Hate: Benchmark data can become an excuse for people to turn off their brains and use benchmarks to justify their spend/metrics - “Everyone else at our stage spends 50% of revenue on engineering so we should too!”
Last month Apple announced that their CFO (Luca Maestri) was stepping down after an incredible 11 year run. During his reign as Apple’s CFO the market cap 10x’d and it is now a $3.4 trillion dollar company.
Below is what Luca had to say on benchmarking and building lean teams. Send this to your leadership team as you go through annual planning for next year.
We run the company very lean. I tell my guys in finance, I don’t want you guys to ever benchmark anybody else because you can only get bad ideas. — Apple CFO
We run the finances of Apple with about half the size of some of the companies I have worked for before that were a tenth the size of Apple. If we have the right people, we don’t need a lot of them. — Apple CFO
Luca is 100% correct. Operators frequently use benchmarking as a crutch and spend justification versus using their brain for what is actually required. Luca could easily justify hiring 5x the number of people on his teams if he compared Apple to benchmarks. Luckily, Luca told his team to ignore benchmarks…
Rather than focusing on benchmarks, companies should be more focused on their own data, its quality, and how they can maximize efficient revenue growth.
Who is benchmarking bad for?
There are two primary audiences of benchmark reports:
Operators - people running companies
Investors - VCs, public market investors, etc
Investors love benchmark data because it helps guide their investment decisions and shows how their portfolio companies stack up against others. There are many software metrics that investors can use to quickly evaluate companies. There are lots of good reasons for investors to use benchmark data.
Operators, on the other hand, usually misuse benchmark data and therefore make bad decisions based on them. Operators focus on benchmark data because of the importance put on them by their investors.
Rarely do I see bootstrapped companies obsess over benchmarks like VC-backed companies…Because bootstrapped companies are forced to use their brain from the beginning since cash is much more meaningful to them. They are much more likely to make decisions based on need and not what everyone else is doing (and what benchmarks say).
Ideally, people are aware of the benchmarks but are able to use their brain to make good decisions. Unfortunately, this is really hard for most people to do.
⚠️Dangers of Benchmarking
By far the biggest issue with operators using benchmarks is what I said above — using it as a crutch and excuse to not use your brain. But there are other issues with benchmarking worth highlighting that can affect both operators and investors:
Data Quality: There are two main considerations:
Inconsistent definitions: Almost every SaaS company has its own definitions of SaaS terms and metrics, which makes benchmarking less meaningful.
ARR, for example, is frequently defined differently or incorrectly. Check out this comprehensive (free) ARR Guide which covers how to define and report on ARR, and some benchmarking data.
Inconsistent accounting data for private companies. Accounting automation tools may help keep your own data consistent but be sure you’re comparing apples to apples when benchmarking. There is a wide range of how things are accounted for at private companies which makes many benchmarks less meaningful. There is even diversity in practice on how public companies do some accounting.
As an example, I frequently hear early-stage companies say that they have world-class gross margins… Yeah, if you don’t add half of the costs that belong in COGS then things will look great :)
Context is critical. The more comparable your company profile to the data set the more meaningful the comparison. Understand who you are comparing against.
Benchmarks are historical looking and may not reflect current economic conditions or impact of things like generative AI. You want to be skating to where the puck is going, not where it has been.
Look at the big picture. It’s easy for companies to get sucked into metrics and comparisons, but you need to step back and understand your company’s objectives. Don’t let investor metrics control your business. Remember to use your brain :)
If you take away anything from this post, remember that benchmarks should never be blindly followed. Benchmarking data can be very useful for investors, but it is often destructive for operators because the data is used improperly and bad decisions are made.
VCs should push benchmarking less and encourage companies to use their brain more and have companies focus on maximizing efficient growth.
Benchmarking Reports
Below is a list of some of my favorite recent cloud benchmark reports that most people probably shouldn’t look at 🤣. But if you do look at them, please remember all my caveats above and that they are just data points.
Private Company
The State of SaaS in 2024 by ICONIQ
This report was released this month. They always puts out amazing content and analysis. My caveat is that ICONIQ has a lot of top-tier companies in their portfolio/benchmarks so remember that when comparing.
ICONIQ Compass — interactive benchmarking
They also released a cool interactive benchmarking tool this week that allows companies to slice the benchmark data to be more comparative.
Bessemer’s State of the Cloud 2024
Published just a couple of months ago, this report includes some great analysis and insights on AI’s impact on cloud companies.
Ray Rike has put together a great interactive benchmarking tool that allows deeper segmentation than many other private company benchmarks, which is critical context.
Public Company
Jamin Ball’s Clouded Judgement Blog
Weekly update on public SaaS company benchmarks and always has insightful commentary on the markets.
Great interactive public cloud company data. They also do some awesome S-1 breakdowns if we ever get more cloud IPOs!
Footnotes:
A couple of things from Brex (today’s sponsor) that may help:
Report more accurately & consistently with accounting automation
Get a peek into the future of B2B payments
For those struggling with defining or how to report ARR, check out - ARR Guide