Code coverage is not a quality metric

Code coverage is not a quality metric

You've just finished a new feature. The pull request is approved, the pipeline is green, and your coverage report proudly shows 100%.

Great. Right?

Well, not necessarily. One of the biggest misconceptions I still encounter in software teams is the belief that a high coverage percentage automatically means high quality software. It doesn't. In fact, I've seen teams proudly report 90%+ coverage while still shipping serious production bugs.

Coverage is useful, but only when you understand what it actually measures.

What is code coverage?

Code coverage tells you how much of your code is executed while running your tests. Most coverage tools report several metrics. These metrics are useful because they help identify completely untested areas of the application. What they don't tell you is whether your tests are actually good. 

Statements

Have all statements been executed at least once?

if (isLoggedIn) {
  showDashboard();
}

Did the test execute the showDashboard() statement?

Branches

Have all possible decision paths been executed?

if (isLoggedIn) {
  showDashboard();
} else {
  showLogin();
}

Did the tests cover both the true and false paths?

Functions

Have all functions been called?

Lines

Have all lines of code been executed?

The dangerous part about coverage

Coverage measures execution. It does not measure correctness.

Consider this example:

it('calculates discount', () => {
  calculateDiscount(100);
});

This test executes the function. But it doesn't verify anything. If the function returns the wrong result, the test still succeeds. From a coverage perspective, everything looks great. From a quality perspective, the test is almost worthless.

This is why chasing a coverage percentage can create a false sense of security.

100% coverage can still hide bugs

Imagine a tax calculation:

function calculateTax(amount) {
  return amount * 0.12;
}

The business rule should have been 0.21.

You can write enough tests to achieve 100% coverage and still ship the wrong implementation.

Coverage cannot tell you:

  • Whether business requirements are implemented correctly
  • Whether edge cases are covered
  • Whether assertions are meaningful
  • Whether users can successfully complete their journey
  • Whether the application works in production

Coverage only tells you that code was executed. Nothing more.

Why coverage matters

At this point you might wonder whether coverage is useful at all. I believe it is. Coverage is an excellent indicator for identifying blind spots. When a new feature is merged with 15% coverage, that should raise questions. When a critical authentication flow has no tests at all, coverage reports make that visible. The mistake is treating coverage as a goal rather than a signal. A team aiming for "80% coverage" is often optimizing for the wrong outcome. A team aiming for confidence in production usually ends up with good coverage naturally.

The actual value of tests

The biggest benefit of testing isn't the coverage report. It's what happens while writing the tests.

Good tests force developers to think about:

  • Responsibilities
  • Edge cases
  • Dependencies
  • Architecture
  • Maintainability

In my experience, code that is difficult to test is often difficult to understand as well. Testing exposes design problems long before they become production issues.

Comments

There are no comments yet, leave yours below.

Leave a comment

Do you have an addition, question or experience related to this article? Share it below.

Comments are briefly reviewed before they appear.

Read more about:

Starten met unit testen in Vue

Vue wordt door steeds meer developers gebruikt. Laravel geeft out of the box de mogelijkheid om gebruik te maken van Vue en Bootstrap, maar ook React of Angular developers kiezen steeds vaker voor Vue. Na het meewerken bij een aantal projecten waar VueJS wordt ingezet begin ik te merken dat er nauwelijks wordt getest. Meest genoemde redenen zijn dat het testen te veel tijd kost en dat unit testen in de frontend moeilijk is. Volgens een…

Lees verder

10 JavaScript one-liners every developer should know

Code should be readable and pragmatic. For me, that means when I look at a piece of code, I should almost instantly understand what it does. Clear code always has good variable names, is predictable, avoids unnecessary logic, and lives in the right place within a project. But sometimes you need to do something a little complex without turning the rest of your code into a mess. That’s where one-liners come in. Small, pragmatic snippets…

Continue reading

The difference between debounce and throttle

Debounce and throttle are powerful techniques, but only if you really understand what they do. I often ask about them in tech interviews, and to my surprise, many developers (even seniors) struggle to explain the difference. The good news is: it’s actually quite simple. In this post, I’ll break down both techniques and share some practical use cases. If you are unfamiliar with the techniques and are asked about them too in your interview…

Continue reading