Your webshop promises free shipping from €50. A customer spends exactly €50 and still pays €4.95 for delivery. The tests pass, both branches are covered, and the coverage report shows 100%.
I've seen teams proudly report 90%+ coverage while still shipping serious bugs. The percentage was correct. The confidence attached to it was not. Somewhere between writing the tests and approving the pull request, the number became evidence that the feature worked.
I want to know what those tests would actually catch. A wrong shipping fee? A payment taken twice? Someone opening another customer's invoice? Those are questions we can answer, but reading the coverage percentage will not do it for us.
Like my articles about frontend development and web accessibility, this is about a useful practice losing its purpose. Coverage helps us find gaps. The bullshit starts when reaching a target becomes enough to approve the work.
What coverage measures
Code coverage records which parts of your code execute during a test run. Most tools report several measurements:
- Statements: which statements ran.
- Branches: which outcomes of decisions were taken.
- Functions: which functions were called.
- Lines: which executable lines were reached.
These measurements describe execution. They cannot tell you whether the expected result in a test matches the requirement. They also cannot tell you whether you chose the inputs that expose a mistake.
Branch coverage gives you more information about decisions than line coverage alone. It still does not mean every input, state or combination of decisions has been tested. That distinction matters even in a function with one line of logic.
The missing assertion
Start with the obvious problem. Suppose the agreed rule for a simplified tax calculation is 21%, but the implementation uses 12%. This test still passes:
function calculateTax(amount) {
return amount * 0.12;
}
it('calculates tax', () => {
calculateTax(100);
});The function executes, but nothing checks its result. An exception could fail the test; returning the wrong amount will not. Adding expect(calculateTax(100)).toBeCloseTo(21, 2) would catch this particular mistake. These examples use Jest-style assertions.
Most developers will spot that problem immediately. It is tempting to conclude that meaningful assertions solve the issue. The shipping example is harder to dismiss, because its tests actually check the results.
The €50 bug
Take the webshop from the introduction. Its rule is simple: delivery costs €4.95 below €50 and is free from €50 onwards. The amounts in this example are expressed in euros.
Both tests pass
function shippingCost(total) {
return total > 50 ? 0 : 4.95;
}
it('charges shipping for a small order', () => {
expect(shippingCost(25)).toBe(4.95);
});
it('offers free shipping for a large order', () => {
expect(shippingCost(75)).toBe(0);
});Both tests pass. The function is called, its return statement runs, and both outcomes of the condition are exercised. For this function, those cases cover all the executable lines and both branches.
The assertions are meaningful too. They would catch an implementation that always returned zero, always charged shipping or used the wrong delivery fee. These tests protect real behaviour. They just do not protect the whole rule.
At exactly €50, the condition total > 50 is false. The customer pays €4.95. The developer implemented "more than fifty", while the requirement says "from fifty".
Test the boundary
We need cases around the point where the behaviour changes:
it('offers free shipping from 50 euros', () => {
expect(shippingCost(49.99)).toBe(4.95);
expect(shippingCost(50)).toBe(0);
expect(shippingCost(50.01)).toBe(0);
});The assertion for €50 fails against the original implementation. Changing the comparison fixes it:
function shippingCost(total) {
return total >= 50 ? 0 : 4.95;
}The new test catches a real bug without increasing the branch coverage. Nothing new had to execute. We needed a better choice of input and a clear expectation for that input.
This is why I want reviewers to read the requirement alongside the tests. Reading the implementation first and deriving test cases from it makes it easy to copy the same misunderstanding. Both pieces of code can agree while the customer still gets charged incorrectly.
There are more questions behind this small rule. Does the threshold apply before or after a discount? Do gift cards count towards it? Those answers depend on the product. No testing tool can decide them for us.
Your mocks passed
Now consider a deliberately incomplete checkout flow. It charges the customer and then saves the order:
async function checkout(order, payments, orders) {
const payment = await payments.charge(order.total);
return orders.save({ ...order, paymentId: payment.id });
}In a unit test, the payment mock resolves successfully and the database mock does too. You assert that the expected amount was charged, that the payment ID was passed to the database and that the saved order was returned. Every statement in this function executes.
After the payment
That test tells you something useful about the successful flow. Now let the payment succeed and the database fail. The checkout rejects, the customer sees an error, and no order has been saved. The payment has already happened. Rejecting a promise does not undo it.
The customer tries again. This implementation calls the payment service again. Whether that creates a second charge depends on the payment integration and its protection against duplicate requests. A mock that always returns the same successful response does not answer that question.
Trying again
I would want the team to decide what should happen in at least these situations:
- The payment succeeds, but saving the order fails.
- The payment provider processes the request, but the response never reaches us.
- The same checkout is submitted again while the first attempt is unresolved.
For each case, define the expected customer outcome and how we recover. Do we recognise the existing payment? Can we resume processing the order? What do we show the customer while the result is uncertain?
A payment API may support idempotency keys to recognise retries of the same operation. Stripe documents that mechanism, including how the same key is reused on a retry. Generating a fresh key on every attempt would defeat that purpose. It also would not solve the separate problem of recovering the missing order.
Mocks are useful here. Make the database mock reject after a successful payment and test the recovery you have designed. Then verify the assumptions about your payment adapter against the provider's contract and test environment. A unit test and an integration test answer different parts of the question.
The problem was never the existence of a mock. It was allowing the behaviour we programmed into the mock to become our entire understanding of the dependency.
Missing requirements
Consider a customer portal with a server-side endpoint for downloading invoices. It accepts an invoice ID, retrieves the matching record and returns it. A test requests invoice 123 and checks that invoice 123 comes back. Another checks the response for an unknown ID. All the existing retrieval logic can be covered.
Now add the requirement everyone assumed was obvious: a customer may only download their own invoices. Suppose that check was never implemented.
There is no ownership check for the coverage report to mark as untested. The missing rule has produced no code, so it contributes nothing to the percentage. The implementation can be fully covered while the feature is incomplete.
At the endpoint level, test the difference between a customer requesting their own invoice and the same customer requesting someone else's. The second request must not return the invoice data. If the system supports shared company accounts or other permissions, make those rules explicit too.
This does not mean every database helper needs to implement authorisation. It means the tests for the protected endpoint must exercise the layer responsible for enforcing it. Hiding another customer's invoice link in the frontend is not enough.
A reviewer asking "whose invoice is this?" can uncover a requirement that another hundred calls to the retrieval function would never test. That question requires an understanding of the product, not a higher coverage target.
Chasing 80%
I understand why teams want a minimum percentage. It is visible, easy to automate and gives a pull request a clear pass or fail. A threshold can help prevent coverage from deteriorating unnoticed. I would keep a useful check like that.
But a target also affects which work gets done. If a pull request needs another two percentage points, the quickest option may be to test a few straightforward formatting functions. Investigating how checkout recovers from a lost payment response takes longer. A total percentage does not distinguish the value of those choices.
The shipping test shows the other side of that problem. It improves the suite without moving the coverage number at all. If the dashboard is how we report progress, useful work can become invisible.
I use coverage to inspect what has not executed, especially in changed code. Then I want to discuss the behaviour, the missing cases and the consequences of a failure. Google's coverage guidance also stresses that covered code can still have missing cases and that a percentage needs to be interpreted alongside risk.
A low score deserves attention. A high score still needs an explanation. "The pipeline accepts it" is a description of our settings, not a review of the feature.
Break your code
Go back to the corrected shipping function. Temporarily change >= to > and run the tests. The original €25 and €75 cases still pass. The new €50 case fails. You have directly checked that the suite detects the mistake you care about.
Mutation testing automates this idea by making small changes and rerunning tests. Tools such as Stryker include comparison changes like this. When a test fails because of the change, the mutation is caught. When the tests still pass, investigate what that tells you.
Some mutations expose a missing case. Others are equivalent mutations that do not change observable behaviour. A surviving mutation needs interpretation. It is not automatically evidence of a bad test.
Mutation testing also cannot invent the invoice ownership requirement for us. If the check is absent, changing operators in the retrieval code will not supply it. This technique is another way to challenge the tests we have.
I would start with important logic and use the results to learn where the suite is weak. Introducing a mandatory mutation score and then writing tests purely to raise it would bring us straight back to the same bullshit.
What I expect
When I review tests, I want to understand the decisions behind them. Why these inputs? Where did the expected result come from? What happens when the dependency fails? These are the same kinds of questions I discuss in my article about code reviews. You do not need to know every detail of a system to notice an assumption that nobody has explained.
Review the behaviour
For the examples here, a useful review would establish a few concrete things:
- The shipping cases cover the agreed boundary, including exactly €50.
- The checkout tests address a successful payment followed by a failed order, including what a retry should do.
- The invoice tests check the requesting customer's permissions at the boundary where access is enforced.
- The team knows which assumptions are covered by mocks and which have been checked against the real integration.
That is also where writing tests helps me most. Choosing the €50 case forces me to read "from fifty" carefully. Working through a failed checkout forces a discussion about what we owe a customer whose money we have already taken. Testing invoice access forces us to define who may see which data. The work starts before the first assertion is written.
When those questions expose an unclear requirement, I want the developer to raise it. I want the product owner to help decide the expected behaviour. I want the reviewer to challenge the test that quietly assumes both external calls will always succeed. A coverage requirement does not remove any of those responsibilities.
Know the limits
We should also be honest about the limits. A unit test will not prove that the payment provider behaves exactly like our mock. A successful integration test will not cover every possible outage. We can still explain what we have verified, what remains uncertain and why we consider the remaining risk acceptable. That is a much more useful conversation than defending a percentage.
If a bug reaches production, use what happened to improve that understanding. A customer charged for shipping at €50 needs the missing boundary case. A payment with no order needs a recovery path. An exposed invoice needs an access rule enforced and tested. Raising the coverage threshold from 80% to 90% would not, by itself, fix any of those failures.
I am happy to see a green pipeline. I want the checks in it to earn the confidence we place in them. When someone tells me a feature is well tested, they should be able to explain which behaviour the tests protect and show a plausible mistake that makes them fail.
If we can explain why coverage went up, but cannot explain which failure we now catch, what exactly did we improve?
