ExcludeFromCodeCoverage considered harmful

Abuse of the ExcludeFromCodeCoverage attribute

I once worked on project that had a mandatory code coverage target. If your commit didn’t maintain the overall coverage ratio of 75% then it was rejected. The reasoning came from good intentions; a high code coverage is good therefore we will create the mandate that is must be high. It had some unfortunate side effects though. The first unintended consequence is that developers only wrote enough tests to keep the value above the target instead of considering how they needed to test their code. The second consequence was the proliferation of [ExcludeFromCodeCoverage] attributes adoring all sorts of classes.

The attribute was originally designed for generated code but more recently I’ve seen it be applied where the code that is hard to test or too simple to test. Ultimately this hides code from testing metrics. Your code coverage metric is no longer accurate nor useful.

Yeah, we are at 85% code coverage ignoring the code we excluded because it was hard to test.

My immediate response to this is “How come it isn’t 100% then?”

I would much rather have a lower, but accurate, code coverage metric so I consider the use of this attribute harmful on any code that isn’t generated.

I wish code analysis tools like SonarCube or the Roslyn analyzers treated this attribute like SuppressMessageAttribute - the Justification property should be filled out when applied. Better still, just fix the issue and avoid the need to use either of them.

Why should I test properties?

The consensus on Should you Unit Test simple properties? seems pretty much for testing and you can use the examples here to test yours. I prefer property based testing though because it can find edge cases you didn’t think of. There is a great intro at Property-Based Testing with C# using FsCheck. Your code will effectively look like:

[Property]
public Property Set_Then_Get_Returns_Same(string exampleValue)
{
    var target = new ClassYouWantToTest();
    target.PropertyToTest = exampleValue;
    return (target.PropertyToTest == exampleValue).ToProperty();
}

FsCheck will generate a bunch of random values to try this test with so these 4 lines of code are resulting in hundreds of unique tests for this property including, for this example: blank string, null value, very short, very long, non-printing, accent characters, etc.

Why should I test … something else?

I will come back and add more examples as I encounter them.

Summary

• Code coverage metrics are only useful if they help improve the software quality.
• Mandatory targets can lead to harmful practices such as excluding code from testing or writing superficial tests.
• ExcludeFromCodeCoverage attributes should be treated in the same way as SuppressMessage - they hide warnings that should really be fixed.

Photo by Luca Bravo on Unsplash