The Cyber Eden

Troubleshooting Fine Code Coverage

Peter Vo's photo
Peter Vo
·

4 min read

The Problem

One day, I found the Fine Code Coverage does not generate coverage report of the project I was testing on. It was weird, because it used to do so. At this point, it only issued the coverage report of the test project, not the tested project.

The Solution (TLDR)

Add the file finecodecoverage-settings.xml with the following content:

<?xml version="1.0" encoding="utf-8"?>
<FineCodeCoverage>
    <Enabled>
        True
    </Enabled>
    <Include>
        [*]*
  </Include>
</FineCodeCoverage>

Additionally, solution to the problem where the tests in MSTest project not detected:

When the MS tests are not detected, first, check the Nuget Packages of the test project, and find if there is a mismatch in versions between 2 Nuget Packages MSTest.TestAdapter and MSTest.TestFramework.

Second, sometimes tests are not detectable due to the *.testrunconfig or *.runsettings files. Go to the menu Test > Configure Run Settings to find where are these files. Delete them.

Troubleshoot Journals

I read its logs, which showed the following:

Run OpenCover ({My test project name})

I opened my playground project PeterDoStuff and ran the tests as well to see any difference. Unlike the test project above, the playground project has Fine Code Coverage provides complete coverage report on both the test project and the tested project. The log has a small difference:

Run Coverlet ({My test project name})

After doing research on this, I find the reason there is a difference there is because:

Under the covers, the extension runs:

Source:github.com/FortuneN/FineCodeCoverage/issues..

Try update Visual Studio Community 2022, from 17.9.0 to 17.9.4.

Not effective.

Look up for more, found this:

@rmed1naFor the solution that you were having issues with, do you have the nuget test adapter in your test projects ?

Source: https://github.com/FortuneN/FineCodeCoverage/issues/33

Proceed to check and compare the Nuget of the 2 projects.

The Pass Project is MSTest for .NET 8. It has Nuget as following:

  • MSTest.TestAdapter: 3.0.4

  • MSTest.TestFramework: 3.0.4

  • coverlet.collector: 6.0.0

The Fail Project is MSTest for .NET Framework. It has Nuget as following:

  • MSTest.TestAdapter: 2.1.1
    Updated to 3.2.2, which causes no test detected. Reverting the version temporarily (which are reverting of 3 files csproj, app.config, and packages.config), which makes the tests detectable.

  • MSTest.TestFramework: 2.1.1
    Updated to 3.2.2,which brings the tests back to detectable.
    However the FCC does not bring back the Fail project coverage report.

Though, new knowledge gained:

When the MS tests are not detected, first, check the Nuget Packages of the test project, and find if there is a mismatch in versions between 2 Nuget Packages MSTest.TestAdapter and MSTest.TestFramework.

Second, sometimes tests are not detectable due to the *.testrunconfig or *.runsettings files. Go to the menu Test > Configure Run Settings to find where are these files. Delete them.

  • coverlet.collector: Not exist
    Proceed to install version 3.0.2. Caused failure in building. If close and open Visual Studio, the test project cannot be loaded with the following error:

Look up: https://stackoverflow.com/questions/67058242/using-coverlet-with-net-framework-generates-an-error-the-expression-system-v

Try resolve by accessing the packages folder of the solution root path, remove the coverlet.collector folder, close the solution completely and reopen it. At first the project can be loaded, but the same error would happen again.

Abort the coverlet direction.

Try uninstall then re-install the Extension. Not effective.

Try install the Extension "Run Coverlet Report VS2022." Not effective.

Attempt to assess the FCC Output logs further. Find the following:

Fine Code Coverage {timestamp}: Run OpenCover ({Test Project name}.Test)
Fine Code Coverage {timestamp}: OpenCover Run ({Test Project name}.Test) Arguments 
...
"-filter:+[{Test Project name}.Test]*"

This filter parameters are not found in the playground project:

Fine Code Coverage 25/3/2024 11:24:08 AM: Run Coverlet (PeterDoStuff.Test)
Fine Code Coverage 25/3/2024 11:24:08 AM: Coverlet Run (PeterDoStuff.Test) - Arguments
"D:\Projects\PeterDoStuff\PeterDoStuff.Test\bin\Debug\net8.0\fine-code-coverage\build-output\PeterDoStuff.Test.dll"
--format "cobertura"
--exclude-by-file "**/Migrations/*"
--exclude-by-attribute GeneratedCode
--include-test-assembly
--target "dotnet"
--threshold-type line
--threshold-stat total
--threshold 0
--output "D:\Projects\PeterDoStuff\PeterDoStuff.Test\bin\Debug\net8.0\fine-code-coverage\coverage-tool-output\PeterDoStuff.Test.coverage.xml"
--targetargs "test  ""D:\Projects\PeterDoStuff\PeterDoStuff.Test\bin\Debug\net8.0\fine-code-coverage\build-output\PeterDoStuff.Test.dll"" --nologo --blame  --results-directory ""D:\Projects\PeterDoStuff\PeterDoStuff.Test\bin\Debug\net8.0\fine-code-coverage\coverage-tool-output"" --diag ""D:\Projects\PeterDoStuff\PeterDoStuff.Test\bin\Debug\net8.0\fine-code-coverage\coverage-tool-output/diagnostics.log""  "

Follow the guide on https://github.com/FortuneN/FineCodeCoverage?tab=readme-ov-file#project-configuration

Add the file finecodecoverage-settings.xml with the following content as a Solution Item:

<?xml version="1.0" encoding="utf-8"?>
<FineCodeCoverage>
    <Enabled>
        True
    </Enabled>
    <Include>
        [*]*
  </Include>
</FineCodeCoverage>

The <Include> then overrides the filter, to be as following:

Fine Code Coverage {timestamp}: Run OpenCover ({Test Project name}.Test)
Fine Code Coverage {timestamp}: OpenCover Run ({Test Project name}.Test) Arguments 
...
"-filter:+[*]* +[{Test Project name}.Test]*"

Now I can see the coverage of the project being tested.

To consider consolidating and reporting to Fine Code Coverage issue board, possibly with a simple test .NET Framework project to replicate first.