Xunit Unit Testing

[Karleen Chura]

xUnitnet is a free, open source, community-focused unit testing tool for the .NET Framework. Written by the original inventor of NUnit v2, xUnit.net is the latest technology for unit testing C#, F#, VB.NET and other .NET languages. xUnit.net works with ReSharper/Rider, CodeRush, TestDriven.NET and Xamarin. It is part of the .NET Foundation, and operates under their code of conduct. It is licensed under Apache 2 (an OSI approved license).

Start Visual Studio, which will bring you to the start splash screen. Under "Get Started", click "Create a new project". This will bring you to the first step of the new project wizard, where you pick your project type:

Find the project in the Solution Explorer (it will be titled "MyFirstUnitTests"), right click it, then click "Edit Project File". This will launch the text editor for your project file. It should look something like this:

Test Explorer is the name of the window that lets you browse and run your tests from within Visual Studio. Open it by choosing Test > Test Explorer from the main menu. You should be greeted with a window that contains a hierarchy of the tests in your project, which should look something like this:

We purposefully wrote both a passing and failing test, and we can see that the results reflect that. By clicking on the failed test, we can see a link both to the top of the unit test (at line 14), but also the failure message (we expected 5 but got 4) as well as a link to the exact line where the failure occurred (line 16).

When you edit the source file, also take note of the fact that CodeLens decorations show up which indicate not only test status (passed/failed) on the test themselves, but also on functions that are called by the code, indicating how often the code is called in tests, and a count of which of those tests have passed or failed:

You may have wondered why your first unit tests use an attribute named [Fact] rather than one with a more traditional name like Test. xUnit.net includes support for two different major types of unit tests: facts and theories. When describing the difference between facts and theories, we like to say:

A good example of this is testing numeric algorithms. Let's say you want to test an algorithm which determines whether a number is odd or not. If you're writing the positive-side tests (odd numbers), then feeding even numbers into the test would cause it fail, and not because the test or algorithm is wrong.

Although we've only written 3 test methods, the test runner actually ran 5 tests; that's because each theory with its data set is a separate test. Note also that the runner tells you exactly which set of data failed, because it includes the parameter values in the name of the test. The Test Explorer UI even shows a new level in the tree, as each row of data becomes a test result underneath its test method.

Sometimes, you want to write tests and ensure they run against several target application platforms. The xUnit.net test runner that we've been using supports .NET Core 1.0 or later, .NET 5.0 or later, and .NET Framework 4.5.2 or later. With a single test project, we can have our tests run against multiple target frameworks. Open the .csproj file and make the following change.

Test Explorer supports any combination of .NET Core (including .NET 5+) and .NET Framework targets. You can even include multiple versions of the same target framework (for example, it's legal to have something like net452;net461;net48;netcoreapp2.1;netcoreapp3.1;net5.0). Application authors will typically only use a single target framework, related to the target framework the application is intended to run on. Library authors are more likely to use several target frameworks, to ensure their tests run successfully on all supported target frameworks.

The Test Explorer tree view will now show your test project multiple times, once for each target framework. You can run individual tests from individual frameworks, or you can simply run all tests for all frameworks.

TL;DR: This article will guide you in creating automated tests with xUnit for your C# applications. You will learn the basics of automated tests and how to create unit and integration tests.

Testing ensures that your application is doing what it's meant to do. It seems a trivial statement, but sometimes this statement is underrated, especially when you change your existing codebase. You have to make sure not only that your changes work as intended, but also that the untouched code continues to do its expected job.

Manual testing is a very demanding task, not only for performing the tests themselves but because you have to execute them a huge number of times. It is a repetitive task, and where there is a repetitive task, you need automation.

Of course, each type of test brings value to ensuring the correctness of the software application, and each one has its strengths and weaknesses. For example, while the unit tests are usually executed really fast, the end-to-end tests are slower and may have various points of failure due to the interaction of multiple systems.

When testing your system, you cannot pretend to be able to cover all possible use cases. You should limit them to a subset due in part to the growth of complexity when passing from a simple unit to a composition of systems, in part to the time required to execute the tests. Usually, the number of tests decreases while passing from unit to end-to-end tests, following the well-known Test Pyramid diagram:

The .NET Core platform supports different testing frameworks. However, xUnit has become the most popular due to its simplicity, expressiveness, and extensibility. The project is supported by the .NET Foundation, and it is part of the more recent versions of .NET Core. This means that you don't need to install anything but the .NET Core SDK.

You may have heard about Test-Driven Development (TDD). It is a software development process that promotes the writing of tests before writing your application code. This approach leads to a short and iterative development cycle based on writing a test and letting it fail, fixing it by writing the application code, and refactoring the application code for readability or performance.

This command will clone only the starting-point-unit-tests branch of the repository in your machine. After the command executes, you will find the unit-integration-test-xunit folder containing a unit-tests subfolder. This subfolder contains the PasswordValidator folder with a project with the same name.

As you can see, the validation logic is implemented by the IsValid() method through a regular expression. The PasswordValidator class represents here a unit of code because it is self-contained and focused on one specific goal.

Here you see the ValidityTest class, which is hosting the unit tests for the IsValid() method of the PasswordValidator class. The only unit test currently implemented is the ValidPassword() method. This method is decorated with the Fact attribute, which tells xUnit that this is a test. The statements in the body of the ValidPassword() method are organized to highlight the AAA pattern mentioned above.

Finally, the Assert step verifies that the returned result is the expected one. This check uses the Assert object, which provides many methods to validate a result. In this case, you are using the True() method, which is successful when its first argument is true. Otherwise, the test fails and displays the string provided as the second argument.

When you are testing your code, you shouldn't just verify the positive cases; that is, the cases where things are fine. You also have to verify negative cases. For the IsValid() method, you have to verify a possible case where the password passed as an argument doesn't comply with the constraints. So, add the new unit test implemented by the method NotValidPassoword() to the ValidityTest class, as shown below:

In this case, you are passing an invalid password, and in the Assert step, you expect that the value returned by the IsValid() method is false. If you run the tests with dotnet test you will get two successful tests. Pretty easy!

The two cases of password validity tested by the unit tests are far from exhaustive. They are just two simple examples of positive and negative cases, but, of course, the possible cases to test are many more. You cannot expect to check every possible case, but you can test a significant subset of typical cases. This helps in having a greater code coverage for your production code. In the password validation example, this means that you should identify a representative set of valid and invalid passwords. For each password in these sets, you should apply one of the tests implemented above.

This approach should ensure significant confidence in the correct behavior of the IsValid() method. But it requires to replicate the same code for each sample password to test. You know that code replication is not a good practice. Fortunately, xUnit can help you with this issue with theories. A theory is a parametric unit test that allows you to represent a set of unit tests sharing the same structure. Theories allow you to implement what is called data-driven testing, which is a testing approach heavily based on input data variation.

The code above shows one single method, ValidatePassword(), in place of the two methods implemented before. This method has two parameters: password and expectedResult. And the application of the Arrange-Act-Assert pattern is based on these parameters.

In addition, you see a set of attributes decorating the method. The first attribute marks the ValidatePassword() method as a theory. The other InlineData attributes represent the data to pass to the method. Each attribute has a couple of values that are mapped to the method's parameters. So, basically, the first value of each InlineData attribute is a possible password, and the second value is the boolean value expected as a result of the IsValid() method.

3a8082e126