In .NET we have 3 dominant unit test frameworks:

• MS Test (v2)
• NUnit (3.x)
• xUnit (2.x)

First one comes with Visual Studio, two other requires additional components installation.

Initialize

First difference we will find in initialization approach. In MS Test there several initialization available (by attributes): AssemblyInitialize, ClassInitialize and TestInitialize. The disadvantage is that initialization method must match to its signature. Otherwise compiler will not build the solution and raise errors. A workaround is using constructor to init a test class. In xUnit, constructor is the only one way to initialize test class. NUnit contains SetUp attribute which can be set on a choosen public method. Also you may use TestFixture on a class but it is not must have.

It is worth to mention that only MSTest requires mark that class contains unit tests by TestClass attribute. Otherwise tests will not be discovered by the runner.

Tear down/cleanup

In MS Test TestCleanup, ClassCleanup and AssemblyCleanup attributes. In NUnit TearDown attributes. xUnit does not contain such attribute. The best way to achieve that is to implement IDisposable interface.

TestMethod/Test/Fact

To mark method as a test (single test case) you can use following attributes:

• TestMethod for MSTest

    [TestMethod]
    public void SimpleAddTest()
    {
        var result = this.math.SimpleAdd(1, 2);
        Assert.AreEqual(3, result);
    }

• Test for NUnit

    [Test]
    public void SimpleAddTest()
    {
        var result = this.math.SimpleAdd(1, 2);
        Assert.AreEqual(3, result);
    }

• Fact for xUnit

    [Fact]
    public void SimpleAddTest()
    {
        var result = this.math.SimpleAdd(1, 2);
        Assert.Equal(3, result);
    }

Supplying parameters to tests

MSTest

In MSTest you can use following attributes to pass parameters:

• DataRow – passing static data like int, string etc.

[TestMethod]
[DataRow(1, 2, 3)]
[DataRow(-1, 1, 0)]
[DataRow(-1, -2, -3)]
public void SimpleAddUsingAttributeTest(int i1, int i2, int expected)
{
     var result = this.math.SimpleAdd(i1, i2);
      Assert.AreEqual(expected, result);
}

• DynamicData – passing dynamic data like use data returned from property (default) or static method (see code samples).

[TestMethod]
[DynamicData(nameof(SimpleAddTestDataProperty), DynamicDataSourceType.Property)]
public void SimpleAddUsingDynamicDataProperty(int i1, int i2, int expected)
{
        var result = this.math.SimpleAdd(i1, i2);
        Assert.AreEqual(expected, result);
}

private static IEnumerable<object[]> SimpleAddTestDataProperty
{
	get
	{
		yield return new object[] { 1, 2, 3 };
		yield return new object[] { -1, 1, 0 };
		yield return new object[] { -1, -2, -3 };
	}
}

• Custom attribute – Implement own attribute class inheriting from Attribute, and implementing ITestDataSource interface.

public class AddClassDataAttribute : Attribute, ITestDataSource
{
    public IEnumerable<object[]> GetData(MethodInfo methodInfo)
    {
        yield return new object[] { 1, 2, 3 };
        yield return new object[] { -1, 1, 0 };
        yield return new object[] { -1, -2, -3 };
    }

    public string GetDisplayName(MethodInfo methodInfo, object[] data)
    {
        return nameof(AddClassDataAttribute);
    }
}

[TestMethod]
[AddClassData]
public void SimpleAddUsingCustomAttribute(int i1, int i2, int expected)
{
	var result = this.math.SimpleAdd(i1, i2);
	Assert.AreEqual(expected, result);
}

NUnit

In NUnit you can use following attributes to pass parameters:

• TestCase – passing static data like int, string etc

    [TestCase(1, 2, 3)]
    [TestCase(-1, 1, 0)]
    [TestCase(-1, -2, -3)]
    public void SimpleAddUsingAttributeTest(int i1, int i2, int expected)
    {
        var result = this.math.SimpleAdd(i1, i2);
        Assert.AreEqual(expected, result);
    }

• TestCaseSource – passing dynamic data from property

[TestCaseSource(nameof(SimpleAddTestDataProperty))]
public void SimpleAddUsingDynamicDataProperty(int i1, int i2, int expected)
{
     var result = this.math.SimpleAdd(i1, i2);
     Assert.AreEqual(expected, result);
}

private static IEnumerable<object[]> SimpleAddTestDataProperty
{
        get
        {
            yield return new object[] { 1, 2, 3 };
            yield return new object[] { -1, 1, 0 };
            yield return new object[] { -1, -2, -3 };
        }
}

• TestFixtureSource – allows to inject parameters into test class (via constructor). It is significant limitation comparing to other frameworks.

[TestFixtureSource(typeof(AddClassFixtureData), "AddParams")]
public class MathTestsWithTestFixtureSource
{
    private int i1;
    private int i2;
    private int expected;
    private Math math;

    public MathTestsWithTestFixtureSource(int i1, int i2, int expected)
    {
        this.i1 = i1;
        this.i2 = i2;
        this.expected = expected;
    }

    [SetUp]
    public void Setup()
    {
        this.math = new Math();
    }

    [Test]
    public void AddWithTextFixtureSource()
    {
        int result = math.SimpleAdd(this.i1, this.i2);
        Assert.AreEqual(expected, result);
    }
}

public class AddClassFixtureData
{
	public static IEnumerable AddParams
	{
		get
		{
			yield return new TestFixtureData(1, 2, 3);
			yield return new TestFixtureData(-1, 1, 0);
			yield return new TestFixtureData(-1, -2, -3);
		}
	}
}

xUnit

In xUnit you can use following parameters:

• InlineData – passing static data like int, string etc

    [Theory]
    [InlineData(1, 2, 3)]
    [InlineData(-1, 1, 0)]
    [InlineData(-1, -2, -3)]
    public void SimpleAddUsingAttributeTest(int i1, int i2, int expected)
    {
        var result = this.math.SimpleAdd(i1, i2);
        Assert.Equal(expected, result);
    }

• MemberData – passing dynamic data from method or property (important: it has to be public)

[Theory]
[MemberData(nameof(SimpleAddTestDataProperty))]
public void SimpleAddUsingDynamicDataProperty(int i1, int i2, int expected)
{
        var result = this.math.SimpleAdd(i1, i2);
        Assert.Equal(expected, result);
}

public static IEnumerable<object[]> SimpleAddTestDataProperty
{
    get
    {
        yield return new object[] { 1, 2, 3 };
        yield return new object[] { -1, 1, 0 };
        yield return new object[] { -1, -2, -3 };
    }
}

• ClassData – additional class which implements interface IEnumerable<object[]> and the method GetEnumerator returns test data

[Theory]
[ClassData(typeof(AddClassData))]
public void SimpleAddUsingDynamicClass(int i1, int i2, int expected)
{
	var result = this.math.SimpleAdd(i1, i2);
	Assert.Equal(expected, result);
}

public class AddClassData : IEnumerable<object[]>
{
    public IEnumerator<object[]> GetEnumerator()
    {
        yield return new object[] { 1, 2, 3 };
        yield return new object[] { -1, 1, 0 };
        yield return new object[] { -1, -2, -3 };
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

Using any of mentioned attributes requires Theory attribute instead of Fact on the test method.

Observation on differences

As you can note, on high level there is not so many differences. Only in very specific usage you may feel the difference. There is just one annoying thing which I observed today: when I run MS test with multiple input data, it does not show each as a separate cases while for other frameworks it is clearly visible.

Ignoring tests

In each framework we can ignore test using an attribute. For MSTest it is Ignore attribute, for NUnit Ignore(“reason”) and for xUnit it is Skip property in Fact attirbute (e.g. Fact(Skip=”reason”)).

Parallel execution

They are two “levels” of parallel execution. One is runner – it can run tests from multiple assemblies in parallel and the second one is on assembly level.

Paralell test execution is supported in each of frameworks. MSTest supports it from v2. It requires an attribute Parallelize(Workers=0, Scope=ExecutionScope) on assembly level. ExecutionScope indicates parallelization mode – or each test class is executed in parallel or all tests in a class are executed in parallel. If you do not want to parallel execute selected test, you can use DoNotParallelize attribute on selected test.

NUnit since version 3.0 supports parallel execution (requirement: .NET Standard 2.0, no support for older). Attribute Parallelizable may be use on class or assembly with defined scope (ParallelScope enum). More details for NUnit test parallel execution can be found here. It is also possible to disable parallel execution using NonParallelizable attribute.

In xUnit this feature is available since version 2.0. It uses a concept of test collection. By default each test class is a single test collection. If for any reason you would like to run several unit test classes as the same test collection, you may use an attribute Collection(“collectionName”). Also it is possible to disable parallel exeuction or set maximum number of threads.

Code coverage

In Visual Studio, to show code coverage, we need Enterprise edition or a plugin like NCover which costs 658 USD (price checked 22.04.2019 on official page) or JetBrains dotCover (a part of ReSharper Ultimate). From freeware, I found AxoCover (no support for .NET Core and Xamarin) and OpenCover. The first one I never used, the second one I used few times and I can recommend it as a stable solution with community support. For sure they are other options which I’m not familiar with. If you know any valuable library, worth to add here, please let me know in comment.

Assertions

All unit test framework contains different set of assertions. Common part is static Assert class with a set of methods (for detailed list please refer to the documentation).

A different approach is using Fluent assertions. I prefer this one as the easiest to maintain and understood. Also it does not depend on unit test framework which might be a plus in some cases.

With FluentAssertions the test method would look like the following one:

public void SimpleAddTestWithFluentAssertion()
{
    var result = this.math.SimpleAdd(1, 2);
    result.Should().Be(3);
}

Code samples

Unit test code samples can be found on gihtub. I encourage you to explore them to learn more on unit testing.

Typical problems

During my career few problems repeated across the projects:

• missing public modifier (tried to create test method as private)
• missing TestClass attribute
• not creating unit tests at all – that is very common and sad practice 🙁
• lack of discipline in creating and maintenance of unit tests

Conclusions

There is no perfect solution. You should match the solution to your needs. xUnit grew up significantly in last years. If we consider usability, MSTest is definetly behind NUnit and xUnit however the integration with Visual Studio is still the best (xUnit is just behind it and NUnit has the worst one in my opinion). Do not afraid to experiment to learn which framework matches to your solution.

Worth to read:

• xUnit unit test framework comparison
• xUnit dynamic parameters
• NUnit documentation