NBomber is a .NET framework for load testing of many kinds of services including web, queues, databases etc. Unfortunately it is not open source but there is free version. It is pretty well documented for F# and a bit worst for C# however it is pretty easy to take F# sample and write the same in C#.

Before we will go into usage, let’s start with definition of load tests:

Load testing is the process of putting demand on a software system and measuring its response. […] Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program concurrently.

https://en.wikipedia.org/wiki/Load_testing

Load tests: Test whether the app can handle a specified load of users for a certain scenario while still satisfying the response goal. The app is run under normal condition

https://docs.microsoft.com/en-US/aspnet/core/test/load-tests?view=aspnetcore-5.0

Code

In this article we will test simple REST API which returns random data. I create an empty API interface which comes as a sample with WeatherForecastController.

[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
    private static readonly string[] Summaries = new[]
    { "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching" };

    [HttpGet]
    public IEnumerable<WeatherForecast> Get()
    {
        var rng = new Random();
        return Enumerable.Range(1, 5).Select(index => new WeatherForecast
        {
            Date = DateTime.Now.AddDays(index),
            TemperatureC = rng.Next(-20, 55),
            Summary = Summaries[rng.Next(Summaries.Length)]
            }).ToArray();
        }
    }

Right now, I willcreate an empty console application, reference NBomber and NBomber.HTTP nuget. We will use also NBomber.PingPlugin for additional reporting.

Let’s consider what do we want to test. In this particular example, it will be single scenario with one step (step name must be unique in scope of scenario) – get weather forecast. We will be checking only if response is valid (HTTP status code OK – 200).

using System;
using System.Net.Http;
using NBomber.Contracts;
using NBomber.CSharp;
using NBomber.Plugins.Network.Ping;

namespace RandomAPILoadTests
{
    class Program
    {
        static void Main(string[] args)
        {
            var step = Step.Create("get weather forecast", async context =>
            {
                var httpClient = new HttpClient();
                httpClient.BaseAddress = new Uri("https://localhost:5001/");
                var result = await httpClient.GetAsync("WeatherForecast");
                return result.StatusCode == System.Net.HttpStatusCode.OK ?
                    Response.Ok() :
                    Response.Fail();
            });

            var scenario = ScenarioBuilder.CreateScenario("get weather", step)
                .WithLoadSimulations(LoadSimulation.NewInjectPerSec(_rate: 5, _during: TimeSpan.FromSeconds(20)))
                .WithWarmUpDuration(TimeSpan.FromSeconds(5));

            var pingConfig = PingPluginConfig.CreateDefault(new[] { "https://localhost:5001" });
            var pingPlugin = new PingPlugin(pingConfig);

            NBomberRunner
                .RegisterScenarios(scenario)
                .WithWorkerPlugins(pingPlugin)
                .Run();

        }
    }
}

I set test duration to 20 seconds with 5 seconds warm up and injection rate set to 5 requests per second. In the sample report below, we can see that all were processed succesfully, with RPS (max requests per seconds handled). As for such simple API, latency is quite high with significant standard deviation (StdDev) so we can guess that system will start to response with higher delay or become unavailable if we will increase load.

On the last report, we may note that RPS = 24 even though 25 was set in test load, which means that application maximum load is 24 requests per second. After that amount, response time increases significantly and some of requests may end up with timeouts.

I run also a test for 40 requests/sec. API was able to handle 5 requests per second. Performance degradation is significant. It’s good to know the application limits to properly optimize infrastructure including scaling and SLAs. Also it’s a good indicator when we should start optimizing our application (if we measure the time difference after code changes).

Multi step scenario

If you would need to pass the data from step to next one, you need to return the data in Response.OK(DATA). Then, in next step, it can be read from the context using GetPreviousStepResponse<T>() method.

Other solutions

The best known competitor is JMeter however it has a bit different usage concept and also is much more powerfull, and available much longer on the market.

In past, Azure DevOps had own tool however it was removed.