Introducing Allure in C# project

Posted on February 25, 2015

Background

About a year ago I worked as a QA automation engineer and, of course, had to deal with lots of test reports. In our company we had both unit- and UI-tests and used MS Test + Team Foundation Server + Microsoft Test Manager 2012 to run our automated tests, collect and view all the reports. But this solution had too many disadvantages for me:

And then I discovered Allure Framework - open-source tool which was developed by Yandex test team and it provides really great reports. You can browse their site, or watch sample report to convince yourself that it’s a very good. Allure consists of 2 important parts - adapter and generator. Adapter produces XML files in specific format for generator to generate the final report.

Unfortunately, by the time I found Allure, there was no way to use it in C# project, because there was no adapter for C#. So if I wanted to use Allure, I had to develop a little addon for testing framework (MS Test or NUnit) which would generate XML report.

But that was not a problem for me, I’m an open-source guy, so I quickly decided to develop adapter myself. Long story short, this is the result: core library for .NET and adapter for NUnit 2. Core library is reusable which makes possible to develop adapters for any .NET testing framework. For example, @someuser77 has developed adapter for MS Test.

Disclaimer: although core library (allure-csharp-commons) is well developed and tested, adapter itself (allure-nunit) may contain some bugs. In case of any troubles, you are welcome to raise an issue in bug tracker. Pull-requests are of course also welcomed.

Ok, so we are good to go.

Getting work done

End-to-end configuration is pretty tricky, but it can be simplified in the future by using plugins for TeamCity or Jenkins. The result will be *.bat-script you can use to run tests and generate report as a build step in you build server. I will describe this process for NUnit because I was personally using it. But it should be pretty much the same for MS Test.

So what exactly do we need?

  1. NUnit 2.6.3
  2. Latest release of allure-nunit
  3. JRE for allure-cli
  4. Latest release of allure-cli
  5. Web-server, for example Nginx
  6. Built *.dll assemblies with tests

First of all, we need to install NUnit, allure-cli, JRE, Nginx and set environmental variables, for example (replace these values with yours):

set ASSEMBLIES_DIR=C:\project\bin\debug
set NUNIT_HOME = C:\NUnit-2.6.3
set ALLURE_CLI_HOME = C:\allure-cli
set JAVA_HOME = C:\Program Files\Java\jre-1.7.0_75
set NGINX_HOME=C:\nginx-1.7.0

Next, we need to install nunit-adapter and configure it. Here are the steps to do it:

  1. Unpack allure-nunit binaries to %NUNIT_HOME%
  2. Addin will NOT be visible in Tools -> Addins.. because it’s built against .NET 4.0
  3. In %NUNIT_HOME%.xml specify absolute path to any folder (this folder will be created automatically) where XML files will be generated (for example <results-path>C:-results</results-path>)
  4. You can also specify in configuration whether you want to take screenshots after failed tests and whether you want to have test output to be written to attachments

After that we need to add new environmental variable OUTPUT_FOLDER pointing to the folder we specified in adapter’s config.xml:

set OUTPUT_DIR=C:\test-results\AllureResults

After we set all variables comes the easy part - running tests and generating report. I’ll just provide the resulting *.bat-script, it’s self-explanatory:

set ASSEMBLIES_DIR=C:\project\bin\debug
set NUNIT_HOME=C:\NUnit-2.6.3
set ALLURE_CLI_HOME=C:\allure-cli\allure-cli.jar
set JAVA_HOME=C:\Program Files\Java\jre-1.7.0_75
set OUTPUT_DIR=C:\test-results\AllureResults
set NGINX_HOME=C:\nginx-1.7.0

%NUNIT_HOME%\bin\nunit-console.exe %ASSEMBLIES_DIR%\YourAssembly.dll /framework=net-4.0
%JAVA_HOME%\bin\java -jar %ALLURE_CLI_HOME%\allure-cli.jar generate -v 1.4.0 %OUTPUT_DIR% -o %NGINX_HOME%\html\

You can run as much *.dll’s as you want or create an *.nuproj project containing all assemblies.

Now we need to start Nginx and thats all! If you’ve done it right, you will see the report at http://localhost:8080 (depends on Nginx configuration).

Note that this is only the basic configuration and it can of course be extended/modified.

Problems

There is one big issue with allure-csharp-commons, specifically, handling some custom attributes like Attachment, Step. We even have a pull-request for this issue, but it’s quite complicated and I personally have no time to dig into it.

If you have any questions, you are welcome to ask it in comments.

comments powered by Disqus