Masterarbeit/QuellCode/CentronERP/docs/guides/development/end-to-end-testing.md

# How to EndToEnd test

## General

EndToEnd tests are supposed to test everything from the `WebServiceBL` layer and downwards (see the diagram [here](GeneralStructure.md)).  
In other words, with EndToEnd tests we are testing the `WebServiceBL`, the `BL` and the `database`.

The project can be found at `/tests/Centron.Tests.EndToEnd` and all the tests are supposed to be in the `Tests` namespace.

For example, there are tests for the c-entron.NET `Receipt Area` are at `/tests/Centron.Tests.EndToEnd/Tests/Receipts/ReceiptTest.cs`.

The tests are executed as part of every Pull-Request.  
If one of the tests fails, the PR cannot be merged.

> Caution!  
> Some tests are a bit flaky sometimes. That means they fail even tho they should not.
> In these cases you can just let the PR get built again, and the tests should run successful.

We should try to make all tests NOT flaky. 

## Different kinds of tests

We can roughly categorize all tests into 2 categories:
1. [General purpose tests](#general-purpose-tests)
2. [Specialized tests](#specialized-tests)

They are only different in what they do, and it's not a hard categorization either.  
You still create both kinds of tests the same way, and they will look the same.  
They both inherit from `EndToEndTest.cs` and only differ in **what** and **how much** they do.

You can also inherit from `CentronTest.cs`, if you don't need database access for your test.  

And also, there is a `PerformanceTest.cs`, that measures the performance of some code, and fails if the performance-goal is not met.  
We can use this kind of test to make sure, that we don't have too harsh performance-regressions when we change some things.

For example, we have a test that makes sure big receipts can be saved in less than 5 seconds.  
If we add something to the SaveReceipt method, or change some other things around, and the receipt suddenly takes 8 seconds to save, then we made a mistake and have to correct it.

### General purpose tests
These tests are general purpose, doing every day things that the user does too.  
For example `creating a new customer`, `adding addresses` and `contact-persons` to it, `renaming contact-persons`, `removing addresses` and so on.

The general purpose tests are supposed to cover **really much code**.  
We want them to fail if we break a workflow that the user is doing every day.

### Specialized tests
The specialized tests are way smaller than the general purpose tests and are usually created as a response to a ticket that was fixed, or a new feature that was created.

For example `make sure that a setting works as expected`, or `make sure that a bug was fixed`.

These specialized tests are supposed to cover **really small amounts of code**, just enough to make sure the specific thing works as expected.  
We want them to fail if we break a specialized setting that some users might be using.

## How does it work?

1. Before each test is executed a database backup (`/Infrastructure/DatabaseBackup.zip`) is restored
2. The `DAOFactory` is configured to use that new database
3. Any new scripts (`IScriptMethod`) are executed
4. The `Test.Execute` method is executed
5. The database backup is removed

This workflow means that every test always has the same starting configuration.  
This fact is extremely useful because it allows us to execute these tests today, or in a month, or in a year, and we will still get the same result in the end.  
If you create new entries in a table for example, they will get the same I3D every time.

Actually, that's the most important thing about these tests, and why they even work at all: **The tests are repeatable!**

If they were not repeatable, they would fail without any real breaking changes, and then they are not useful to find the breaking changes.

## How to run the tests on your machine

Because we restore the database backup before each test, and the backup is on your local machine, the **SQL Server also needs to be on your local machine**.  
So if I work on my own machine `CS-ULM-DANHAE`, and I want to run some tests, I absolutely need a SQL Server installed on my own machine (`CS-ULM-DANHAE\SQL2017` in my case).  
**You cannot use a SQL Server on a different machine.**  

> You need at least an SQL Server 2017.

To **configure the connection to the SQL Server** you have to **change some small things** in the `/Infrastructure/Database.cs` class.

You will see three properties at the top: `Server`, `Username` and `Password`. They all look similar to this:
``` csharp
public static string Server
{
    get
    {
        var server = GetEnvironmentVariable("CENTRON_TESTS_DATABASE_SERVER");

        if (string.IsNullOrWhiteSpace(server) == false)
            return server;

        throw new Exception("You have to configure the SQL server for EndToEndTests");
        return "YOUR-MACHINE\\SQL2017"; //Enter your local SQL server here
    }
}
```

You can see, it at first tries to get the configured value from an **environment variable** - that is exactly how the code gets the connection when it's run by the **build server**.  
The build-server will set the environment variables to a SQL Server installed on the build-server itself (so it's locally to where the tests will be executed).

But if the environment variable is not set, it will **throw an Exception**. 

If you want to configure the EndToEnd tests to use your own local SQL Server, you just have to **comment out the exception**, and **change the return value of the property**.

```csharp
public static string Server
{
    get
    {
        var server = GetEnvironmentVariable("CENTRON_TESTS_DATABASE_SERVER");

        if (string.IsNullOrWhiteSpace(server) == false)
            return server;

        //throw new Exception("You have to configure the SQL server for EndToEndTests");
        return "CS-ULM-DANHAE\\SQL2017"; //Enter your local SQL server here
    }
}
```

If you do the same thing for the `Username` and `Password` properties, you are done and can execute the tests locally on your machine.

> :exclamation: Make sure to not commit these changes! :exclamation:

## How data is validated

When executing the test, you will probably create some data or some objects.  
For example, you will create a new customer, a new ticket, or a new receipt.  

The test has to validate, that these objects look exactly how expected.

This is done with a class called `CentronVerifier`. It has a `Verify` method that takes the data you want to verify, and a name for it.  
```csharp
this.Verifier.Verify("CreatedOffer", offer);
```

This will create a file called `CreatedOffer.actual.txt` that is basically the `offer` json serialized.  
It will then compare this file, to a `CreatedOffer.expected.txt` file.  
If they **differ**, **something has changed**, and the **test fails**.  
If they are **the same**, **everything still works** as expected, and the **test continues**.

The first time you execute a test, of course there will be no `*.expected.txt` file, and thus the test will fail.  
You're supposed to go ahead, and rename the `*.actual.txt` one to `*.expected.txt`.  
So when you execute the test again, a new `*.actual.txt` file will be created, and compared to the `*.expected.txt`.

This also means, the `*.expected.txt` should be added to source-control, but the `*.actual.txt` files should **NOT** be added to source-control.

You can also verify data from a table by using the `VerifyTable` method.
```csharp
this.Verifier.VerifyTable("TheCustomer", "dbo.Kunden", where: "I3D = 12345");
```
With that you have to provide a name again, the table that you want to verify, and a optional where filter.  
A `SELECT * FROM [Table] WHERE [Filter]` statement will be executed, and the result will be used as data for the `Verify` method.  
So everything with the `*.expected.txt` and `*.actual.txt` files works exactly the same as with the `Verify` method.

## Tips and other useful things

### Run any sql statement in your test

The base class for all tests `EndToEndTest.cs` provides a useful method `Sql` to run any sql statement in your test.  
You can use it for example if you easily wanna change a setting in the database before running your test. Or you want to change a configuration property on a customer or something else.

```csharp
this.Sql("UPDATE Stammdat SET Wert = 1 WHERE I3D = 1234");
```

### Debugging a test

You can also very easily debug a test by setting breakpoints in your code, and then `Right-Click` -> `Debug` the test in the `Test Explorer`.

![Debug a test](./.attachments/How-to-end-to-end-test/DebugTest.png)

### Diff view for *.expected.txt and *.actual.txt

If a test fails because something has changed between a `*.expected.txt` and a `*.actual.txt` file, you can see a diff between those two files.  
This allows you to more easily spot the differences.

It is pretty easy to do. If a test fails because of a `this.Verifier.Verify` call, you just have to [debug the test](#debugging-a-test).

Now if the `Verify` call fails the test will fail as usual, but at the same time a Diff-View will open in Visual Studio with both the `*.expected.txt` and the `*.actual.txt` in it.

You can then more easily decide whether the changes are fine, or whether you broke something.

## How to create a test

Lets say we want to write a [general purpose test](#general-purpose-tests) for some `Helpdesk` stuff - creating a new ticket for a customer, setting some values and saving it. 

> The real general purpose HelpdeskTest is of course supposed to cover even more of the helpdesk code.  
> For example forwarding it, adding documents and history entries, and closing the ticket.

### Step 1
Make sure to [configure the database connection](#How-to-run-the-tests-on-your-machine) so you can run them on your computer.

### Step 2
Create a new namespace/folder under the `/Tests` namespace: `WikiHelpdesk`.

### Step 3
In that namespace, create a new class `WikiHelpdeskTests.cs`.

### Step 4
Make that class inherit from `EndToEndTest`.

### Step 5
Implement the constructor for the class, and override the `Execute` method.  
Your class should look like this now:
```csharp
using Centron.Tests.EndToEnd.Infrastructure;
using Xunit.Abstractions;

namespace Centron.Tests.EndToEnd.Tests.WikiHelpdesks
{
    public class WikiHelpdeskTests : EndToEndTest
    {
        public WikiHelpdeskTests(ITestOutputHelper testOutputHelper) 
            : base(testOutputHelper)
        {
        }

        public override void Execute()
        {
        }
    }
}
```

> This is the basic setup you always have to do for a test.  
> Everything that follows now is actually implementing the test and [validating data](#how-data-is-validated).

### Step 6
Create a new method `CreateTicket`

```csharp
private HelpdeskDTO CreateTicket()
{
    using (var session = new BLSession())
    {
        //Get a LoggedInUser for the current user using the base class method
        var loggedInUser = this.GetLoggedInUser(11); // Default is 11 if no parameter is provided

        //Create a new ticket
        var ticket = session.GetBL<HelpdeskWebServiceBL>().GetNewTicketFromCustomer(loggedInUser.User, 10022);
        
        //Set some values
        ticket.ShortDescription = "Mein super wichtiges Ticket!";
        ticket.Description = "Mit einer nicht sehr ausführlichen Beschreibung.";

        //Save the helpdesk
        var saveHelpdeskResult = session.GetBL<HelpdeskWebServiceBL>().SaveHelpdesk(ticket, loggedInUser);
        //Verify the saveHelpdeskResult
        this.Verifier.Verify("SaveHelpdeskResult", saveHelpdeskResult);

        //Load the complete helpdesk object
        var savedTicket = session.GetBL<HelpdeskWebServiceBL>().GetHelpdeskByI3D(saveHelpdeskResult.Data, loggedInUser);
        //Verify the savedTicket
        this.Verifier.Verify("SavedTicket", savedTicket);

        return savedTicket.Data;
    }
}
```

I wanna especially highlight the two lines that use the `this.Verifier`:

```csharp
this.Verifier.Verify("SaveHelpdeskResult", saveHelpdeskResult);
```

and 

```csharp
this.Verifier.Verify("SavedTicket", savedTicket);
```

These calls make sure that the [data is actually validated](#how-data-is-validated).

### Step 7
Call the new `CreateTicket` method in the `Execute` method.
```csharp
public override void Execute()
{
    var ticket = this.CreateTicket();
}
```

You created your first test! :tada: 

### Step 8
As you know from the chapter on [how data is actually validated](#how-data-is-validated) the test will fail right now.  
There is no `SaveHelpdeskResult.expected.txt` and no `SavedTicket.expected.txt`.

So lets run the test by opening the `Test Explorer` in Visual Studio and `Right-Click` -> `Run` the test.

![Run the test](./.attachments/How-to-end-to-end-test/RunTest.png)

It will take a little while, but after like 40 seconds you should get a failed test result.

![Failed test run](./.attachments/How-to-end-to-end-test/FailedTest.png)

### Step 9
If you activate `Show All Files` in the solution explorer for the `Centron.Tests.EndToEnd` project, you should see the `SaveHelpdeskResult.actual.txt` file.

![SaveHelpdeskResult.actual.txt file in solution explorer](./.attachments/How-to-end-to-end-test/SaveHelpdeskResult-actual-txt.png)

Go ahead and rename it to `SaveHelpdeskResult.expected.txt`.  
It automatically gets picked up by the csproj file too.

![SaveHelpdeskResult.expected.txt file in solution explorer](./.attachments/How-to-end-to-end-test/SaveHelpdeskResult-expected-txt.png)

### Step 10
Run the test again!

...

As expected, it still fails because of the second `Verify` call.  
But you now know what to do.

### Step 11
A new file `SavedTicket.actual.txt` was created.  

Feel free to open it, and look at the data that is verified.  
Every single line in that JSON document has to stay exactly the same.  
Every property in that file is not allowed to change.  

![SavedTicket.actual.txt file in solution explorer](./.attachments/How-to-end-to-end-test/SavedTicket-actual-txt.png)

Rename it to `SavedTicket.expected.txt`.

![SavedTicket.expected.txt file in solution explorer](./.attachments/How-to-end-to-end-test/SavedTicket-expected-txt.png)

### Step 12
Run the test once again!

...

IT WORKS! :tada:

![Successful test](./.attachments/How-to-end-to-end-test/SuccessfulTest.png)

The test is finished!  
You can now add even more logic to the `Execute` method.