# DataQualityService

## Purpose
The `DataQualityService` is an ASP.NET Core `BackgroundService` that runs periodically to:
- Clean up outdated data
- Fix data inconsistencies
- Update missing database values
- Validate and repair relationships between database entities
- Perform regular maintenance tasks on database records

## Implementation Details

### Core Structure
- Inherits from `BackgroundService` (Microsoft.Extensions.Hosting)
- Runs every hour on a continuous schedule while the application is active
- Each maintenance task is executed sequentially in the `ExecuteAsync` method

### Execution Pattern
- The service runs in an infinite loop until a cancellation is requested
- Tasks are executed with 1-hour intervals between full cycle executions
- Cancellation is checked between each task to allow for graceful shutdown

## Task Implementation Rules

When implementing tasks in the `DataQualityService`:

### 1. Session Management
- Each task must create its own `BLSession` instance within a `using` statement
- Sessions should be short-lived and disposed immediately after the task completes
```csharp
using (var session = new BLSession())
{
    // Task implementation here
}
```

### 2. Error Handling
- Every task must be wrapped in a try-catch block
- Exceptions should be logged with the specific task name in the error message
- Tasks should not crash the service; errors should be contained
```csharp
try
{
    // Task implementation
}
catch (Exception e) 
{
    Logger.Error(e, "Error while executing data quality service - [TASK NAME]");
}
```

### 3. Cancellation Checking
- After each task, check if cancellation has been requested
- Break the execution loop if cancellation is detected
```csharp
if (stoppingToken.IsCancellationRequested)
{
    break;
}
```

### 4. Business Logic Access
- Use the session's `GetBL<T>()` method to access the appropriate business logic class
- Call only methods that are designed for data quality operations
```csharp
session.GetBL<BusinessLogicClassName>().DataQualityMethodName();
```

## Adding New Data Quality Tasks

When adding a new task to the `DataQualityService`:

### 1. Task Placement
- Add the new task after existing tasks
- Follow the established pattern of try-catch-cancellation check

### 2. Business Logic Implementation
- Create a dedicated method in the appropriate BL class
- Prefix data quality specific methods with `DataQuality` (e.g., `DataQualityCleanupSecondaryStockArticles`)
- For specialized operations that need filters, create appropriate filter classes (e.g., `TicketPatternUpdateCustomerMappingsFilter`)

### 3. Performance Considerations
- Design tasks to be efficient and lightweight
- Avoid long-running operations that might block other tasks
- Consider implementing pagination for large datasets

### 4. Documentation
- Add a descriptive error message that clearly identifies the task
- Comment the task implementation if it performs complex operations

## Existing Task Reference

The `DataQualityService` currently handles these tasks:

1. **Ticket pattern customer mappings updates**
   - Updates customer mappings for ticket patterns
   - Uses `TicketPatternUpdateCustomerMappingsFilter`

2. **Directory checks**
   - Executes directory validation and checking
   - Uses `DirectoryCheckBL.ExecuteDirectoryCheck()`

3. **Cleanup of Centron notifications**
   - Removes old or expired notifications
   - Uses `CentronNotificationsBL.CleanupCentronNotifications()`

4. **Checklist customer mappings updates**
   - Updates customer mappings for checklists
   - Uses `CentronChecklistUpdateCustomerMappingsFilter`

5. **Profiler entries reorganization**
   - Reorganizes and optimizes profiler entries
   - Uses `ProfilerBL.ReorganizeProfilerEntries()`

6. **Todo-list missing Account I3D filling**
   - Fills missing Account I3Ds in Todo items
   - Uses `ToDoBL.DataQualityFillAccountI3D()`

7. **AccountTypeToAccounts table repair**
   - Checks and repairs AccountTypeToAccounts relationships
   - Uses `AccountBL.CheckAndRepairAccountTypeToAccountsTable()`

8. **HelpdeskTimer properties updates**
   - Updates missing properties in HelpdeskTimer records
   - Uses `HelpdeskTimerBL.DataQualityUpdateMissingHelpdeskTimerProperties()`

9. **Secondary stock articles cleanup**
   - Cleans up secondary stock article records
   - Uses `SecondStockArticleBL.DataQualityCleanupSecondaryStockArticles()`

## Best Practices

### 1. Task Independence
- Each task should be independent of other tasks
- Failure in one task should not affect subsequent tasks

### 2. Resource Utilization
- Consider the server load when implementing new tasks
- Use database indexes for heavy operations
- Run operations during off-peak hours if possible

### 3. Logging
- Log the start and completion of tasks at the Debug or Info level for monitoring
- Include sufficient context in error messages for troubleshooting

### 4. Testing
- Test new tasks thoroughly in a development environment
- Verify that new tasks fix the intended issues without side effects

## Example Implementation

Adding a new data quality task:

```csharp
if (stoppingToken.IsCancellationRequested)
{
    break;
}

try
{
    using (var session = new BLSession())
    {
        session.GetBL<YourBusinessLogic>().DataQualityYourNewTask();
    }
}
catch (Exception e)
{
    Logger.Error(e, "Error while executing data quality service - Your new task description");
}
```