Masterarbeit/QuellCode/CentronERP/docs/Background Service/DataQualityService.md

DataQualityService

Purpose

The DataQualityService is an ASP.NET Core BackgroundService that runs periodically to:

• Clean up outdated data
• Fix data inconsistencies
• Update missing database values
• Validate and repair relationships between database entities
• Perform regular maintenance tasks on database records

Implementation Details

Core Structure

• Inherits from BackgroundService (Microsoft.Extensions.Hosting)
• Runs every hour on a continuous schedule while the application is active
• Each maintenance task is executed sequentially in the ExecuteAsync method

Execution Pattern

• The service runs in an infinite loop until a cancellation is requested
• Tasks are executed with 1-hour intervals between full cycle executions
• Cancellation is checked between each task to allow for graceful shutdown

Task Implementation Rules

When implementing tasks in the DataQualityService:

1. Session Management

• Each task must create its own BLSession instance within a using statement
• Sessions should be short-lived and disposed immediately after the task completes

using (var session = new BLSession())
{
    // Task implementation here
}

2. Error Handling

• Every task must be wrapped in a try-catch block
• Exceptions should be logged with the specific task name in the error message
• Tasks should not crash the service; errors should be contained

try
{
    // Task implementation
}
catch (Exception e) 
{
    Logger.Error(e, "Error while executing data quality service - [TASK NAME]");
}

3. Cancellation Checking

• After each task, check if cancellation has been requested
• Break the execution loop if cancellation is detected

if (stoppingToken.IsCancellationRequested)
{
    break;
}

4. Business Logic Access

• Use the session's GetBL<T>() method to access the appropriate business logic class
• Call only methods that are designed for data quality operations

session.GetBL<BusinessLogicClassName>().DataQualityMethodName();

Adding New Data Quality Tasks

When adding a new task to the DataQualityService:

1. Task Placement

• Add the new task after existing tasks
• Follow the established pattern of try-catch-cancellation check

2. Business Logic Implementation

• Create a dedicated method in the appropriate BL class
• Prefix data quality specific methods with DataQuality (e.g., DataQualityCleanupSecondaryStockArticles)
• For specialized operations that need filters, create appropriate filter classes (e.g., TicketPatternUpdateCustomerMappingsFilter)

3. Performance Considerations

• Design tasks to be efficient and lightweight
• Avoid long-running operations that might block other tasks
• Consider implementing pagination for large datasets

4. Documentation

• Add a descriptive error message that clearly identifies the task
• Comment the task implementation if it performs complex operations

Existing Task Reference

The DataQualityService currently handles these tasks:

1. Ticket pattern customer mappings updates

   • Updates customer mappings for ticket patterns
   • Uses TicketPatternUpdateCustomerMappingsFilter

2. Directory checks

   • Executes directory validation and checking
   • Uses DirectoryCheckBL.ExecuteDirectoryCheck()

3. Cleanup of Centron notifications

   • Removes old or expired notifications
   • Uses CentronNotificationsBL.CleanupCentronNotifications()

4. Checklist customer mappings updates

   • Updates customer mappings for checklists
   • Uses CentronChecklistUpdateCustomerMappingsFilter

5. Profiler entries reorganization

   • Reorganizes and optimizes profiler entries
   • Uses ProfilerBL.ReorganizeProfilerEntries()

6. Todo-list missing Account I3D filling

   • Fills missing Account I3Ds in Todo items
   • Uses ToDoBL.DataQualityFillAccountI3D()

7. AccountTypeToAccounts table repair

   • Checks and repairs AccountTypeToAccounts relationships
   • Uses AccountBL.CheckAndRepairAccountTypeToAccountsTable()

8. HelpdeskTimer properties updates

   • Updates missing properties in HelpdeskTimer records
   • Uses HelpdeskTimerBL.DataQualityUpdateMissingHelpdeskTimerProperties()

9. Secondary stock articles cleanup

   • Cleans up secondary stock article records
   • Uses SecondStockArticleBL.DataQualityCleanupSecondaryStockArticles()

Best Practices

1. Task Independence

• Each task should be independent of other tasks
• Failure in one task should not affect subsequent tasks

2. Resource Utilization

• Consider the server load when implementing new tasks
• Use database indexes for heavy operations
• Run operations during off-peak hours if possible

3. Logging

• Log the start and completion of tasks at the Debug or Info level for monitoring
• Include sufficient context in error messages for troubleshooting

4. Testing

• Test new tasks thoroughly in a development environment
• Verify that new tasks fix the intended issues without side effects

Example Implementation

Adding a new data quality task:

if (stoppingToken.IsCancellationRequested)
{
    break;
}

try
{
    using (var session = new BLSession())
    {
        session.GetBL<YourBusinessLogic>().DataQualityYourNewTask();
    }
}
catch (Exception e)
{
    Logger.Error(e, "Error while executing data quality service - Your new task description");
}