In dashboard speedup pt 1, fetchEntityIssueCounts was identified as a performance bottleneck. This was resolved by switching to the endpoint_dataset_issue_type_summary table in the performance database. Initially, this appeared to work correctly with issues displaying properly on the dashboard.
However, during testing, it was discovered that Salford showed an incorrect issue count for the trees dataset. Further investigation revealed the root cause:
- A separate function, fetchEntryIssueCounts, runs a slow LEFT JOIN SQL operation on the issue table
- The results from fetchEntryIssueCounts are merged with fetchEntityIssueCounts into a single array
- Another function counts the number of rows for each unique dataset to calculate the task numbers displayed on the dashboard
The switch to the performance database for fetchEntityIssueCounts inadvertently caused issue count duplication whenever fetchEntryIssueCounts returns results. This bug went undetected because the issue table returns no results for most LPAs. Why Salford is the only LPA returning data from the issue table requires further investigation.
Solution: Remove fetchEntryIssueCounts from the middleware chain and fetchEntryCounts (Separate function with no use). This change:
- Further improves dashboard performance
- Eliminates the duplication bug
- Provides clearer insight into the middleware chain's operations
Future improvement: The internal pipeline could be used instead of the performance database for even better performance.
In dashboard speedup pt 1, fetchEntityIssueCounts was identified as a performance bottleneck. This was resolved by switching to the endpoint_dataset_issue_type_summary table in the performance database. Initially, this appeared to work correctly with issues displaying properly on the dashboard.
However, during testing, it was discovered that Salford showed an incorrect issue count for the trees dataset. Further investigation revealed the root cause:
The switch to the performance database for fetchEntityIssueCounts inadvertently caused issue count duplication whenever fetchEntryIssueCounts returns results. This bug went undetected because the issue table returns no results for most LPAs. Why Salford is the only LPA returning data from the issue table requires further investigation.
Solution: Remove fetchEntryIssueCounts from the middleware chain and fetchEntryCounts (Separate function with no use). This change:
Future improvement: The internal pipeline could be used instead of the performance database for even better performance.