Skip to content

optimize countUsersByQuery when not querying for specific permissions#3429

Open
thomasgl-orange wants to merge 1 commit intoSonarSource:masterfrom
thomasgl-orange:fix-slow-countUsersByQuery-with-no-permission
Open

optimize countUsersByQuery when not querying for specific permissions#3429
thomasgl-orange wants to merge 1 commit intoSonarSource:masterfrom
thomasgl-orange:fix-slow-countUsersByQuery-with-no-permission

Conversation

@thomasgl-orange
Copy link
Copy Markdown
Contributor

See support ticket #69600.

This PR is an optimisation (800ms down to a handful of ms) for a slow SQL query (countUsersByQuery), when counting users without any additional filtering criteria (no specific permission).

The original SQL query, as spotted in our pg_stat_statements:

select count(distinct(u.uuid))
from users u
left join user_roles ur on ur.user_uuid = u.uuid
left join (
(select
p.uuid as uuid, p.kee as kee, p.name as name, p.private as isPrivate, p.description as description, p.qualifier as qualifier, 
$1 as authUuid  from projects p)
UNION
(select
p.uuid as uuid, p.kee as kee, p.name as name, p.private as isPrivate, p.description as description,  case when p.parent_uuid is null then $2 else $3 end as qualifier,
case when p.root_uuid != p.uuid then p.root_uuid else $4 end as authUuid
from portfolios p where p.parent_uuid is null)
) entity on ur.entity_uuid = entity.uuid
WHERE  u.active = $5;

A closer look shows that parameters $1, $2, $3 and $4 only affect values of some sub-queries results which are not actually used in the parent query, so it can first be simplified like this:

select count(distinct(u.uuid))
from users u
left join user_roles ur on ur.user_uuid = u.uuid
left join (
  (select p.uuid as uuid from projects p)
  UNION
  (select p.uuid as uuid from portfolios p where p.parent_uuid is null)
) entity on ur.entity_uuid = entity.uuid
WHERE  u.active = $1;

Furthermore, because the selected result is only a count(distinct(users.uuid)), and the main query where condition is only about this users table, whatever is left-outer joined don't actually influence the final result!

So an equivalent much simpler query goes like this:

select count(distinct(u.uuid))
from users u
WHERE  u.active = $1;

In terms of exec plan, the original query is indeed a bit crazy for what it actually does: https://explain.dalibo.com/plan/5245ac7bg4a93b9g

The proposed change get rid of these unnecessary left outer join when NOT query.withAtLeastOnePermission(), meaning when the left-outer joined tables can't contribute anything which would be used for filtering, and thus which could influence the count(distinct) result. (A more verbose version of this explanation is in the referenced support ticket.)

This SQL query is involved when processing GET api/permissions/users?projectKey=... requests (with no additional permission filter). It's something which happens from the project permissions page in the SonarQube UI, where we've indeed observed a slow (800ms) reply from the server in our browser dev tools.

Furthermore, but that's a very specific use-case, we happen to periodically execute a script which exports users permissions for all our SonarQube projects, as part of our monitoring/auditing tools for SQ. That's thousands of executions of this query every day, and the main reason we've seen it at a top spot in our pg_stat_statements.


  • Please explain your motives to contribute this change: what problem you are trying to fix, what improvement you are trying to make
  • Use the following formatting style: SonarSource/sonar-developer-toolset
  • Provide a unit test for any code you changed
    • the query is covered by various tests already, and this optimisation does not change its results, so no new unit tests
    • I've checked ./gradlew server:sonar-db-dao:test is still passing
  • If there is a JIRA ticket available, please make your commits and pull request start with the ticket ID (SONAR-XXXX)
    • I can't create a SONAR-XXXX ticket in Jira; if you create one, let me know and I will amend the commit title (or feel free to amend the commit yourself)

@sonar-review-alpha
Copy link
Copy Markdown

sonar-review-alpha Bot commented Apr 29, 2026

Summary

Conditional SQL optimization for counting users without permission filters.

The countUsersByQuery query is optimized by skipping expensive LEFT JOINs to user_roles and entity tables when no permission filtering is applied. A single-line MyBatis conditional: when withAtLeastOnePermission() is false, the query simplifies from joining across 4 tables to just counting distinct active users. The performance improvement is dramatic (800ms → single-digit ms) because the joins have no effect on the count result when not filtering by permissions. The change is low-risk and well-scoped to one file.

What reviewers should know

Review focus:

  • The change is in UserPermissionMapper.xml — a simple MyBatis conditional (<choose>/<when>/<otherwise>)
  • When permission filtering is disabled, only the simplified from users u path executes; the full join-heavy query still runs when needed
  • The logic is safe: the left joins only contribute when filtering by user roles/permissions. Without that filter, the joins produce rows that don't affect the final count(distinct)

Testing:

  • No new tests added (intentional) — the existing test suite already covers this query path across various permission scenarios
  • Author verified ./gradlew server:sonar-db-dao:test passes

  • Generate Walkthrough
  • Generate Diagram

🗣️ Give feedback

sonar-review-alpha[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant