perf: batch DigestStore serialization during Maven/Gradle project import - #3839
Open
chagong wants to merge 4 commits into
Open
perf: batch DigestStore serialization during Maven/Gradle project import#3839chagong wants to merge 4 commits into
chagong wants to merge 4 commits into
Conversation
DigestStore.updateDigest(Path) serialized the entire digest map to disk on every changed build file, producing O(N) full-map serializations during a single import of an N-module workspace. Add DigestStore.updateDigests(Collection<Path>) that applies all changes under the existing lock and serializes at most once. updateDigest(Path) now delegates to it, preserving existing per-file behavior. MavenProjectImporter and GradleProjectImporter collect build-file paths during import and flush them with a single batched call. The per-file boolean gate in MavenBuildSupport, GradleBuildSupport and StandardProjectsManager is unchanged. Signed-off-by: Changyong Gong <chagon@microsoft.com>
chagong
force-pushed
the
perf/batch-digest-store
branch
from
July 7, 2026 04:48
23fd8aa to
9937572
Compare
…related to this change) The Integration Tests / Jenkins Build failures on the prior CI run were caused by MavenProjectMetadataFileTest#testMetadataFileSync and #testDeleteClasspath, which are known to fail intermittently independent of this change (see eclipse-jdtls#1443 and eclipse-jdtls#1251). Reproduced the same failures on eclipse-jdtls/eclipse.jdt.ls main branch CI (unrelated PR), confirming this is not caused by the DigestStore batching change in this PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
During project import,
DigestStore.updateDigest(Path)was called once per build file, and each call that detected a change serialized the entire digest map to disk viaObjectOutputStream. For a workspace with N modules this produced O(N) full-map serializations during a single import (and O(N²) cumulative bytes written as the map grows), all on the import thread.This change adds a batched API and uses it in the import paths so the digest map is serialized at most once per import, instead of once per changed build file.
Changes
DigestStore: addupdateDigests(Collection<Path>), which computes all requested digests, applies changes under the existingsynchronizedblock, and serializes the map only once if anything changed.updateDigest(Path)now delegates to it, so existing per-file callers keep identical behavior (including the boolean "changed" return).MavenProjectImporter: collect the POM paths of new/re-imported projects during classification and flush them with a singleupdateDigests(...)call after the loop, instead of callingupdateDigest(...)per project.GradleProjectImporter: collect thebuild.gradle(.kts)/settings.gradle(.kts)paths across all imported Gradle projects and seed them with a singleupdateDigests(...)call.The build-support / file-change paths (
MavenBuildSupport,GradleBuildSupport,StandardProjectsManager) intentionally keep using the per-fileupdateDigest(Path)boolean gate — their behavior is unchanged.Testing
MavenProjectImporterTest.testDigestStoreBatchUpdatecovering batch update, no-op when unchanged, persistence across a newDigestStoreinstance, and change detection after a file edit../mvnw verifyfororg.eclipse.jdt.ls.tests(MavenProjectImporterTest: 35 tests, 0 failures)..file-digestscontained the expected build-file entries.