feat: Implement granular compile-time caching for LightTypeTag in Scala 3#569
feat: Implement granular compile-time caching for LightTypeTag in Scala 3#569AriajSarkar wants to merge 12 commits into
Conversation
Implements three-level compile-time caching for both Scala versions: - LTT cache: Caches complete LightTypeTag results - FullDB cache: Caches inheritance hierarchy (basesDb) - InheritanceDB cache: Caches unapplied type inheritance data Scala 3 changes: - Inspect.scala: LTT-level cache with structural key generation - FullDbInspector.scala: FullDB-level cache - InheritanceDbInspector.scala: InheritanceDB-level cache Scala 2 changes: - LightTypeTagImpl.scala: All three cache levels with structural key Cache implementation details: - Uses ConcurrentHashMap with SoftReference for memory efficiency - Structural key normalization ensures type equivalence across contexts - Controlled via -Dizumi.reflect.rtti.cache.compile property (default: true) - Cache statistics available via getCacheStats/resetCacheStats methods New configuration properties in DebugProperties.scala: - izumi.reflect.rtti.cache.macro - izumi.reflect.rtti.cache.ltt - izumi.reflect.rtti.cache.db Added comprehensive tests for both Scala versions: - CacheStressTest.scala: Stress tests with repeated Tag materializations - CompileTimeCacheBenchmarkTest.scala: Correctness verification tests Benchmark results: - Scala 3: ~1% clean build improvement, ~66% warm build improvement - Scala 2.13: ~9% clean build improvement, ~64% warm build improvement
Implements three-level compile-time macro caching: - LTT cache: Caches final LightTypeTag results - FullDB cache: Caches full database computations - InheritanceDB cache: Caches inheritance database computations Key features: - Uses ConcurrentHashMap with SoftReference wrappers for memory efficiency - Structural cache keys using type representation + base classes with source position - Source position (file path + byte offset) ensures unique keys for local types - Configurable via izumi.reflect.rtti.cache.compile property (default: true) Benchmark results (clean compilation): - Scala 3.3.6: 49.07s -> 47.95s (2.3% faster) - Scala 2.13.14: 43.90s -> 41.02s (6.6% faster) Note: issue350/ folder contains benchmark documentation for review only
…erties Cleanup: - Removed cacheHits/cacheMisses counters and getCacheStats/resetCacheStats methods - Removed unused config properties (cache.macro, cache.ltt, cache.db) from DebugProperties - Simplified cache lookup code using getOrElse pattern The cache stats were only useful for debugging and add unnecessary overhead. Only the essential izumi.reflect.rtti.cache.compile property is needed.
Overall, 2..6% improvement doesn't look significant.
|
- Add individual cache flags (ltt, db.full, db.inheritance, macro) - Remove unused PickleImpl import from Inspect.scala - Add benchmark scripts for reproducible testing - Update .gitignore for generated stress test files
Updated ImplementationAddressed all feedback from the previous review. Here are the latest benchmark results: Benchmark Results
Environment: Windows 11, i5-12500H, JDK 17, sbt 1.11.2 Key Implementation NotesScala 3 uses String-based cache keys because Scala 2 uses How to Reproduce# Generate stress test (1755 Tag materializations)
.\scripts\generate-stress-test.ps1
# Run benchmark
.\scripts\benchmark-compile-time.ps1Request@pshirshov @neko-kai Could you please test on your machines as well? I'm curious about performance differences across CPU architectures (especially on macOS/Linux). The numbers above are from Windows on Intel 12th gen hybrid architecture. |
|
@AriajSarkar Re: path dependency of Quotes, that's not an issue, much less a "fundamental" one, just use a type projection e.g. |
|
Thank you for the insight Sir @neko-kai! I wasn't aware type projections could work here i'll look into changing the cache key from String to Quotes#reflectModule#TypeRepr and update the PR. Would using TypeRepr directly as the key provide better performance than the current string-based approach? |
|
It should because you won't need to serialize. |
|
I may have overthought this earlier! 😅 Made the change to use Quotes#reflectModule#TypeRepr as suggested and got these results:
Thanks for the tip - learned something new about type projections! |
|
Ok, could you provide results for individual cache types? |
|
Scala 3.3.6:
Scala 2.13.14: ALL ON 41.2s ( -4.8%) Scala 2 shows slight overhead - should we disable caching by default for Scala 2, or is this expected for a small test suite? |
- Use Quotes#reflectModule#TypeRepr as cache key instead of String - Remove makeTypeKey function (no longer needed) - Clean up comments - Remove benchmark files and stress tests (not needed for PR)
|
Scala 3 shows consistent ~12% improvement across cache types. Scala 2 shows slight overhead on this test suite. Again test - ALL ON 55.5s ( -1.1%) Note: The previous String-based key approach showed improvement for both Scala versions (~15% Scala 3, ~22% Scala 2). The new TypeRepr projection approach is cleaner but performs better only for Scala 3, with slight overhead on Scala 2 for this test suite. |
|
@AriajSarkar Re: benchmarks, something is wrong with your setup as the results vary a lot between your posts with seemingly no changes. You should probably first make sure that you have Turbo Boost or similar dynamic frequency control disabled on your PC and also run way more iterations per test (1000+) |
|
It might be difficult to run multiple iterations in the regular sense, because the nature of the benchmark assumes invoking the compiler and the startup overhead might be significant. I would suggest to generate sufficiently large test case file with enough duplication. The variations in the results (+22% vs -3.6% for the same Scala 2 test) suggest that something is wrong with your environment or methodology. I would suggest:
|
|
Thank you for the detailed feedback! I'm looking into the tree-level caching approach from #2331. For the benchmark variance - I'm running on a laptop (charger connected(although it always plugged)). Will try:
Will report back with improved methodology. Edit: Scala 3: Benchmarks (3000 tags, -Xmx4G, warmup):
~2% improvement. Modest gains in high-heap env, but architecture matches your request. |
- Scala 2: Cache c.typecheck results in a synchronized WeakHashMap to avoid re-typing. - Scala 3: Cache underlying Terms with path-independent types (Term-cast technique) to bypass ScopeException and avoid re-typing. - Verified with benchmarks showing 1-2% compile-time improvement in high-heap environments.
|
As @pshirshov mentioned above, you have to construct a specific test file that would provoke many cache hits. e.g. Compilation time of a file with thousands of lines of derivations for a tag for the same custom case class ought to be significantly improved by caching. Either previous Scala 2 cache or your new caches or a tree-based cache or at the very least just avoiding macro call by storing the derived tag in implicit val: |
|
Sir @neko-kai, ran the benchmark you requested - comparing macro calls vs implicit val baseline for a custom case class. Simple Case Class - 3000 Tag callsType:
Cache doesn't help for simple types - both WITH and WITHOUT cache take same time. Why?Verified cache IS hitting (added debug output, saw cache hits). The problem is the tag computation for simple case classes is already fast - the overhead comes from Scala 3's Also tested complex types - 500 Tag callsTried with deeply nested generics (inheritance hierarchies, multiple type params, type lambdas):
~10% improvement here. Complex types have expensive LightTypeTag computation so caching helps. But that's not what you asked for. Commands I used# WITH CACHE
$env:SBT_OPTS = "-Xmx4G"
sbt clean
sbt "++3.3.6" "izumi-reflectJVM/Test/compile"
# WITHOUT CACHE
$env:SBT_OPTS = "-Xmx4G -Dizumi.reflect.rtti.cache.compile=false"
sbt clean
sbt "++3.3.6" "izumi-reflectJVM/Test/compile"Test file structurecase class CustomCaseClass(id: Int, name: String, value: Double)
// BASELINE - no macro calls after initial derivation
object BaselineWithImplicitVal {
implicit val cachedTag: Tag[CustomCaseClass] = Tag[CustomCaseClass]
val t1 = implicitly[Tag[CustomCaseClass]]
val t2 = implicitly[Tag[CustomCaseClass]]
// ... 3000 total
}
// MACRO CALLS - each is a macro expansion
object MacroCallsWithCache {
val t1 = Tag[CustomCaseClass]
val t2 = Tag[CustomCaseClass]
// ... 3000 total
}Bottom lineFor simple case classes, macro calls can't approach the implicit val baseline because the inline expansion overhead dominates, not the tag computation. Cache works and hits correctly, but there's nothing expensive to cache. Note: Benchmarks run on Windows 11. Absolute numbers may vary but the relative comparison (WITH vs WITHOUT on same run) should be valid. Happy to have someone verify on a dedicated benchmark setup. Let me know what you think or if you want me to try something different! |
|
@AriajSarkar Could you just commit reproducible benchmark files into the branch, for manual checking? |
- Remove version-specific CacheStressTest2.scala and CacheStressTest3.scala - Create izumi-reflect-stress-baseline module (implicit val resolution benchmark) - Create izumi-reflect-stress-macrocalls module (Tag macro cache benchmark) - Both modules are cross-version and unpublished (benchmark-only) 🤖 Generated with Claude Code
|
@AriajSarkar Your benchmark was flawed. I committed a fix just now. I'm getting 3 second compile time with implicit val ( |
|
@neko-kai Thanks Sir, for fixing the benchmark setup! Good to see the cache actually provides ~23% improvement 🎉 So the implementation was right, just my benchmarking methodology needed work 😅 , having both objects in the same file meant compiling everything twice instead of testing them separately.
Anything else needed from my side, or is this ready for final review?
|
|
@AriajSarkar No, it's not ready - there are multiple caching strategies present, but we don't know which of them actually work (hit), nor which are more effective than, or subsume, others. Also, manually running sbt compile once or twice, especially, in my case, on a a laptop, doesn't give us an accurate performance picture - I suggest looking at how compiler benchmarks for https://github.com/scala/scala3 are made and to see if something can be lifted from there. Also, I would suggest to read https://bcantrill.dtrace.org/2025/12/05/your-intellectual-fly-is-open/ and avoid writing replies directly with LLMs or pasting their output raw, without delineating from your own writing - it's jarring to read and not as helpful as the verbosity of their output would suggest. In fact I just skip reading everything that looks pasted from an LLM because it's too much work to figure out whether I'm reading something that's supposed to make sense or a nonsensical hallucination. |
the term cache (tree-level, hit ratio is 99.9985%) handles all reuse. @neko-kai Want me to push the instrumention? so you can run it yourself and verify? cache disabled run: stats: Benchmark results :
re: proper benchmarks - Scala/Scala3 uses JMH via the bench/ directory. That’s the correct approach for statistically valid measurements. I can try set this up to validate the cache behavior properly. |
|
The other caches should hit in other scenarios - e.g. LTT cache should hit when CustomCaseClass is inspected as a type parameter of another type. DB caches should hit when it's a base type of another type; they ought to be relevant for inheritors of Scala collection types, which have very large inheritance hierarchies. |
|
Tested type parameter and collection inheritance scenarios. DB caches don't hit because they're keyed by full type (List[Int]), not component types. When inspecting List[Int] vs Vector[Int], the shared Iterable ancestry isn't a separate cache lookup - it's computed inside buildFullDb. termCache intercepts first for any same-type reuse. The other caches only help if we had different TypeReprs producing identical LightTypeTags, which doesn't happen in practice. you can verify with: Re: JMH - still needed, or is termcache data sufficient? |
|
@AriajSarkar val t15_1 = implicitly[Tag[Base]]
val t15_2 = implicitly[Tag[Level1]]
val t15_3 = implicitly[Tag[Level2]]
val t15_4 = implicitly[Tag[Level3]]
val t15_5 = implicitly[Tag[Leaf1]]
val t15_6 = implicitly[Tag[Leaf2]]
val t15_7 = implicitly[Tag[Leaf3]]
val t15_8 = implicitly[Tag[Leaf4]]
val t15_9 = implicitly[Tag[Service]]
val t15_10 = implicitly[Tag[DatabaseService]]
val t15_11 = implicitly[Tag[CacheService]]
val t15_12 = implicitly[Tag[ApiService]]
val t15_13 = implicitly[Tag[PostgresService]]
val t15_14 = implicitly[Tag[RedisService]]
val t15_15 = implicitly[Tag[RestApiService]]
val t15_16 = implicitly[Tag[List[Base]]]
val t15_17 = implicitly[Tag[List[Level1]]]
val t15_18 = implicitly[Tag[List[Level2]]]
val t15_19 = implicitly[Tag[List[Level3]]]
val t15_20 = implicitly[Tag[List[Leaf1]]]
val t15_21 = implicitly[Tag[List[Leaf2]]]
val t15_22 = implicitly[Tag[List[Service]]]
val t15_23 = implicitly[Tag[Option[Base]]]
val t15_24 = implicitly[Tag[Option[Leaf1]]]
val t15_25 = implicitly[Tag[Option[Service]]]
val t15_26 = implicitly[Tag[Map[String, Base]]]
val t15_27 = implicitly[Tag[Map[String, Service]]]
|
|
yea you're right - the DB caches weren't checking for cached component types during recursive processing. now checking the cache inside Results - |
…level entry point
|
@neko-kai Three caches working, two are dead:
lttCache never hits because termCache uses the same key and is checked first. serializedCache never hits because it uses IdentityHashMap, each fresh LightTypeTag is a new object so identity comparison never matches. should I remove lttCache and serializedCache (and their flags), keeping only the 3 that work? |
3 participants
/claim #350
Implemented
izumi.reflect.rtti.cache.compileBenchmark Results
Not yet done
Submitting for early review to confirm direction before additional work.
Fixes #350