Changed auto generated index name by shubhamsrkdev · Pull Request #503 · mikemccand/luceneutil

shubhamsrkdev · 2026-01-06T17:39:13Z

The auto generated index name though informative is very long leading to errors like:

Could not start recording, not able to write to file /local/home/username/lucenebenchy/logs/baseline_vs_patch/2026.01.06.11.14.39/bench-index-baseline_vs_patch-wikimediumall.lucene_candidate.c
andidate.facets.taxonomy:Date.taxonomy:Month.taxonomy:DayOfYear.sortedset:Date.sortedset:Month.sortedset:DayOfYear.taxonomy:RandomLabel.sortedset:RandomLabel.Lucene90.Lucene104.dvfields.nd33.3326M.jfr. /local/home/username/lucenebenchy/logs/baseline_vs_patch/2026.01.06.11.14.39/bench-index-baseline_vs_patch-wikimediumall.lucene_candidate.candidate.facets.taxonomy:Date.taxonomy:Month.taxonomy:DayOfYear.sortedset:Date.sortedset:Month.sorted
set:DayOfYear.taxonomy:RandomLabel.sortedset:RandomLabel.Lucene90.Lucene104.dvfields.nd33.3326M.jfr: File name too long 
Error occurred during initialization of VM

This PR reduces the file name size and also added a metadata file to verify if the index is not reduced.

Fixes #198

mikemccand

Thank you for tackling this! It is super annoying when it strikes!

As a short-term workaround, you can tell your local Competition (or Index instance maybe) which index name you want to use, overriding this overly-verbose default.

mikemccand · 2026-01-07T17:44:21Z

Looks like you need a make tidy maybe? Or possibly make ruff-fix?

github-actions · 2026-02-07T00:14:24Z

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

mikemccand

Looks great, super close!

mikemccand · 2026-05-27T11:35:17Z

  JAVA_COMMAND = "%s -server -Xms2g -Xmx2g --add-modules jdk.incubator.vector -XX:+HeapDumpOnOutOfMemoryError -XX:+UseParallelGC" % JAVA_EXE
 else:
-  print("use java command %s" % JAVA_COMMAND)  # pyright: ignore[reportUnboundVariable] # TODO: fix how variables are managed here
+  print("use java command %s" % JAVA_COMMAND)  # pyright: ignore[reportUndefinedVariable] # TODO: fix how variables are managed here


LOL this is so painful -- it fails on GitHub actions this way, but passes for our dev boxes. I go back and forth which is the lesser evil. And I wish we could explain why it's not consistent b/w our dev boxes and GitHub actions... we fix the version of basedpyright=1.29.0 in requirements.txt.

Yeah, it was failing in the build on Github, shall we keep it like this?

mikemccand · 2026-05-27T11:36:42Z

 # /root/jdk1.6.0_31/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC

 /opt/zing/zingLX-jdk1.6.0_31-5.2.0.0-18-x86_64/bin/java -verbose:gc -Xms40G -Xmx40G -cp .:$LUCENE_HOME/build/core/classes/java:$LUCENE_HOME/build/highlighter/classes/java:$LUCENE_HOME/build/test-framework/classes/java:$LUCENE_HOME/build/queryparser/classes/java:$LUCENE_HOME/build/suggest/classes/java:$LUCENE_HOME/build/analysis/common/classes/java:$LUCENE_HOME/build/grouping/classes/java perf.SearchPerfTest \
-    -indexPath /large/indices/wikimediumall.lucene4x.Lucene40.nd33.3326M/index \


Is this just noise/wrong? Since we turn it into hash, this new version (removing the Lucene40 part) won't be right either?

mikemccand · 2026-05-27T11:37:05Z

 import org.apache.lucene.util.fst.Util;

-// javac -cp lucene/core/build/libs/lucene-core-10.0.0-SNAPSHOT.jar IndexToFST.java; java -cp .:lucene/core/build/libs/lucene-core-10.0.0-SNAPSHOT.jar IndexToFST /l/indices/wikimediumall.trunk.facets.taxonomy:Date.taxonomy:Month.taxonomy:DayOfYear.taxonomy:RandomLabel.taxonomy.sortedset:Date.sortedset:Month.sortedset:DayOfYear.sortedset:RandomLabel.sortedset.Lucene90.Lucene90.dvfields.nd33.3326M/index
+// javac -cp lucene/core/build/libs/lucene-core-10.0.0-SNAPSHOT.jar IndexToFST.java; java -cp .:lucene/core/build/libs/lucene-core-10.0.0-SNAPSHOT.jar IndexToFST /l/indices/wikimediumall.trunk.nd33.3326M/index


mikemccand · 2026-05-27T11:37:49Z

      raise

+    # After successful indexing, write the config file
+    write_index_config(fullIndexPath, current_config)


Can we assert can_reuse_index(...) here?

mikemccand

Very close! Thank you @shubhamsrkdev!

github-actions · 2026-06-11T00:38:28Z

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

github-actions · 2026-06-28T00:34:02Z

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

Changed auto generated file name

24b58c2

mikemccand reviewed Jan 6, 2026

View reviewed changes

Comment thread src/python/competition.py Outdated

mikemccand reviewed Jan 6, 2026

View reviewed changes

Added metadata check

8da5024

shubhamsrkdev marked this pull request as ready for review January 7, 2026 14:31

shubhamsrkdev changed the title ~~Changed auto generated file name~~ Changed auto generated index name Jan 7, 2026

mikemccand reviewed Jan 7, 2026

View reviewed changes

Comment thread src/python/competition.py Outdated

shubhamsrkdev added 4 commits January 16, 2026 16:58

Added hash in file name

78ba5bc

Added extra parts

b25d1bb

Fixed test

5228e84

tidy

bc0f3de

github-actions Bot added the Stale label Feb 7, 2026

mikemccand reviewed May 26, 2026

View reviewed changes

Comment thread src/python/competition.py Outdated

github-actions Bot removed the Stale label May 27, 2026

shubhamsrkdev added 3 commits May 27, 2026 11:15

Merge

b53a10c

Comment

720d61b

Lint error fix

9e6ceff

mikemccand reviewed May 27, 2026

View reviewed changes

mikemccand approved these changes May 27, 2026

View reviewed changes

Comment

114eb1f

github-actions Bot added the Stale label Jun 11, 2026

Merge branch 'main' into changeIndexFileName

781dc9d

github-actions Bot removed the Stale label Jun 13, 2026

github-actions Bot added the Stale label Jun 28, 2026

Conversation

shubhamsrkdev commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mikemccand left a comment

Choose a reason for hiding this comment

Uh oh!

mikemccand commented Jan 7, 2026

Uh oh!

Uh oh!

github-actions Bot commented Feb 7, 2026

Uh oh!

mikemccand left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mikemccand May 27, 2026

Choose a reason for hiding this comment

Uh oh!

shubhamsrkdev May 27, 2026

Choose a reason for hiding this comment

Uh oh!

mikemccand May 27, 2026

Choose a reason for hiding this comment

Uh oh!

mikemccand May 27, 2026

Choose a reason for hiding this comment

Uh oh!

mikemccand May 27, 2026

Choose a reason for hiding this comment

Uh oh!

mikemccand left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

github-actions Bot commented Jun 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shubhamsrkdev commented Jan 6, 2026 •

edited

Loading