3.3

• TokenProducer: allow marking/resetting the input stream via TokenControl.
• Upgrade to JUnit 5.13.2 and checkstyle 10.26.0.

3.2

• Add a generic minification handler.
• Upgrade Gradle wrapper to 8.14.2.

3.1.1

• Add a StringBuilder constructor to CommentRemovalHandler.
• Use constants instead of numbers in a few places.
• Remove a few unneeded type casts.
• Add a checkstyle check.
• Gradle: add a task to run tests on Java 8.
• Upgrade Gradle wrapper to 8.14.1.
• Upgrade JUnit to 5.13.1.

3.1

The ContentHandler, ControlHandler and TokenErrorHandler interfaces were introduced, to give more flexibility than the big handler interface that was used in prior versions.

• Introduce the ContentHandler, ControlHandler and TokenErrorHandler interfaces.
• Default to Linux line endings for source files.
• Gradle: switch to gradle/actions/wrapper-validation v4.
• Fix a Gradle warning.
• Add scm section to Maven POM data.
• Upgrade Gradle wrapper to 8.13.
• Upgrade Jazzer to 0.24.0.
• Upgrade to JUnit 5.12.1.
• Upgrade to actions/setup-java v4.
• Upgrade CodeQL actions to v3.
• Run CI with Java 11 and 21.
• Bump year to 2025 in copyright notices.

3.0

Add a TokenProducer3 which works with a new TokenHandler3 interface.

Most token handlers will report problems through error handlers and throw no checked exceptions, in which case you should use TokenProducer together with a TokenHandler or TokenHandler2. In other use cases your handler may want to throw checked exceptions, and then you must use TokenProducer3 with a TokenHandler3 instead.

2.0.2

This release fixes a compilation error with Java 8. The binaries in previous versions may not produce any problems when running on a Java 8 JRE, but 2.0.2 may be safer to use.

Detail of changes

• Fix compilation error with Java 8.
• Use advanced for loop in TokenProducer.
• Tests: use diamond operator.
• Compile for --release 8.
• Upgrade to JUnit 5.10.0.
• Upgrade to Jazzer 0.20.1.
• Upgrade Gradle wrapper to 8.3.
• Bump actions/checkout from 3 to 4.
• changes.sh: add a dot at the end of each item.
• Set up a dependabot upgrade task.
• Set up a CodeQL analysis action.
• Add a Security Policy.

Version 2.0.1

This is essentially 2.0 but with improved backwards compatibility.

Detail of changes

• Rename the updated handler as TokenHandler2, to avoid backwards compatibility issues.
• Upgrade Gradle wrapper to 8.1.

Version 2.0

Highlights

Tokenproducer 2.0 replaces the methods openGroup() and closeGroup() with leftParenthesis()/rightParenthesis() and others. Using the new event methods may allow you to write clearer code, although you can also have your old TokenHandler implementation inherit from the new LegacyTokenHandler, which is compatible with the 1.x API and provides a simpler upgrade path.

Detail of changes

• Introduce new events like leftParenthesis()/rightParenthesis() as a replacement for openGroup() and closeGroup().
• Upgrade to jazzer-junit 0.16.1.
• Upgrade Gradle wrapper to 8.0.2.

Version 1.2

• Add method skipNextCodepoint() to TokenControl.
• Migrate to a directory layout compatible with Gradle's defaults.
• Upgrade to Gradle 8.0.1

Version 1.1.3

• Do not report as error an EOF found after a string quote if EofEndingQuoted is accepted.
• Tests: convert the tests to JUnit 5.
• Add a Jazzer-based fuzzer.
• Upgrade Gradle wrapper to 8.0