Fix 2609 binary userdata lists#2785
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a unique identification system for UserData to ensure consistent serialization of list, map, and array fields, replacing the previous hardcoded indexing. It also enables list consistency testing in JmeExporterTest. A critical issue was identified in BinaryInputCapsule.java, where the logic added to readLong incorrectly handles special markers, potentially leading to stream corruption when reading specific long values.
| if (value == BinaryOutputCapsule.NULL_OBJECT | ||
| || value == BinaryOutputCapsule.DEFAULT_OBJECT) { | ||
| index -= 4; | ||
| } |
There was a problem hiding this comment.
This change introduces a critical bug that will lead to stream corruption when reading long values of -1 or -2.
The index -= 4 logic is intended to compensate for the way BinaryOutputCapsule.deflate handles int values of -1 (NULL_OBJECT) or -2 (DEFAULT_OBJECT) by converting them into a single-byte marker. However, deflate only applies this special handling to 4-byte values (integers). For 8-byte long values, it writes the full (or partially deflated) bytes without using the magic markers.
Consequently, when readLong encounters a full 8-byte -1L, it will correctly increment the index by 9 bytes (1 length byte + 8 data bytes), but then this new code will incorrectly backtrack the index by 4 bytes. This causes the next read operation to start in the middle of the previous long value, corrupting the entire import process.
References
- Avoid using magic numbers (e.g., -1) as markers or default values in serialization to detect missing keys or special states, as this can lead to stream corruption or mask bugs.
xinzhoudev
left a comment
There was a problem hiding this comment.
Motivation
The BinaryExporter in jMonkeyEngine currently fails to correctly deserialize Savable objects that contain multiple UserData lists. When a scene object carries more than one such list, the binary import process misreads the stream, causing subsequent data to be parsed from the wrong offset. This leads to corrupted deserialization results and potential crashes, as reported in issue #2609.
The root cause lies in how BinaryInputCapsule handles the index pointer after reading certain values. The existing logic did not account for cases where multiple consecutive list structures are present, causing the read cursor to fall out of sync with the actual byte stream layout.
Summary
This PR makes two changes to address the issue:
- BinaryInputCapsule.java — Index correction on NULL_OBJECT / DEFAULT_OBJECT sentinel values
When reading a long value that matches the NULL_OBJECT (-1) or DEFAULT_OBJECT (-2) sentinel constants defined in BinaryOutputCapsule, the index is now rewound by 4 bytes. This compensates for the fact that BinaryOutputCapsule.deflate() encodes these special int-sized markers in a compressed form, so the reader must not advance the full 8-byte stride in such cases.
- JmeExporterTest.java — Enable list serialization tests
The testLists flag in the exporter consistency test suite is changed from false to true, ensuring that list serialization and deserialization (including multiple UserData lists) is now covered by automated tests, preventing regression.
JNightRider
left a comment
There was a problem hiding this comment.
I believe this PR contains the same changes as this PR - #2610, only this one slightly modifies the format of the prefix proposed by the previous pull request.
I'm not so sure about the other one, @riccardobl was there any reason to close that PR - #2610?
| if (value == BinaryOutputCapsule.NULL_OBJECT | ||
| || value == BinaryOutputCapsule.DEFAULT_OBJECT) { | ||
| index -= 4; | ||
| } |
There was a problem hiding this comment.
I tested these changes without this code snippet; the test runs without issues, so I suspect it's not related to the problem.
|
|
||
| if (uniqueId == null) { | ||
| synchronized (this) { | ||
| if (uniqueId == null) { |
There was a problem hiding this comment.
I don't understand the double validation; isn't one validation enough?
No description provided.