Add MQTT message payload compression and decompression functionality#26
Add MQTT message payload compression and decompression functionality#26KAVINDYADEVINDI wants to merge 2 commits into
Conversation
| log.warn("Empty MqttMessage Received"); | ||
| return; | ||
| } | ||
| int startIndex = mqttMsgString.indexOf("{"); |
There was a problem hiding this comment.
The current implementation relies on indexOf/lastIndexOf to extract JSON, which is not a safe parsing strategy.
This can lead to incorrect extraction when payloads contain additional braces or non-JSON content. Recommend using a proper JSON parsing library (e.g:- Jackson) to validate and deserialize the payload instead of manual string manipulation.
| } | ||
| } | ||
|
|
||
| public static int findGzipHeaderOffset(byte[] payload) { |
There was a problem hiding this comment.
Add Java Doc comment
| return -1; | ||
| } | ||
|
|
||
| public static String decompressMqttMsgFromOffset(byte[] payload, int offset) throws IOException { |
There was a problem hiding this comment.
Add Java Doc comment
| } | ||
| } | ||
|
|
||
| public static byte[] compressMqttMessage(byte[] data) throws IOException { |
There was a problem hiding this comment.
Add Java Doc comment
| org.wso2.carbon.event.output.adapter.core.exception;version="[5.3,6)", | ||
| org.wso2.carbon.user.api;version="[1.0,2)" | ||
| org.wso2.carbon.user.api;version="[1.0,2)", | ||
| io.entgra.device.mgt.core.device.mgt.core.config;version="[5.2,6)", |
There was a problem hiding this comment.
Don't hard code device-mgt core version, instead of add "${io.entgra.device.mgt.core.version.range}"
|
|
||
| public static int findGzipHeaderOffset(byte[] payload) { | ||
| // Find the offset of the gzip header (gzip magic bytes -> 0x1f 0x8b) | ||
| for (int i = 0; i < payload.length - 1; i++) { |
There was a problem hiding this comment.
Maintain backward compatibility with both:
plain JSON messages, and
compressed MQTT payloads.
How is this achieved?, Instead of only using 4KB size to automatically use gzip compression, there need to be a variable to completely enable or disable gzip compression in device mgt core , cdm-config.xml and that need to be used here. Only the existing deployments relying on this listeners can then be safely tested for when gzip compression is enabled or disabled, otherwise when payload size is above the mentioned 4kb , gzip compression is forced.
| int compressionThresholdKb = (configuredThresholdKb > 0) ? configuredThresholdKb : | ||
| DEFAULT_COMPRESSION_THRESHOLD_KB; | ||
|
|
||
| if (rawBytes.length >= compressionThresholdKb * 1024) { |
5 participants
Purpose
To improve MQTT message reliability and performance.
Related Prs: entgra/device-mgt-core#170
Related Issue ticket: https://roadmap.entgra.net/issues/15144
Approach