During the LAVA @artifact_processor conversion we hit real-world settings_secure.xml (and other) files that contain characters invalid in XML 1.0 — raw control bytes (\x00–\x08, \x0b, \x0c, \x0e–\x1f) or unescaped & embedded in setting values — which make a plain xml.etree.ElementTree.parse() raise ParseError: not well-formed (invalid token) and error the whole artifact. As a tactical fix that avoided touching the framework mid-conversion (and kept us aligned with iLEAPP), we added an identical module-level _parse_xml(file_found) recovery helper to each affected artifact: it tries a normal parse, and on ParseError strips the invalid control characters, escapes bare ampersands, re-parses, and — if still unparseable — logs and returns an empty element so the run continues instead of crashing. This logic is now duplicated across 23 artifacts (plus two module-level regex constants, INVALID_XML_CHARS and BARE_AMPERSAND, in each). The framework-level follow-up is to promote a single canonical helper into scripts/ilapfuncs.py (next to the existing XML helpers abxread/checkabx) — e.g. parse_xml_with_recovery(file_found) -> Element that also transparently handles the ABX branch — then migrate the per-file copies to import it and delete the duplication. Whoever does this should keep behavior identical (recover-then-skip, never raise), run the loader smoke test (python3 -c "from scripts.plugin_loader import PluginLoader; PluginLoader()"), and note that CI lints changed files with pylint … --disable=C,R, so the migration PR must leave each touched file at 0 W/E.
Artifacts currently carrying the per-file _parse_xml fix (migration reference — 23 files, all under scripts/artifacts/):
settingsSecure.py (original fix)
gmail.py, kijijiLocalUserInfo.py, usageHistory.py, atrackerdetect.py, ulrUserprefs.py, suggestions.py, setupWizardinfo.py
citymapper.py, LinkedIn.py, GarminUser.py
roles.py, runtimePerms.py, permissions.py, appopSetupWiz.py, discreteNative.py
protonVPN.py, rarlabPreferences.py, bittorrentClientpref.py, ChessComAccount.py, ChessComGames.py, mewe.py, deviceHealthServices_AppUsage.py
Not yet hardened (excluded — handle first): OrnetBrowser.py, Grok.py, TorBrowser.py also call ET.parse but carry pre-existing pylint W-level debt (bad indentation, unused args, broad excepts) that CI flags on any edit; they need a lint cleanup before the same fix can be applied.
During the LAVA
@artifact_processorconversion we hit real-worldsettings_secure.xml(and other) files that contain characters invalid in XML 1.0 — raw control bytes (\x00–\x08,\x0b,\x0c,\x0e–\x1f) or unescaped&embedded in setting values — which make a plainxml.etree.ElementTree.parse()raiseParseError: not well-formed (invalid token)and error the whole artifact. As a tactical fix that avoided touching the framework mid-conversion (and kept us aligned with iLEAPP), we added an identical module-level_parse_xml(file_found)recovery helper to each affected artifact: it tries a normal parse, and onParseErrorstrips the invalid control characters, escapes bare ampersands, re-parses, and — if still unparseable — logs and returns an empty element so the run continues instead of crashing. This logic is now duplicated across 23 artifacts (plus two module-level regex constants,INVALID_XML_CHARSandBARE_AMPERSAND, in each). The framework-level follow-up is to promote a single canonical helper intoscripts/ilapfuncs.py(next to the existing XML helpersabxread/checkabx) — e.g.parse_xml_with_recovery(file_found) -> Elementthat also transparently handles the ABX branch — then migrate the per-file copies to import it and delete the duplication. Whoever does this should keep behavior identical (recover-then-skip, never raise), run the loader smoke test (python3 -c "from scripts.plugin_loader import PluginLoader; PluginLoader()"), and note that CI lints changed files withpylint … --disable=C,R, so the migration PR must leave each touched file at 0 W/E.Artifacts currently carrying the per-file
_parse_xmlfix (migration reference — 23 files, all underscripts/artifacts/):settingsSecure.py(original fix)gmail.py,kijijiLocalUserInfo.py,usageHistory.py,atrackerdetect.py,ulrUserprefs.py,suggestions.py,setupWizardinfo.pycitymapper.py,LinkedIn.py,GarminUser.pyroles.py,runtimePerms.py,permissions.py,appopSetupWiz.py,discreteNative.pyprotonVPN.py,rarlabPreferences.py,bittorrentClientpref.py,ChessComAccount.py,ChessComGames.py,mewe.py,deviceHealthServices_AppUsage.pyNot yet hardened (excluded — handle first):
OrnetBrowser.py,Grok.py,TorBrowser.pyalso callET.parsebut carry pre-existing pylint W-level debt (bad indentation, unused args, broad excepts) that CI flags on any edit; they need a lint cleanup before the same fix can be applied.