Skip to content

Promote per-file _parse_xml recovery into a shared framework helper #790

Description

@abrignoni

During the LAVA @artifact_processor conversion we hit real-world settings_secure.xml (and other) files that contain characters invalid in XML 1.0 — raw control bytes (\x00–\x08, \x0b, \x0c, \x0e–\x1f) or unescaped & embedded in setting values — which make a plain xml.etree.ElementTree.parse() raise ParseError: not well-formed (invalid token) and error the whole artifact. As a tactical fix that avoided touching the framework mid-conversion (and kept us aligned with iLEAPP), we added an identical module-level _parse_xml(file_found) recovery helper to each affected artifact: it tries a normal parse, and on ParseError strips the invalid control characters, escapes bare ampersands, re-parses, and — if still unparseable — logs and returns an empty element so the run continues instead of crashing. This logic is now duplicated across 23 artifacts (plus two module-level regex constants, INVALID_XML_CHARS and BARE_AMPERSAND, in each). The framework-level follow-up is to promote a single canonical helper into scripts/ilapfuncs.py (next to the existing XML helpers abxread/checkabx) — e.g. parse_xml_with_recovery(file_found) -> Element that also transparently handles the ABX branch — then migrate the per-file copies to import it and delete the duplication. Whoever does this should keep behavior identical (recover-then-skip, never raise), run the loader smoke test (python3 -c "from scripts.plugin_loader import PluginLoader; PluginLoader()"), and note that CI lints changed files with pylint … --disable=C,R, so the migration PR must leave each touched file at 0 W/E.

Artifacts currently carrying the per-file _parse_xml fix (migration reference — 23 files, all under scripts/artifacts/):

  • settingsSecure.py (original fix)
  • gmail.py, kijijiLocalUserInfo.py, usageHistory.py, atrackerdetect.py, ulrUserprefs.py, suggestions.py, setupWizardinfo.py
  • citymapper.py, LinkedIn.py, GarminUser.py
  • roles.py, runtimePerms.py, permissions.py, appopSetupWiz.py, discreteNative.py
  • protonVPN.py, rarlabPreferences.py, bittorrentClientpref.py, ChessComAccount.py, ChessComGames.py, mewe.py, deviceHealthServices_AppUsage.py

Not yet hardened (excluded — handle first): OrnetBrowser.py, Grok.py, TorBrowser.py also call ET.parse but carry pre-existing pylint W-level debt (bad indentation, unused args, broad excepts) that CI flags on any edit; they need a lint cleanup before the same fix can be applied.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions