break away bigfix pre-commits to their own repo

[jgstew]

Break away the BigFix pre-commit hooks to their own dedicated repo.

There are currently 2 BigFix pre-commit hooks in this repo and I'd like to add more:

• https://github.com/jgstew/pre-commit-jgstew/blob/main/pre_commit_jgstew/validate_bes.py — validates BES XML against the official XSDs (thin wrapper around validate_bes_xml)
• https://github.com/jgstew/pre-commit-jgstew/blob/main/pre_commit_jgstew/bigfix_bes_check_conventions.py — the check-bes-conventions hook with the W2xx content/convention checks (prefetch line shape W206, prefetch https W207, </ActionScript> whitespace W205/W208, dynamic download W211, etc.)

The plan is to grow this into a hook dedicated to ActionScript and another dedicated to Relevance. That's easier and more sensible in a standalone BigFix-focused repo than inside this general-purpose hooks collection.

Context

This issue is the landing spot for the content-level validation backlog that was closed out of validate_bes_xml (which is staying deliberately XSD-only — the schema treats ActionScript/Relevance bodies as opaque strings, so content checks don't belong there):

• Use ActionScriptSyntax\ExpandedActionScript.schclass to validate BES file actionscript validate_bes_xml#5 — validate actionscript command verbs using the ActionScript.schclass / ExpandedActionScript.schclass grammar
• Use RegEx to validate BES file syntax validate_bes_xml#4 — prefetch block structure validation (line-based checker, not a monster regex)
• Validate Relevance for Syntax Errors validate_bes_xml#2 — relevance syntax validation via QnA (machinery now lives in EvaluateRelevance.py)
• Generate validation warnings if actionscript prefetch uses http instead of https validate_bes_xml#1 — http vs https warnings (partially shipped as W207; escalation + broader scanning remain)

Plan

1. Create a new repo (something like pre-commit-bigfix) with its own .pre-commit-hooks.yaml, moving validate_bes.py and bigfix_bes_check_conventions.py over with their tests.
2. Keep the hook ids stable (validate-bes, check-bes-conventions) so consumers only have to change the repo: URL in their .pre-commit-config.yaml.
3. Deprecate the BigFix hooks here: leave them working for a release or two with a deprecation note in the README pointing at the new repo, then remove.
4. New ActionScript hook work happens in the new repo (schclass-driven per-line verb validation, </ActionScript> whitespace auto-fix, http→https escalation for no-hash prefetch/downloads, scanning Description CDATA for http://).
5. New Relevance hook work happens in the new repo (thin glue over EvaluateRelevance.py — see comment below).

[jgstew]

ActionScript hook

The actionscript validator should use the official BigFix grammar definitions as the source of truth:

• ExpandedActionScript.schclass
• ActionScript.schclass

These are machine-readable lexer definitions — the command verbs can be parsed out of them, then validation is a per-line check: the first token of each line must be a known command verb, a // comment, a substitution, or a continuation. This is the maintainable alternative to the giant multiline regex approach I rejected in jgstew/validate_bes_xml#4 (too brittle with nesting, add nohash prefetch item, collect prefetch items, continuations, and interleaved comments).

Also in scope for this hook: the </ActionScript> closing-tag whitespace trap from jgstew/validate_bes_xml#5 — the only thing allowed on the same line as ]]></ActionScript> is a newline, a comment, or a valid actionscript command; no spaces or tabs. Pretty-printing the XML injects whitespace there and then BigFix complains, which is why tons of my templates end with //END]]></ActionScript>. W205 (check_actionscript_blank_lines) and W208 (check_empty_actionscript) already cover part of this; the extension is a check for a bad token immediately before the closing tag plus an auto-fixer that inserts a newline or the //END marker.

Related:

• https://forum.bigfix.com/t/actionscript-code-snippets/21572

[jgstew]

Relevance hook

Related to a relevance linter / hook: jgstew/validate_bes_xml#2 (comment)

The hard parts are already built and proven in EvaluateRelevance.py:

• evaluate_relevance_string() / evaluate_relevance_array() shell out to qna -t -showtypes on stdin
• get_path_qna() probes standard QnA install locations cross-platform
• parse_raw_result_array() splits the A:/E:/I:/T: output lines, so a syntax error surfaces as an E: line

Multi-platform CI coverage exists via run_qna.yaml in the tools repo (macOS/Windows/Ubuntu matrix plus RHEL-family/Debian containers and sibling workflows for Solaris/Raspbian/aarch64/s390x/ppc64le/AIX), and there's a REST path via besapi session relevance for confirming statements against a live root.

What the pre-commit hook needs is just thin glue: given a .bes/.ojo, extract each <Relevance> (and MIMEField relevance) in document order, call evaluate_relevance_string() on each, and stop the "meaningful" syntax check at the first FALSE result — everything after that point may legitimately error on an unsupported platform, so errors there are expected and not meaningful. Reuse the already-parsed lxml tree, and keep the QnA/REST machinery in EvaluateRelevance.py/besapi so the hook stays dependency-light.

[jgstew]

http → https checks

Should warn on any use of http URLs in BigFix content, not just in prefetches: jgstew/validate_bes_xml#1 (comment)

Current state: W207 (check_prefetch_https) already warns on http for prefetch / add prefetch item lines, with an inline prefetch-https-ok opt-out.

Two refinements remain, both destined for the new repo:

1. Escalation for no-hash downloads: a no-hash prefetch/download over http has no integrity check at all, so it should be a hard error, while hashed ones stay warnings. W211 (check_dynamic_download) already identifies the inherently no-hash dynamic downloads, so the hook has what it needs to special-case this.
2. Scan everywhere, not just prefetch lines: http:// links in HTML <Description> CDATA bodies and "click to go to URL" action links should also warn, with the same inline opt-out marker. There may still be rare hosts where only http works, so the exception mechanism matters to avoid hard errors on known cases.

Related:

• https://forum.bigfix.com/t/content-modification-updates-for-windows-applications-published-2021-11-03/39818/2

[jgstew]

Building a python schclass ActionScript linter

Scope of this comment: a generic schclass tokenizer engine — takes any .schclass file as config, tokenizes input text exactly the way the console's editor does, and emits tokens plus "unlexable here" errors. Reusable beyond actionscript: point it at any schclass and it lexes that language. Higher-level actionscript linting (line grammar, block pairing, convention checks) is out of scope here and builds on top of this.

The .schclass files aren't a BigFix invention — they're the lex schema format of the Codejock Xtreme ToolkitPro SyntaxEdit control that the console embeds, so the dialect is documented outside BigFix.

Reference copies preserved here:

• https://github.com/jgstew/bigfix-proxyagent-servermon/blob/master/bigfix/reference-files/ActionScript.schclass — ships with the console at BES Console\Reference (98 built-in commands as token:tag entries)
• https://github.com/jgstew/bigfix-proxyagent-servermon/blob/master/bigfix/reference-files/ExpandedActionScript.schclass — generated per-operator by the console (found under %LOCALAPPDATA%\BigFix\Enterprise Console\<ROOTSERVER>\<USER>\ActionScriptSyntax). Verified to be the base file + the union of https://github.com/jgstew/bigfix-proxyagent-servermon/blob/master/bigfix/reference-files/ProxyPluginCommands.json appended as 225 extra token:tag entries (323 total; only surrender device id from the JSON is missing — likely a generator quirk)

Existing python libraries?

None. Nothing on PyPI for schclass, and a GitHub-wide search finds only repos vendoring the Codejock toolkit's schema files — no parser for the format in any language. This would be the first, but the format is simple: lexClass: blocks of key = value lines (start:Tag, end:Tag, skip:Tag, previous:tag, token:tag, separators, parent/child containment). A loader is a ~50-line function; the tokenizer engine driven by it maybe ~150 lines, stdlib-only.

What the engine does

1. Load a .schclass into lexClass definitions: entry/exit/escape tags, separators, previous:tag context constraints, token:tag keyword lists, parent/child containment.
2. Tokenize input text by walking those states — comments, strings, {...} substitutions (honoring skip:Tag escape rules), keywords matched longest-first (commands are multi-word, e.g. add nohash prefetch item), respecting previous:tag = @eol so line-start-only tokens only match at line start.
3. Emit a token stream with class names + positions, and errors for text that falls into no class where one was expected, and for states entered but never exited (unterminated string/substitution).

The error emission is the one judgment the format itself withholds: a highlighting grammar is deliberately total — everything highlights, unmatched text just renders as default. The engine makes "default-class text where a token was expected" and "unterminated state" reportable instead of silent. What stays out of the engine because a lexical grammar can't express it: argument shapes, block pairing, semantics — those are consumers of the token stream, not part of it.

One caveat: schclass files are tuned for display, not validation (e.g. the base url lexClass only matches http:, not https:), so consumers should treat the grammar as primary input plus a small override file, not unquestionable truth.