Skip to content

fix(linter): exclude string-literal contents from paren/quote balance checks#50

Open
trungminhdo4-glitch wants to merge 3 commits into
alphacrack:developmentfrom
trungminhdo4-glitch:fix/linter-string-literal-false-positives
Open

fix(linter): exclude string-literal contents from paren/quote balance checks#50
trungminhdo4-glitch wants to merge 3 commits into
alphacrack:developmentfrom
trungminhdo4-glitch:fix/linter-string-literal-false-positives

Conversation

@trungminhdo4-glitch

@trungminhdo4-glitch trungminhdo4-glitch commented Jul 20, 2026

Copy link
Copy Markdown

Problem

The linter in maithili_dsl/transpiler/linter.py flagged false warnings for unbalanced parentheses and quotes when those characters appeared inside string literals. For example, valid code like छपाउ("(क") produced पंक्ति 1: गोल ब्रैकेट असंतुलित अछि.

Additionally, only double-quote balance was checked; single quotes were never validated at all.

Root cause

lint_maithili_code used stripped.count("(") and stripped.count('"') on the entire line including string literal contents. Characters inside "..." and '...' strings should not contribute to the balance count.

Change

  • Reuse _tokenize_preserving_strings from maithili_dsl.transpiler.transpile — the same string-aware tokenizer already used by the transpiler.
  • Build a code-only version of each line by joining only the non-string tokens.
  • Count parentheses and quote balance only in the code-only portion.
  • Add single-quote (') balance checking alongside the existing double-quote check.

Files changed:

  • maithili_dsl/transpiler/linter.py (+12/-4, functional changes only)
  • tests/test_linter.py (+6 regression tests)
  • tests/test_security.py (+2 security regression tests)
  • CHANGELOG.md (Unreleased entry)

Tests

Before fix: lint_maithili_code('छपाउ("(क")\n')['पंक्ति 1: गोल ब्रैकेट असंतुलित अछि'] (false positive)

After fix: same input returns [] (no errors). Genuinely unbalanced parens and quotes are still caught.

New tests:

  • test_parens_inside_double_quoted_string_not_flagged
  • test_parens_inside_single_quoted_string_not_flagged
  • test_quote_inside_string_not_flagged
  • test_unbalanced_single_quote_flagged (new capability)
  • test_genuinely_unbalanced_parens_still_flagged
  • test_genuinely_unbalanced_quotes_still_flagged
  • test_linter_tokenizer_does_not_expand_trust_boundary (security)
  • test_actual_dangerous_code_outside_strings_still_blocked (security)

Test results: 152 passed. 5 pre-existing Windows smoke-test failures (cp1252 encoding of Devanagari subprocess output) confirmed identical on unmodified development.

Security Considerations

This change only reuses the existing string-aware tokenizer for lint balance checks. It does not alter transpilation, import validation, or sandbox execution. Security regression tests in tests/test_security.py confirm that:

  • Code-like content inside string literals remains data (no false linter positives, no execution path).
  • Actual disallowed code outside strings is still rejected by the existing _validate_imports gate.

The trust boundary is unchanged.

Risk

Minimal. The tokenizer is the same function used by the transpiler's keyword-replacement path, which has existing test coverage for string literal preservation. No new dependencies, no API change.

Closes #30

… checks

The linter counted parentheses and quotes inside string literals,
producing false positive "unbalanced" warnings for code like छपाउ("(क").

Use _tokenize_preserving_strings from the transpiler to extract only
the code portions of each line before counting. Also add single-quote
balance checking (previously only double quotes were checked).

Regression tests cover: parens/quotes inside strings (no error),
genuinely unbalanced lines (still error), single-quote balance,
apostrophe in double-quoted strings.

Closes alphacrack#30
@trungminhdo4-glitch trungminhdo4-glitch changed the title \ %TITLE%\ fix(linter): exclude string-literal contents from paren/quote balance checks Jul 20, 2026

@alphacrack alphacrack left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really solid contribution — the fix reuses _tokenize_preserving_strings exactly as #30 suggested, adds the missing single-quote balance check, registers against the existing security marker correctly (the project runs --strict-markers, and security is declared in pyproject, so you're safe), and includes genuine negative controls. Thank you!

One test needs a fix before merge — it silently doesn't test what its name says:

def test_quote_inside_string_not_flagged():
    code = 'छपाउ("it''s ok")\n'

Inside a single-quoted Python literal, '' closes and reopens the string — adjacent-literal concatenation. The value is actually छपाउ("its ok") (verified: repr(code) shows no apostrophe), so the apostrophe-inside-double-quotes case is never exercised. The underlying behavior is correct (I checked: छपाउ("it's ok") lints clean with your change), the test just needs to construct the string it claims to:

code = 'छपाउ("it\'s ok")\n'   # escaped apostrophe

Style suggestion while you're in there (non-blocking): the rest of test_linter.py uses literal Devanagari rather than \uXXXX escapes — literals would make these tests much easier to read for the Maithili-speaking contributors this project hopes to attract.

Happy to approve as soon as the one-line test fix lands. CI run is approved on my end.

@alphacrack alphacrack added bug Something isn't working area:linter maithili_dsl/transpiler/linter.py P3 Low priority: nice to have labels Jul 20, 2026
@alphacrack alphacrack added this to the v0.4.0 milestone Jul 20, 2026
@trungminhdo4-glitch

Copy link
Copy Markdown
Author

Thanks for catching that. You were right—the adjacent string literals removed the apostrophe, so the test wasn't exercising the intended case. I've corrected the test, added an assertion that verifies the apostrophe is present in the constructed input, reran the focused linter/security tests, and pushed the update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:linter maithili_dsl/transpiler/linter.py bug Something isn't working P3 Low priority: nice to have

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Linter counts parentheses/quotes inside string literals — false unbalanced warnings

2 participants