File: maithili_dsl/transpiler/transpile.py (_tokenize_preserving_strings)
The tokenizer only knows single-line "..." / '...' strings and # comments. Triple-quoted strings are parsed as pairs of empty strings, so when the content itself contains a quote, part of the string body is treated as code and keywords inside it get replaced:
Input: x = """क " में " ख"""
Output: x = """क " in " ख""" # में replaced inside a string literal
(Verified against v0.3.0. Simple triple-quoted strings without inner quotes survive only by accident.)
Fix: extend the tokenizer regex to match """...""" and '''...''' (with re.DOTALL-style content) before the single-quote alternatives. Consider f-string prefixes (f"...") while in there.
Tests: triple-quoted string with keywords, with inner quotes, multi-line docstring on a कार्य.
File:
maithili_dsl/transpiler/transpile.py(_tokenize_preserving_strings)The tokenizer only knows single-line
"..."/'...'strings and#comments. Triple-quoted strings are parsed as pairs of empty strings, so when the content itself contains a quote, part of the string body is treated as code and keywords inside it get replaced:(Verified against v0.3.0. Simple triple-quoted strings without inner quotes survive only by accident.)
Fix: extend the tokenizer regex to match
"""..."""and'''...'''(withre.DOTALL-style content) before the single-quote alternatives. Consider f-string prefixes (f"...") while in there.Tests: triple-quoted string with keywords, with inner quotes, multi-line docstring on a
कार्य.