gh-139489: Add xml.is_valid_text()

[serhiy-storchaka]

• Issue: Add utility functions to test for legal XML characters and names #139489

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #32547402 | 📁 Comparing 2523e52 against main (426f99c)

  🔍 Preview build  

10 files changed · ± 10 modified

± Modified

• c-api/type.html
• library/datatypes.html
• library/difflib.html
• library/pprint.html
• library/ssl.html
• library/unittest.mock.html
• library/xml.html
• tutorial/stdlib2.html
• whatsnew/3.15.html
• whatsnew/changelog.html

[vstinner]

What do you think of adding also a sanitize function which takes a callback? Example:

import re

ILLEGAL_XML_CHARS_RE = re.compile(
    '['
    # Control characters; newline (\x0A and \x0D) and TAB (\x09) are legal
    '\x00-\x08\x0B\x0C\x0E-\x1F'
    # Surrogate characters
    '\uD800-\uDFFF'
    # Special Unicode characters
    '\uFFFE'
    '\uFFFF'
    # Match multiple sequential invalid characters for better efficiency
    ']+')

def sanitize(text, replace_func):
    def callback(regs):
        return replace_func(regs[0])

    return ILLEGAL_XML_CHARS_RE.sub(callback, text)

def escape(text):
    return ''.join(f'\\x{ord(char):02x}' for char in text)

invalid = '\x00'
test = f'a{invalid}b'
print(sanitize(test, lambda text: '#' * len(text)))
print(sanitize(test, escape))

It would be useful for sanitize_xml() of Lib/test/libregrtest/utils.py.

[serhiy-storchaka]

What do you think of adding also a sanitize function which takes a callback?

I left it for a separate PR. I have similar function in a different branch. But it accepts also the name of the registered error handler ('strict', 'ignore', 'replace, 'backslashreplace', etc) and the callback has different interface. It needs a separate discussion.