Fix/ip address and string

CHANGELOG.md

@@ -1,6 +1,14 @@
 # Changelog
 
 ## NEXT
+* **Fixes:**
+    * `X509GeneralName.IpAddress` now accepts the RFC 5280 §4.2.1.10 name-constraints encoding (address **plus** subnet mask: 8 octets for IPv4, 32 for IPv6) in addition to bare `subjectAltName`/`issuerAltName` addresses (4/16 octets). It previously rejected the 8/32-octet form outright.
+    * `Asn1String.IA5.isValid` now accepts the complete 7-bit IA5 alphabet `0x00`–`0x7F`, **including DELETE (`0x7F`)** (ITU-T T.50 / ISO 646); DELETE was previously rejected.
+    * `Asn1String.UTF8.isValid` now reports UTF-8 well-formedness (RFC 3629) via a canonical round-trip instead of testing for the replacement character. A value that legitimately contains U+FFFD is no longer rejected, and malformed/overlong byte sequences are detected reliably.
+    * `Asn1String.Teletex`, `Asn1String.General`, and `Asn1String.Graphic` no longer reject legitimate 8-bit or multi-byte content (e.g. Latin-1 or CJK graphic characters). Their true repertoires (ISO 2022 / T.61) cannot be validated exactly, so `isValid` now performs best-effort recognition: `true` for a recognized subset, `null` ("unknown") otherwise, and **never `false`**; their `String` constructors consequently never throw.
+        * **API change:** `isValid` on these three types widened from `Boolean` to `Boolean?` (the base-class type; `null` denotes "not validated / unknown").
+* **Other Changes:**
+    * Document `Asn1String` repertoires and validation semantics — a per-type table, best-effort/limitation warnings, and X.680 §41 / ITU-T T.50 / RFC 3629 references — in the low-level guide and API docs.
 
 ## 0.6.0
 * **Features:**

core/src/commonMain/kotlin/at/asitplus/awesn1/Asn1String.kt

@@ -77,16 +77,27 @@ sealed class Asn1String(
     val value: String by lazy { String.decodeFromAsn1ContentBytes(rawValue) }
 
     /**
-     * Returns whether this string is valid:
+     * Returns whether this string's [rawValue] is valid for its concrete ASN.1 string type:
      * - `true`: validation succeeded
      * - `false`: validation failed
-     * - `null`: no validation implemented
+     * - `null`: no validation implemented (see [Universal], [BMP], [Unrestricted], [Videotex])
+     *
+     * With the sole exception of [UTF8] (which validates the raw UTF-8 bytes directly), validation is evaluated
+     * against the decoded [value].
+     *
+     * **Accuracy caveat:** [Teletex], [General], and [Graphic] cannot be validated exactly (their true repertoires
+     * are multi-byte / ISO 2022). They perform *best-effort* recognition: `true` for a recognized subset, `null`
+     * for anything else, and **never `false`** — so they never reject potentially-valid input. [UTF8], [IA5],
+     * [Visible], [Printable], and [Numeric] validate their exact repertoires and do report `false` for violations.
      */
     abstract val isValid: Boolean?
 
 
     /**
-     * UTF8 STRING (verbatim String)
+     * UTF8 STRING (ISO/IEC 10646, encoded as UTF-8 per RFC 3629).
+     *
+     * [isValid] reports whether [rawValue] is **well-formed UTF-8**; it deliberately does *not* reject the
+     * legitimate replacement character U+FFFD.
      */
     @Serializable(with = Asn1Utf8StringSerializer::class)
     class UTF8 private constructor(
@@ -95,8 +106,17 @@ sealed class Asn1String(
     ) : Asn1String(rawValue, performValidation) {
         override val tag = BERTags.UTF8_STRING.toULong()
 
+        /**
+         * `true` iff [rawValue] is well-formed UTF-8, via a canonical round-trip: [value] decodes [rawValue]
+         * (replacing any malformed sequence with U+FFFD), and re-encoding reproduces [rawValue] exactly **iff**
+         * the input was well-formed. Bytes that genuinely encode U+FFFD round-trip and are reported valid;
+         * malformed/overlong sequences re-encode differently and are reported invalid.
+         *
+         * **Limitation:** the [String] constructor cannot observe malformed bytes (any Kotlin [String] is
+         * encodable), so it never rejects; byte-level validation is meaningful only for values decoded from ASN.1.
+         */
         override val isValid: Boolean by lazy {
-            !value.contains('\uFFFD')
+            value.encodeToByteArray().contentEquals(rawValue)
         }
 
         /**
@@ -134,7 +154,10 @@ sealed class Asn1String(
     }
 
     /**
-     * VISIBLE STRING (checked)
+     * VISIBLE STRING (a.k.a. ISO646String).
+     *
+     * The visible/printing subset of ITU-T T.50: graphic characters plus SPACE, i.e. `0x20`–`0x7E`
+     * (no control characters, no DELETE).
      */
     @Serializable(with = Asn1VisibleStringSerializer::class)
     class Visible private constructor(
@@ -160,7 +183,10 @@ sealed class Asn1String(
     }
 
     /**
-     * IA5 STRING (checked)
+     * IA5 STRING.
+     *
+     * The IA5 alphabet is the International Reference Alphabet (ITU-T T.50 / ISO 646): the full 7-bit range
+     * `0x00`–`0x7F`, **including DELETE (`0x7F`)**. [isValid] accepts every code point `<= 0x7F`.
      */
     @Serializable(with = Asn1Ia5StringSerializer::class)
     class IA5 private constructor(
@@ -169,8 +195,9 @@ sealed class Asn1String(
     ) : Asn1String(rawValue, performValidation) {
         override val tag = BERTags.IA5_STRING.toULong()
 
+        /** `true` iff every character of [value] is in the 7-bit IA5 range `0x00`–`0x7F`. */
         override val isValid: Boolean by lazy {
-            Regex("[\\x00-\\x7E]*").matches(value)
+            Regex("[\\x00-\\x7F]*").matches(value)
         }
 
         /**
@@ -186,8 +213,14 @@ sealed class Asn1String(
     }
 
     /**
-     * TELETEX STRING (checked).
-     *  This string format is deprecated for HTTPS certificates and its use in generally discouraged in favor of UTF-8 strings (see [Asn1String.UTF8]).
+     * TELETEX (T61) STRING.
+     *
+     * Deprecated for HTTPS certificates; prefer UTF-8 (see [Asn1String.UTF8]).
+     *
+     * **Best-effort validation.** True T.61/Teletex is a multi-byte coded character set (ITU-T T.61) that awesn1
+     * does not fully model. [isValid] only *recognizes* the Latin-1 subset (`0x00`–`0xFF`): it returns `true` for
+     * recognized content and `null` ("unknown") otherwise, and **never returns `false`** — so it never rejects
+     * potentially-valid input, and the `String` constructor never throws.
      */
     @Serializable(with = Asn1StringSerializer::class)
     class Teletex private constructor(
@@ -196,16 +229,16 @@ sealed class Asn1String(
     ) : Asn1String(rawValue, performValidation) {
         override val tag = BERTags.T61_STRING.toULong()
 
-        override val isValid: Boolean by lazy {
-            Regex("[\\u0000-\\u00FF]*").matches(value)
+        override val isValid: Boolean? by lazy {
+            if (Regex("[\\u0000-\\u00FF]*").matches(value)) true else null
         }
 
         /**
          * @throws Asn1Exception if illegal characters are provided
          */
         @Throws(Asn1Exception::class)
         constructor(value: String) : this(value.encodeToByteArray(), true) {
-            if (!isValid) throw Asn1Exception("Input contains invalid chars: '$value'")
+            if (isValid == false) throw Asn1Exception("Input contains invalid chars: '$value'")
         }
 
         @PublishedApi
@@ -235,7 +268,15 @@ sealed class Asn1String(
     }
 
     /**
-     * GENERAL STRING (checked)
+     * GENERAL STRING.
+     *
+     * GeneralString (X.680 §41) comprises all registered ISO 2022 graphic (G) and control (C) sets plus SPACE and
+     * DELETE — effectively unbounded, including 8-bit and multi-byte content.
+     *
+     * **Best-effort validation.** awesn1 does not model the full ISO 2022 repertoire. [isValid] only *recognizes*
+     * the 7-bit subset (`0x00`–`0x7F`, which includes DELETE): it returns `true` for recognized content and `null`
+     * ("unknown") otherwise, and **never returns `false`** — so it never rejects potentially-valid 8-bit or
+     * multi-byte input, and the `String` constructor never throws.
      */
     @Serializable(with = Asn1StringSerializer::class)
     class General private constructor(
@@ -244,24 +285,32 @@ sealed class Asn1String(
     ) : Asn1String(rawValue, performValidation) {
         override val tag = BERTags.GENERAL_STRING.toULong()
 
-        override val isValid: Boolean by lazy {
-            Regex("[\\x00-\\x7E]*").matches(value)
+        override val isValid: Boolean? by lazy {
+            if (Regex("[\\x00-\\x7F]*").matches(value)) true else null
         }
 
         /**
          * @throws Asn1Exception if illegal characters are provided
          */
         @Throws(Asn1Exception::class)
         constructor(value: String) : this(value.encodeToByteArray(), true) {
-            if (!isValid) throw Asn1Exception("Input contains invalid chars: '$value'")
+            if (isValid == false) throw Asn1Exception("Input contains invalid chars: '$value'")
         }
 
         @PublishedApi
         internal constructor(rawValue: ByteArray) : this(rawValue, false)
     }
 
     /**
-     * GRAPHIC STRING (checked)
+     * GRAPHIC STRING.
+     *
+     * GraphicString (X.680 §41) comprises all registered ISO 2022 graphic (G) sets plus SPACE (no control
+     * characters, no DELETE).
+     *
+     * **Best-effort validation.** awesn1 does not model the full ISO 2022 repertoire. [isValid] only *recognizes*
+     * the 7-bit graphic subset (`0x20`–`0x7E`): it returns `true` for recognized content and `null` ("unknown")
+     * otherwise, and **never returns `false`** — so it never rejects potentially-valid input such as accented
+     * Latin-1 letters or CJK characters, and the `String` constructor never throws.
      */
     @Serializable(with = Asn1StringSerializer::class)
     class Graphic private constructor(
@@ -270,16 +319,16 @@ sealed class Asn1String(
     ) : Asn1String(rawValue, performValidation) {
         override val tag = BERTags.GRAPHIC_STRING.toULong()
 
-        override val isValid: Boolean by lazy {
-            Regex("[\\x20-\\x7E]*").matches(value)
+        override val isValid: Boolean? by lazy {
+            if (Regex("[\\x20-\\x7E]*").matches(value)) true else null
         }
 
         /**
          * @throws Asn1Exception if illegal characters are provided
          */
         @Throws(Asn1Exception::class)
         constructor(value: String) : this(value.encodeToByteArray(), true) {
-            if (!isValid) throw Asn1Exception("Input contains invalid chars: '$value'")
+            if (isValid == false) throw Asn1Exception("Input contains invalid chars: '$value'")
         }
 
         @PublishedApi
@@ -330,7 +379,9 @@ sealed class Asn1String(
     }
 
     /**
-     * PRINTABLE STRING (checked)
+     * PRINTABLE STRING.
+     *
+     * The PrintableString repertoire (X.680 §41): `A`–`Z`, `a`–`z`, `0`–`9`, SPACE and `' ( ) + , - . / : = ?`.
      */
     @Serializable(with = Asn1PrintableStringSerializer::class)
     class Printable private constructor(
@@ -356,7 +407,9 @@ sealed class Asn1String(
     }
 
     /**
-     * NUMERIC STRING (checked)
+     * NUMERIC STRING.
+     *
+     * The NumericString repertoire (X.680 §41): digits `0`–`9` and SPACE.
      */
     @Serializable(with = Asn1NumericStringSerializer::class)
     class Numeric private constructor(

core/src/commonTest/kotlin/at/asitplus/awesn1/Asn1StringValidatorFixTest.kt

@@ -0,0 +1,95 @@
+package at.asitplus.awesn1
+
+import at.asitplus.testballoon.matrix.matrixSuite
+import io.kotest.assertions.throwables.shouldNotThrowAny
+import io.kotest.assertions.throwables.shouldThrow
+import io.kotest.matchers.shouldBe
+
+/*
+ * Regression ("A/B") coverage for two fixed [Asn1String] validators:
+ *
+ *  1. IA5 — the alphabet is ITU-T T.50 / ISO 646, the full 7-bit range 0x00–0x7F **including DELETE (0x7F)**.
+ *     Before the fix the upper bound was 0x7E, so DELETE was wrongly rejected.
+ *  3. UTF8 — [Asn1String.UTF8.isValid] must report UTF-8 *well-formedness*, not "contains U+FFFD". Before the fix,
+ *     a value that legitimately contains U+FFFD was wrongly rejected, and malformed bytes were only caught by
+ *     accident via the decode-replacement marker.
+ *
+ * Each case is a pair: the input that behaves *differently* after the fix (A), plus a neighbouring input that must
+ * keep its old behaviour (B), so the boundary is pinned from both sides.
+ */
+val Asn1StringValidatorFixTest by matrixSuite {
+
+    val DEL = 0x7f.toChar()             // U+007F DELETE — legal IA5, was rejected before the fix
+    val TILDE = 0x7e.toChar()           // U+007E '~'     — top of the old range, still legal
+    val PAD = 0x80.toChar()             // U+0080         — first code point outside IA5
+    val REPLACEMENT = 0xfffd.toChar()   // U+FFFD REPLACEMENT CHARACTER — a legitimate UTF-8 character
+
+    // ---- Fix 1: IA5 ----
+
+    "IA5 (A) accepts DELETE 0x7f — via both the String and the raw-bytes path" {
+        shouldNotThrowAny { Asn1String.IA5(DEL.toString()) }
+        Asn1String.IA5(DEL.toString()).isValid shouldBe true
+        Asn1String.IA5(byteArrayOf(0x7f)).isValid shouldBe true      // raw byte 0x7f
+    }
+
+    "IA5 (B) still accepts 0x7e and still rejects the first out-of-range code point 0x80" {
+        Asn1String.IA5(byteArrayOf(0x7e)).isValid shouldBe true
+        // 0x80 as a well-formed UTF-8 char (C2 80) decodes cleanly but is outside the IA5 range
+        Asn1String.IA5(byteArrayOf(0xc2.toByte(), 0x80.toByte())).isValid shouldBe false
+        shouldThrow<Asn1Exception> { Asn1String.IA5(PAD.toString()) }
+    }
+
+    // ---- Fix 3: UTF8 ----
+
+    "UTF8 (A) accepts a genuine U+FFFD — via both the String and the raw-bytes path" {
+        shouldNotThrowAny { Asn1String.UTF8(REPLACEMENT.toString()) }
+        Asn1String.UTF8(REPLACEMENT.toString()).isValid shouldBe true
+        // the canonical UTF-8 encoding of U+FFFD is EF BF BD
+        val efbfbd = byteArrayOf(0xef.toByte(), 0xbf.toByte(), 0xbd.toByte())
+        Asn1String.UTF8(efbfbd).let {
+            it.isValid shouldBe true
+            it.value shouldBe REPLACEMENT.toString()
+        }
+    }
+
+    "UTF8 (B) rejects malformed byte sequences and accepts well-formed multi-byte" {
+        // malformed: lone continuation byte, truncated lead byte, overlong '/' (C0 AF)
+        Asn1String.UTF8(byteArrayOf(0x80.toByte())).isValid shouldBe false
+        Asn1String.UTF8(byteArrayOf(0xc3.toByte())).isValid shouldBe false
+        Asn1String.UTF8(byteArrayOf(0xc0.toByte(), 0xaf.toByte())).isValid shouldBe false
+
+        // well-formed: "ä" == C3 A4, and plain ASCII
+        Asn1String.UTF8(byteArrayOf(0xc3.toByte(), 0xa4.toByte())).let {
+            it.isValid shouldBe true
+            it.value shouldBe "ä"
+        }
+        Asn1String.UTF8(byteArrayOf(0x41)).isValid shouldBe true       // 'A'
+    }
+
+    // ---- Best-effort types (Teletex, General, Graphic): never reject potentially-valid input ----
+    //
+    // Their true repertoires are multi-byte / ISO 2022 and cannot be validated exactly, so isValid returns
+    // `true` for a recognized subset and `null` ("unknown") otherwise, and NEVER `false`. The String constructor
+    // must therefore never throw for these types.
+
+    // Each `.isValid` below is reached through the throwing `String` constructor, so a `null` result also proves
+    // the constructor did not reject the input.
+
+    "Graphic recognizes its 7-bit subset and never rejects higher graphic characters" {
+        Asn1String.Graphic("abc123").isValid shouldBe true          // recognized 0x20-0x7e subset
+        // é (Latin-1) and CJK are legitimate graphic characters outside the recognized subset -> null, not false
+        Asn1String.Graphic("é23456").isValid shouldBe null
+        Asn1String.Graphic("テスト").isValid shouldBe null
+    }
+
+    "General recognizes its 7-bit subset (incl. DELETE) and never rejects higher content" {
+        Asn1String.General("abc").isValid shouldBe true
+        Asn1String.General(DEL.toString()).isValid shouldBe true    // DELETE is a member of GeneralString
+        Asn1String.General("é").isValid shouldBe null               // 8-bit content: unknown, not rejected
+    }
+
+    "Teletex recognizes its Latin-1 subset and never rejects beyond it" {
+        Asn1String.Teletex("é").isValid shouldBe true               // within the Latin-1 recognized subset
+        Asn1String.Teletex("テスト").isValid shouldBe null           // beyond Latin-1: unknown, not rejected
+    }
+}

core/src/jvmTest/resources/asn1strings.json

@@ -25,8 +25,8 @@
       "invalid": []
     },
     "graphic": {
-      "valid": ["123456", "Graphic123"],
-      "invalid": ["é23456", "テスト"]
+      "valid": ["123456", "Graphic123", "é23456", "テスト"],
+      "invalid": []
     },
     "bmp": {
       "valid": ["User", "s", "TestUser"],