Please, document, how to define a custom schema types

[svetlyak40wt]

For example, I'm using YASON to parse data from API and then want to define a schema which will unserialize this data into CL objects or serialize them back to arrays and hash-tables.

To make this process reverable, I have to use the feature of YASON, when it for booleans returns yason:true or yason:false symbols and for null returns a :null keyword.

I want these fields to be represented on CL side as NIL, and T. And to do this, I'd like to define some way a new schemas - something like yason-boolean and yason-null. There is no reason to put these definitions into schemata library I just want to extend it in my library which need this.

[svetlyak40wt]

For simplest case I was able to do the following serializer:

(defclass yason-null-schema (schemata:schema)
  ())

(defmethod schemata::parse-schema-type ((schema-type (eql 'yason-null)) schema-spec)
  (make-instance 'yason-null-schema))


(defmethod schemata::schema-validate ((schema yason-null-schema) data)
  (unless (equalp data :null)
    (schemata:validation-error "~s is not equal to :NULL" data)))


(defmethod schemata:unserialize-with-schema ((schema yason-null-schema) data format)
  nil)


(defmethod schemata::%serialize-with-schema ((schema yason-null-schema) serializer input stream)
  (generic-serializer:serialize-value serializer :null stream))

However as I do not see a way to get back my arrays and hash-tables as serialization result, because schemata relies on generic-serializer which outputs to a stream and uses cl-json which does not know about yason:true and yason:false.

May be I miss something and it is possible to transform data back to arrays and hash tables using schemata? This way it will be possible to use the same JSON library for serialization/unserialization, using simple lisp datastructures as an intermediate layer.

[svetlyak40wt]

Here is another schema for true/false:

(defclass yason-boolean-schema (schemata:schema)
  ())

(defmethod schemata::parse-schema-type ((schema-type (eql 'yason-boolean)) schema-spec)
  (make-instance 'yason-boolean-schema))


(defmethod schemata::schema-validate ((schema yason-boolean-schema) data)
  (unless (member data '(yason:true yason:false))
    (schemata:validation-error "~s is not yason:true or yason:false" data)))


(defmethod schemata:unserialize-with-schema ((schema yason-boolean-schema) data format)
  (ecase data
    (yason:true t)
    (yason:false nil)))


(defmethod schemata::%serialize-with-schema ((schema yason-boolean-schema) serializer input stream)
  (let ((transformed-input
          (cond
            (input
             t
             ;; generic-serializer uses cl-json which does not recognize
             ;; symbols from Yason:
             ;; yason:true
             )
            (t
             nil
             ;; yason:false
             ))))
    (generic-serializer:serialize-value serializer transformed-input stream :type :boolean)))

Also, I've found that generic-serializer incorrectly serializes false value. It uses a code like this:

(defmethod serialize-value ((serializer (eql :json)) value stream &key type formatter &allow-other-keys)
  (let ((formatted-value (or (and formatter (funcall formatter value))
                             value)))
    (case type
      (:list (json:with-array (stream)
               (loop for elem in value
                     do
                        (json:as-array-member (stream)
                          (serialize elem serializer stream)))))
      (:boolean (if value
                  (json:encode-json t stream)
                  (json:encode-json :false stream)))
      (t (json:encode-json formatted-value stream)))))

but cl-json serializes :false symbol as string:

CL-USER> (with-output-to-string (s) (json:encode-json :false s))
"\"false\""

CL-USER> (with-output-to-string (s) (json:encode-json t s))
"true"

There is an issue in cl-json repository since 2017: hankhero/cl-json#22 but the problem is not fixed and seems will not be fixed (that is why I prefer Yason).

[mmontone]

Ok. Let me look. Without looking much, first thing that occurs to me is that perhaps a yason serializer/unserializer could be an option.

[mmontone]

There's also GENERIC-SERIALIZER:SERIALIZE-VALUE and SCHEMATA:UNSERIALIZE-WITH-SCHEMA that you can try to specialize ? (no new schema needed)

[svetlyak40wt]

I'll try to do this with GENERIC-SERIALIZER:SERIALIZE-VALUE