Grim's Scythe
Marcus Griep

Writings on software engineering.

Recent posts


Synchronous messaging and lightweight threading


Powerful JSON processing with computation expressions


Using Chiron to serialize types that you can't control


Getting started with Chiron


Getting started — all over again

Chiron: Taming Types in the Wild

In my last post, I gave an overview of Chiron, described how to parse JSON, and demonstrated how to write ToJson and FromJson static methods to support serialization of custom types. At the end of the article, I left a question hanging: What if you don't control the data type that you want to serialize? What if you can't add the static ToJson/FromJson methods required by Chiron? That's where serialization extensions come in.

As an example, let's consider the NodaTime library. NodaTime provides a preferable set of types for interacting with date/time values when compared to the BCL, and I regularly reference NodaTime anywhere that I need to work with or manipulate time. While it is possible to convert an Instant to a DateTimeOffset and then serialize that value, it would be much nicer to be able to serialize an Instant directly. We will use the representation defined in ISO-8601 for serializing Instant values to JSON strings. We could choose any other form, like WCF's \/Date(1234567890)\/, but representing a date/time in any format other than the ISO standard generally leads to confusion.

Using NodaTime's facilities for formatting and parsing date/time strings we can define a serialization extension for an Instant:

1: 
2: 
let instantToJson i =
  String <| InstantPattern.GeneralPattern.Format i

GeneralPattern provides serialization of Instants in the ISO-8601 format. If you prefer a compatible representation with sub-second resolution, you can use the ExtendedIsoPattern instead.

The instantToJson function has the type signature Instant -> Json. This is different from the monadic Json<'a> signature that was used in the serializers in the previous post. Instead of calling Json.serialize, instantToJson can be used as a drop-in replacement.

1: 
2: 
3: 
4: 
let instantJson =
  SystemClock.Instance.Now
  |> instantToJson
  |> Json.format
""2016-04-13T14:30:46Z""

Next we define the deserialization function. In doing so, I will also define an active pattern to encapsulate NodaTime's pattern parsing logic. This will help to keep the intent of the deserialization function clear.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
let (|ParsedInstant|InvalidInstant|) str =
  let parseResult = InstantPattern.GeneralPattern.Parse str
  if parseResult.Success then
    ParsedInstant parseResult.Value
  else
    InvalidInstant

let instantFromJson = function
  | String (ParsedInstant i) -> Value i
  | json ->
    Json.formatWith JsonFormattingOptions.SingleLine json
    |> sprintf "Expected a string containing a valid ISO-8601 date/time: %s"
    |> Error

instantFromJson has the type signature Json -> JsonResult<Instant>. Together with the signature of instantToJson, these functions provide the complimentary facets of the Json<'a> state monad. JsonResult<'a> holds the working value while Json stores the state of the JSON data that we are serializing or deserializing.

Now we can try to convert instantJson back to an Instant to validate that our serialization can round-trip.

1: 
2: 
3: 
4: 
let instantRoundTrip =
  instantJson
  |> Json.parse
  |> instantFromJson
Value 2016-04-13T14:30:46Z

We can also check that an invalid value produces a relevant error message during deserialization:

1: 
2: 
3: 
4: 
let instantError =
  """ "Tomorrow at 13:30" """
  |> Json.parse
  |> instantFromJson
Error
  "Expected a string containing a valid ISO-8601 date/time: "Tomorrow at 13:30""

This looks good so far. Of course, it is rare to want to serialize something like an Instant all on its own. More commonly, the data is incorporated into a larger JSON object or array. In this case, there are a few additional read and write functions that provide points to inject custom (de)serialization functions: readWith and writeWith. To demonstrate their use, we will consider a trivial type containing an Instant and add FromJson and ToJson static methods.

1: 
2: 
3: 
4: 
5: 
6: 
7: 
type MyType =
  { Time : Instant }
  static member ToJson (x:MyType) =
    Json.writeWith instantToJson "time" x.Time
  static member FromJson (_:MyType) =
        fun t -> { Time = t }
    <!> Json.readWith instantFromJson "time"

Note how instantToJson and instantFromJson are injected as the first argument to Json.writeWith and Json.readWith, respectively. Now we can demonstrate round-trip serialization:

1: 
2: 
3: 
4: 
let myTypeJson =
  { Time = SystemClock.Instance.Now }
  |> Json.serialize
  |> Json.format
"{"time":"2016-04-13T14:30:46Z"}"
1: 
2: 
3: 
4: 
let myTypeRoundTrip : MyType =
  myTypeJson
  |> Json.parse
  |> Json.deserialize
{Time = 2016-04-13T14:30:46Z;}

We now have serializers that can round-trip an external type either alone or as part of a type we control. What if the external type is contained in yet another external type? To demonstrate this case, we will consider serializing an Instant list:

1: 
2: 
3: 
4: 
let listOfInstantJson =
  [ Instant(); SystemClock.Instance.Now ]
  |> Json.serialize
  |> Json.format

Before we even get to running this code, the compiler has already started disagreeing with us:

1: 
Error: No overloads match for method 'ToJson'. …

Where did we go wrong? The issue is that an a' list is serializable through Chiron's default functions if and only if 'a contains the necessary ToJson/FromJson functions. Instant doesn't have the needed hooks needed for Chiron's default serialization functions. Since we wrote our own serialization for Instant, we need need to write a function to serialize our list too. Instead of defining a specialized Instant list serializer, we can instead write a generic 'a list serializer and include a parameter so that we can plug in an arbitrary serializer.

1: 
2: 
let listToJsonWith serialize lst =
  Array <| List.map serialize lst

The deserializer is a little more complicated because we can't just map it over the list. We need a function that maps Json -> JsonResult<Instant list>. Chiron already has a function that fits this need: fromJsonFold, which it uses to support default serialization of arrays and lists. fromJsonFold iterates over a Json list wrapped by a Json.Array and produces a JsonResult over the list. This function is marked internal, though, so we don't have direct access to it. Instead, we can extract the function's logic and refactor it to fit our needs. Replacing fromJson with a new deserialize parameter gives us a generic function for applying a custom deserializer over a Json list.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
let fromJsonFoldWith deserialize fold zero xs =
    List.fold (fun r x ->
      match r with
      | Error e -> Error e
      | Value xs ->
        match deserialize x with
        | Value x -> Value (fold x xs)
        | Error e -> Error e) (Value zero) (List.rev xs)

let listFromJsonWith deserialize = function
  | Array l -> fromJsonFoldWith deserialize (fun x xs -> x::xs) [] l
  | _ -> Error "Expected an array"

fromJsonFoldWith is likely to be added in to Chiron in a future version, but for now, our custom serialization functions suffice as demonstrated by another round-trip:

1: 
2: 
3: 
4: 
let listOfInstantJson =
  [ Instant(); SystemClock.Instance.Now ]
  |> listToJsonWith instantToJson
  |> Json.format
"["1970-01-01T00:00:00Z","2016-04-13T14:30:46Z"]"
1: 
2: 
3: 
4: 
let listOfInstantRoundTrip =
  listOfInstantJson
  |> Json.parse
  |> listFromJsonWith instantFromJson
Value [1970-01-01T00:00:00Z; 2016-04-13T14:30:46Z]

Thus far, I've only been creating custom serializers for records and tuples, a.k.a. product types, and none of these serializers would deal well if they were given an object with missing data or null values:

1: 
2: 
3: 
let result : Choice<MyType,_> =
  Json.parse "{}"
  |> Json.tryDeserialize
Choice2Of2
  "Error deserializing JSON object; Missing required member 'time': {  }"

In order to handle missing data or null values and other discriminated unions, a.k.a. sum types, we will need to learn about a few more tricks that Chiron has up its sleeve. In my next post, I will focus on the Chiron features that allow you to provide defaults for missing values and serialize the disjoint cases of a discriminated union.

This post is a continuation of my post for the F# Advent Calendar in English. Many thanks to Sergey Tihon for promoting this event. For more posts on F# and functional programming throughout December, check out the list of posts on his site.

module Chiron
module Operators

from Chiron
namespace NodaTime
namespace NodaTime.Text
val instantToJson : i:Instant -> Json

Full name: 12-15-chiron-taming-types-in-the-wild_.instantToJson
val i : Instant
Multiple items
union case Json.String: string -> Json

--------------------
module String

from Microsoft.FSharp.Core
type InstantPattern =
  member Format : value:Instant -> string
  member Parse : text:string -> ParseResult<Instant>
  member PatternText : string
  member WithCulture : cultureInfo:CultureInfo -> InstantPattern
  member WithMinMaxLabels : minLabel:string * maxLabel:string -> InstantPattern
  static member Create : patternText:string * cultureInfo:CultureInfo -> InstantPattern
  static member CreateNumericPattern : cultureInfo:CultureInfo * includeThousandsSeparators:bool -> InstantPattern
  static member CreateWithCurrentCulture : patternText:string -> InstantPattern
  static member CreateWithInvariantCulture : patternText:string -> InstantPattern
  static member ExtendedIsoPattern : InstantPattern
  ...

Full name: NodaTime.Text.InstantPattern
property InstantPattern.GeneralPattern: InstantPattern
InstantPattern.Format(value: Instant) : string
val instantJson : string

Full name: 12-15-chiron-taming-types-in-the-wild_.instantJson
type SystemClock =
  member Now : Instant
  static val Instance : SystemClock

Full name: NodaTime.SystemClock
field SystemClock.Instance
property SystemClock.Now: Instant
Multiple items
module Json

from Chiron.Mapping

--------------------
module Json

from Chiron.Formatting

--------------------
module Json

from Chiron.Parsing

--------------------
module Json

from Chiron.Optics

--------------------
module Json

from Chiron.Functional

--------------------
type Json =
  | Array of Json list
  | Bool of bool
  | Null of unit
  | Number of decimal
  | Object of Map<string,Json>
  | String of string
  static member Array_ : Prism<Json,Json list>
  static member private Array__ : (Json -> Json list option) * (Json list -> Json)
  static member Bool_ : Prism<Json,bool>
  static member private Bool__ : (Json -> bool option) * (bool -> Json)
  static member Null_ : Prism<Json,unit>
  static member private Null__ : (Json -> unit option) * (unit -> Json)
  static member Number_ : Prism<Json,decimal>
  static member private Number__ : (Json -> decimal option) * (decimal -> Json)
  static member Object_ : Prism<Json,Map<string,Json>>
  static member private Object__ : (Json -> Map<string,Json> option) * (Map<string,Json> -> Json)
  static member String_ : Prism<Json,string>
  static member private String__ : (Json -> string option) * (string -> Json)

Full name: Chiron.Json

--------------------
type Json<'a> = Json -> JsonResult<'a> * Json

Full name: Chiron.Functional.Json<_>
val format : json:Json -> string

Full name: Chiron.Formatting.Json.format
val str : string
val parseResult : ParseResult<Instant>
InstantPattern.Parse(text: string) : ParseResult<Instant>
property ParseResult.Success: bool
property ParseResult.Value: Instant
val instantFromJson : _arg1:Json -> JsonResult<Instant>

Full name: 12-15-chiron-taming-types-in-the-wild_.instantFromJson
active recognizer ParsedInstant: string -> Choice<Instant,unit>

Full name: 12-15-chiron-taming-types-in-the-wild_.( |ParsedInstant|InvalidInstant| )
union case JsonResult.Value: 'a -> JsonResult<'a>
val json : Json
val formatWith : options:JsonFormattingOptions -> json:Json -> string

Full name: Chiron.Formatting.Json.formatWith
type JsonFormattingOptions =
  {Spacing: StringBuilder -> StringBuilder;
   NewLine: int -> StringBuilder -> StringBuilder;}
  static member Compact : JsonFormattingOptions
  static member Pretty : JsonFormattingOptions
  static member SingleLine : JsonFormattingOptions

Full name: Chiron.Formatting.JsonFormattingOptions
property JsonFormattingOptions.SingleLine: JsonFormattingOptions
val sprintf : format:Printf.StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf
union case JsonResult.Error: string -> JsonResult<'a>
val instantRoundTrip : JsonResult<Instant>

Full name: 12-15-chiron-taming-types-in-the-wild_.instantRoundTrip
val parse : (string -> Json)

Full name: Chiron.Parsing.Json.parse
val instantError : JsonResult<Instant>

Full name: 12-15-chiron-taming-types-in-the-wild_.instantError
type MyType =
  {Time: Instant;}
  static member FromJson : MyType -> Json<MyType>
  static member ToJson : x:MyType -> Json<unit>

Full name: 12-15-chiron-taming-types-in-the-wild_.MyType
MyType.Time: Instant
Multiple items
type Instant =
  struct
    new : ticks:int64 -> Instant
    member CompareTo : other:Instant -> int
    member Equals : obj:obj -> bool + 1 overload
    member GetHashCode : unit -> int
    member InUtc : unit -> ZonedDateTime
    member InZone : zone:DateTimeZone -> ZonedDateTime + 1 overload
    member Minus : other:Instant -> Duration + 1 overload
    member Plus : duration:Duration -> Instant
    member PlusTicks : ticksToAdd:int64 -> Instant
    member Ticks : int64
    ...
  end

Full name: NodaTime.Instant

--------------------
Instant()
Instant(ticks: int64) : unit
static member MyType.ToJson : x:MyType -> Json<unit>

Full name: 12-15-chiron-taming-types-in-the-wild_.MyType.ToJson
val x : MyType
val writeWith : toJson:('a -> Json) -> key:string -> value:'a -> Json<unit>

Full name: Chiron.Mapping.Json.writeWith
static member MyType.FromJson : MyType -> Json<MyType>

Full name: 12-15-chiron-taming-types-in-the-wild_.MyType.FromJson
val t : Instant
val readWith : fromJson:(Json -> JsonResult<'a>) -> key:string -> Json<'a>

Full name: Chiron.Mapping.Json.readWith
val myTypeJson : string

Full name: 12-15-chiron-taming-types-in-the-wild_.myTypeJson
val serialize : a:'a -> Json (requires member ToJson)

Full name: Chiron.Mapping.Json.serialize
val myTypeRoundTrip : MyType

Full name: 12-15-chiron-taming-types-in-the-wild_.myTypeRoundTrip
val deserialize : json:Json -> 'a (requires member FromJson)

Full name: Chiron.Mapping.Json.deserialize
val listOfInstantJson : obj

Full name: chirontamingtypesinthewild.listOfInstantJson
val listToJsonWith : serialize:('a -> Json) -> lst:'a list -> Json

Full name: 12-15-chiron-taming-types-in-the-wild_.listToJsonWith
val serialize : ('a -> Json)
val lst : 'a list
Multiple items
union case Json.Array: Json list -> Json

--------------------
module Array

from Microsoft.FSharp.Collections
Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
type List<'T> =
  | ( [] )
  | ( :: ) of Head: 'T * Tail: 'T list
  interface IEnumerable
  interface IEnumerable<'T>
  member GetSlice : startIndex:int option * endIndex:int option -> 'T list
  member Head : 'T
  member IsEmpty : bool
  member Item : index:int -> 'T with get
  member Length : int
  member Tail : 'T list
  static member Cons : head:'T * tail:'T list -> 'T list
  static member Empty : 'T list

Full name: Microsoft.FSharp.Collections.List<_>
val map : mapping:('T -> 'U) -> list:'T list -> 'U list

Full name: Microsoft.FSharp.Collections.List.map
val fromJsonFoldWith : deserialize:('a -> JsonResult<'b>) -> fold:('b -> 'c -> 'c) -> zero:'c -> xs:'a list -> JsonResult<'c>

Full name: 12-15-chiron-taming-types-in-the-wild_.fromJsonFoldWith
val deserialize : ('a -> JsonResult<'b>)
val fold : ('b -> 'c -> 'c)
val zero : 'c
val xs : 'a list
val fold : folder:('State -> 'T -> 'State) -> state:'State -> list:'T list -> 'State

Full name: Microsoft.FSharp.Collections.List.fold
val r : JsonResult<'c>
val x : 'a
val e : string
val xs : 'c
val x : 'b
val rev : list:'T list -> 'T list

Full name: Microsoft.FSharp.Collections.List.rev
val listFromJsonWith : deserialize:(Json -> JsonResult<'a>) -> _arg1:Json -> JsonResult<'a list>

Full name: 12-15-chiron-taming-types-in-the-wild_.listFromJsonWith
val deserialize : (Json -> JsonResult<'a>)
val l : Json list
val listOfInstantJson : string

Full name: 12-15-chiron-taming-types-in-the-wild_.listOfInstantJson
val listOfInstantRoundTrip : JsonResult<Instant list>

Full name: 12-15-chiron-taming-types-in-the-wild_.listOfInstantRoundTrip
val result : Choice<MyType,string>

Full name: 12-15-chiron-taming-types-in-the-wild_.result
Multiple items
type Choice<'T1,'T2> =
  | Choice1Of2 of 'T1
  | Choice2Of2 of 'T2

Full name: Microsoft.FSharp.Core.Choice<_,_>

--------------------
type Choice<'T1,'T2,'T3> =
  | Choice1Of3 of 'T1
  | Choice2Of3 of 'T2
  | Choice3Of3 of 'T3

Full name: Microsoft.FSharp.Core.Choice<_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4> =
  | Choice1Of4 of 'T1
  | Choice2Of4 of 'T2
  | Choice3Of4 of 'T3
  | Choice4Of4 of 'T4

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4,'T5> =
  | Choice1Of5 of 'T1
  | Choice2Of5 of 'T2
  | Choice3Of5 of 'T3
  | Choice4Of5 of 'T4
  | Choice5Of5 of 'T5

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4,'T5,'T6> =
  | Choice1Of6 of 'T1
  | Choice2Of6 of 'T2
  | Choice3Of6 of 'T3
  | Choice4Of6 of 'T4
  | Choice5Of6 of 'T5
  | Choice6Of6 of 'T6

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_,_,_>

--------------------
type Choice<'T1,'T2,'T3,'T4,'T5,'T6,'T7> =
  | Choice1Of7 of 'T1
  | Choice2Of7 of 'T2
  | Choice3Of7 of 'T3
  | Choice4Of7 of 'T4
  | Choice5Of7 of 'T5
  | Choice6Of7 of 'T6
  | Choice7Of7 of 'T7

Full name: Microsoft.FSharp.Core.Choice<_,_,_,_,_,_,_>
val tryDeserialize : json:Json -> Choice<'a,string> (requires member FromJson)

Full name: Chiron.Mapping.Json.tryDeserialize