Extend support for TKey in Dictionaries to non-string types

Motivation

Most of our users that serialize dictionary use Dictionary<string, TKey>; however there is a significant amount that relies on Dictionary<TKey, TValue> where TKey is a primitive other than string, e.g: int or Guid, most of them came form Newtonsoft.Json which offers a plenty amount of support for using several types as the TKey, other popular .Net serializers also offer support for other types, the most common are integers (int, uint, long, etc.), and enums (including Flags enums).

Goals

• 80%+ of dictionaries with non-string keys work out of the box, especially if they can round-trip.
• Remain high performance.

Non-goals

• Complete parity with Newtonsoft.Json capabilities, especially in how string support is extended; any extension point can be through JsonConverter<MyDictionary<non-string, TValue>>.

Sample

// (De)serialize into a dictionary with a non-string key.
Dictionary<int, string> root = new Dictionary<int, string>();
root.Add(1, "value");

string json = JsonSerializer.Serialize(root);
// JSON
// {
//   "1":"value"
// }

Dictionary<int, string> rootCopy = JsonSerializer.Deserialize<Dictionary<int, string>>(json);
Console.WriteLine(rootCopy[1]);
 // Prints
 // value

Strawman proposal

KeyConverter

Implement an internal custom mechanism that is in charge of converting a defined set of types to be supported as the dictionary TKey; more or less like internal JsonConverters work but for dictionary keys to JSON property names and viceversa.

• The alternative that offers the best performance.

• We need to define a criteria to choose what types we should support, I suggest to do as Utf8JsonReader/Writer and support the types supported by the Utf8Parser/Formatter.

• Supported types (Types supported by Utf8Formatter/Parser + a few others that are popular):

  • Boolean
  • Byte
  • DateTime
  • DateTimeOffset
  • Decimal
  • Double
  • Enum
  • Guid
  • Int16
  • Int32
  • Int64
  • Object (Only on Serialization and if the runtime type is one of the supported types in this list)
  • SByte
  • Single
  • String
  • UInt16
  • UInt32
  • UInt64

• https://github.com/Jozkee/runtime/tree/TKey_CustomConverter

Alternative considered

TypeConverter

Use TypeConverter to parse and write the string representation of the type and use that as the JSON property name.

• Many .NET types count with TypeConverter support that provides a string interpretation of the type.
• Users can hook up TypeConverters for their own types.
• Not a high-performance alternative, involves unnecessary boxing.
• https://github.com/Jozkee/runtime/tree/TKeySupport_TypeConverter

Benchmark results

Using a dictionary that contains 100 elements.

Serialize/Write

The custom KeyConverter that calls Utf8Parser underneath performs slightly faster than calling TypeConverter, keep in mind that KeyConverter is a naive implementation, it also calls Encoding.UTF8.GetString since JsonNamingPolicy.ConvertName only takes strings, this could be fixed if we add an internal method that can take a ROS<byte>, also the allocations are currently super high; this might be alleviated by moving the KeyConverter store to the JsonSerializerOptions.

Dictionary<String, TValue> results show the same numbers across branches since that still uses DictionaryOfStringTValueConverter.

main:

KeyConverter:

TypeConverter:

Deserialize/Read

main:

KeyConverter:

TypeConverter:

Prior-art

Newtonsoft.Json

On write:

• if the TKey is a concrete primitive type*:

  • it calls Convert.ToString()   except for the next types:
    • DateTime (uses DateFormatHandling specified in options)
    • DateTimOffset
    • Double (uses double.ToString("R")) // 'R' stands for round-trip
    • Single
    • Enum (uses an internal helper method)

• If the TKey is object or non-primitive.   * it calls the TypeConverter of the TKey runtime type.   Except for :     * Type, which returns the AssemblyQualifiedName.   * If the type does not have a TypeConverter, it calls ToString() on the TKey instance.

* A primitive type is a value cataloged as such by Json.Net from this list.

On read:

• If the TKey is a concrete type.
  • If is a primitive that implements IConvertible:
    • Calls Convert.ChangeType(propertyName, concreteType) But first tries to manually convert on these types:
      • Enum
      • DateTime
      • BigInteger
    • If the type does not implement IConvertible:
      • It tries to manually convert or cast to the concrete type using a few custom helper methods.
• If the TKey is object, the entries' keys will be of type string if they are quoted (Newtonsoft supports unquoted properties).

Utf8Json

Supported types:

• String
• Numerics (int, float, etc.)
• Enum
• Guid
• Boolean
• Nullables of all the above types

Jil

Supported types:

• String
• Numerics (int, long, etc. Note: does not support floating point types)
• Enum

Notes

1. DictionaryKeyPolicy will apply to the resulting string of the non-string types.
2. Should we provide a way to allow users to customize the EnumKeyConverter behavior, as it is done in JsonStringEnumConverter? As of now KeyConverters are meant to be internal types, to enable the previously described behavior we either pass the options through JsonSerializerOptions or through an attribute.
3. Discuss support for object as the TKey type on deserialization, should we support it in this enhancement? object is treated as a JsonElement on deserialization and is not part of the supported types on the Utf8Parser/Formatter. Consider to defer it when we add support for intuitive types (parse keys as string, etc. instead of JsonElement).