Msgspec

Recently I explored the msgspec library. I used an RRWeb-structured input as my testing data. The size of the input was roughly three megabytes. These were my findings.

Performance

When deserializing without a schema, msgspec performs similarly to orjson. It performs significantly better than the standard library's implementation by about 2 times. When deserializing with a schema (struct definition), msgspec performs 2.5 times better than the standard library and 30% better than orjson. What's particularly amazing about this fact is that you can traverse an object tree with certainty that it is defined exactly as you specified in the type. You get the rewards of type checking in addition to a significant performance boost.

Msgspec can ignore fields when deserializing in exchange for a performance speed up. Specifically fields you do not define on the struct. In my case, I ran an experiement where I did not care about the serialized HTML. The performance of the deserializer was 10 times faster than the standard library's implementation.

Usage

Msgspec is a very pleasant library to use. Define a type and go. For my case, I essentially copy-pasted the RRWeb types from the Typescript source library and converted them to Python types. After applying a few msgspec-specific requirements to the schema I was rewarded with a performant RRWeb deserializer.

Optimizing deserilaization was as easy as commenting out (or removing) fields I did not care about. I did not have a long tail of debugging problems. Every issue I had with msgspec was caught at "compile-time". By which I mean the program would fail because it was not valid Python. Once Python was happy, the deserializer worked immediately on a large, extremely complex, and recursive data type.

Conclusion

Msgspec is powered by twenty-two thousand lines of C code exposed through a declarative, type-driven interface. It delivers succintly on its promise.