protobuf

protobuf‘s Python implementation has been known for its slowness, but that might be changing. From a 2010-11-01 changelog:

  Python
  * Added an experimental  C++ implementation for Python messages via a Python
    extension. Implementation type is controlled by an environment variable
    PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION (valid values: "cpp" and "python")
    The default value is currently "python" but will be changed to "cpp" in
    future release.
  * Improved performance on message instantiation significantly.
    Most of the work on message instantiation is done just once per message
    class, instead of once per message instance.
  * Improved performance on text message parsing.

– http://code.google.com/p/protobuf/source/detail?r=349

Also, if you like Protocol Buffers and JSON, check out Protojson.

Google’s Protocol Buffers serializes to and deserializes from a compact binary format. As of this writing, there’s an open ticket for JSON serialization support. JSON is very useful when transporting data to web applications, or if you want human-readable bytes on the wire.

I recently wrote Protojson, a protobuf Message<->lists encoder/decoder in Python. You can get it at github.

Protojson requires the google.protobuf Python module, because it works with google.protobuf.message.Messages. Right now, Protojson only supports the PbLite (PbJsLite) format. You can use Protojson to send and receive Messages from web applications with Closure Library’s goog.proto2, or even use it for non-webapp purposes. If you’re interested in goog.proto2, see this thread, which links to a .proto->.pb.js compiler.

I haven’t benchmarked Protojson, but I wouldn’t be surprised if Protojson+simplejson was faster than the binary serialization (at least for Python).

Try it out and let me know how it works.