Minerva is a robust Comet server and client. You can use it to build web applications that need a socket-like abstraction. Your application simply needs to send and receive strings. The Minerva server is built on top of Twisted. The JavaScript client uses Closure Library.

Some uses for Minerva: chat applications, games, real-time collaboration, dashboards, monitoring systems.

Update 2013-07-11: Minerva still works, but it is no longer updated. There is some minimal documentation (tutorial files and DemosMinerva) for the curious. We recommend using WebSocket, RTC Data Channels, Flash socket fallback, and/or a reliable Comet protocol like Google's BrowserChannel. It's important to use a Comet implementation that guarantees in-order delivery of all data using ACKs in both directions.

View demos Tutorial (soon!) On GitHub Bug tracker

Jump to: Highlights • Disclaimers • FAQ • Contact

Highlights

• Reliable and in-order message delivery over HTTP. Both client and server use TCP-like ACK logic to ensure reliability. Minerva doesn't make these incorrect assumptions:
  • If an HTTP response is sent, it will be received by the client.
  • If an HTTP response is received, it will be fully received.
  • If HTTP requests A and B are made, the server will see A first.
• Support for many connections per domain: To work around browsers' per-domain active request limits (usually just 4-6 requests), Minerva can send HTTP requests to random subdomains. Your web application will work as usual when you open it in a dozen tabs. (For HTTPS, a wildcard cert is required.)
• Serialization-agnostic: You're free to use JSON, Base64, or your own serialization format. And there's no double-serialization to get in your way.
• Cross-browser: Minerva works in old and new versions of IE, Firefox, Chrome, Safari, and Opera. We aim to keep Minerva working in every JavaScript runtime that can make HTTP requests.
• Robust timeout logic: Stuck requests and sockets are detected and aborted.
• Licensed under Apache License, Version 2.0.

Disclaimers

Go ahead and try out Minerva because it should work well, but keep in mind that right now:

• The API isn't stable yet. It will be soon, after some initial feedback.
• There's no cross-domain support yet. Right now, website.com can only connect to a Minerva server at *.website.com. If you really want cross-domain support, you can use Closure Library's cross-domain support (goog.net.xpc) combined with the Minerva client in an iframe.
• If you don't want to use Twisted or Closure Library, you'll have to do some legwork. We'll soon have better support for non-Twisted and non-Closure Library users.
• Despite HTTP streaming and Flash sockets working, there's no automatic negotiation between HTTP long-polling/HTTP streaming/Flash sockets yet. For Internet-facing applications, use just long-polling for now.

FAQ

What about WebSocket?

Minerva's HTTP transport (which uses XHR on the client) means that if a user can load your website, they can also connect to your Minerva server, as long as they have JavaScript enabled. This isn't the case with WebSocket, because (1) browser support is still poor (2) even in browsers that support it, many networks make WebSocket connections impossible. Google reported success rates of 67% (port 80) to 95% (port 443).

As far as we can tell, there's no good reason to implement a Minerva transport that uses WebSocket. Your web application can just use Minerva as a fallback for when a WebSocket connection fails. Alternatively, it can use Minerva as the initial connection method, for potentially faster connection setup. We'll soon document some ways to use both Minerva and WebSocket.

Note that WebSocket is vastly superior to HTTP transports in this use case: sending small messages frequently from the client to the server. If you use HTTP transports for this, the client wastes bandwidth by sending the same HTTP headers for every message. But for most applications, Comet-style HTTP transports work fine.

Do I need to use Twisted?

For now, yes. In the future, we might have an TCP gateway and a WebSocket gateway. These will allow you to use Minerva's reliable HTTP transports without writing any Python/Twisted server code; you will just specify a TCP/WebSocket server for Minerva to proxy to.

How many simultaneous connections can the server handle?

Somewhere around 10,000-20,000 per Minerva server. If you run your own benchmarks, remember to: (1) select the best Twisted reactor for your OS (epoll on Linux) (2) think about TIME_WAIT sockets: they might clog up your benchmark (3) try PyPy.

Why not 100,000 or more? Minerva was built for correctness, not for maximizing connection count. There are three mitigating factors:

1. You can run a Minerva server for every CPU you have.
2. Minerva works on PyPy, and it'll keep getting faster.
3. WebSocket will soon work for more users, and you can bypass Minerva entirely for your WebSocket connections.

The client uses Closure Library. Does this use Google's BrowserChannel?

No, but the Minerva protocol does solve the same problems as BrowserChannel.

I'd like to read the code. Where do I start?

The guts of the server are implemented in mserver.py and the client in client.js. Consider reading it in IDEA so that can jump to function definitions quickly. Your understanding may be assisted by a sketch of how Minerva works, the frame types used by the Minerva protocol, and by playing with /chatapp/?mode=http on the minerva_site server. Understanding the separation between the stream and the transports is critical. You're welcome to ask questions about how Minerva works, or why it works a certain way.

Why are Minerva strings restricted to 95 codepoints (" " 0x20 to "~" 0x7E)?

This design decision simplifies Minerva and helps keep the API stable if new transports are added in the future. The codepoint range doesn't seriously impact most applications: you can use JSON (encoded as ASCII-only), or Base64, or even higher Base* encodings. This decision is left to you.

Supporting more than this codepoint range would make Minerva more fragile to changes in the transports it uses. For example, XMLHttpRequest can't receive 0x00 in IE and Opera. XMLHttpRequest's readyState 3 and Unicode often don't mix (especially in older Firefox versions). XDomainRequest (IE8+) replaces a large number of non-Character Unicode codepoints.

What other software has any kind of Comet implementation?

Free: Lift's Comet support, Divmod Nevow's Athena (homepage dead), Atmosphere, gwt-comet, Socket.IO, nginx_http_push_module, Meteor, Perservere's Comet support, gwteventservice, Orbited, HookBox, js.io, APE (Ajax Push Engine), DWR (Direct Web Remoting), ICEpush, Google's BrowserChannel (several servers available).

Commercial: GAE's Channel API, Migratory Push Server (supports very high connection counts), StreamHub, WebSync (.NET), Liberator, Kaazing, Lightstreamer, ZK's push support.