Some uses for Minerva: chat applications, games, real-time collaboration, dashboards, monitoring systems.
Update 2012-06-27: Documentation and PyPI packages are coming; please bear with us.View demos Tutorial (soon!) On GitHub Bug tracker
- Reliable and in-order message delivery: Both client and server attach sequence numbers to messages and send/receive ACKs. Messages never get lost or re-ordered, even when the HTTP transports are used.
- Serialization-agnostic: Your application isn't forced to use JSON, XML, or anything else.
- Cross-browser: Minerva works in old and new versions of IE, Firefox, Chrome, Safari, and Opera. Minerva disables the page-loading spinner in all browsers.
- Multiple transports: Minerva runs over HTTP long-polling, HTTP streaming, and Flash sockets. Its design easily supports new transports.
- Robust timeout logic: Stuck requests and sockets are detected and aborted.
- Handles tab addicts: To work around browsers' per-domain active request limits (usually just 4-6 requests), Minerva can send HTTP requests to random subdomains. Your web application will work as usual when you open it in a dozen tabs. If you want to use this on your HTTPS site, a wildcard cert is required.
- Licensed under Apache License, Version 2.0.
Mailing list and IRC
If you want to subscribe without a Google account, send an email to email@example.com; you should receive a confirmation within ~10 minutes. Then, send your message to firstname.lastname@example.org. To unsubscribe, email email@example.com.
The IRC channel is
##minerva (two hashes) on
irc.freenode.net. You may want to stick around for a day or two
(or just use the mailing list). Logs will be available later. If you have a
Twisted-related question, try asking in
Go ahead and try out Minerva because it should work well, but keep in mind that right now:
- The API isn't stable yet. It will be soon, after some initial feedback.
- There's no cross-domain support yet. Right now, website.com can only connect to a Minerva server at *.website.com. If you really want cross-domain support, you can use Closure Library's cross-domain support (goog.net.xpc) combined with the Minerva client in an iframe.
- If you don't want to use Twisted or Closure Library, you'll have to do some legwork. We'll soon have better support for non-Twisted and non-Closure Library users.
- Despite HTTP streaming and Flash sockets working, there's no automatic negotation between HTTP long-polling/HTTP streaming/Flash sockets yet. For Internet-facing applications, use just long-polling for now.
What about WebSocket?
As far as we can tell, there's no good reason to implement a Minerva transport that uses WebSocket. Your web application can just use Minerva as a fallback for when a WebSocket connection fails. Alternatively, it can use Minerva as the initial connection method, for potentially faster connection setup. We'll soon document some ways to use both Minerva and WebSocket.
Note that WebSocket is vastly superior to HTTP transports in this use case: sending small messages frequently from the client to the server. If you use HTTP transports for this, the client wastes bandwidth by sending the same HTTP headers for every message. But for most applications, Comet-style HTTP transports work fine.
Do I need to use Twisted?
For now, yes. In the future, we might have an TCP gateway and a WebSocket gateway. These will allow you to use Minerva's reliable HTTP transports without writing any Python/Twisted server code; you will just specify a TCP/WebSocket server for Minerva to proxy to.
How many simultaneous connections can the server handle?
Somewhere around 10,000-20,000 per Minerva server. If you run your own benchmarks, remember to: (1) select the best Twisted reactor for your OS (epoll on Linux) (2) think about TIME_WAIT sockets: they might clog up your benchmark (3) try PyPy.
Why not 100,000 or more? Minerva was built for correctness, not for maximizing connection count. There are three mitigating factors:
- You can run a Minerva server for every CPU you have.
- Minerva works on PyPy, and it'll keep getting faster.
- WebSocket will soon work for more users, and you can bypass Minerva entirely for your WebSocket connections.
The client uses Closure Library. Does this use Google's BrowserChannel?
No, but the Minerva protocol does solve the same problems as BrowserChannel.
HttpFace uses twisted.web. Should I expose
twisted.web to the Internet?
twisted.web generally works well, but you probably don't want to expose it to the Internet (though many do). Consider putting it behind nginx (or similar) for three reasons:
- nginx has better support for unusual/old/broken HTTP clients.
- nginx has better HTTPS support: SSL sessions are cached, and compatibility options are enabled.
- nginx is a lot harder to DoS than twisted.web.
I'd like to read the code. Where do I start?
The guts of the server are implemented in newlink.py and the client in client.js. Consider reading it in IDEA so that can jump to function definitions quickly. Your understanding may be assisted by a sketch of how Minerva works, the frame types used by the Minerva protocol, and by playing with
/chatapp/?mode=http on the
minerva_site server. Understanding the separation between the Stream and the transports is critical. You're welcome to ask questions about how Minerva works, or why it works a certain way.
Why are Minerva strings restricted to 95 codepoints (" " 0x20 to "~" 0x7E)?
This design decision simplifies Minerva and helps keep the API stable if new transports are added in the future. The codepoint range doesn't seriously impact most applications: you can use JSON (encoded as ASCII-only), or Base64, or even higher Base* encodings. This decision is left to you.
Supporting more than this codepoint range would make Minerva more fragile to changes in the transports it uses. For example, XMLHttpRequest can't receive 0x00 in IE and Opera. XMLHttpRequest's readyState 3 and Unicode often don't mix (especially in older Firefox versions). XDomainRequest (IE8+) replaces a large number of non-Character Unicode codepoints.
What other software has any kind of Comet implementation?
Free: Lift's Comet support, Divmod Nevow's Athena (homepage dead), Atmosphere, gwt-comet, Socket.IO, nginx_http_push_module, Meteor, Perservere's Comet support, gwteventservice, Orbited (homepage dead), HookBox, js.io, APE (Ajax Push Engine), DWR (Direct Web Remoting), ICEpush, Google's BrowserChannel (no server available).
We don't endorse any of these, but many have interesting implementations and features.
Ivan Kozik: firstname.lastname@example.org. Consider mailing the mailing list for most questions.