In many ways, this is a golden age for web developers: we have a bunch of good, high-level frameworks for writing apps in highly-productive dynamic languages and a solid corpus of best practices for testing, service API design, and data serialization. We don’t have to deal with dog-slow CGI scripts, complicated J2EE stacks, or proprietary ColdFusion code that only runs atop expensive application servers.
Unfortunately, all is not wine and roses (or scotch and bacon, or whatever). The major dynamic webapp frameworks push you by convention into doing the bulk of your application work syncronously in the request-processing loop, rather than asynchronously in a background thread. All of the accumulated wisdom about building responsive graphical user interfaces gets thrown out and re-discovered by each framework’s user community, resulting in a multitude of solutions for the basic problem of pushing work into a queue and dealing with it later.
As the fine folks at Twitter so famously discovered, synchronous processing puts a hard upper limit on how much (and how quickly) you can scale an application. Even at the much more modest loads my current project at work receives, there are quite a few performance problems that can’t be solved by simply throwing more stuff in memcached and hoping for the best.
Some folks are starting to catch on, and bake asynchronous processing into their frameworks by default, but the solutions tend to either be limited to very particular deployment and application models, or esoteric in the extreme. Meanwhile, desktop application authors continue to politely chuckle at all of our bumbling, and old-skool enterprise developers look at our hackish background-worker implementations and (rightly) consider them to be toys compared to the classic “big boy” message queueing solutions, or even the newer open source alternatives.
The next generation of web application frameworks should be designed around the idea that work is done asynchronously by default, with a fallback to syncronous jobs only in cases where a user needs to see the result immediately. Since applications also need to scale across a potentially large and heterogenous set of CPUs and servers, those delayed jobs also may not be running in the same memory space as the web application itself. That means machine and language-agnostic serialization, fast network IPC, and callback and event-driven programming.
Developers who grok these concepts now will have a leg up on the competition when building tomorrow’s crop of web applications.
Excellent point. My feeling is you’re onto a trend that a growing number of people are just starting to address. It appears an ‘asynchronous by default’ framework is easy to build incorrectly however. I wonder how many will have to land with a thud before something really catches on?
Good post! We’ve been using Beanstalkd for quite a while now with Rails, anything that can be done asyncronously we push it into the queue, it’s quite reliable with some monitoring to make sure everything is working as it should.
There are actually two levels at which easily approachable asynchronous processing in web apps would be (or already is) a major game changer.
The first one, as you described, is concerned with throwing jobs into a queue and processing them behind the scenes.
The other one is low-level and is concerned about how the actual request is being processed by the application and application server. This is where grizzly, grizzly comet (atmosphere) and JSR 315 (Servlet 3.0) come into play.
The future is definitely asynchronous. It’s in fact surprising that it took us relatively long time to figure that out.
Um, no. As anyone that’s programmed threads or lots of async stuff can tell you, it introduces lots of complicated problems. Sure, you can deal with them, but it makes apps much more complex conceptually on many levels.
I will certainly reserve the right to change my mind if someone comes up with an async framework that doesn’t make synchronous workflow hard and/or makes it really easy to manage things that happen in unpredictable order.
However, I think that problems like race conditions, out-of-order processing, and producing a “synchronous-feeling” front-end with an asynchronous model is much harder to deal with both mentally and in actual code than doing things synchronously.
Alan
Amen, brother. Some similar stuff here: http://code.flickr.com/blog/2008/09/26/flickr-engineers-do-it-offline/
@Alan: you’re absolutely right that threads can introduce a lot of problems. Erlang, Clojure, and other functional languages point the way towards a more sensible future, though. Removing shared state from the picture makes a lot of things possible that would be difficult or impossible in a stateful system.
I was just talking with another developer about this the other day. We were chatting about how current frameworks and servers handle large request and response bodies. Most of them do way too much buffering at each phase that it adds extra latency into the process.
For example, many frameworks will not handle a 1 GB request to a non-existent URI very efficiently. They will read the entire response, buffering it (sometimes even in memory!) and then process the request only to find there is no end-point and returning a 404.
A better approach would be to read the stream asyncronously and figure out if there’s an end-point matching the request line (the first line of the request) failing fast with a 404 if not. As each header is read it should be figuring out if it can satisfy the response before doing anything else — for example maybe the User-Agent sends an Accept header for something the end-point can’t satisfy. Rather than reading in even one more byte it should return a 406 Not Acceptable immediately.
Things are similarly poor for responses. We shouldn’t be buffering the entire response in memory — as each chunk of the template is rendered, send it out the wire.
I’m not sure why this isn’t the norm, but it would probably result in a modest speed improvement, because we could begin handling the request immediately as the first few bytes are received, rather than waiting for the last byte to be received.
@Dan
CouchDB provides a framework for doing just that. There’s never a good reason to buffer in process, especially when you could have millions of rows. CouchDB provides a way to render view rows as HTML (or other content-types) one row at a time, so that the amount that can be buffered is limited to the HTML rendered by a single row. The rendering functions also cannot have any side effects on the CouchDB database.
Bonus points! You get to write your server code in JavaScript. My blog (linked from my name) is running on just a copy of CouchDB, rendering the HTML etc. There’s no reason it would have trouble, even rendering an Atom feed with millions of items in it.
CouchDB has an interesting background-job model as well. Interesting in that *you can’t do anything except update documents*, but you can subscribe to events generated by updates, and of course your subscribed process could create more updates. Working with documents as state-machines makes asynchronous processing almost child’s play.
@Chris: CouchDB has one of the better stories out there right now for single-node document storage and processing, and even for loosely-coupled replicated storage across a modest number of nodes. The piece I don’t see being demonstrated at the moment is the option to spread computation and I/O across a bunch of machines, which is a much harder problem.
There’s also the question of what happens when you need to talk to some sort of stateful external component. Could I maintain the same smooth scaling of a CouchDB-based application when every third or fourth request triggers a call into a painfully slow network service that may take 2-3 seconds to return a result?
Even when the data might possibly live in one place, there are still messy legacy data and protocols to deal with — not everything can be trivially lifted into the JSON-over-HTTP worldview. I want something that can work with existing, imperfect subsystems like relational tables, Java libraries, and UNIX command-line scripts.
It seems like something is missing, no?
pretty accrate