No More App Engine

Since my last post, I've been busy setting up my own Python web development environment. Just how easy is it to get 500 request per second? This is just an overview of what I've accomplished so far, and I'll provide more details as I tune things up. Maybe I'll even release my code so that others can get started. I wanted to bring up a Python environment and a database, to be able to get a reasonable comparison with App Engine. This is my setup:

1. Apache 2.2 with the worker MPM.
2. MySQL 5.0 with the thread-safe client.
3. mod_wsgi running in embedded mode.
4. I'm using the MySQLdb python module.

I tuned Apache by adding more processes and threads to handle more connections. I upped the max_connections for MySQL for the same reason. Everything is running on the same machine. I ran two tests, one from the smallest EC2 instance (2.0Ghz CPU, 1.7G RAM) and the other from largest instance (8 3.0Ghz CPU's, 7.5G RAM).

Small instance driver: ab -n 100000 -c 500
Large instance driver: ab -n 1000000 -c 500

I ran the test from one EC2 instance to the other. The bandwidth between EC2 instances is pretty good, so it does push Apache. I did boost the large instance to -n 5000000 and -c 1000, and it handled it with no problem. I ran the test 3-4 times for each configuration, and averaged the numbers.

Here are my numbers:

Small instance: 1650, 1000, 780 Requests per second
Large instance: 4800, 4000, 3100 Request per second

1. The first number just returns a 'hello world' string.
2. The second number does a SELECT COUNT(*) on a large MySQL table.
3. The third number inserts a row into a MySQL table.

I was amazed by these numbers. I'm not that experienced with Apache (although I do know a little bit about MySQL), and yet I was easily able to blow away 500 requests per second even on a small EC2 instance.

The Google App Engine Hoax

I heard about Google releasing the paid quotas on App Engine the other day, and just got around to reading the details. I was horrified - HORRIFIED. App Engine is the biggest hoax ever perpetrated on unsuspecting developers.

For months now, I have been writing App Engine applications, waiting for the scalability that Google has been promising. The quotas up until now have been painful to deal with, but hope was right around the corner - right? When the paid quotas are released, I'll be able to pay for what I need - right? Wrong. Read the maximum request quota - 500 request per second. Is Google fucking joking? I can get better throughput on my laptop. Here is the "Automatic Scalability" paragraph straight from Google:

"For the the first time your applications can take advantage of the same scalable technologies that Google applications are built on, things like BigTable and GFS. Automatic scaling is built in with App Engine, all you have to do write your application code and we'll do the rest. No matter how many users you have or how much data your application stores, App Engine can scale to meet your needs."

Note to Google - remove this paragraph. It's a stinking lie. You have been misleading developers for months, leading us to believe that App Engine was the way to go, that all our scalability problems were solved. You're a bunch of fucking liars. 500 requests per second? That's "scaling is built in"? Ha, ha, ha, ha, ha, ha, ha, ha!!!!!!

Azure here I come.