Scale PHP on Ec2 to 30,000 Concurrent Users / Server
RockThePost.com is a LAMP stack hosted on Ec2. We're preparing to be featured in an email which will be sent to ~1M investors... all at the same time. For our 2 person engineering department, that meant we had to do a quick sanity check to see just how many people we can support concurrently.
Our app uses PHP's Zend Framework 2. Our web servers are two m1.medium Ec2 machines with an ELB in front of them setup to split the load. On the backend, we're running a master/slave MySQL database configuration. Very typical for most of the small data shops I've worked at.
Here are my opinions and thoughts in random order.
Use PHP's APC feature. I don't understand why this isn't on by default but having APC enabled is really a requirement in order for a website to have a chance at performing well.
Put everything that's not a .php request on a CDN. No need to bog down your origin server with static file requests. Our deployer puts everything on S3 and we abs path everything to CloudFront. Disclaimer: CloudFront has had some issues lately and we've recently set the site to instead serve all static materials directly from S3 until the CloudFront issues are resolved.
Don't make connections to other servers in your PHP code, such as the database and memcache servers, unless it's mandatory and there's really NO other way to do what you're trying to do. I'd guess that the majority of PHP web apps out there use a MySQL database for the backend session management and a memcache pool for caching. Making connections to other servers during your execution flow is not efficient. It blocks, runs up the CPU, and tends to hold up the line, as it were. Instead, use the APC key/value store for storing data in PHP and Varnish for full page caching.
I repeat... Use Varnish. In all likelihood, most of the pages on your site do not change, or hardly ever change. Varnish is the Memcache/ModRewrite of web server caching. Putting Varnish on my web server made the single biggest difference to my web app when I was load testing it.
Use a c1.xlarge. The m1.medium has only 1 CPU to handle all of the requests. I found that upgrading to a c1.xlarge, which has 8 CPU's, really paid off. During my load test, I did an apt-get install nmon and then ran nmon and monitored the CPU. You can literally watch the server use all 8 of its cores to churn out pages.
Google Analytics shows us how many seconds an average user spends on an average page. Using this information, we ran some load tests with Siege and were able to extrapolate that with the above recommendations, we should be able to handle 30,000 users concurrently per web server on the "exterior" of the site. For "interior" pages which are PHP powered, we're expecting to see something more along the lines of 1,000 concurrent sessions per server. In advance of the email blast, we'll launch a bunch of c1.xlarges and then kill them off slowly as we get more comfortable with the actual load we're seeing on blast day.
Finally, we will do more work to make our code itself more scaleable... however, the code is only about 4 months old and this is the first time we've ever been challenged to actually make it scale.
About RockThePost.com
We help entrepreneurs raise money for a new venture. If you're thinking about crowdfunding your next venture, use coupon code "HACKERNEWS" for a 50% discount on any of our plans.
Written by Jonathan Block
Related protips
9 Responses
Nice article Jonathan, it supports my own opinions.
@luceos Thank you!
ck2k on hacker news also added the following comment to this article which I thought was interesting.. he wrote:
.....
Stop using APC and switch to the "new" Zend Opcache.
It can serve 10-30% more requests per second than APC and has several layers of optimization.
It's bundled with PHP 5.5 but will work just fine with PHP 5.3 and 5.4
ps. also turn off file stat in the opcache on production servers
pps. make sure you are using PHP-FPM
Nice article!
I just want to ask a question: How do you "use the APC key/value store for storing data in PHP" instead of using a mysql database, can you provide a use case?
ntdunglc - Sure - lets say for example you want to display the list of the 50 states. It makes no sense to query your database for that. It's really slow. Put the list of the 50 states in APC with a high cache TTL. There probably won't be any new states joining the union any time soon. ;)
Thank you !
Nice and misleading title, you claim to power 30k concurrent connections on PHP then near the end you state it's on "exterior" pages which are either coming from a CDN or in your varnish cache.
Interesting article, but you say 'just use APC to store key-value data', how do you get it in there / manage it in the first place? What happens if you bring a new server up / need to restart one - surely APC isn't safe to use there?
While it certainly helps an enormous amount on a single server, APC (and all other in-process caches) isn't a scalable solution because it can't maintain cache coherence beyond a single host. As soon as you have more than one web server, you need to use something distributed such as memcached or redis.
For front ends, both nginx and haproxy are faster than varnish and will scale to higher connection counts. They don't necessarily do the same thing, but there is a lot of overlap for proxying, balancing, failover, caching, SSL unwrapping etc that they can make a better job of than varnish.
