Easypromos: Tricks to improve your LAMP servers scalability
Easypromosapp is Facebook application with more than 500.000 monthly active users and therefore must achieve a high level of scalability in order to serve a huge amount of requests from numerous simultaneously active promotions.
As there are many large promotions in the system, each with a time limit, there is the possibility that the server might become overloaded if it is not dimensioned and prepared for scaling.
The most important way to achieve high scalability is to optimize the application. You must find bottlenecks early and make swift decisions about caching commonly used resources. It’s very important to optimize the most-used URLs, even generating static files when necessary.
Another important aspect is the base software stack; we use a LAMP stack with some special features to assure a good response time and the least possible resource consumption.
Basic software stack:
The basic software stack is obvious and easy to install and configure.
Note: It’s important with Apache to know the limit of processes that your server can reach before it starts thrashing. We’ve observed, for example, that a typical dual-core server with 4Gbs of RAM and the full stack (Apache+PHP+MySQL) installed in a single instance, can support up to 600 Apache processes concurrently (MaxClients 600) before collapsing. This means that if you are reaching that limit and queuing requests, then you should move quickly to share services with other servers and start a load balancer in order to serve requests from more than one server at a time.
Our particular stack is not rocket science, but we can give you some guidelines to help increase your scalability, speed and monitoring if you add some pieces to the standard configuration.
Memcache is a simple, but powerful, memory cache. You can store and retrieve objects with a simple API and reserve a fixed amount of server memory to maintain the objects according to LRU.
Depending on the PHP framework you are using, or whether you use an O/R Mapping tool, or make standard MySQL calls in your simple PHP, you can use this tool in a relatively automated way to keep a high speed cache of database objects (or even html pieces) in memory and serve pages without worrying about MySQL at all.
Lighttpd is a very fast and scalable web server and is known for its simplicity and low resource consumption. We use Lighttp to serve static content from different subdomains to speed up the downloading of the whole page. This technique is usually called “Client optimization” or “Browser optimization”. We use 10 subdomains (for example: i0.easypromosapp.com, i1.easypromosapp.com, …, i9.easypromosapp.com), all served by the same lighttpd instance. You can create a hash function to serve each static resource from one of its subdomains; a good example is the last digit of the URL: this maps each URL uniquely with the same subdomain.
/css/all.css –> length=12 –> i2.easypromosapp.com
/js/jquery.min.js –> length=17 –> i7.easypromosapp.com
This is very useful for an important reason: modern browsers launch 5 concurrent http connections for each different domain found on a page (including subdomains) this means
Eaccelerator is a powerful cache system that precompiles your PHP code and keeps a copy of this compiled code in memory to avoid doing the same repetitive task each time a request is made. It’s extremely fast and even with the standard parameters you can see an increase in the loading speed of your PHP pages right away. The best thing is that it’s totally transparent to the developer and works with lots of standard frameworks.
Simply install the PHP extension and you will see the changes for yourself.
Monit is a straightforward monitoring tool that allows you to establish simple rules and warns you via email when any resource limits are reached. It’s capable of even more, also allowing you to restart failing processes, as well as many other options, but we only use it as a resource monitoring and alerting system.
With this simple tools you can improve the scalability of your site with very little effort and at no cost.