Pinterest is a Big Lesson on Scalability

Pinterest is a Big Lesson on Scalability – Pinterest is one of those startups who to had to deal with an exponential growth of 10 billion page views. Going from 0 to 10 billion page views in less than 2 years needed to redesign their infrastructure several times before achieving full stability. Previously we wrote about the topic Scalability in Cloud Computing. Also, it is must to be mentioned that; Pinterest is the perfect practical example of what we wrote before – Scalability and Service Continuity Are Not Equivalent in Cloud Computing.

Pinterest, Scalability and Cloud Computing

Pinterest uses cloud computing technologies and open source softwares to manage more than 10 billion page views every day, with all the problems involved in managing data the huge data, their backup and redundancy of the same, the points should not be underestimated when it comes to cloud computing. The initial infrastructure in 2011 included :

1. Amazon EC2 + S3 + CloudFront
2. Nginx setup for redundancy, not really for load
3. 1 Master MySQL DB + 1 Slave (in case the master goes down)
4. 1 Task 2 Task Queue
5. 1 MongoDB (for counters)
6. 2 Engineers

Pinterest is a Big Lesson on Scalability

At a time when the website started crunching numbers to be growing, the team had to work by setting up a sort of emergency solution made up of many different technologies, NoSQL databases and open source software which in fact has complicated the management of the infrastructure, all while remaining within the Amazon AWS for everything. The final solution has taken into consideration that they always the use Amazon EC2 and S3, with the use of multiple availability zones. Ultimately it has resulted in a further growth of the infrastructure, which was in late 2012 composed of:

1. Amazon EC2 + S3 + Edge Cast, Akamai, Level 3
2. Ngnix + 240 180 Web API Engines
3. 88 MySQL DBs + 1 slave for each
4. 110 Redis Instances
5. 200 Memcache Instances
6. Redis + 4 Task Processors
7. 40 Engineers

Pinterest said how they learned to manage the infrastructure gradually, it is clear that increasing the complexity of the infrastructure is more difficult to intervene properly. The choice of MySQL is interesting: the team of Pinterest seems to have preferred the most stable and well known components of the infrastructure, for which goes “it is easier to find educated engineers”, for many technologies as Redis, which are younger, then it is difficult to find people who have already operated very large infrastructure and thus have the lack of necessary experience.