How to Scale Application to support Millions User
Designing a system that supports millions of user is a process that requires continuous improvement and refinement. In this article, we will discuss about high level system architecture for single user first and then scale our system to millions of user.
Single Server Setup
Let’s start from basic setup where everything is running on just only one web-server: web app, database and cache.
Here, User request website using its domain name url to DNS and in return it gets an unique IP address where our website is hosted on the web. With IP address it makes a HTTP request for content to server and then Server makes request to Database to perform CRUD operation and in return get data and then server send a response to our client in HTML page or JSON Object.
But there is one problem here, with growth of user base, one server is not enough to entertain all the request coming from client side and as well as if our server stops running for any of the reason, our whole system is going to down which is not ideal for any business. To solve this problems, we need to scale our application architecture.
Vertical Scaling vs Horizontal Scaling
Vertical Scaling is known as scale-up which means we can increase our CPU power or increase in our RAM for our server.
Horizontal Scaling is known as scale-out which allows you to scale by adding more server into our pool.
Vertical Scaling is great option when traffic is low and it is easy to achieve but it comes with very serious limitations.
- It is impossible to add unlimited CPU power and RAM to any single server and adding CPU power and RAM to server is costly too at the same time.
- Vertical Scaling does not support any kind of failover to our server. If our only server goes down, the whole system down with it completely. [In 2013, StackOverFlow goes down due to this problem]
Horizontal Scaling is more desirable for massive user base due to limits of Vertical Scaling.
In the previous design, User will be unable to access the website if web-server is offline or many users access the website at the same time, There is very high chance that user will get slower response and when server reaches load limit it may stop working and our system can go down completely.
So in Horizontal Scaling, Now we have added more server in our pool and Introduce Load Balancer between our application and web-server pool. Here, Application connects with Public IP that we get from DNS and then Load Balancer connect to our server with private IP, which can not be accessible outside of our system and then will route the request to our server. There are several strategies and algorithm that Load Balancer uses to route the request to particular server. Round Robin is one of the popular one that it uses to give request to server. So now, Even any of the server fails due to any reason, Load Balancer will identify it and sends request to another server and also if our website traffic grows rapidly and three server are not enough to handle traffic, all we need to do it just add one more server in our pool and Load Balancer will handle it gracefully.
Now our web-tier looks good, what about data-tier? Right now we just have one database that supports whole website, we also need to scale up otherwise when it reaches its load limit or fails for any reason our system can go down entirely. Database Replication is one of the common technique to address those problem.
Database Replication is used to scale database application using master-slave system. Here, master is our main database for that server and slaves are copies of the main database.
In this system, master database can only be used to perform write operation such as create, update or delete. and slave database is used for read operation. Because in any system, read operation ratio is much higher than write operation.
Advantages of Data Replication
Better Performance:- in master-slave model, by separating read and write operation, more queries can run in parallel and it gives better performance for our system.
Reliability:- If anyone database server destroyed by any reasons such as flood, earthquake or any terrorist attack, your data is still available and safe because we have multiple copies of the database across multiple locations.
If one of our slave database goes offline, all the read operation can be transferred to another slave database.
But now main question is what if our master database goes offline due to any reasons??? In this case, a new healthy slave database can be prompted to master database and acts as main database for temporary or permanent based on condition of destroyed previous master database. But there is one more problem here, our new master database might not be up to date. in this case, we have to run data recovery script. This is challenging so in industry we can have other replication method like circular replication or multi-master set up can help and it is complicated to achieve.
Now our web-tier and data-tier looks awesome. In the next article, we will understand how we can improve website load/response time by adding Cache and CDN layers.
