System Design Fundamentals: Understanding Scalability
Scalability explained in simple words
Introduction
Recently, Reliance Jio witnessed more than 30 million concurrent users while streaming the IPL(Indian Premier League) final. For those of you who don’t know IPL, its similar to premier league for cricket in India. Indians are crazy about cricket and often streets are deserted during interesting matches.
Reliance Jio streaming platform telecasted the match seamlessly to 30 million users. Their engineering team deserves kudos for the effort. Moreover, the cloud platform also played a significant role in ensuring successful live streaming of the match.
Over the past two decades, internet companies have simplified our lives. Feeling hungry ? Open the food delivery app & order your favourite. Want to shop for an item ? Go to Amazon, find the item and place the order. Feeling bored ? Start Netflix and binge watch your favourite series.
We have everything at the click of a button. As the internet penetration increases, more users across the globe will have access to the internet products. With the user growth, the amount of data will also grow by leaps and bounds.
We don’t frequently face issues while using these products. We are able to shop on Amazon during prime day sales, order food during New Year’s eve, watch Netflix during weekends when most of the people watch simultaneously. Even with the user growth, these platforms deliver us high quality experience.
How are these companies able to do this ? The answer is Scalability. These companies ensure that their platform scales with the user & data growth.
In this article, we will understand what Scalability is, & why it is important. We will also look at the different approaches to scale the system.
What is Scalability ?
Scalability is the ability of the software system to increase capacity to meet the increase in the demand. Every software system is unique and each has it’s own requirement. Scalability for a software system depends on the dimensions in which the system operates.
Let’s take an example of a real-time chat application such as WhatsApp. The challenge in building such a system is to meet the growing number of users who are online simultaneously. Also, it must handle events such as festivals where people share and forward a lot of messages. For WhatsApp, the system is scalable if its able to handle large number of concurrent users and messages from the users.
Companies like Youtube, Instagram handle large number of uploads throughout the globe. They need to persist the user content indefinitely. Further, users also watch the videos regularly on the platforms. The scalability challenge for such companies is the amount of data that they manage and process. Videos are large in size and the systems need to handle increasing amount of videos uploaded everyday.
We can classify a system’s scalability depending on it’s operational dimension as follows :-
Handle increasing number of requests from the users simultaneously.
Process and store large amount of data.
Keep the response time within a threshold with the increasing load.
Why is Scalability important ?
In today’s digital age, majority of the companies are driven by technology. The digital acceleration is taking place at a rapid pace. Companies are rapidly adopting technology, innovating and taking data driven decisions.
During the industrial revolution, manufacturing work used to take place in the factories. To meet the increasing demands of customers, companies had to increase the production. They achieved this by employing more people on the assembly line. If one person could pack one item, employing ten people would result in 10x output.
In today’s world, there is no human intervention and most of the applications are powered by computers. When you shop on any e-commerce website, the webpage showing the product is being fetched, processed and rendered by some remote computer. Behind the scenes, there is a lot of complexity and millions of API invocations are involved in showing the customers the right experience.
Imagine that you are shopping on an e-commerce website and the page takes more than a minute to load. What would be your reaction ? Below, was my reaction when I was trying to open a website and it didn’t load even after 5 minutes.
As customers we want quick response from the applications that we use. Any delays would result in poor user experience and frustrate the customers. They would search for alternatives and think twice before opening your website again. As a result, the website won’t be able to retain the users. This would result in fall in the business’s revenue numbers.
In today’s internet age, its easy to start a business online and acquire customers. Customers prefer companies that provide them the right value and get their work done. So, as the business acquires more customers, it would also experience a proportionate load on it’s systems. It’s important for a company to invest in scalable solutions to meet the increasing user demand.
How to build scalable systems ?
There are multiple ways to build scalable systems. We will discuss the two techniques that are commonly used to scale a system.
Horizontal scaling
Horizontal scaling is also known as scaling out. In this technique, additional computing resources are added to handle the growing load. We achieve scalability by adding more machines. Let’s understand this in detail.
Let’s say your system is able to process 10 requests/second. And there is a requirement to handle 30 requests/second. One way to meet this demand is to add 3 instances of the same service. Also, adding a load balancer would help distribute the traffic evenly among the 3 instances.
If the capacity of your system is to store 10 TB of data, and you are expecting to store 30 TB, then you will add 3 database instances each having a capacity of 10 TB. So, the overall capacity of your system will be 30TB. And it will be able to store the additional data.
With Horizontal scaling, the system also avoids a single point of failure. If one machine fails, the work will be done by another machine. The downside of horizontal scaling is the cost associated with adding more machines.
Horizontal scaling is analogous to adding more workers on the assembly line to increase the production.
Vertical scaling
In this approach, we add more resources such as CPU, memory or hard-disks to the same machine. Vertical scaling improves the performance of the system & leaves the system with additional capacity to handle the increasing load. It is also known as scaling up.
Let’s assume you are able to process 10 requests/second on your machine with 1 CPU core. If you add 4 cores to your machine, it will be able to process 40 requests/second. This will be cheaper than buying, deploying and managing software on new machines.
There is an upper bound to vertical scaling. We can’t add infinite number of CPU cores, or increase the RAM size to TBs. At one point, vertical scaling will hit limits and we need to horizontally scale the system.
A simple analogy of Vertical scaling is that of introducing more machines in the factory. The output will increase with more number of machines. However, the factory won’t accommodate infinite number of machines. At some point, you would have to start a new factory at some other place.
Technologies for scalable systems
Cloud providers such as AWS, Azure and GCP have built multi-tenant systems that are scalable. They provide different offerings such as IaaS (Infrastructure as a Service), and PaaS (Platform as a Service) which customers can leverage to build scalable systems.
Most of the cloud providers offer auto-scaling capabilities. With such offerings, customers can specify how to scale the system i.e use better hardware or add more machines to handle the additional load. As the traffic increases, the customer’s workloads are scaled up automatically. Once the traffic reduces, the workloads are scaled down to save the costs.
Businesses can also use the managed database offerings such as DynamoDB, Azure CosmoDB, etc for storing the data. Amazon’s DynamoDB serves 10s of millions of customers at the same time throughout the globe. The database systems are elastic and can be configured as per the business needs.
Dockers allow software to be run in any environment and provides OS-level virtualization. Most of the companies use docker containers for deploying their workloads. Similarly, Kubernetes is used as a container orchestration engine to manage the containers. These solutions can also be adopted to improve the availability and scalability of the system.
Summary
In this article, we understood the importance of Scalability for Internet businesses. It enables businesses to grab the opportunities for growth and expansion. It also impacts the customer experience.
We can build scalable systems by either scaling up or scaling out. Each of the approaches have their own pros/cons. We need to choose the solution that works best for the use case.
Scalability is closely associated with the business costs. Harnessing more computing resources adds to the costs. However, by adopting cloud technologies, businesses don’t have to worry about the costs anymore. They only need to pay for what they use and can take advantage of capabilities such as auto-scaling.
References
Thanks for reading the article! Before you go:
You can share your thoughts on the article in the comments below.