How to Build a Scalable Application up to 1 Million Users on AWS

Suppose you’ve built a web application and started getting few customers. After some feedback and suggestions, you are ready with a full-fledged product. Now, your marketing team shares your app on product hunt to acquire new customers. Suddenly, thousands of visitors are using your app and at one point they are unable to use your app.

You’ve tested your app and it is working fine. So what happened?

“This is not a bug but a problem of scalability. Your cloud architecture is not designed to scale with increasing load.”

I’ve seen many companies that usually focus more on features and less on scalability. Creating applications that are both resilient and scalable is an essential part of any application architecture. In this blog, you will learn how to build highly scalable web application architecture that can scale with increasing load. 

Scalability refers to the ability of a system to give a reasonable performance under growing demands (This can be larger data-sets, higher request rates, the combination of size and velocity, etc). It should work well with 1 user or 1 million users and handles spikes in traffic automatically. By adding and removing the resources only when needed, scalable apps consume only the resources necessary to meet demand.

When talking about scalability in cloud computing, you will often hear about two main ways of scaling – horizontal or vertical. Let’s look deeper into these terms.

Vertical scaling (Scaling Up)

Scaling up or vertical scaling refers to resource maximization of a single unit to expand its ability to handle the increasing load. In hardware terms, this includes adding processing power and memory to the physical machine running the server. In software terms, scaling up may include optimizing algorithms and application code.

Horizontal scaling (Scaling Out)

Scaling out or horizontal scaling refers to resource increment by the addition of units to the app’s cloud architecture. This means adding more units of smaller capacity instead of adding a single unit of larger capacity. The requests for resources are then spread across multiple units thus reducing the excess load on a single machine.

Whether you are looking to scale horizontally or vertically, scalable web architecture will be the base you will need. The scalable architecture enables web applications to adapt according to the user demand and offer the best performance. 

All the components in a monolithic architecture are tightly coupled. So, a single failure in any of the several layers can cause the entire application to fail. On the other hand, scalable web architecture requires a loosely coupled structure of different modules interacting through lightweight mechanisms. 

A typical example of scalable web architecture is a one using the LAMP (Linux, Apache, MySQL. PHP) stack. It offers vital benefits like scalability, high availability, loosely coupled services (fault-tolerant), and distributed computing. 

Scalability- A scalable web architecture like LAMP can enable horizontal scalability. The distributed system can quickly expand or contract resource pools as per scaling needs by adding or removing nodes.

High Availability- Higher uptime and lower downtime are essential to any business. Especially for eCommerce businesses, an hour’s downtime can lead to massive revenue loss. A scalable web architecture needs consideration for redundancy of critical components and rapid disaster recovery even in a partial system failure to ensure higher availability. 

Fault-tolerant- There is no single point of failure, and the system should be able to run efficiently even if a component fails. 

Share Nothing –  This is an architecture pattern where each component is independent and capable of carrying out specific functions. It can be achieved for different web application layers like, 

• For database layer through partitioning or sharding 
• For cache layer- Memcache client-side partitioning that stores a portion of application data in cache and rest is managed by RDBMS(Relational DataBase Management Systems)
• For computing layer- Job queues that allow the tasks to be in queues until a bare minimum for execution are met.

A typical scalable web architecture will have four key layers,

1. Web servers
2. Database servers
3. Load balancers
4. Shared file Servers 

Each of these layers is scaled independently, with the database layer being the toughest to scale. Employing the master-slave replication approach is an excellent way to scale databases efficiently. Each master node is a powerful machine that can read/write data, whereas a slave node can only read data. At the same time, a load balancer will ensure the distribution of load across master nodes. 

For optimal performance, the caching hierarchy is important. Here, the application’s local cache can handle common tasks like bootstrapping, while application server caches can handle complex and big queries. 

Further, job queues are a great way to control the high CPU operations needed for complex data processing. Finally, a Content Distribution Network (CDN) can be leveraged for static public content, and the design system should be fault-tolerant. 

Similarly, when it comes to cloud-based scalable architectures, Amazon Web Services have everything you may need. Not just the application scalability, but AWS also offers several service integration that enhances the functionality.

As mentioned earlier, a scalable application needs different layers to scale independently rather than as a stack of tightly coupled components. Now, let’s look at some successful applications that leveraged such scalable web architecture. 

Examples of Top Scalable Applications:

• Evernote- It is a note-taking web app that uses a microservices architecture to improve the velocity of developing new features and support more than 5 million notes and 12 billion file attachments 
• Bitly-The URL shortening service leverages microservices architecture for the creation of decentralized structures and isolates the failure of components to support 20 billion clicks per month.
• Dropbox-Uses an orchestration service like Atlas to manage distributed database services through self-contained clusters to store more than one million files every 15 minutes.
• Slack- Leverages microservices to enable real-time communication, video calls, and screen sharing for more than 12 million daily active users.
  

There are several challenges to building a scalable web application like the choice of framework, testing, deployment, and even infrastructure needs. 

Right Framework

Choosing the proper framework can be challenging to build scalable web applications. First, you need to analyze the system architecture along with application features and a technology stack. Post this, one can decide whether a particular framework is meeting all the requirements. 

Frameworks like Django and Ruby on Rails are also great options to build scalable web applications. However, if you compare Ruby on Rails with Node.Js, the latter offers an excellent capacity to run complex asynchronous queries. Other frameworks that you can try are, Asp.net, Angular JS, React.js, Laravel, etc.

Testing Needs

Testing is an essential part of the entire web application development lifecycle. Load testing simulations are especially crucial because they allow you to understand the scaling capabilities of your application. 

In addition, it enables the measurement of response times, throughput rates, and resource consumption to identify the app’s breaking point. However, the complexity of the protocols on which the load tests are based makes it challenging to execute. For example, if you are performing HTTP-based load testing, there are several dynamic parameters to consider, like cookies or session IDs. 

However, there are several other performance testing requirements for measuring and enhancing app scalability. From operating system compatibility of the application to real-time error detections, you need to create tests for several different conditions and even simulate app usage experience for many users.

One possible solution is choosing a Test-driven Development (TDD) to leverage the iterative development approach. It is a software development approach that relies on converting web app requirements into test cases before fully developing them. It allows developers to use different test cases and track all the web app development through iterations. 

Due to the “Test First” approach of TDD, there is rapid feedback to the functionality of applications predefined by test codes. So, when you scale the application and if a function does not perform as expected, you can quickly change the code before the next iteration.

Deployment Challenges

Deployment of multiple services interacting with each other needs a reliable communication mechanism which is challenging. That is why you need to test these communication mechanisms and orchestration services simultaneously to ensure lower disruptions. Failure to do efficiently can slow down your development process due to parallel development and testing activities. 

One possible solution is the usage of CI/CD systems. Such a system will inherently allow you to build features without worrying about testing and deployment activities. 

Infrastructure Needs

It is challenging to forecast the exact number of users. So, gauging the actual infrastructure requirements to scale accordingly is challenging. However, depending on the situation, you can employ two types of scaling- horizontal and vertical. Both of these have unique use cases, which are discussed below,  

You can choose horizontal scaling when, 

• You can’t risk downtime due to a single point of failure
• Your scope of upgrading the application is higher
• You don’t want a vendor-lockin and want to explore multiple vendor services
• You want to improve the resilience of systems by using multiple services

At the same time, you can choose vertical scaling when,

• You need a simple architecture with reduced operational costs
• You want a system that can scale without consuming much power
• You need an easy to install and scalable system with lower licensing costs
• You want the compatibility of applications maintained. 

What does scalability look like? There are some areas where an app needs to excel to be considered scalable.

Performance

First and foremost, the application must operate well under stress with low latency. The speed of a website affects usage and user satisfaction, as well as search engine rankings, a factor that directly correlates to revenue and retention. As a result, creating a scalable web application architecture that is optimized for fast responses and low latency is key.

Availability and Reliability

These are closely related and equally necessary. Scalable apps rarely if ever go down under stress. They need to reliably produce data upon request and not lose stored information.

Manageability

The manageability of the cloud architecture equates to the scalability of operations: maintenance and updates. Things to consider for manageability are the ease of diagnosing and understanding problems when they occur, ease of making updates or modifications, and how simple the system is to operate. (i.e., does it routinely operate without failure or exceptions?)

Cost

Highly scalable applications don’t have to be unreasonably expensive to build, maintain, or scale. Planning for scalability during development allows the app to expand as demand increases without causing undue expenses.

You’ve plenty of options to choose the cloud provider while building the high-performance web application architecture. The three leading cloud computing vendors, AWS, Microsoft Azure and Google Cloud, each have their own strengths and weaknesses that make them ideal for different use cases.

In this blog, I’ve chosen AWS to show you how to build web scalable application. AWS is a subsidiary of the renowned company, Amazon, it provides different services that are cloud-centered for various requirements. AWS holds the highest 33% market share of cloud computing. They provide excellent documentation on each of their services, helpful guides and white papers, reference architectures for common apps.