Node.js Architecture and Best Practices for Node.js Application Development

Netflix, one of the leading streaming services in the world, observed several issues in its software framework which were inhibiting business growth.

• The monolithic architecture wasn’t ideal for scaling.
• JAVA’s heavy-weight back-end was a resistance to speedy development/deployment.
• There was a lack of smooth transition between the front-end and back-end, resulting in higher load time and latency.

One of the significant steps Netflix took to overcome these challenges was to adopt Node.js.

After adopting Node.js, Netflix reduced the startup loading time by 70%. In addition, there was a tremendous improvement in front-end to back-end transition, resulting in reduced latency. Lastly, scaling became easier after adopting Node.js, which is asynchronous and non-blocking. As a result, the product could handle millions of requests concurrently.

NASA, Trello, PayPal, Uber, Twitter, etc., have also reaped similar benefits by adopting Node.js. However, you need to know Node.js architecture and best practices to attain such results. In this article, we tell you all there is to know about Node.js architecture and the best practices for Node.js application development.

Node.js uses ‘Single Threaded Event Loop’ architecture to handle multiple concurrent clients. The Node.js processing model is based on a Javascript-event-based model and a callback mechanism. This mechanism based on the event loop allows Node.js to run blocking I/O operations in a non-blocking way. Moreover, scaling is much simpler with a single thread than one thread/new thread per request under ordinary web loads. Now, let’s understand the key elements that make up Node.js architecture.

Key elements of Node.js architecture

1. Requests –  Based on the specific tasks users need to perform in a web application, the requests can either be blocking (complex) or non-blocking (simple).

2. Nodejs server – It accepts user requests, processes them, and returns the results to the corresponding users.

3. Event queue – It stores the incoming requests and passes them sequentially to the Event Pool.

4. Event pool – After receiving the client requests from the event queue, it sends responses to corresponding clients.

5. Thread pool – It contains the threads available for performing those operations necessary to process requests.

6. External resources – The external resources are used for blocking client requests and can be used for computation, data storage, and more.

What makes Node.js architecture the right choice for applications?

Scalability

The non-blocking I/O model and the system kernel’s worker pool allow you to build highly scalable Node.js applications. These apps can handle thousands of concurrent client requests without crippling the system. Its lightweight nature enables leveraging microservices architecture.

Walmart implemented the microservices architecture with Node.js to witness:

• 20% conversion growth in general
• 100% uptime on Black Friday sales handling over 500 million page views
• Saving up to 40% on hardware.

Speed and performance

The event-driven programming of Node.js is one of the main reasons for its higher speed compared to other technologies. It helps in synchronizing the occurrence of multiple events and building a simple program. Moreover, its non-blocking, input-output operations contribute to its speed. As the code runs faster, it improves the entire run-time environment.

Flexibility

When developers make a change in Node.js, only a specific Node is affected. In comparison, other frameworks require making changes to the core programming. As a result, this Node.js feature benefits the initial build stage and ongoing maintenance.

These were some of the benefits that Node.js brings to the play. However, to reap those benefits, you got to follow best practices.

1. Incorporate the publisher-subscriber model

As a data exchange pattern, it includes two communicating systems– publishers and subscribers. The publishers send out messages through specific channels without any information about the receiving system. Subscribers deal with one or more of these channels without the knowledge of the publishers. Adopting this model allows you to manage multiple children operations connected to a single action.

Let’s try to understand this concept with a few use cases. As you know, sending event notifications to a significantly large user base can sometimes be highly challenging. You will require systems to deliver the messages to clients rapidly, which becomes an issue when the user base keeps increasing. In addition, sometimes, there is a chance that users may miss out on the message as they’re not available. So, it’s the system’s responsibility to ensure that clients receive the message when they come online.

The pub/sub model follows a loose coupling principle allowing the publisher to send event notifications without worrying about the client’s status. The model also ensures robust implementation of brokers, which can persist messages and deliver them exactly once. So, publishers can simply “fire and forge,” while brokers will provide the message when clients need them.

Another famous use case of the pub/sub model is distributed caching. Caches play a significant role in improving the system’s performance. Depending on your application, you need to distribute cache across the system. With the pub/sub model, you can implement distributed caching efficiently. You don’t need to wait for the client’s request to seed the cache. Instead, you can do that in advance asynchronously. It saves time and also improves app performance.

2. Adopt a layered approach

With the popular Node.js framework Express.js, developers can follow the “separation of concerns” principle during development. By dividing the codebase into three categories – business logic, database, and API routes, which fall into three different layers, service layer, controller layer, and data access layer. The idea is to separate the business logic from Node.js API routes to avoid complex background processes.

Controller layer – This layer defines the API routes. The route handler functions let you deconstruct the request object, collect the necessary data pieces, and pass them on to the service layer for further processing.

Service layer – This layer is responsible for executing the business logic. It includes classes and methods that follow S.O.L.I.D programming principles, perform singular responsibilities, and are reusable. The layer also decouples the processing logic from the defining point of routes.

Data access layer – Responsible for handling database, this layer fetches from, writes to, and updates the database. It also defines SQL queries, database connections, models, ORMs, etc.

The three-layered setup acts as a reliable arrangement for Node.js applications. These stages make your application easier to code, maintain, debug and test.

3. Use dependency injection

Dependency injection is a software design pattern that advocates passing (injecting) dependencies (or services) as parameters to modules instead of requiring or creating specific ones inside them. This is a fancy term for a fundamental concept that keeps your modules more flexible, independent, reusable, scalable, and easily testable across the application.

Let’s take an example to understand this concept:

// Constructor injection
class Car{
constructor(engine){
this.engine = engine;
}
}

// Setter injection
class Car{
constructor(){

}
setEngine(engine){
this.engine = engine;
}
}

// Taking an input argument and binding a function
function createCarWithEngine(engine){
// create a car with the engine
}

let createCar = createCarWithEngine.bind(null, engine);

As you can see, we have focused here on creating a car, so we don’t care about the dependencies of the engine. Dependency injection is all about focusing on the core functionality and not caring care where our dependencies are, how they are made, or even what they are. It greatly benefits code decoupling, reusability, testing, and readability.

4. Utilize third-party solutions

Node.js has a vast developer community across the world. In addition, Node.js offers a package manager, NPM, which is full of feature-rich, well-maintained, well-documented frameworks, libraries, and tools. Therefore, it’s convenient for developers to plug these existing solutions into their code and make the most of their APIs.

Some other Node.js libraries that may help you to enhance your coding workflows:

• Moment (date and time)
• Nodemon (automatically restarts the app when there’s a code update
• Agenda (job scheduling)
• Winston (logging)
• Grunt (automatic task runner)

There are many options available regarding the third-party solution when you talk about Node.js. However, knowing these solutions’ purpose, benefits, and dependencies is paramount. Only after being doubly sure, try to import them intelligently. And don’t over-rely on third-party solutions, as some may also be unsafe and introduce too many dependencies.

5. Apply a uniform folder structure

We have already discussed adopting a layered approach for Node.js applications. A folder structure can help you to transform that into reality. Here, different modules will get organized into separate folders. In addition, it clarifies various functionalities, classes, and methods used in the application.

Here is a basic folder structure you should follow while setting up a new application in Node.js:

src
├── app.js           app entry point
├── /api               controller layer: api routes
├── /config          config settings, env variables
├── /services        service layer: business logic
├── /models          data access layer: database models
├── /scripts            miscellaneous NPM scripts
├── /subscribers    async event handlers
└── /test                 test suites

The directories API, Services, and Models represent controller, service, and data access layers. /config is for the directory to store the configuration of the environment variables, while /scripts stores workflow automation scripts. /test directory is for storing all the test cases, and /subscribers store the event handlers in the pub/sub pattern.

6. Use linters, formatters, style guide, and comments for clean coding

Linting and formatting:

Code linters are static code analysis tools that automatically check programming errors, styles, bugs, and suspicious constructs. These tools can help you identify your code’s potential bugs and harmful patterns. Formatters ensure consistent formatting and styling across your whole project. The main goal of both these tools is to improve code quality and readability.

JSLint and ESLint are some of the most-used linter tools for inspecting Javascript code. For code formatting, you can look at Prettier. Nowadays, most IDEs or code editors provide linters and formatters as plugins. They also provide intelligent code completion, auto imports, on-hover documentation support, debugging tools, code navigation, refactorings, etc.

Style guides:

Why not adopt one of the fantastic style guides that Google or Airbnb have provided for Javascript? Then, reading code will become incredibly accessible, and you won’t get frustrated trying to understand how to position that curly brace correctly. These style guides cover everything from naming conventions formatting to file system encodings. In addition, using style guides will help you follow the standards used by some of the top developers across the globe.

Adding comments:

Another way of ensuring clean coding and enhanced readability is to put comments about the code you’ve written intelligently. It will help other developers understand the purpose of writing particular code. Also, it can help you when you refer to the same code after a while. Lastly, comments are one of the best ways to document your project details, such as APIs, class, functions, author name, version, etc.

7. Rectify errors with unit testing, logging, and error-handling

Unit testing:

Unit testing isolates a section of the code and verifies its accuracy, validity, and robustness, thereby improving the whole flow of the project. It also improves the code quality, helps identify problems in the code early, and reduces the cost and time spent on debugging.

Unit testing is the foundation of any testing setup that allows you to verify whether individual components are performing desired actions. You can conduct unit testing in Node.js applications by developing various test cases. For creating these test cases, you can use test frameworks like Jest, Mocha, or Jasmine.

Logging and error-handling:

Errors are an integral part of programming. Every time there is an error, it gets accompanied by a straightforward error message about the error and possible solution for its fixation. In addition, programming languages are attested with built-in logging systems.

These logging systems log important information throughout all the stages of Nodejs development and helps keep a record of essential information required when analyzing your application’s performance. In addition, this makes debugging process effortless.

The most common way of logging information in Node.js is by using the console.log() method. The command simply logs the message in your console. The process of information logging in Node.js checks through three streams – stdin, stdout, and stderr.

• Stdin – deals with input to the process (example – keyboard or mouse)
• Stdout – deals with the output to the console or a separate file
• Stderr – deals specifically with error messages and warnings.

For smooth development using Node.js architecture, one must be conscious of how they handle code errors. For example, async javascript is often susceptible to what’s known as “callback hell.” This notorious callback issue forces you to check errors in each nested loop.

Build a centralized error-handling component for error handling in Node.js architecture. Creating such a component will avoid possible code duplication errors. The component is also responsible for sending error messages to system admins, transferring events to monitoring services, and making a log of every event.

8. Practice writing asynchronous code

If you’re someone who has been using JavaScript for a long time, then you must know about callback functions. It helps you to define asynchronous behavior. However, the problem with callbacks is that as there is an increase in the number of chained operations, the code gets chunkier and more challenging to handle. It’s known as a callback hell situation.

ES6 came up with the idea of Promises API to eliminate this situation. Also, with ES8, there is a provision of async/await syntax that further simplifies things.

The callback function requires you to process the function within the function thrice before reaching the end goal. While the async/await mode has cleaner code, better readability, and easier error handling.

9. Using config file and environment variables

As your application scales, you’ll require global configuration options that every module can access. You can use a separate file and put everything inside a config folder in the project structure. The /config directory contains different configuration options as shown below:

/config
├── index.js
├── module1.js
└── module2.js

These options include basic settings, API keys, database connections, etc. It gets stored in a .env file as environment variables in the form of key-value pairs.

DB_HOST=localhost
DB_USER=xyz
DB_PASS=me@123

You can access these environment variables in your code using NPM’s dotenv package. However, the standard practice is to import environment variables into one place. The reason is that if any change is required, it will happen in one place and gets reflected in the application.

10. Employ Gzip compression

Gzip is a lossless compression mechanism used for compressing files so that they can get transferred quickly. Node.js serve heavy-weight web development pages that contain multimedia files, so using Gzip compression help reduce the load and make the process faster.

Express.js, a Node.js framework, helps you implement this tactic quickly. In addition, every Experss.js documentation has an advisory to use Gzip compression. The reason behind that is the compression middleware used by Express.js, making it a natural progression.

11. Take APM tools into account

For enterprise-level applications developed with Node.js, ensuring a world-class user experience and high performance is paramount. If this is the primary goal of your application, then a better understanding of how users interact with the application becomes essential.

That’s where APM (Application Performance Monitoring) tools have a significant role to play. They help you monitor the app performance continuously and detect errors or bottlenecks that hamper the user experience. And they also provide real-time insights.

With continuous monitoring and follow-up on various issues, there’s every chance that your application will perform at its best for a more extended period. Appmetrics, Prometheus, Express Status Monitor, etc., are some popular Node.js monitoring tools.

We’ve helped multiple clients build enterprise-level apps with Node.js. SenTMap, a sentiment-based market analytics engine, is one such app. As the product was related to stock market analytics, there was a need to process vast amounts of data in real time. So, our engineering team decided to build a robust back-end using Node.js. Due to the asynchronous and non-blocking nature of Node.js, the application could process terabytes of information efficiently. It also helped SenTMap to reduce RAM server requirements, and it could handle 1 Million concurrent users.

Simform is a top-rated web application development company that helps you build modernized, scalable, and reliable applications. Hire our expert Node.js developers and avail a 360-degree backend engineering solution for your organization. Feel free to contact us for more information!