How to Scale Your Node.js Application Using the Cluster Module

In this article, I will walk you through how to scale up your node.js application in the easiest way, that is the cluster to serve more requests.

Node.js runs on a single thread, it means by default, it runs on 1 CPU. If your application receives a lot of requests or it has a long live blocking function. Your entire application can be unresponsive although your server still has a lot of resources.

For example:

This simple Node.js application has 2 endpoints

1. Get Fibonacci 45th Number, this is a cpu bound task, blocking function
2. Get Server Information

javascriptCopy
const express = require("express");
const os = require("os");

const app = express();
const PORT = 3000;

// Simple recursive Fibonacci (not optimized for large numbers)
function fib(n) {
  if (n <= 1) return n;
  return fib(n - 1) + fib(n - 2);
}

// Endpoint to return the 45th Fibonacci number
app.get("/fib", (req, res) => {
  const start = process.hrtime();
  const result = fib(45);
  const end = process.hrtime(start);
  const executionTime = (end[0] * 1000 + end[1] / 1000000).toFixed(2); // Convert to milliseconds

  res.json({
    fibonacciNumber: result,
    processingTime: `${executionTime}ms`,
    processId: process.pid,
  });
});

// Endpoint to return server info
app.get("/info", (req, res) => {
  res.json({
    name: os.hostname(),
    numberOfCpus: os.cpus().length,
    processId: process.pid,
  });
});

// Start the server
app.listen(PORT, () => {
  console.log(`Server is running on http://localhost:${PORT}`);
});

Since the Get Fibonacci Number is the blocking function, and it is written poorly, whenever it is executed, it occupy the entire thread, making application is unresponsive to the requests. So if you place the second request to get server information, it returns the result only when the get fib number function is executed successfully

The application is not responding when you perform 2 requests at the same time, get fibbonaci number 45th, and get server information, even that the get info is very lightweight endpoint.

In the image below, demonstrate that for a server which has 10 Threads, one thread is 100%, while the other is just idle.

So it comes to the question, how can you use other other cpus to handle more requests, because is is obviously, there are 9 idles cpus, sitting and doing nothing, while one cpu is working ass off?

That is when cluster come for the rescue.

What is Node.js Cluster?

• The Node.js cluster allows you to create the child process (workers), that share the same server ports and run simutatenoulys on multiple CPU cores. With clustering, you can scale up your application all available CPU cores to improve performance and handle more concurrent requests.
• Each Worker has each own event loop, but shares the same server port.
• The master (primary) process will act as the load balancer. It is responsible for managing and co-ordinating the worker process. It does not handle any incoming HTTP requests it selft. That is the job that worker do.

With a bit of refactoring, our code can become.

javascriptCopy
const express = require("express");
const os = require("os");
const cluster = require("cluster");

// Check if current process is primary
if (cluster.isPrimary) {
  const numCPUs = os.cpus().length;

  // Fork workers for each CPU
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  // Log when a worker dies and fork a new one
  cluster.on("exit", (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died`);
    cluster.fork();
  });

  console.log(`Primary ${process.pid} is running`);
} else {
  const app = express();
  const PORT = 3000;

  function fib(n) {
    if (n <= 1) return n;
    return fib(n - 1) + fib(n - 2);
  }

  app.get("/fib", (req, res) => {
    const start = process.hrtime();
    const result = fib(45);
    const end = process.hrtime(start);
    const executionTime = (end[0] * 1000 + end[1] / 1000000).toFixed(2);

    res.json({
      fibonacciNumber: result,
      processingTime: `${executionTime}ms`,
      processId: process.pid,
    });
  });

  // Endpoint to return server info
  app.get("/info", (req, res) => {
    res.json({
      name: os.hostname(),
      numberOfCpus: os.cpus().length,
      processId: process.pid,
    });
  });

  // Start the server
  app.listen(PORT, () => {
    console.log(`Worker ${process.pid} is running on http://localhost:${PORT}`);
  });
}

As you can see, in this version, the code can run on multiple threads, with the number of threads equal to the number of cpus of the system.

so when you perform the

bashCopy
GET localhost:3000/fib
GET localhost:3000/info

It can run at the time time, although the fib endpoint can make a CPU go to 100%, but the info still can be handled by other thread.

Each time you access the endpoint, you may noticed that the processId will be different since it is handled by different threads

For the full code, you can checkout at:

https://github.com/mt26691/dalabs-sample-files/tree/main/nodejs-cluster-example