How to develop highload services on Node.js

Posted on by

This post is also available in: Russian

In this post, we are going to focus on developing based on a relatively new technology, Node.js. It is rapidly gaining in popularity, and not unreasonably. Node.js allows you to implement faster Web services capable to hold extreme load, which is critical to video service projects. For example, in Node.js you can easily build real-time statistics collectors , large-scale chats, etc. Of course, services written in C++ have a higher performance margin, but are too costly to implement. On the contrary, Node.js allows you to quickly create complex systems. In this post, we will use Node.js to build a simple application that can hold very high load.

Node.js development specifics

This technology is based on JavaScript, so it has its particular requirements and development paradigms. First of all, the functions of core modules in Node.js are asynchronous. This means that the function does not block the stream, but is executed in the background. This immediately evokes the concept of a callback function, a procedure that is run after execution of the code in the stream. Node.js really excels in tasks involving a large number of I/O operations. Examples are: reading a file, writing to the database.

Installation

To use Node.js, you first need to install it. Perhaps, in any Linux distribution now, you will find Node.js packages. In Debian or Ubuntu, just run the following commands to proceed with installation:

sudo aptitude update
sudo aptitude install node

Modular architecture

Another advantage of the Node that it hosts a multitude of modules for any occasion. Modules are similar to libraries you may use in other languages. Some Node.js modules are pre-installed, while others need separate deployment. For their installation you’ll need a special utility called npm:

npm install

The modules can be installed either locally (for the current project only) or globally (for all projects at once). We strongly recommend that you install modules only locally, as they are actively developed and hence compatibility between the versions is often lost.

Your First Application

Let’s not be original and just say "Hello, world!" in our first node. To do this, create a new file index.js. Type the following code in it:

console.log('Hello, World!');

To launch the script, just run its file using the command:

node index.js

Making things more complex

Of course, ‘Hello, World’ is fine, but sure we want something more. Even better still, is to create a real highload project. Chat is a perfect fit to this end.

First, select a stack of technologies we are going to use to create our messaging system. Apart from Node.js we need to store the messages somewhere. Redis is a perfect match for this purpose. Moreover, this storage offers built-in collection lifetime support, which we are going to use to timely purge the historical data, and the Pub/Sub mechanism which is helpful to enable immediate instant message delivery.

For transport of our protocol messages let’s use Websocket.

var webSocketServer = require ('websocket'). server;

Sets the node to listen to connections:

var http = require('http');

var server = http.createServer();

server.listen(config.port, function() {
    console.log('Chat started on port ' + config.port);
});

var wsServer = new webSocketServer({
    httpServer: server,
    autoAcceptConnections:false
});

wsServer.on('request', function(req) {
    var connection = req.accept();

    connection.on('open', function(con) {

    });

    connection.on('message', function(message) {

    });

    connection.on('close', function() {

    });
});

Note that by this code we not just enable a service on sockets, but also create event handlers. Let’s now dwell into one of them, message arrival.

On arrival of chat messages, it is necessary: 1) to save it in the history 2) send it to other users

Although the first item is obvious, the second one is far from it. The first thing that you can imagine is to keep a list of users in an array, and as the message arrives just send it to them. This solution would be adequate in case when we run a single instance of the service. But we have Highload, right?

What about Scalability?

Node.js-services have an excellent feature: they can run in a cluster mode, several processes at once. Each of them occupies the processor core. To enable this feature, just use the built-in cluster module.

var cluster = require('cluster');
var numCPUs = require('os').cpus().length;

cluster.setupMaster({
    exec:'./chat.js'
});

for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
}

Now the users will be distributed across the chat instances. This will improve load characteristics. However, now our approach with multicasting messages to all users by going through an array, will obviously not work. Node.js is unaware of who is connected to an adjacent instance.

There is a solution, however. This is the Redis Pub/Sub mechanism. It allows us to create a channel in a storage and subscribe to it. Now, on arrival of a message to the channel, all subscribed nodes will invoke a callback method to message the users. As you know, the Node.js service serves as transport for Redis, as it just writes to it and distributes messages across users.

Despite the fact that Redis is quick, we still can have a situation where it would create a bottleneck in our system. To avoid this, deploy the hashring mechanism.

The principle of hashring is that multiple Redis instances are combined into a single system. Also, collections are distributed among them. Hashring knows where each collection is stored, so we can just send a request to it and get a link to the Redis we need.

More on accessing Redis from Node.js

Do you remember that Node.js code runs asynchronously? And you know what? This sets some specifics on accessing the data storage.

It is important that all of your operations with it are atomic. It means that you cannot first read the value of a collection, edit it and then save it back. Such an approach will entail a lot of conflicts: e.g., between read and write operations, another execution stream may change the value so that the data is inconsistent.

Instead of this, you should use such commands as zadd, incr, decr. They allow you to change the collection without reading the data from it ; as a result, the number of collisions will be brought to zero.

How to develop a messaging protocol

To expand the features of your chat, you have not just to exchange messages between clients, but to enable interaction with them using service messages. For this end, you have to develop a messaging protocol. As a basis for it, let’s use JSON, offering an easy way to save objects. Now, instead of plain text you can send objects containing a variety of fields. For example, by adding a type field, you can enable processing of different service messages received from the server. Everything is limited by your imagination only.

Cleaning of Redis collections

Sooner or later you will run into the problem that the Redis collections are growing. The volume of old data will continuously increase. For instance, it is not likely that anyone will need to view week-old chat messages.

Of course, you can clean messages either by the chat node itself (this would put extra load on it) or by third-party services (load + time to implement it). However, Redis provides a tool to set collection lifetime using the expire command. By that time, the collection is completely removed, with all messages erased. To avoid this, it is necessary to separate history among several collections with time based discretization. For instance, using 5-minute step. The ID of collections will differ by suffixes that are equal to the time closest to current and evenly divisible by the discretization step. Each of the collections has a preset lifetime. Now the messages will be deleted not all at once, but in small groups, with current messages always present in the storage.

Why Highload?

So, to write code is half the battle. Now we need to make sure that the service can actually hold the load.

In our Web socket projects we use easy-to-use benchmark called wsbench. It allows you to empirically measure the load capacity of the service. With it you can create connections to the service and deliver messages. (You can learn more at the benchmark page.)

The Wsbench will help us measure the following service indicators:

  1. Maximum number of connections
  2. RAM usage
  3. Incoming message response time
  4. Load capacity.

Here we go!

1) To determine the maximum number of users you have to initiate them with the benchmark. There you can specify the number of users to create and the rate of service connection growth. As it has been shown, the server processor is critical here. 4-core Intel Core i5 2.5 GHz has held up to 30,000 concurrent connections. And all this, with the service run from within a virtual machine!

2) To determine the required amount of RAM, let’s connect 10,000 users to the service and measure RAM usage before and after that. Then divide the difference of the two values by the number of connections. Then be sure to multiply this value by 2: your service has not just to "hold" users but send messages to them.  Node.js based services are greedy for memory, so allocate it lavishly. In case of insufficient memory, swapping will begin, which will substantially slow Node.js down. The value of 1-2 MB per user should not embarrass you.

3) To determine response time, let’s launch one instance of the service, create the maximum number of connections, send 1 message to Redis, note sending time and compare it with the time of message delivery to all users.

4) To evaluate the total load capacity of the service. create the maximum number of connections to Node.js and make them communicate intensively. Perhaps, once in every 5 seconds is enough. Along with that, connect one real user to the service, and look for new messages in the browser. It is important to limit the amount of messages sent, as the service should not be run on large test sets. With the processor indicated in item 1, a stream of 6,000 messages per second has been processed quite easily. It’s true that with a real customer the browser hang, but most likely, the load in a chat room would not be so high. (Operations to add messages to DOM are not quick. For more detail, please read the article on "Optimization of HTML5 applications.")

Conclusion

In this post, we have described basic approaches to implementing highload applications in Node.js in conjunction with Redis. By following our guidelines, you can easily build your own highload service. On Github, we have published a ready chat app which is based on the principles presented in this paper.

DChat Demo

Helpful links

Leave a Reply