NodeJS and its use cases
NodeJS has many use cases.
It is an easy to pickup and use runtime.
The user does not have to worry about threading.
This is taken care off by the runtime.
You write your code and make use of the many asynchronous operations provided by Node.
This will take care of any multithreading for you.
However, as you will read later in this blog post, making use of multiple Node instances to divide work is still possible!
More on that later!
Node can be used for a variety of tasks:
- Small yet efficient web server
- Code playground, test something quickly
- Automation and tooling, instead of using ruby/python/…
- IoT, Raspberry pi’s and other devices that can run Node!
However, you should not use node for computationally heavy tasks! While the V8 engine is highly performant, there are other much more performant options available for computationally heavy operations!
This blog post is not meant for people who have no NodeJS experience! Below are some resources for those that are new to the platform:
This has some pros and cons.
While it is an easy language to pick up, it can be hard to master.
It also does not require any compilation, which makes running your code very easy.
However, this also removes any help from the compiler as no compile time checks are performed.
No type checking, no checking for illogical structures or things that will just not work.
Code for Node can be run by simple opening a command prompt or terminal window and typing
This will start a Node instance and present you with an interpreter.
You can now type commands and press return to execute them.
This can be handy to test something quickly.
However, most of the time you will not be using this way of running code. Most of the time you will use npm to install your dependencies and start the node instance:
This reads the package.json file and executes the scripts contained inside it.
Extensive documentation about the package.json file can be found on the NPM website
TypeScript you say!?
TypeScript has been around for some years now.
It uses the same syntax but adds among other things compile time type checking.
It also adds a more Object Oriented model.
Mozilla Developer website
TypeScript developed mainly by Microsoft and is completely open source! This means developers can make suggestions and report bugs (and even fix these bugs if they want).
TypeScript is very well documented and getting started with the language is fairly easy. A lot of common development tools have support for TypeScript syntax checking. These include, but are not limited to:
- Visual Studio Code
As Node applications regularly use other NPM dependencies it is required for the TypeScript compiler to know about these dependencies and what types they use. You could make or generate these typings yourself. However, you can easily find these typings on TypeSearch website. The most commonly used dependencies have their typings available here!
You can add the typings to the dependencies in the package.json file.
Making it all work: An example
- WeatherGenie This was the initial implementation, a simple weather web application for checking the weather conditions for any city in Belgium
- LoRa-IoT-Demo The second, extended iteration, based on the code from the WeatherGenie application. Because with the advent of IoT we needed a simple to extend/run/maintain solution to create IoT demos for clients.
- NodeSimpleServer The third and current iteration. Written from the ground up in TypeScript and completely reworked to work better and be more maintainable. This is the application that will be detailed below!
Node Simple Server: High level architecture
The Application starts in app.ts under the main src folder.
This is the entry point for the application.
This file contains the actual master instance code.
The master instance is in charge of forking the workers and reviving them if they die.
The master is also used to pass messages between the workers For this a specialized
MessageHandler singleton is used.
MessageHandler instance (one per worker) is used to relay messages.
The master instance itself will not execute any application logic.
Its purpose is to manage the other workers and be the message bridge.
The master will create a number of workers:
HttpWorkeris an endpoint for requests to be received. There will always be a minimum of two HttpWorkers created. If more CPU cores are available, more HttpWorkers are created.
DataBroker: For the application there is one
DataBrokerworker instance. This worker handles CRUD operations for data (for now in memory only).
IntervalWorker: For the application there is one
IntervalWorkerinstance. This worker can run code periodically and is used to connect to other devices such as Arduino’s and the Raspberry Pi I/O pins.
These workers are created by a
WorkerFactory, as the master forks new Node instances, a process variable is set, the factory uses this to see which type the node instance should become. Each type of worker instance implements the basic
NodeWorkerinterface. Each implementation will be detailed below.
Handling HTTP requests: The HttpWorker
HttpWorker instance will create a
This instance will be used to receive HTTP requests.
Node will automatically load balance requests between all instances that register a server on the same port.
Simply put all HttpWorkers compete for the next request, the least burdened process (depending on OS/CPU process affinity) will be given the next Http request to handle.
Server class will also register the endpoints that are known to the application and can be handled.
EndpointManager is used to register endpoints.
EndPoint has a path, a method to execute and optional parameters.
Parameter is provided with a Generic type for compile time type checking, a name which should be used in the url, a description that provides information what the parameter should contain and an optional
ParameterValidator is used to validate the
Parameter at runtime.
If the check fails an error is shown to the user.
Server instance forwards all requests to the
As the name suggests this will perform the routing.
It will see if a resource is requested or and endpoint has been called.
If a resource is requested it will be served if found.
If an endpoint has been called, that endpoint will be executed and passed the parameters that were entered, but only after the correct amount of parameters has been passed and they are all valid.
Handling data: The DataBroker
DataBroker is the Node instance in the application that will save and retrieve data.
For the time being it is sufficient to only have in memory ‘caches’ on which basic CRUD operations can be performed.
All methods on the
DataBroker are called by sending an
IPCRequest with the data that needs to be saved of the instruction for what data should be retrieved.
DataBroker will reply to the original worker by sending an
IPCReply with the result of the operation.
DataBroker for now only has a concept of caches.
A cache has a name, type and values (of said type).
Values can be retrieved, added, updated and deleted from the caches.
Caches can be retrieved, added and deleted at runtime.
Handling asynchronous tasks: The IntervalWorker
IntervalWorker as its name suggest performs tasks at a certain interval.
It is also used for other asynchronous workloads, such as connecting to an Arduino and running Arduino/Raspberry pi Johhny-Five scenarios.
IntervalWorker is handy when you need for example to update the content of a cache every so often.
It can also run Arduino scenarios. These are Implementations that contain logic to perform actions on the Arduino or in response to something that happens on the Arduino. The
IntervalWorker picks up what type of Arduino
Scenario you want to run and starts the logic.
There are two Arduino implementations available.
Both can execute a
The first and simplest implementation is the Johnny-Five Arduino implementation.
This allows you to make use of the Johnny-Five framework to write dynamic code for the Arduino that can change at runtime.
This is possible because it uses the StandardFirmata firmware.
Johnny-Five supports a lot of components and peripherals.
Their website has extensive documentation and very clear examples.
Johnny-Five also supports the Raspberry PI I/O pins.
This allows it to be used on a Raspberry pi also.
The second Arduino implementation uses no framework and communication is done via regular serial. In the type of scenarios you have to handle all the serial communication yourself. You also have to write Arduino firmware and thus it cannot be dynamically updated at runtime. Use this Arduino implementation if some component is incompatible or not supported by Johnny-Five.
Inter Process Messaging: Communicating between different Node instances
Having all these different worker instances is quite handy.
However they are of not much use if there cannot be any communication between them.
Each Node instance has its own allocated memory and cannot access variables or call methods on other instances.
The Node cluster and process framework provide the option to send messages between Node instances.
IPCMessage instances that are sent exist in two forms.
IPCRequest: This is the initial message that is sent to a target.
IPCReply: This is the response (if any) from the target back to the original caller.
This allows for easy two way communication and identification whether the message was a reply to an earlier message.
Messages can be sent with or without a callback.
The callback is executed when a reply to the original message is received.
Because only basic data types can be sent across Node instances the
MessageManager instance of the caller stores the callback reference and generates an unique id for said callback.
This allows the application to send the callback ID across Node instances and execute it when it arrives back at the caller.
Every worker has an instance of the
MessageHandler, it in its turn has an event emitter on which events from the messages are broadcast.
The actual worker implementations register themselves on the emitter to receive said events.
In a future version the message handling should be split up, because now a single file (with an instance on each Node instance) handles both master and slave messages.
In conclusion; It is perfectly possible to make a more complex application for NodeJS with TypeScript.
By using TypeScript you gain compile time type checking and a more robust and better readable codebase.
Fewer errors and strange bugs are encountered because TypeScript ‘forces’ you to write better code.
The Node Simple Server application was a great way to learn the ‘new’ TypeScript language. The project is not finished, as some parts could use some more work, but it should stand as a solid starting point. Feel free to fork the codebase, submit issues or start some discussion.