Within this tutorial, we’ll cover Queues in JavaScript. We'll make an introduction to what Queue is, look at different scenarios that need the implementation of a Queue, write code to demonstrate how Queues works, and explore the different parts of the code structure.
There are common scenarios where an application will have a lot of workloads that should be processed asynchronously from application flows.
These asynchronous operations may contain asynchronous actions within them, too. This easily poses a common issue to handle in Node.js, but with Queueing as a technique in Node.js, we can easily and effectively handle asynchronous operations.
Table of contents
Prerequisites
- Basic understanding of JavaScript
- Know the difference between synchronous programming and asynchronous programming
- Basic understanding of Node.js
Introduction to Queues
Data Structure can be defined as the group of data elements that provides an efficient approach to storing and organizing data in the computer so that it can be retrieved and used efficiently.
A queue is also a linear data structure — linear data structures have data elements arranged in an orderly sequential manner and each member is connected to its previous and next sibling element.
This connectivity between the elements helps to traverse the linear data structure in a single flow and in a single run.
A queue is a linear data structure used in Node.js to appropriately and progressively organize asynchronous operations.
It is an ordered list of asynchronous operations where an asynchronous operation is inserted at the end of the queue and is removed from the front of the queue.
These operations exist in different forms, including HTTP requests, read or write file operations, streams, and more. Handling asynchronous operations within Node.js applications can be daunting to manage.
JavaScript functions like native timers and other asynchronous operations can take an indefinite duration to complete their asynchronous operation. But with queues, we handle asynchronous operations synchronously by inserting a new element and removing an existing element on an array.
The call stack, event loop, and the callback queues help us implement Queues within a program. With the call stack, we can easily record and keep detailed track of the current function being executed and where the execution is happening from.
A function gets added to the call stack when it is about to be executed within the program. The event loop is what allows our Node application to perform non-blocking I/O operations — despite the fact that JavaScript is single-threaded.
The operations are offloaded to the system kernel whenever possible, constantly checking the call stack for when it’s empty to take a function from the callback queue and add it to the call stack. The event loop only checks the queues when all synchronous operations have been executed. The callback queues are also queues that store callback functions — asynchronous operations, once any operation has been completed behind the scene.
Introduction To The Types Of Callback Queues
Before we introduce the types of queues available, we’ll quickly give the order of priority given to them, then go ahead to discuss each one following their order of priority. There’s timer, close, I/O, microtask and check queues. The microtask queue is given the most priority, followed by the timer queue, the I/O queue, the check queue, and, lastly, the close queue.
Microtask Queue
Within this type of callback queue, we’re constantly using the event loop to keep checking the queue for delayed functions before proceeding to other queues. The queue is first-in-first-out: tasks enqueued first run first. We implement this with two helpful different functions:
- process.nextTick: this is a function that would execute a different function within the queue at every next tick — the next iteration of the program. The structure of the microtask queue storing these functions makes it so easy that they can be executed at every next tick. The first queue holds this function.
- Promise: this represents the eventual completion (or failure) of an asynchronous operation. Promise handling is always asynchronous, as all promise actions pass through the internal “promise jobs” queue, also called “microtask queue”.
Timer Queue
Node.js provides us with a Timers module, containing functions that execute a program after a set period of time. This provides us with the means to have time-related operations that are asynchronous in nature.
setTimeout(() => {
//Executed last
...
}, 0);
console.log("Executed first");The event loop goes on executing all synchronous operations within the program, then returns to the callback queues only when all synchronous operations have been completely executed to execute the asynchronous operations.
I/O Queue
Here, we’re dealing with operations that involve interacting with external environments outside our internal program. These operations are asynchronously handled by Node.js after the synchronous code has finished running.
Examples of these operations include read and write file operations, network operations, and other operations that reach outside your program to fetch data for use within your program. I/O callback queue transfers the operation to the event loop which in turn transfers to the call stack for execution.
Check Queue
This queue allows immediate execution of the callback queue immediately after the callback functions in the IO queue have been executed. With this, we continually check the call stack if it becomes idle and scripts have been queued with setImmediate(), the event loop may continue to execute the check callback queue rather than waiting.
setImmediate() is actually a special timer that runs in a separate phase of the event loop. It uses a libuv API that schedules callbacks to execute after the poll phase has been completed.
Close Queue
This queue stores functions that are associated with close event operations. One example is stream, an abstract interface for working with streaming data in Node.js. For instance, the stream close event, which is emitted when the stream has been closed. It signifies that no more events will be emitted, and the HTTP close event, which is emitted when the server closes.
Implementing Queue in Javascript Using OOP
We’ll be implementing the FIFO — First in First Out, nature of Queues as a data structure. The basic concept at the root of this implementation is adding elements at the end of the queue and removing the elements at the frontal section of the queue. Arrays are used to implement queues in JavaScript. Let’s explore it:
Class Queue {
constructor() {
this.list = [];
}
}We just created a Queue class that contains an array, list, representing the bare minimum structure of a queue, using the constructor function to implement the array that would be used for the queue in this program.
Next, we start defining different methods that would exist on the queue data structure using conventional naming for each one.
The first function would be used to add an element at the end of a queue. We’ll use the Array.prototype.push() method to implement it.
Class Queue {
...
enqueue = (el) => {
this.list.push(el);
}
}In the above code, we’re defining a function with the name enqueue that accepts an argument. This argument is what gets added to the end of the list array.
The next function would remove elements from the front of the queue. We’ll use the Array.prototype.shift() method to implement this functionality.
Class Queue {
...
dequeue = () => this.list.length !== 0 ? this.list.shift() : “No executable element”;
}We implemented a function, dequeue, within the Queue class to always check if the array is empty. The array length isn’t equal to zero, before running the shift method to remove an element from the front of the array — queue.
The next thing we'll define is the helper functions for the Queue class we just created. These helper functions will be useful when working with the queue data structure.
This function will help to implement a functionality that would return an element with the position passed in as an argument within the queue:
Class Queue { ...
queueIndex = (index) => this.list.length !== 0 ? this.list[index] : "No executable element";}For the above method, we’re simply passing in the position of the element we want to retrieve, carefully ensuring the array isn’t empty.
We have constantly encountered the need to always check first if the array is empty before implementing the intended operation we want to carry out on the queue. To solve this, we’ll implement a helper function to constantly check the emptiness of the queue before implementing any operation:
Class Queue {
...
isEmpty = () => this.list.length !== 0;
}With this function, we can easily restructure our Queue method to use this function to check if the list array is empty or not.
This next function lies on what purpose the queue we’re implementing would serve for us within our application or program.
Class Queue {
...
execAll = () => {
this.list.forEach(item => item())
}
}Our program assumes to be executing asynchronous operations synchronously. Meaning the elements within the queue are functions that would be executed within our program.
We’ll use the queue and all the methods defined within it to demonstrate a simple program that executes functions stored within the queue.
const queue = new Queue();
//add an element
queue.enqueue(() => "Hello world program");
queue.enqueue(() => "Demo program");
queue.enqueue((a, b) => {
switch (a) {
case a === b:
return `${a} and ${b} are absolute equals`;
case a !== b:
return `${a} and ${b} are absolute unequal`;
default:
return "No data passed in";
}
});
queue.enqueue(() => ({
first_name: "Amanda",
last_name: "Petes",
email: "[email protected]",
}));
queue.dequeue();
let executable = queue.queueIndex(2);
console.log(executable());
queue.execAll();
console.log(queue);
/**{
first_name: 'Amanda',
last_name: 'Petes',
email: '[email protected]'
}
Demo program
No data passed in
{
first_name: 'Amanda',
last_name: 'Petes',
email: '[email protected]'
}
Queue {
enqueue: [Function: enqueue],
dequeue: [Function: dequeue],
queueIndex: [Function: queueIndex],
isEmpty: [Function: isEmpty],
execAll: [Function: execAll],
list: [
[Function (anonymous)],
[Function (anonymous)],
[Function (anonymous)]
]
} */In the above code, we implemented a queue variable with a Queue class. We filled this queue with functions or better still operations that would be executed synchronously.
These functions are assumed to be asynchronous functions. We could achieve that with the enqueue method on the Queue class. We then removed an element from the front of the queue using the dequeue method that also exists on the Queue class.
We used the queueIndex method passing in a parameter of 2 into it. This returned an element or function at the index position of 2.
We saved this anonymous function in a variable called executable, and then executed the function. Then finally called the function that executes all the functions within the queue. The execAll method loops over all the elements within the queue.list element, calling each element within it.
Conclusion
Are you a Node.js software developer interested in expanding your knowledge and keeping your skills up-to-date? Here's what you should read next:
- MongoDB Aggregation Pipeline (detailed explanation with examples)
- Nodejs Architecture (most common problems and challenges)
- Containerizing Node.js Applications with Docker (a practical guide)
