Please support this book: buy it or donate

36. Asynchronous programming in JavaScript

This chapter explains the foundations of asynchronous programming in JavaScript.

36.1. A roadmap for asynchronous programming in JavaScript

This section provides a roadmap for the content on asynchronous programming in JavaScript.

36.1.1. Synchronous functions

Normal functions are synchronous: the caller waits until the callee is finished with its computation. divideSync() in line (A) is a synchronous function call:

  1. function main() {
  2.   try {
  3.     const result = divideSync(12, 3); // (A)
  4.     assert.equal(result, 4);
  5.   } catch (err) {
  6.     assert.fail(err);
  7.   }
  8. }

36.1.2. JavaScript executes tasks sequentially in a single process

By default, JavaScript tasks are functions that are executed sequentially in a single process. That looks like this:

  1. while (true) {
  2.   const task = taskQueue.dequeue();
  3.   task(); // run task
  4. }

This loop is also called the event loop, because events such as clicking a mouse add tasks to the queue.

Due to this style of cooperative multi-tasking, we don’t want a task to block other tasks from being executed while, e.g., it waits for results coming from a server. The next section explores how to handle this case.

36.1.3. Callback-based asynchronous functions

What if divide() needs a server to compute its result? Then the result should be delivered in a different manner: The caller shouldn’t have to wait (synchronously) until the result is ready, it should be notified (asynchronously) when it is. One way of delivering the result asynchronously is by giving divide() a callback function that it uses to notify the caller.

  1. function main() {
  2.   divideCallback(12, 3,
  3.     (err, result) => {
  4.       if (err) {
  5.         assert.fail(err);
  6.       } else {
  7.         assert.equal(result, 4);
  8.       }
  9.     });
  10. }

This is what happens after an asynchronous function call divideCallback(x, y, callback):

  • divideCallback() sends a request to a server.
  • Then the current task is finished (for now) and other tasks can be executed.
  • When a response from the server arrives, it is either:
    • An error err: Then the following task is added to the queue.
  1. taskQueue.enqueue(() => callback(err));
  • A result r: Then the following task is added to the queue.
  1. taskQueue.enqueue(() => callback(null, r));

36.1.4. Promise-based asynchronous functions

Promises are two things:

  • A standard pattern that makes working with callbacks easier.
  • The mechanism on which async functions (the topic of the next section) are built.
    Invoking a Promise-based function looks as follows.
  1. function main() {
  2.   dividePromise(12, 3)
  3.     .then(result => assert.equal(result, 4))
  4.     .catch(err => assert.fail(err));
  5. }

36.1.5. Async functions

One way of looking at async functions is as better syntax for Promise-based code:

  1. async function main() {
  2.   try {
  3.     const result = await dividePromise(12, 3); // (A)
  4.     assert.equal(result, 4);
  5.   } catch (err) {
  6.     assert.fail(err);
  7.   }  
  8. }

The dividePromise() we are calling in line A is the same Promise-based function from the previous section. But we now have synchronous-looking syntax for making the call. await can only be used inside a special kind of function, an async function (note the keyword async in front of the keyword function). await pauses the current async function and returns from it. Once the awaited result is ready, the execution of the function continues where it left off.

36.1.6. Next steps

36.2. The call stack

Whenever a function calls another function, we need to remember where to return to, after the latter function is finished. That is typically done via a stack, the call stack: The caller pushes onto it the location to return to, and the callee jumps to that location after it is done.

This is an example where several calls happen:

  1. function h(z) {
  2.   const error = new Error();
  3.   console.log(error.stack);
  4. }
  5. function g(y) {
  6.   h(y + 1);
  7. }
  8. function f(x) {
  9.   g(x + 1);
  10. }
  11. f(3);
  12. // done

Initially, before running this piece of code, the call stack is empty. After the function call f(3) in line 11, the stack has one entry:

  • Line 12 (location in top-level scope)
    After the function call g(x + 1) in line 9, the stack has two entries:

  • Line 10 (location in f())

  • Line 12 (location in top-level scope)
    After the function call h(y + 1) in line 6, the stack has three entries:

  • Line 7 (location in g())

  • Line 10 (location in f())
  • Line 12 (location in top-level scope)
    Creating the exception in line 2 is yet another call. That’s why the call stack that is recorded inside the exception error includes a location inside h(). Logging error produces the following output (note that stack traces record where calls are made, not return locations):
  1. Error
  2.     at h (demos/async-js/stack_trace.js:2:17)
  3.     at g (demos/async-js/stack_trace.js:6:3)
  4.     at f (demos/async-js/stack_trace.js:9:3)
  5.     at <top level> (demos/async-js/stack_trace.js:11:1)

Afterwards, each of the functions terminates and each time the top entry is removed from the stack. After function f is done, we are back in top-level scope and the call stack is empty. When the code fragment ends then that is like an implicit return. If we consider the code fragment to be a task that is executed, then returning with an empty call stack ends the task.

36.3. The event loop

By default, JavaScript runs in a single process – in both web browsers and Node.js. The so-called event loop sequentially executes tasks (pieces of code) inside that process. The event loop is depicted in fig. 20.

Figure 20: Task sources add code to run to the task queue, which is emptied by the event loop.

Figure 20: Task sources add code to run to the task queue, which is emptied by the event loop.

Two parties access the task queue:

  • Task sources add tasks to the queue. Some of those sources run concurrently to the JavaScript process. For example, one task source takes care of user interface events: If a user clicks somewhere and that triggers JavaScript code, then that code is added to the task queue.

  • The event loop runs continuously, inside the JavaScript process. It takes a task out of the queue and executes it. Once the call stack is empty and there is a return, the current task is finished. Control goes back to the event loop, which then retrieves the next task from the queue and executes it. Etc.

The following JavaScript code is an approximation of the event loop:

  1. while (true) {
  2.   const task = taskQueue.dequeue();
  3.   task(); // run task
  4. }

36.4. How to avoid blocking the JavaScript process

36.4.1. The user interface of the browser can be blocked

Many of the user interface mechanisms of browsers also run in the JavaScript process (as tasks). Therefore, long-running JavaScript code can block the user interface. Let’s look at a web page that demonstrates that. There are two ways in which you can try out that page:

  • You can run it online.
  • You can open the following file inside the repository with the exercises: demos/async-js/blocking.html
    The following HTML is the page’s user interface:
  1. <a id="block" href="">Block</a>
  2. <div id="statusMessage"></div>
  3. <button>Click me!</button>

The idea is that you click “Block” and a long-running loop is executed via JavaScript. During that loop, you can’t click the button, because the browser/JavaScript process is blocked.

A simplified version of the JavaScript code looks like this:

  1. document.getElementById('block')
  2.   .addEventListener('click', doBlock); // (A)
  4. function doBlock(event) {
  5.   // ···
  6.   setStatus('Blocking...');
  7.   // ···
  8.   sleep(5000); // (B)
  9.   setStatus('Done');
  10. }
  12. function sleep(milliseconds) {
  13.   const start = Date.now();
  14.   while ((Date.now() - start) < milliseconds);
  15. }
  16. function setStatus(status) {
  17.   document.getElementById('statusMessage')
  18.     .textContent = status;
  19. }

These are the key parts of the code:

  • Line A: We tell the browser to call doBlock() whenever the HTML element is clicked whose ID is block.
  • doBlock() runs a loop for 5000 milliseconds (line B).
  • sleep() does the actual looping.
  • setStatus() displays status messages inside the <div> whose ID is statusMessage.

36.4.2. How can we avoid blocking the browser?

There are several ways in which you can prevent a long-running operation from blocking the browser:

  • The operation can deliver its result asynchronously: Some operations, such as downloads, can be performed concurrently to the JavaScript process. The JavaScript code triggering such an operation registers a callback, which is invoked with the result once the operation is finished. The invocation is handled via the task queue. This style of delivering a result is called asynchronous, because the caller doesn’t wait until the results are ready. Normal function calls deliver their results synchronously.

  • Perform long computations in separate processes: This can be done via so-called Web Workers. Web Workers are heavyweight processes that run concurrently to the main process. Each one of them has its own runtime environment (global variables etc.). They are completely isolated and must be communicated with via message passing. Consult MDN web docs for more information.

  • Take breaks during long computations. The next section explains how.

36.4.3. Taking breaks

The following global function executes its parameter callback after a delay of ms milliseconds (the type definition is simplified – setTimeout() has more features):

  1. function setTimeout(callback: () => void, ms: number): any

The function returns a handle (an ID) that can be used to clear (cancel) it, via the following global function:

  1. function clearTimeout(handle?: any): void

setTimeout() is available on both browsers and Node.js. The next section shows it in action.

36.4.4. Run-to-completion semantics

JavaScript makes the following guarantee for tasks:

Each task is always finished (“run to completion”) before the next task is executed.

That means that tasks don’t have to worry about their data being changed while they are working on it (concurrent modification). That simplifies programming in JavaScript.

The following example demonstrates this guarantee:

  1. console.log('start');
  2. setTimeout(() => {
  3.   console.log('callback');
  4. }, 0);
  5. console.log('end');
  7. // Output:
  8. // 'start'
  9. // 'end'
  10. // 'callback'

setTimeout() puts its parameter into the task queue. The parameter is therefore executed sometime after the current piece of code (task) is completely finished.

The parameter ms only specifies when the task is put into the queue, it does not specify when exactly the task runs. It may even never run, if there is a task before it in the queue that never terminates. That explains, why the previous code logs 'end' before 'delayed', even though the delay is zero milliseconds.

36.5. Patterns for delivering asynchronous results

These are three popular patterns for delivering results asynchronously in JavaScript:

  • Events
  • Callbacks
  • Promises
    The first two patterns are explained next. Promises are explained in the next chapter.

36.5.1. Delivering asynchronous results via events

Events as a pattern work as follows:

  • They are used to deliver values asynchronously.
  • They do so zero or more times.
  • There are three roles in this pattern:
    • The event (an object) carries the data to be delivered.
    • The event listener is a function that receives events via a parameter.
    • The event source sends events and lets you register event listeners.
      Multiple variations of this pattern exist in the world of JavaScript. We’ll look at three examples next.
36.5.1.1. Events: IndexedDB

IndexedDB is a database that is built into web browsers. This is an example of using it:

  1. const openRequest = indexedDB.open('MyDatabase', 1); // (A)
  3. openRequest.onsuccess = (event) => {
  4.   const db = event.target.result;
  5.   // ···
  6. };
  8. openRequest.onerror = (error) => {
  9.   console.error(error);
  10. };

indexedDB has an unusual way of invoking operations:

  • Each operation has an associated method for creating request objects. For example, in line A, the operation is “open”, the method is .open() and the request object is openRequest.

  • The parameters for the operation are provided via the request object, not via parameters of the method. For example, the event listeners (functions) are stored in the properties .onsuccess and .onerror.

  • The invocation of the operation is added to the task queue via the method (in line A). That is, we configure the operation after its invocation has already been added to the queue. Only run-to-completion semantics save us from race conditions here and ensure that the operation runs after the current code fragment is finished.

36.5.1.2. Events: XMLHttpRequest

The XMLHttpRequest API lets you make downloads from within a web browser. This is how you download the file http://example.com/textfile.txt:

  1. const xhr = new XMLHttpRequest(); // (A)
  2. xhr.open('GET', 'http://example.com/textfile.txt'); // (B)
  3. xhr.onload = () => { // (C)
  4.   if (xhr.status == 200) {
  5.     processData(xhr.responseText);
  6.   } else {
  7.     assert.fail(new Error(xhr.statusText));
  8.   }
  9. };
  10. xhr.onerror = () => { // (D)
  11.   assert.fail(new Error('Network error'));
  12. };
  13. xhr.send(); // (E)
  15. function processData(str) {
  16.   assert.equal(str, 'Content of textfile.txt\n');
  17. }

With this API, you first create a request object (line A), then configure it, then send it (line E). The configuration consists of:

  • Specifying which HTTP request method to use (line B): GET, POST, PUT, etc.
  • Registering a listener (line C) that is notified if something could be downloaded. Inside the listener, you still need to determine if the download contains what you requested or informs you of an error. Note that some of the result data is delivered via xhr. I’m not a fan of this kind of mixing of input and output data.
  • Registering a listener (line D) that is notified if there was a network error.
36.5.1.3. Events: DOM

We have already seen DOM events in action, in the section on blocking browser UIs. The following code also handles click events:

  1. const element = document.getElementById('my-link'); // (A)
  2. element.addEventListener('click', clickListener); // (B)
  4. function clickListener(event) {
  5.   event.preventDefault(); // (C)
  6.   console.log(event.shiftKey); // (D)
  7. }

We first ask the browser to retrieve the HTML element whose ID is my-link (line A). Then we add a listener for all click events (line B). In the listener, we first tell the browser not to perform its default action – going to the target of the link (line C). Then we log to the console if the shift key is currently pressed (line D).

36.5.2. Delivering asynchronous results via callbacks

Callbacks are another pattern for handling asynchronous results. They are only used for one-off results and have the advantage of being less verbose than events.

As an example, consider a function readFile() that reads a text file and returns its contents asynchronously. This is how you call readFile() if it uses Node.js-style callbacks:

  1. readFile('some-file.txt', {encoding: 'utf8'},
  2.   (error, data) => {
  3.     if (error) {
  4.       assert.fail(error);
  5.       return;
  6.     }
  7.     assert.equal(data, 'The content of some-file.txt\n');
  8.   });

There is a single callback that handles both success and failure. If the first parameter is not null then an error happened. Otherwise, the result can be found in the second parameter.

36.6. Asynchronous code: the downsides

In many situations, on either browsers or Node.js, you have no choice: You must use asynchronous code. In this chapter, we have seen several patterns that such code can use. All of them have two disadvantages:

  • Asynchronous code is more verbose than synchronous code.
  • If you call asynchronous code, your code must become asynchronous, too. That’s because you can’t wait synchronously for an asynchronous result. Asynchronous code has an infectious quality.
    The first disadvantage becomes less severe with Promises (covered in the next chapter) and mostly disappears with async functions (covered in the chapter after the next one).

Alas, the infectiousness of async code does not go away. But it is mitigated by the fact that switching between sync and async is easy with async functions.

36.7. Resources