How to Convert Node.js Buffer to String

Buffer Objects

For many years, JavaScript lacked support for handling arrays of binary data. That changed with ECMAScript 2015, when typed arrays were introduced to handle those cases more effectively.

All these types descend from an abstract TypedArray class. Each one specializes in specific integer word sizes and signing, and some provide floating-point support. Float64Array, for example, represents an array of 64-bit floating-point numbers; Int16Array represents an array of 16-bit signed integers.

Of all typed arrays, Uint8Array is probably the most useful. It allows JavaScript to handle binary data, image files, networking protocols, and video streaming, in a similar way to other programming languages.

In the Node.js world, there’s another class, Buffer, that descends from Uint8Array. It adds some utility methods like concat(), allocUnsafe(), and compare(), and methods to read and write various numeric types, mimicking what’s offered by the DataView class (eg: readUint16BE, writeDoubleLE, and swap32).

Text Encoding

Character encoding is a vast and complex subject, which is beyond the scope of this article.

Conceptually, text is composed of characters: the smallest pieces of meaningful content, usually letters, though some languages use very different symbols. Each character is assigned a unique code point number used to identify it in a one-to-one relationship.

Finally, each code point may be represented by a sequence of bits according to an encoding. An encoding is a table that maps each code point to its corresponding bit string.

It may look like simple mapping, but multiple encodings were created in recent decades to accommodate different computer architectures and character sets.

The most widespread encoding is ASCII.

It includes the basic Latin characters (A-Z, upper and lower case), digits, and many punctuation symbols, besides some control characters. ASCII is a 7-bit encoding, but computers store data in 8-bit blocks called bytes, so many 8-bit encodings were created, extending the first 128 ASCII characters with another 128 language-specific characters.

A very common encoding is ISO-8859-1, also known by several other names like “latin1”, “IBM819”, “CP819”, and even “WE8ISO8859P1”. ISO-8859-1 adds some extra symbols to ASCII and many accented letters like Á, Ú, È, and Õ.

Those language-specific encodings worked reasonably well for some situations but failed miserably in multi-language contexts.

Data corruption and information loss were common as programs tried to process text using the wrong file encoding.

Even displaying the file contents was difficult, because text files don’t include information about the encoding used to create them.

The Unicode standard was introduced in the 1980s as a solution to this problem. Its objective was to map every character used in modern language, and many historical scripts, by assigning each one a single code point.

The Unicode standard also defines a number of generic encodings that are able to encode every Unicode code point.

The most common Unicode encodings are UTF-16, UCS-2, UTF-32, and UTF-8. UCS-2 and UTF-32 use a fixed width for all code points, while UTF-8 and UTF-16 use a variable number of bits for each code point. UCS-2 has a limited range and is unable to represent all Unicode characters.

On the web, the most widely used Unicode encoding is UTF-8, because it’s reasonably efficient for many uses. It uses 1 byte for the ASCII characters, and 2 bytes for most European and Middle-East scripts - but it’s less efficient for Asian scripts, requiring 3 bytes or more. For example, the French saying “Il vaut mieux prévenir que guérir” is equivalent to the following sequence of bytes when it’s encoded using UTF-8 (the bytes are shown in hexadecimal):

Convert Buffer to String Node.js

The content of a Buffer may be quickly converted to a String using the toString() method:

const b = Buffer.from([101, 120, 97, 109, 112, 108, 101]);
console.log(b.toString()); // example

The Buffer class uses UTF-8 by default when converting to/from strings, but you can also choose another one from a small set of supported encodings:

const b = Buffer.from([101, 120, 97, 109, 112, 108, 101]);
console.log(b.toString('latin1')); // example

In most cases, UTF-8 is the best option for both reading and writing. Here is the full list of encodings supported by Node.js as of September/2021 (names are not case sensitive):

Convert Node.js String to Buffer

You can also convert data in the opposite direction by creating a new Buffer from a string. If the encoding is not specified, Node.js uses UTF-8 by default:

const s = Buffer.from('example', 'utf8');
console.log(s); // <Buffer 65 78 61 6d 70 6c 65>

If you need to write text to an existing Buffer object, you can use its write() method:

const b = Buffer.alloc(10);
console.log(b);
// <Buffer 00 00 00 00 00 00 00 00 00 00>

b.write('example', 'utf8');
console.log(b);
// <Buffer 65 78 61 6d 70 6c 65 00 00 00>

You can even set the starting position (offset):

const b = Buffer.alloc(10);
b.write('test', 4, 'utf8');
console.log(b);
// <Buffer 00 00 00 00 74 65 73 74 00 00>

Conclusion

In a nutshell, it’s easy to convert a Buffer object to a string using the toString() method. You’ll usually want the default UTF-8 encoding, but it’s possible to indicate a different encoding if needed. To convert from a string to a Buffer object, use the static Buffer.from() method - again optionally passing the encoding.

If you’re a Node.js developer interested in advancing your knowledge, you may enjoy reading Queues in Node.js (all types explained).

FAQs

Q: Is Buffer a string?

A string is a sequence of characters, but a Buffer is a sequence of bytes. Even though a buffer might contain the encoded content of a string value, it may also encode other kinds of values or any binary data.

Q: What do Buffer objects do in Node.JS?

Buffers allow storing and manipulating byte arrays, especially when working with files: functions like fs.readFile return Buffer objects when reading binary files. Buffers are also important for handling network communications and image processing.

Q: What is a Buffer object in programming?

A Buffer object is a way to abstract sequences or arrays of bytes. Besides common array operations like getting and changing one or more elements of the byte array, Buffers usually have advanced methods to read and write more complex values such as integers, floating point numbers and strings.