Glossary / Node.js buffers

Node.js buffers

Buffer is Node.js's byte sequence type, implemented as a Uint8Array subclass, for binary data such as files, sockets, cryptographic input, and protocol frames. A Buffer represents bytes, not text; encoding and ownership decisions determine how those bytes are interpreted and copied.

Construction and encoding

Use Buffer.from() for known input or Buffer.alloc() for zero-filled memory. Avoid Buffer.allocUnsafe() unless every byte is overwritten before any read or exposure; it may contain old process memory contents. Buffer.from(string, encoding) encodes text; buffer.toString(encoding) decodes it. UTF-8 characters may span chunks, so decoding each chunk independently can corrupt split multibyte characters; use StringDecoder or streaming TextDecoder for incremental decoding.

const bytes = Buffer.from('hello', 'utf8');
const copy = Buffer.from(bytes);        // copies bytes
const view = bytes.subarray(1, 3);      // shares backing memory

slice() historically returns a view sharing storage; subarray() explicitly does too. Mutating a view affects the shared backing bytes. To make an independent copy, use Buffer.from(view).

Memory, limits, and security

Buffers may use pooled memory for small allocations; do not infer lifetime or isolation from their allocation shape. Be careful when exposing slices because they can retain a larger backing allocation and reveal neighboring bytes if code exposes a larger range than intended. Validate lengths before allocating from untrusted inputs and impose protocol limits. Concatenating many chunks repeatedly can create quadratic copying; accumulate chunks and concatenate once, or stream them.

For typed arrays, be precise about byte offset and length: Buffer.from(arrayBuffer, offset, length) can share an ArrayBuffer; Buffer.from(typedArray) copies element values as bytes rather than necessarily preserving the original view's backing layout. Check API semantics for the specific overload.

Engineering guidance

Keep data binary until a known text boundary. Specify encodings explicitly. Bound allocation sizes and avoid unsafe allocation unless fully initialized. Test non-ASCII split boundaries, empty buffers, views, and large payloads. Use stream pipelines for large files instead of reading entire content into memory.

Interview questions

  1. Is a Buffer text? No; it is bytes interpreted using an encoding when converted to or from strings.
  2. Does subarray() copy? No; it creates a view sharing underlying memory.
  3. Why avoid allocUnsafe? It may contain uninitialized bytes unless overwritten, risking accidental data disclosure.
  4. How can UTF-8 decoding fail across chunks? A multibyte sequence may be split; incremental decoders retain incomplete bytes.
  5. How to avoid repeated concatenation costs? Collect bounded chunks and concatenate once, or process as a stream.

References