Glossary / Transform streams

Transform streams

A TransformStream couples a writable input to a readable output through a transformation algorithm. It is useful when data should be processed incrementally—for example, compression, encoding, filtering, parsing, or redaction—while preserving stream composition and backpressure.

Transform lifecycle

A transformer can implement start(controller), transform(chunk, controller), and flush(controller). start initializes state. transform receives each input chunk and may enqueue zero, one, or many output chunks. It can return a promise to apply asynchronous work and constrain progress. flush runs when the writable side is closed, allowing final buffered state to be emitted; it is not called as a normal completion step after every kind of abort or error.

const lines = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk.toUpperCase());
  },
});
const output = input.pipeThrough(lines);

The readable and writable sides have independent queuing strategies. Piping links their progress, but an implementation that eagerly emits many chunks or stores unbounded state can still consume excessive memory. A transform is not necessarily one-input-to-one-output: it may buffer incomplete records, discard input, or emit multiple outputs.

Errors and cancellation

If transform throws or rejects, the transform errors and piping propagates failure according to the pipe options. A transform may error explicitly using its controller. Cancellation and abort behavior flows through the connected pipeline unless prevented by options. Always decide how partial state is handled on close, cancellation, and error; for example, a parser may reject an incomplete final record rather than silently drop it.

Engineering guidance

Keep transform state bounded and make chunk-boundary behavior explicit. Preserve encoding state for multibyte text and framing state for records. Apply asynchronous work with controlled concurrency; launching every chunk at once defeats backpressure and can reorder outputs. Test empty input, partial records, errors, cancellation, and large chunks.

Interview questions

  1. Can a transform emit multiple outputs for one input? Yes; transform can enqueue zero or more chunks.
  2. What is flush for? Emitting or validating final buffered state during normal writable close.
  3. Does TransformStream guarantee bounded memory? No; queuing strategies and transformer state must be bounded.
  4. How do you handle async transformations? Return/await the work in the transform algorithm so the stream can apply pressure and preserve intended sequencing.
  5. What happens if transform throws? The stream errors and connected pipe operations reject according to propagation rules.

References