Glossary / Unicode and UTF-16

Unicode and UTF-16

JavaScript strings are sequences of UTF-16 code units, not bytes, Unicode code points, or user-perceived characters. A visible grapheme may consist of multiple code points—for example a base letter plus combining mark or an emoji sequence joined with zero-width joiners. Correct string handling depends on whether a task concerns storage, code units, code points, grapheme clusters, normalization, or display.

Length and iteration

const emoji = "😀";
emoji.length; // 2 UTF-16 code units
[...emoji].length; // 1 code point

Index access can split a surrogate pair. for...of iterates code points, but it can still split a grapheme formed from several code points. Intl.Segmenter can segment grapheme clusters, words, and sentences according to locale-sensitive rules. User-facing character limits should generally count graphemes, not string.length.

Visually identical text can have different code-point sequences (precomposed versus base-plus-combining mark). Unicode normalization such as NFC can help comparison/search, but should be applied consistently and deliberately; normalization is not a complete security solution. Confusable characters can impersonate identifiers across scripts. Encoding, database collation, case folding, and sorting are separate concerns.

Engineering guidance

Test with combining marks, non-Latin scripts, emoji modifiers, family emoji, RTL text, and malformed surrogate data where relevant. Avoid slicing strings by arbitrary UTF-16 index for user-visible text. Use Unicode-aware regex and segmentation APIs intentionally, and document which unit a limit or cursor position measures.

Interview questions

  1. What does string.length count? UTF-16 code units.
  2. Does for-of iterate grapheme clusters? It iterates code points; a grapheme may contain multiple code points.
  3. How count user-perceived characters? Use grapheme segmentation such as Intl.Segmenter.
  4. Why normalize Unicode? Canonically equivalent sequences can otherwise compare differently; apply a deliberate consistent policy.

References