utf16

utf16 encodes and decodes UTF-16 code units, including surrogate pairs.

Use encode/decode when you already have []u16 code units (for example, from a language or format that exposes UTF-16 text as 16-bit units). Use encode_le, encode_be, decode_le and decode_be for the byte-oriented UTF-16LE and UTF-16BE representations used by files and network protocols.

Invalid code points are encoded as U+FFFD. decode is lenient and replaces invalid surrogate sequences with U+FFFD, matching the behaviour of Go's unicode/utf16; decode_strict returns an error instead.

rune_len returns the number of units produced by encode_rune: one for BMP scalar values and invalid code points, and two for U+10000 through U+10FFFF. encode replaces malformed UTF-8 input in the same way as string.runes().

The byte decoders reject odd byte lengths and replace unpaired surrogates. The endian-specific functions do not add, remove, or detect a byte order mark; U+FEFF is preserved as text, and the selected function determines the byte order.

import encoding.utf16

units := utf16.encode('V 😀')
assert utf16.decode(units) == 'V 😀'
assert utf16.encode_le('A') == [u8(0x41), 0]
assert utf16.decode_be([u8(0), 0x41]) or { panic(err) } == 'A'

fn append_rune(mut units []u16, r rune)

append_rune appends the UTF-16 code units for r to units. Invalid code points and surrogate values are appended as U+FFFD.

fn decode(units []u16) string

decode converts UTF-16 code units to a string. Unpaired surrogates and truncated surrogate pairs are replaced with U+FFFD.

fn decode_be(bytes []u8) !string

decode_be decodes big-endian UTF-16 bytes into a string. An odd number of bytes is an error; invalid surrogate sequences are replaced with U+FFFD.

fn decode_le(bytes []u8) !string

decode_le decodes little-endian UTF-16 bytes into a string. An odd number of bytes is an error; invalid surrogate sequences are replaced with U+FFFD.

fn decode_rune(units []u16) (rune, int)

decode_rune decodes the first code point in units. It returns the decoded rune and the number of code units consumed. An unpaired surrogate or a truncated surrogate pair decodes to U+FFFD and consumes one unit.

fn decode_strict(units []u16) !string

decode_strict converts UTF-16 code units to a string, returning an error when an unpaired surrogate or a truncated surrogate pair is present.

fn encode(s string) []u16

encode returns the UTF-16 code units of s.

fn encode_be(s string) []u8

encode_be returns the big-endian byte representation of s.

fn encode_le(s string) []u8

encode_le returns the little-endian byte representation of s.

fn encode_rune(r rune) []u16

encode_rune returns the UTF-16 code units for r. Invalid code points and surrogate values are encoded as U+FFFD.

fn is_surrogate(r rune) bool

is_surrogate reports whether r is a UTF-16 surrogate code point (U+D800..U+DFFF). Surrogates are not valid Unicode scalar values, but they do appear as individual code units inside UTF-16 data.

fn is_valid_rune(r rune) bool

is_valid_rune reports whether r is a Unicode scalar value, i.e. a code point that can be encoded as UTF-16 without loss.

fn rune_len(r rune) int

rune_len returns the number of UTF-16 code units needed to represent r. Valid code points in the BMP need one unit, code points above U+FFFF need two. Invalid code points are represented by a single replacement character.