encoding.binary #

Description

encoding.binary contains utility functions for converting between an array of bytes ([]u8) and unsigned integers of various widths (u16, u32, and u64).

Also, it provide functions encode_binary[T]() and decode_binary[T]() which can converting between an array of bytes ([]u8) and generic type T.

There are two ways in which bytes can be encoded:

1. Little endian: The least significant bytes are stored first, followed by the most significant bytes.
2. Big endian: The most significant bytes are stored first, opposite to the little endian convention.

For example, let us take the number 0x12345678. In little endian, the bytes are extracted as 0x78, 0x56, 0x34, and 0x12. In big endian, the bytes are 0x12, 0x34, 0x56, and 0x78.

We follow a similar procedure when we want to go the other way around. Consider the second sequence of bytes in the previous example: 0x12, 0x34, 0x56, and 0x78. If we encode this sequence in little endian format, we get the integer 0x78563412. If we encode this sequence in big endian, we get 0x12345678.

Note
The endian decode and put functions require enough bytes within the supplied slice for the
requested integer width (2, 4, or 8 bytes). The _at variants also require a nonnegative
offset and enough bytes after it; the _end variants use the final bytes of the slice.
Invalid lengths or offsets panic before any read or write, even when the slice has a larger
backing array. little_endian_f32_at follows the same rule for its 4-byte value.

IEEE 754 floats can be read or written at an offset with little_endian_f32_at, big_endian_f32_at, little_endian_f64_at, big_endian_f64_at, and their little_endian_put_f32_at, big_endian_put_f32_at, little_endian_put_f64_at, big_endian_put_f64_at counterparts. These preserve the exact floating point bits, including negative zero, subnormal values, infinities, and NaN payloads. The _at bounds requirements above apply to all of them; f32 needs four bytes and f64 needs eight.

For generic T data encoding/decoding, you can use encode_binary[T]() and decode_binary[T]():

module main

import encoding.binary

struct MyStruct {
    g_u8 u8
}

struct ComplexStruct {
mut:
    f_u8      u8
    f_u32     u32 @[serialize: '-'] // this field will be skipped
    f_u64     u64
    f_string  string
    f_structs []MyStruct
    f_maps    []map[string]string
}

fn main() {
    a := ComplexStruct{
        f_u8:      u8(10)
        f_u32:     u32(1024)
        f_u64:     u64(2048)
        f_string:  'serialize me'
        f_structs: [
            MyStruct{
                g_u8: u8(1)
            },
            MyStruct{
                g_u8: u8(2)
            },
            MyStruct{
                g_u8: u8(3)
            },
        ]
        f_maps:    [
            {
                'abc': 'def'
            },
            {
                '123': '456'
            },
            {
                ',./': '!@#'
            },
        ]
    }

    b := binary.encode_binary(a)!
    mut c := binary.decode_binary[ComplexStruct](b)!

    // because there skipped field in `a`, a != c
    assert a != c

    c.f_u32 = u32(1024)
    assert a == c
}

For Go-style fixed-size stream I/O, you can use read(), write(), and size() with binary.little_endian or binary.big_endian:

module main

import encoding.binary
import io

struct Header {
    version u16
    flags   u16
    length  u32
}

struct Buffer {
mut:
    data []u8
    pos  int
}

fn (mut b Buffer) read(mut out []u8) !int {
    if b.pos >= b.data.len {
        return io.Eof{}
    }
    n := if out.len < b.data.len - b.pos { out.len } else { b.data.len - b.pos }
    copy(mut out[..n], b.data[b.pos..b.pos + n])
    b.pos += n
    return n
}

fn (mut b Buffer) write(src []u8) !int {
    b.data << src
    return src.len
}

fn main() {
    header := Header{
        version: 1
        flags:   2
        length:  32
    }
    mut buf := Buffer{}
    binary.write(mut buf, binary.big_endian, header)!
    assert binary.size(header) == 8

    buf.pos = 0
    mut decoded := Header{}
    binary.read(mut buf, binary.big_endian, mut decoded)!
    assert decoded == header
}

fn big_endian_f32_at(b []u8, o int) f32

big_endian_f32_at reads an IEEE 754 f32 from four bytes at offset o in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_f64_at(b []u8, o int) f64

big_endian_f64_at reads an IEEE 754 f64 from eight bytes at offset o in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_get_u16(v u16) []u8

big_endian_get_u16 creates u8 array from the unsigned 16-bit integer v in big endian order.

fn big_endian_get_u32(v u32) []u8

big_endian_get_u32 creates u8 array from the unsigned 32-bit integer v in big endian order.

fn big_endian_get_u64(v u64) []u8

big_endian_get_u64 creates u8 array from the unsigned 64-bit integer v in big endian order.

fn big_endian_put_f32_at(mut b []u8, v f32, o int)

big_endian_put_f32_at writes an IEEE 754 f32 to four bytes at offset o in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_put_f64_at(mut b []u8, v f64, o int)

big_endian_put_f64_at writes an IEEE 754 f64 to eight bytes at offset o in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_put_u16(mut b []u8, v u16)

big_endian_put_u16 writes a u16 to the first two bytes in the array b in big endian order. Panics if b is shorter than the value width.

fn big_endian_put_u16_at(mut b []u8, v u16, o int)

big_endian_put_u16_at writes a u16 to the two bytes in the array b at the specified offset in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_put_u16_end(mut b []u8, v u16)

big_endian_put_u16_end writes a u16 to the last two bytes in the array b in big endian order.

fn big_endian_put_u16_fixed(mut b [2]u8, v u16)

big_endian_put_u16_fixed writes a u16 to the fixed array b in big endian order.

fn big_endian_put_u32(mut b []u8, v u32)

big_endian_put_u32 writes a u32 to the first four bytes in the array b in big endian order. Panics if b is shorter than the value width.

fn big_endian_put_u32_at(mut b []u8, v u32, o int)

big_endian_put_u32_at writes a u32 to four bytes in the array b at the specified offset in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_put_u32_end(mut b []u8, v u32)

big_endian_put_u32_end writes a u32 to the last four bytes in the array b in big endian order.

fn big_endian_put_u32_fixed(mut b [4]u8, v u32)

big_endian_put_u32_fixed writes a u32 to the fixed array b in big endian order.

fn big_endian_put_u64(mut b []u8, v u64)

big_endian_put_u64 writes a u64 to the first eight bytes in the array b in big endian order. Panics if b is shorter than the value width.

fn big_endian_put_u64_at(mut b []u8, v u64, o int)

big_endian_put_u64_at writes a u64 to eight bytes in the array b at the specified offset in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_put_u64_end(mut b []u8, v u64)

big_endian_put_u64_end writes a u64 to the last eight bytes in the array b at the specified offset in big endian order.

fn big_endian_put_u64_fixed(mut b [8]u8, v u64)

big_endian_put_u64_fixed writes a u64 to the fixed array b in big endian order.

fn big_endian_u16(b []u8) u16

big_endian_u16 creates a u16 from the first two bytes in the array b in big endian order. Panics if b is shorter than the value width.

fn big_endian_u16_at(b []u8, o int) u16

big_endian_u16_at creates a u16 from two bytes in the array b at the specified offset in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_u16_end(b []u8) u16

big_endian_u16_end creates a u16 from two bytes in the array b at the specified offset in big endian order.

fn big_endian_u16_fixed(b [2]u8) u16

big_endian_u16_fixed creates a u16 from the first two bytes in the fixed array b in big endian order.

fn big_endian_u32(b []u8) u32

big_endian_u32 creates a u32 from four bytes in the array b in big endian order. Panics if b is shorter than the value width.

fn big_endian_u32_at(b []u8, o int) u32

big_endian_u32_at creates a u32 from four bytes in the array b at the specified offset in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_u32_end(b []u8) u32

big_endian_u32_end creates a u32 from the last four bytes in the array b in big endian order.

fn big_endian_u32_fixed(b [4]u8) u32

big_endian_u32_fixed creates a u32 from four bytes in the fixed array b in big endian order.

fn big_endian_u64(b []u8) u64

big_endian_u64 creates a u64 from the first eight bytes in the array b in big endian order. Panics if b is shorter than the value width.

fn big_endian_u64_at(b []u8, o int) u64

big_endian_u64_at creates a u64 from eight bytes in the array b at the specified offset in big endian order. Panics if o is negative or the value extends past the end of b.

fn big_endian_u64_end(b []u8) u64

big_endian_u64_end creates a u64 from the last eight bytes in the array b in big endian order.

fn big_endian_u64_fixed(b [8]u8) u64

big_endian_u64_fixed creates a u64 from the fixed array b in big endian order.

fn decode_binary[T](b []u8, config DecodeConfig) !T

decode_binary decode a u8 array into T type data. for decoding struct, you can use @[serialize: '-'] to skip field.

Note: shared fields in struct will be skipped.

fn encode_binary[T](obj T, config EncodeConfig) ![]u8

encode_binary encode a T type data into u8 array. for encoding struct, you can use @[serialize: '-'] to skip field.

Note: shared fields in struct will be skipped.

fn little_endian_f32_at(b []u8, o int) f32

little_endian_f32_at creates an f32 from four bytes at offset o in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_f64_at(b []u8, o int) f64

little_endian_f64_at reads an IEEE 754 f64 from eight bytes at offset o in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_get_u16(v u16) []u8

little_endian_get_u16 creates u8 array from the unsigned 16-bit integer v in little endian order.

fn little_endian_get_u32(v u32) []u8

little_endian_get_u32 creates u8 array from the unsigned 32-bit integer v in little endian order.

fn little_endian_get_u64(v u64) []u8

little_endian_get_u64 creates u8 array from the unsigned 64-bit integer v in little endian order.

fn little_endian_put_f32_at(mut b []u8, v f32, o int)

little_endian_put_f32_at writes an IEEE 754 f32 to four bytes at offset o in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_put_f64_at(mut b []u8, v f64, o int)

little_endian_put_f64_at writes an IEEE 754 f64 to eight bytes at offset o in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_put_u16(mut b []u8, v u16)

little_endian_put_u16 writes a u16 to the first two bytes in the array b in little endian order. Panics if b is shorter than the value width.

fn little_endian_put_u16_at(mut b []u8, v u16, o int)

little_endian_put_u16_at writes a u16 to the two bytes in the array b at the specified offset in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_put_u16_end(mut b []u8, v u16)

little_endian_put_u16_end writes a u16 to the last two bytes of the array b in little endian order.

fn little_endian_put_u16_fixed(mut b [2]u8, v u16)

little_endian_put_u16_fixed writes a u16 to the fixed array b in little endian order.

fn little_endian_put_u32(mut b []u8, v u32)

little_endian_put_u32 writes a u32 to the first four bytes in the array b in little endian order. Panics if b is shorter than the value width.

fn little_endian_put_u32_at(mut b []u8, v u32, o int)

little_endian_put_u32_at writes a u32 to the four bytes in the array b at the specified offset in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_put_u32_end(mut b []u8, v u32)

little_endian_put_u32_end writes a u32 to the last four bytes in the array b in little endian order.

fn little_endian_put_u32_fixed(mut b [4]u8, v u32)

little_endian_put_u32_fixed writes a u32 to the fixed array b in little endian order.

fn little_endian_put_u64(mut b []u8, v u64)

little_endian_put_u64 writes a u64 to the first eight bytes in the array b in little endian order. Panics if b is shorter than the value width.

fn little_endian_put_u64_at(mut b []u8, v u64, o int)

little_endian_put_u64_at writes a u64 to the eight bytes in the array b at the specified offset in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_put_u64_end(mut b []u8, v u64)

little_endian_put_u64_end writes a u64 to the last eight bytes in the array b at in little endian order.

fn little_endian_put_u64_fixed(mut b [8]u8, v u64)

little_endian_put_u64_fixed writes a u64 to the fixed array b in little endian order.

fn little_endian_u16(b []u8) u16

little_endian_u16 creates a u16 from the first two bytes in the array b in little endian order. Panics if b is shorter than the value width.

fn little_endian_u16_at(b []u8, o int) u16

little_endian_u16_at creates a u16 from two bytes in the array b at the specified offset in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_u16_end(b []u8) u16

little_endian_u16_end creates a u16 from the last two bytes of the array b in little endian order.

fn little_endian_u16_fixed(b [2]u8) u16

little_endian_u16_fixed creates a u16 from the fixed array b in little endian order.

fn little_endian_u32(b []u8) u32

little_endian_u32 creates a u32 from the first four bytes in the array b in little endian order. Panics if b is shorter than the value width.

fn little_endian_u32_at(b []u8, o int) u32

little_endian_u32_at creates a u32 from four bytes in the array b at the specified offset in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_u32_end(b []u8) u32

little_endian_u32_end creates a u32 from the last four bytes in the array b in little endian order.

fn little_endian_u32_fixed(b [4]u8) u32

little_endian_u32_fixed creates a u32 from the fixed array b in little endian order.

fn little_endian_u64(b []u8) u64

little_endian_u64 creates a u64 from the first eight bytes in the array b in little endian order. Panics if b is shorter than the value width.

fn little_endian_u64_at(b []u8, o int) u64

little_endian_u64_at creates a u64 from eight bytes in the array b at the specified offset in little endian order. Panics if o is negative or the value extends past the end of b.

fn little_endian_u64_end(b []u8) u64

little_endian_u64_end creates a u64 from the last eight bytes in the array b in little endian order.

fn little_endian_u64_fixed(b [8]u8) u64

little_endian_u64_fixed creates a u64 from the fixed array b in little endian order.

fn read[T](mut reader io.Reader, order ByteOrder, mut data T) !

read decodes fixed-size values from reader using order.

fn size[T](data T) int

size returns the number of bytes needed to encode data, or -1 for unsupported types.

fn write[T](mut writer io.Writer, order ByteOrder, data T) !

write encodes fixed-size values to writer using order.

interface ByteOrder {
	u16(b []u8) u16
	u32(b []u8) u32
	u64(b []u8) u64
	put_u16(mut b []u8, value u16)
	put_u32(mut b []u8, value u32)
	put_u64(mut b []u8, value u64)
}

ByteOrder decodes and encodes fixed-size integers using a specific byte order.

struct BigEndian {}

BigEndian implements ByteOrder using big-endian encoding.

fn (_ BigEndian) u16(b []u8) u16

u16 decodes a 16-bit unsigned integer from b using big-endian byte order.

fn (_ BigEndian) u32(b []u8) u32

u32 decodes a 32-bit unsigned integer from b using big-endian byte order.

fn (_ BigEndian) u64(b []u8) u64

u64 decodes a 64-bit unsigned integer from b using big-endian byte order.

fn (_ BigEndian) put_u16(mut b []u8, value u16)

put_u16 encodes value into b using big-endian byte order.

fn (_ BigEndian) put_u32(mut b []u8, value u32)

put_u32 encodes value into b using big-endian byte order.

fn (_ BigEndian) put_u64(mut b []u8, value u64)

put_u64 encodes value into b using big-endian byte order.

struct DecodeConfig {
pub mut:
	buffer_len int = 1024
	big_endian bool // use big endian decode the data
}

struct EncodeConfig {
pub mut:
	buffer_len int = 1024
	big_endian bool // use big endian encoding the data
}

struct LittleEndian {}

LittleEndian implements ByteOrder using little-endian encoding.

fn (_ LittleEndian) u16(b []u8) u16

u16 decodes a 16-bit unsigned integer from b using little-endian byte order.

fn (_ LittleEndian) u32(b []u8) u32

u32 decodes a 32-bit unsigned integer from b using little-endian byte order.

fn (_ LittleEndian) u64(b []u8) u64

u64 decodes a 64-bit unsigned integer from b using little-endian byte order.

fn (_ LittleEndian) put_u16(mut b []u8, value u16)

put_u16 encodes value into b using little-endian byte order.

fn (_ LittleEndian) put_u32(mut b []u8, value u32)

put_u32 encodes value into b using little-endian byte order.

fn (_ LittleEndian) put_u64(mut b []u8, value u64)

put_u64 encodes value into b using little-endian byte order.