toml.scanner #

fn new_scanner(config Config) !&Scanner

new_scanner returns a new heap allocated Scanner instance, based on the file in config.input.file_path, or based on the text in config.input.text .

fn new_simple(config Config) !Scanner

new_simple returns a new stack allocated Scanner instance.

fn new_simple_file(path string) !Scanner

new_simple_file returns a new stack allocated Scanner instance ready for parsing TOML in file read from path.

fn new_simple_text(text string) !Scanner

new_simple_text returns a new stack allocated Scanner instance ready for parsing TOML in text.

struct Config {
pub:
	input               input.Config
	tokenize_formatting bool = true // if true, generate tokens for `\n`, ` `, `\t`, `\r` etc.
}

Config is used to configure a Scanner instance. Only one of the fields text and file_path is allowed to be set at time of configuration.

struct Scanner {
pub:
	config Config
	text   string // the input TOML text
mut:
	col        int // current column number (x coordinate)
	line_nr    int = 1 // current line number (y coordinate)
	pos        int // current flat/index position in the `text` field
	header_len int // Length, how many bytes of header was found
	// Quirks
	is_left_of_assign bool = true // indicates if the scanner is on the *left* side of an assignment
}

Scanner contains the necessary fields for the state of the scan process. the task the scanner does is also referred to as "lexing" or "tokenizing". The Scanner methods are based on much of the work in vlib/strings/textscanner.

fn (mut s Scanner) scan() !token.Token

scan returns the next token from the input.

fn (mut s Scanner) free()

free frees all allocated resources.

fn (s &Scanner) remaining() int

remaining returns how many characters remain in the text input.

fn (mut s Scanner) next() u32

next returns the next character code from the input text. next returns end_of_text if it can't reach the next character.

fn (mut s Scanner) skip()

skip skips one character ahead.

fn (mut s Scanner) skip_n(n int)

skip_n skips ahead n characters. If the skip goes out of bounds from the length of Scanner.text, the scanner position will be sat to the last character possible.

fn (s &Scanner) at() u32

at returns the current character code from the input text. at returns end_of_text if it can't get the current character. unlike next(), at() does not change the state of the scanner.

fn (s &Scanner) peek(n int) u32

peek returns the character code from the input text at position + n. peek returns end_of_text if it can't peek n characters ahead.

fn (mut s Scanner) reset()

reset resets the internal state of the scanner.

fn (s &Scanner) excerpt(pos int, margin int) string

excerpt returns a string excerpt of the input text centered at pos. The margin argument defines how many chacters on each side of pos is returned

fn (s &Scanner) state() State

state returns a read-only view of the scanner's internal state.

struct State {
pub:
	col     int // current column number (x coordinate)
	line_nr int = 1 // current line number (y coordinate)
	pos     int // current flat/index position in the `text` field
}

State is a read-only copy of the scanner's internal state. See also Scanner.state().