toml.scanner #
Constants #
const digit_extras = [`_`, `.`, `x`, `o`, `b`, `e`, `E`]
const end_of_text = u32(~0)
fn new_scanner #
fn new_scanner(config Config) !&Scanner
new_scanner returns a new heap allocated Scanner
instance, based on the file in config.input.file_path, or based on the text in config.input.text .
fn new_simple #
fn new_simple(config Config) !Scanner
new_simple returns a new stack allocated Scanner
instance.
fn new_simple_file #
fn new_simple_file(path string) !Scanner
new_simple_file returns a new stack allocated Scanner
instance ready for parsing TOML in file read from path
.
fn new_simple_text #
fn new_simple_text(text string) !Scanner
new_simple_text returns a new stack allocated Scanner
instance ready for parsing TOML in text
.
struct Config #
struct Config {
pub:
input input.Config
tokenize_formatting bool = true // if true, generate tokens for `\n`, ` `, `\t`, `\r` etc.
}
Config is used to configure a Scanner instance. Only one of the fields text
and file_path
is allowed to be set at time of configuration.
struct Scanner #
struct Scanner {
pub:
config Config
text string // the input TOML text
mut:
col int // current column number (x coordinate)
line_nr int = 1 // current line number (y coordinate)
pos int // current flat/index position in the `text` field
header_len int // Length, how many bytes of header was found
// Quirks
is_left_of_assign bool = true // indicates if the scanner is on the *left* side of an assignment
}
Scanner contains the necessary fields for the state of the scan process. the task the scanner does is also referred to as "lexing" or "tokenizing". The Scanner methods are based on much of the work in vlib/strings/textscanner
.
fn (Scanner) scan #
fn (mut s Scanner) scan() !token.Token
scan returns the next token from the input.
fn (Scanner) free #
fn (mut s Scanner) free()
free frees all allocated resources.
fn (Scanner) remaining #
fn (s &Scanner) remaining() int
remaining returns how many characters remain in the text input.
fn (Scanner) next #
fn (mut s Scanner) next() u32
next returns the next character code from the input text. next returns end_of_text
if it can't reach the next character.
fn (Scanner) skip #
fn (mut s Scanner) skip()
skip skips one character ahead.
fn (Scanner) skip_n #
fn (mut s Scanner) skip_n(n int)
skip_n skips ahead n
characters. If the skip goes out of bounds from the length of Scanner.text
, the scanner position will be sat to the last character possible.
fn (Scanner) at #
fn (s &Scanner) at() u32
at returns the current character code from the input text. at returns end_of_text
if it can't get the current character. unlike next()
, at()
does not change the state of the scanner.
fn (Scanner) peek #
fn (s &Scanner) peek(n int) u32
peek returns the character code from the input text at position + n
. peek returns end_of_text
if it can't peek n
characters ahead.
fn (Scanner) reset #
fn (mut s Scanner) reset()
reset resets the internal state of the scanner.
fn (Scanner) excerpt #
fn (s Scanner) excerpt(pos int, margin int) string
excerpt returns a string excerpt of the input text centered at pos
. The margin
argument defines how many chacters on each side of pos
is returned
fn (Scanner) state #
fn (s Scanner) state() State
state returns a read-only view of the scanner's internal state.
struct State #
struct State {
pub:
col int // current column number (x coordinate)
line_nr int = 1 // current line number (y coordinate)
pos int // current flat/index position in the `text` field
}
State is a read-only copy of the scanner's internal state. See also Scanner.state()
.