Parsing

Main-module functions cannot be named dump, sizeof, typeof, or isreftype. These names have builtin syntax, so the compiler reports a redefinition error at the function declaration. Methods, C declarations, and qualified functions in other modules can use these names.

Parenthesized expressions may span lines, including a line break before the closing ). Line comments in a multiline condition preserve its grouping and any following && or ||.

A struct initializer may put its opening { on the next line after the type name, including qualified names and generic types. An explicit ; still ends the preceding statement.

Parser.preload_source(path, source) supplies a source snapshot for subsequent parsing and sibling declaration scans. clear_preloaded_sources() restores disk reads for those paths. Parallel prepasses and full parses use the same preloaded text, including empty sources. Set Parser.keep_source_texts to keep parsed text in FlatAst.source_texts, keyed by file ID, after parser buffers are released. The checker can index import lines from these snapshots. The compiler driver preloads module-validation reads only when the module cache is disabled; cache-enabled builds retain disk reads so parsing and cache signatures use the existing flow.

Bare builtin compile-time flags, such as $if windows, retain their compile-time meaning even when a local variable has the same name. This applies to $if statements and expressions.

fn read_batch_source(path string) string

read_batch_source reads the regular file at path with plain system calls into memory of the C allocator, or returns '' when it cannot; the caller then reads the file the ordinary way, which reports the error. Every stream that fopen opens takes a process-wide lock, which the tasks of a parallel parse contended for.

fn Parser.new(prefs &pref.Preferences) &Parser

new creates a Parser value for parser.

fn TranslatedSizeofShared.new() &TranslatedSizeofShared

struct Diagnostic {
pub:
	file           string
	pos            token.Pos
	line           int
	column         int
	severity       string
	message        string
	details        []string
	detail_pos     token.Pos
	detail_message string
}

Diagnostic is a structured source-ingestion, lexical, or parse diagnostic.

struct Parser {
	prefs &pref.Preferences
mut:
	s                            scanner.Scanner
	pending_children             []flat.NodeId
	tok                          token.Token
	lit                          string
	tok_pos                      int
	tok_end                      int
	prev_tok_end                 int
	peek_tok                     token.Token = .eof
	peek_lit                     string
	peek_pos                     int
	peek_end                     int
	has_peek                     bool
	cur_file                     string
	cur_file_id                  int
	next_file_id                 int = 1
	cur_module                   string
	is_translated                bool
	is_generated                 bool         // `@[generated] module x`: type names need not be capitalized
	file_type_names              map[u32]bool // source spelling IDs of types declared so far in the current file
	file_type_names_indexed      int          // nodes before this index are in `file_type_names`
	cur_fn                       string
	cur_fn_offset                int = -1
	unclosed_fn_offset           int = -1 // cur_fn_offset of the last function reported as missing its `}`
	cur_fn_generic_params        []string
	cur_veb_ctx_name             string // source-level name of the active veb request context
	veb_tmpl_counter             int    // monotonic id for unique `$veb.html`/`$tmpl` builder var names
	has_veb_template             bool
	pending_template_diagnostics []TemplateDiagnostic
	may_have_local_types         bool
	cur_struct                   string   // receiver type name of the current method, for `@STRUCT`
	cur_method_is_static         bool     // distinguishes `Type.method()` from `(x Type) method()` for `@LOCATION`
	defer_depth                  int      // >0 while parsing a `defer` block body; gates `$res()` to defer contexts only
	defer_result_allowed         bool     // true when the active defer is guaranteed to run during function return
	unsafe_depth                 int      // >0 while parsing an `unsafe` block body
	defer_unsafe_depth           int      // `unsafe_depth` at the start of the innermost `defer` body
	nested_block_depth           int      // lexical block depth below the current function body's outer scope
	comptime_for_vars            []string // active `$for` loop variables; a `$if` that reads one is deferred to unroll time
	comptime_method_var          string   // innermost active `$for method in Type.methods` loop variable
	comptime_const_values        map[string]string
	comptime_string_consts       map[string]bool
	comptime_string_imports      map[string]string
	comptime_local_values        map[string]string
	imported_module_names        map[string]bool // import aliases in the current file; not captured by inlined template closures
	named_variant_import_types   map[u64]bool
	named_variant_import_scans   map[string]bool
	file_method_names            map[string]bool // qualified method names declared so far in the current file, for duplicate diagnostics
	check_imports                bool            // enabled by the compiler driver, but not by syntax-only parser clients
	// local_binding_* track the variable/parameter names currently in scope, so an inlined
	// template closure captures a bare callee only when it is an actual local binding (a
	// function-valued parameter/local) rather than a module/top-level function. Scoped like
	// comptime_value_*: a name is in scope when local_binding_counts[name] > 0.
	local_binding_counts              map[string]int
	global_names                      map[string]bool
	parse_batch_paths                 []string
	translated_sizeof_type_names      map[u64]bool
	translated_sizeof_const_names     map[string]bool
	translated_sizeof_global_names    map[string]bool
	translated_sizeof_scanned_modules map[string]bool
	// Context and type spelling IDs share the key layout of translated_sizeof_type_key.
	// Earlier lookups never rescan nodes already indexed.
	parsed_type_decls               map[u64]bool
	parsed_type_decls_end           int
	parsed_type_decls_dir           string
	parsed_type_decls_module        string
	parsed_type_decls_context       u32
	type_declaration_context_file   string
	type_declaration_context_module string
	type_declaration_context        u32
	// The sibling-file declarations the workers of one parse batch share (see
	// scan_translated_sizeof_declarations), and this worker's place among them.
	translated_sizeof_shared       &TranslatedSizeofShared = unsafe { nil }
	translated_sizeof_worker       int
	translated_sizeof_branch_evals int
	// The token buffer that scan_translated_sizeof_source reuses for every source
	// it indexes, and whether a scan holds it right now.
	sizeof_scan_tokens []InlineAsmScanToken
	sizeof_scan_busy   bool
	// The line table that the resolvers of a const prepass share.
	prepass_lines PrepassLineFile
	// The sources that a parallel parse batch read once for all of its passes
	// (see parse_files_dispatch); nil outside of such a batch.
	batch_sources &BatchSources = unsafe { nil }
	// The files that the caller expects in the batch after the next one, and the
	// parse of such files that the batch before this one did ahead of time (see
	// speculate_next_batch).
	next_batch_paths []string
	speculated_batch voidptr // &SpeculatedBatch
	// Set for a worker that parses files of the batch after the current one.
	parses_ahead bool
	// Texts that the caller read for the next parallel parse batch, which owns
	// them from then on (see preload_batch_source), and the index of each by path.
	batch_preloads                    []string
	batch_preload_index               map[string]int
	local_binding_undos               []LocalBindingUndo
	local_binding_scopes              []int
	active_lambda_param_counts        map[string]int
	comptime_value_undos              []ComptimeValueUndo
	comptime_value_scopes             []int
	pending_flag                      bool
	pending_json_as_number            bool // `@[json_as_number]` seen before the next enum decl
	pending_params                    bool
	pending_typedef                   bool
	pending_export                    string
	pending_noreturn                  bool
	pending_soa                       bool
	pending_has_aligned               bool
	pending_aligned                   string
	skip_next_decl                    bool
	disable_fn_body                   bool
	pending_decl_pub                  bool
	pending_decl_attrs                []string
	pending_decl_attr_kinds           []int
	pending_decl_attr_sources         []string
	in_for_container                  bool
	in_select_branch_condition        int
	in_match_branch_condition         int
	in_array_literal                  int
	inside_array_init_type_expr       bool
	in_map_value                      int
	in_struct_init_value              int
	in_enum_field_value               int
	unsupported_inline_asm_guards     map[int]bool
	parsing_inferred_fixed_array_type bool
	parsing_struct_field_type         bool
	in_const_value                    bool
	parsing_c_struct_fields           bool
	seen_default_param_value          bool
	diagnostic_limit_reached          bool
	local_type_names                  flat.TypeTextBindings
	local_type_decls_by_block         map[int][]string
	local_type_decls_indexed          bool
	local_type_scopes                 []LocalTypeScope
	anonymous_struct_types            AnonymousStructShapeCache
	anonymous_union_types             AnonymousStructShapeCache
	anonymous_struct_count            int
	sql_query_data_aliases            map[string]bool
	export_records                    []ExportRecord
	// The offsets of the `#line` directives of the file that are between statements, or in
	// skipped code; report_misplaced_line_directives reports the others.
	accepted_line_directives map[int]bool
	// The probe caches that canonicalize the texts of each file of a batch (see
	// intern_parsed_node_texts): nil outside of a batch.
	text_value_cache &flat.TextProbeCache = unsafe { nil }
	text_type_cache  &flat.TextProbeCache = unsafe { nil }
pub mut:
	a &flat.FlatAst = unsafe { nil }
	// no_source_digests leaves the SHA-256 of each parsed source out (see
	// token.File.index_line_starts): the compiler driver sets it for a build
	// that stages no fallback manifest, the only reader of those digests.
	no_source_digests bool
	// quick_source_sums records the quick_sum of each parsed source instead of
	// its SHA-256 (see token.File.index_lines_with_quick_sum).
	quick_source_sums bool
	// keep_source_texts records the text of each parsed source in
	// FlatAst.source_texts, for the stages that would read the file again.
	keep_source_texts bool
	// preloaded_sources holds the text of files that the caller read already and
	// is about to parse (see preload_source).
	preloaded_sources          map[string]string
	parsed_v_files             int
	parsed_v_file_paths        []string
	parsed_v_header_files      int
	parsed_v_header_file_paths []string
	diagnostics                []Diagnostic
}

Parser represents parser data used by parser.

fn (mut p Parser) reserve_selfhost_ast()

reserve_selfhost_ast prepares the shared AST for a compiler-sized input without retaining successively doubled backing arrays during transform.

fn (mut p Parser) enable_import_diagnostics()

enable_import_diagnostics enables parser diagnostics that require a complete source file rather than syntax-only parsing.

fn (mut p Parser) parse_file(path string) &flat.FlatAst

parse_file reads parse file input for parser.

fn (mut p Parser) parse_files(paths []string) &flat.FlatAst

parse_files reads parse files input for parser.

fn (mut p Parser) release_source_storage()

release_source_storage canonicalizes retained metadata and drops parser/scanner references to raw file buffers. Parsed node text is canonicalized as each file is completed and when parallel worker ASTs are merged.

fn (mut p Parser) parse_files_with_starts(paths []string) []int

parse_files_with_starts parses paths in order like parse_files, recording the first node id of each file's region. Import resolution uses the starts to bound per-file post-processing (module-name canonicalization) when files are parsed in batches instead of one at a time.

fn (mut p Parser) parse_into(path string)

parse_into reads parse into input for parser.

fn (mut p Parser) parse_text(name string, src string) &flat.FlatAst

parse_text parses V source text held in memory, without reading a file. name is recorded as the file name: pass a .v name (or .vsh for script mode), since extension checks behave exactly as in parse_into. The AST takes ownership of a copy of src. Useful where no filesystem exists, like the Emscripten build. Reusing a Parser appends files to the same AST and retains earlier diagnostics, like parse_file; inspect diagnostics for parser errors.

fn (mut p Parser) preload_source(path string, source string)

preload_source hands the parser the text of a file that the caller has read already, so that parsing path does not read the file a second time. The text has to stay valid for as long as the parsed AST does.

fn (mut p Parser) clear_preloaded_sources()

clear_preloaded_sources drops the texts that preload_source recorded.

fn (mut p Parser) resolve_comptime_string_declarations() map[int]bool

resolve_comptime_string_declarations resolves static declaration guards once imported constants are parsed. It runs before declaration collection and does not fold unknown values or function-local bindings. It returns imports whose scalar guards still need a dependency's constants.

fn (mut p Parser) parse_files_dispatch(paths []string, allow_parallel bool) ([]int, bool)

parse_files_dispatch parses paths in order, appending to p.a exactly like a serial parse_into loop, across worker threads when there is enough work. Returns each file's first node id in p.a and whether threads were used.

fn (mut p Parser) speculate_next_batch(paths []string)

speculate_next_batch names the files that the caller expects in the batch after the next one. A parallel parse of the next batch parses them as well and keeps the result for the batch that follows, which takes it only when it holds exactly these files and the parser is in the state that the parse assumed (see merge_speculated_batch); otherwise it parses its files itself.

fn (mut p Parser) discard_speculated_batch()

discard_speculated_batch releases a parse ahead that no batch took.

fn (mut p Parser) preload_batch_source(path string, source string)

preload_batch_source hands the next parse batch the text of path, which the caller read with read_batch_source. The batch owns the text: a parallel batch releases it with the sources it reads itself, and any other parse drops it and reads the file.

struct TranslatedSizeofShared {
mut:
	mu    &sync.Mutex = sync.new_mutex()
	files map[string]&TranslatedSizeofFile
}

TranslatedSizeofShared holds, for the workers of one parse batch, the names each sibling file adds to the sizeof declaration tables. Indexing a directory reads and scans all of its files; shared, the workers that need the same directory split that work instead of each doing all of it.