net.html #

net/html is an HTML Parser written in pure V.

Usage

import net.html

fn main() {
    doc := html.parse('<html><body><h1 class="title">Hello world!</h1></body></html>')
    tag := doc.get_tags(name: 'h1')[0] // <h1>Hello world!</h1>
    println(tag.name) // h1
    println(tag.content) // Hello world!
    println(tag.attributes) // {'class':'title'}
    println(tag.str()) // <h1 class="title">Hello world!</h1>
}

For tags with nested markup, tag.content preserves the inner HTML:

import net.html

fn main() {
    doc := html.parse('<p>before <span>in between</span> after</p>')
    tag := doc.get_tags(name: 'p')[0]
    println(tag.content) // before <span>in between</span> after
    println(tag.text()) // before in between after
}

More examples found on parser_test.v and html_test.v

Comments are omitted from the parsed tree. Text before and after comments remains in order, including text wrapped by hydration markers such as <!--[-->hello<!--]-->. Comments do not create empty text nodes or change the surrounding elements.

Calling get_tag_by_class_name without class names returns none; get_tags_by_class_name without class names returns an empty array.

Script and style elements preserve their raw bodies in tag.content and when serialized with tag.str(). Text following a closing script or style tag belongs to the surrounding document.

Document searches include the root element in name, class, and attribute matches. Class searches match all supplied class names and return elements in document order. Searches on a Tag inspect only its descendants.

Attribute searches match attributes by presence, including boolean attributes such as disabled and attributes with an empty value. Searching for an empty value matches only tags that have the attribute. Searches on a Tag recursively inspect its children.

Unterminated script and style elements retain their collected body when parsing ends.

Newlines inside tags separate names and attributes like spaces. Consecutive separators preserve a pending quoted attribute value.

Trailing text is retained when parsing ends, including text after a closing tag and text in an unclosed element. Text after a closing tag belongs to the surrounding element.

Attribute names are case insensitive. Quoted and unquoted attribute values preserve their case, and both forms populate class selectors. HTML whitespace separates attributes; whitespace around = is accepted. Empty attribute names are omitted.

fn parse(text string) DocumentObjectModel

parse parses and returns the DOM from the given text.

Note: this function converts tags to lowercase. E.g. <MyTag>content</MyTag> is parsed as <mytag>content</mytag>.

fn parse_file(filename string) DocumentObjectModel

parse_file parses and returns the DOM from the contents of a file.

Note: this function converts tags to lowercase. E.g. <MyTag>content</MyTag> is parsed as <mytag>content</mytag>.

fn CloseTagType.from[W](input W) !CloseTagType

from returns the CloseTagType value of an integer or of a field name, or an error if there is none.

enum CloseTagType {
	in_name
	new_tag
}

struct DocumentObjectModel {
mut:
	root           &Tag = unsafe { nil }
	constructed    bool
	btree          BTree
	all_tags       []&Tag
	all_attributes map[string][]&Tag
	close_tags     map[string]bool // add a counter to see count how many times is closed and parse correctly
	attributes     map[string][]string
	tag_attributes map[string][][]&Tag
	tag_type       map[string][]&Tag
	debug_file     os.File
}

The W3C Document Object Model (DOM) is a platform and language-neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document.

https://www.w3.org/TR/WD-DOM/introduction.html

fn (dom &DocumentObjectModel) get_root() &Tag

get_root returns the root of the document.

fn (dom &DocumentObjectModel) get_tags(options GetTagsOptions) []&Tag

get_tags returns all tags stored in the document.

fn (dom &DocumentObjectModel) get_tags_by_class_name(names ...string) []&Tag

get_tags_by_class_name retrieves all tags recursively in the document root that have the given class name(s).

fn (dom &DocumentObjectModel) get_tags_by_attribute(name string) []&Tag

get_tags_by_attribute retrieves all tags in the document that have the given attribute name.

fn (mut dom DocumentObjectModel) get_tags_by_attribute_value(name string, value string) []&Tag

get_tags_by_attribute_value retrieves all tags in the document that have the given attribute name and value.

struct GetTagsOptions {
pub:
	name string
}

struct Parser {
mut:
	dom                DocumentObjectModel
	lexical_attributes LexicalAttributes = LexicalAttributes{
		current_tag: &Tag{}
	}
	filename           string = 'direct-parse'
	initialized        bool
	tags               []&Tag
	debug_file         os.File
}

Parser is responsible for reading the HTML strings and converting them into a DocumentObjectModel.

fn (mut parser Parser) add_code_tag(name string)

This function is used to add a tag for the parser ignore it's content. For example, if you have an html or XML with a custom tag, like <script>, using this function, like add_code_tag('script') will make all script tags content be jumped, so you still have its content, but will not confuse the parser with it's > or <.

fn (mut parser Parser) split_parse(data string)

split_parse parses the HTML fragment

fn (mut parser Parser) parse_html(data string)

parse_html parses the given HTML string

fn (mut parser Parser) finalize()

finalize flushes pending text and finishes the parsing stage.

fn (mut parser Parser) get_dom() DocumentObjectModel

get_dom returns the parser's current DOM representation.

struct Tag {
pub mut:
	name               string
	content            string
	children           []&Tag
	attributes         map[string]string // attributes will be like map[name]value
	last_attribute     string
	class_set          datatypes.Set[string]
	parent             &Tag = unsafe { nil }
	position_in_parent int
	closed             bool
	close_type         CloseTagType = .in_name
mut:
	text_content          string
	content_is_inner_html bool
}

Tag holds the information of an HTML tag.

fn (tag &Tag) text() string

text returns the text contents of the tag and its children. A br element (including the parser's br/ form) contributes a newline.

fn (tag &Tag) str() string

fn (tag &Tag) get_tag(name string) ?&Tag

get_tag retrieves the first found child tag in the tag that has the given tag name.

fn (tag &Tag) get_tags(name string) []&Tag

get_tags retrieves all child tags recursively in the tag that have the given tag name.

fn (tag &Tag) get_tag_by_attribute(name string) ?&Tag

get_tag_by_attribute retrieves the first found child tag in the tag that has the given attribute name.

fn (tag &Tag) get_tags_by_attribute(name string) []&Tag

get_tags_by_attribute retrieves all child tags recursively in the tag that have the given attribute name.

fn (tag &Tag) get_tag_by_attribute_value(name string, value string) ?&Tag

get_tag_by_attribute_value retrieves the first found child tag in the tag that has the given attribute name and value.

fn (tag &Tag) get_tags_by_attribute_value(name string, value string) []&Tag

get_tags_by_attribute_value retrieves all child tags recursively in the tag that have the given attribute name and value.

fn (tag &Tag) get_tag_by_class_name(names ...string) ?&Tag

get_tag_by_class_name retrieves the first found child tag in the tag that has the given class name(s).

fn (tag &Tag) get_tags_by_class_name(names ...string) []&Tag

get_tags_by_class_name retrieves all child tags recursively in the tag that have the given class name(s).