A document-buffered pull XML parser for MoonBit, inspired by quick-xml.
- Pull-parser model - Read XML events one at a time (like StAX in Java)
- Document-buffered input - Constructors load the full input, then callers pull events one at a time
- Multi-backend - Works on wasm, wasm-gc, js, and native
- XML 1.0 + Namespaces 1.0 - Unicode names plus namespace-aware events
- Source-aware parsing - Authored ranges for events and attributes, plus contextual spans for errors
let xml = "<root><item id=\"1\">Hello</item></root>"
let reader = @xml.Reader::from_string(xml)
while true {
let event = reader.read_event()
match event.kind {
Start(elem) => println("Start: \{elem.name}")
End(name) => println("End: \{name}")
Text(content) => println("Text: \{content}")
Eof => break
_ => continue
}
}Callers obtain XML text from files, network responses, or other sources before passing it to Reader::from_string or NamespaceReader::from_string.
Use NamespaceReader when callers need namespace URI, prefix, and local-name information. The original Reader remains available for raw qualified names and namespace declaration attributes.
let reader = @xml.NamespaceReader::from_string(
"<p:root xmlns:p=\"urn:example\" p:id=\"1\"/>",
)
match reader.read_event().kind {
Empty(element) => {
println(element.name.local_name) // root
println(element.name.namespace_uri) // Some("urn:example")
}
_ => ()
}Namespace declarations are exposed through NamespaceElement::namespace_declarations and are not included in its normal attributes. Default namespaces apply to element names but not to unprefixed attribute names.
Writer validates XML names, characters, delimiter sequences, and document structure as output is added. Its write methods and to_string raise WriterError instead of returning malformed XML; to_string also requires exactly one complete root element.
Every event returned by Reader::read_event includes its authored source range. Event and attribute spans are half-open; offsets count UTF-16 code units, so they can slice the original MoonBit String directly.
let input = "<root id='a&b'/>"
let reader = @xml.Reader::from_string(input)
let parsed = reader.read_event()
let authored = input[parsed.span.start.offset:parsed.span.end.offset]
assert_eq(authored, input)
guard parsed.kind is Empty(element) else { abort("expected empty element") }
assert_eq(element.attributes[0].value, "a&b")Each XmlAttribute contains the whole attribute span plus separate name and unquoted value spans. Parse failures raise XmlError::At, which contains an XmlErrorKind and a relevant authored source span. Syntax failures normally cover input consumed while detecting the error, while an unclosed-element error points to the unmatched opening tag. Events produced by entity expansion point to the authored entity reference. Defaulted attributes point to their definitions in the internal DTD subset.
Event contains an EventKind and a SourceSpan. The EventKind variants are:
| Event | Description |
|---|---|
Start(XmlElement) |
Opening tag <name> |
End(String) |
Closing tag </name> |
Empty(XmlElement) |
Self-closing tag <name/> |
Text(String) |
Text content (entities decoded) |
CData(String) |
CDATA section <![CDATA[...]]> |
Comment(String) |
Comment <!-- ... --> |
PI(target, data) |
Processing instruction <?target data?> |
Decl(version, encoding, standalone) |
XML declaration |
DocType(String) |
DOCTYPE declaration |
Eof |
End of document |
Tests are compiled from the complete W3C XML Conformance Test Suite 20130923 catalog. Expected acceptance and rejection come from the catalog; output comparisons use the suite's official OUTPUT files, rather than expectations generated by another parser.
Local validation: 1,821 tests pass on wasm, wasm-gc, js, and native; one known conformance failure is explicitly skipped.
| Category | Tests | Assertion |
|---|---|---|
| W3C accepted documents | 771 | Includes valid and DTD invalid documents; 246 also compare official canonical output |
| W3C malformed documents | 899 | Reject through the reader selected by namespace metadata |
| Unit and property tests | 152 | Raw events, reader, writer, escape, namespaces, source spans, resources, properties and mutation fuzzing |
One of the 771 acceptance tests, rmt-e3e-13, remains a known failure, with its original W3C expectation intact. After a skipped parameter entity, an undeclared general entity can be a validity error rather than a well-formedness error. The current API rejects it and has no unresolved-entity event. The generated test has a named #skip reason, and can be run explicitly:
moon test --target native --include-skipped --filter 'w3c/invalid/rmt-e3e-13'
# Currently fails: undefined entity reference &ent2;The coverage ledger accounts for all 2,585 catalog entries, including every fixture URI, classification, assertion, exclusion reason and known failure. Included collections cover XMLTEST, Sun, OASIS/NIST, IBM, XML 1.0 edition errata, Namespaces 1.0 and namespace errata. The upstream root catalog's incorrect base for eduni/misc is corrected explicitly.
The test adapter implements the suite's Second Canonical Form. It retains qualified names and namespace declarations, sorts attributes by Unicode code point, escapes data, expands empty elements and removes comments as required by that format. It does not verify raw lexical event boundaries or source spans; separate public-API regressions check those. The 13 official outputs containing NOTATION declarations cannot be reproduced by the current event API; their acceptance tests remain enabled and their output checks are explicitly recorded as unavailable. Accepted cases without a usable official output check acceptance only.
Property tests run 1,280 generated cases with fixed seeds. Mutation fuzzing runs 3,000 mutated inputs through both readers, checking source ranges on accepted documents and errors and failing on panics. CI also runs 12 generator tests and checks reproducibility against the pinned W3C archive.
The generated MoonBit tests embed their fixtures and need no runtime file I/O or Python:
moon test --target all --deny-warnPython 3.12+, lxml and moonfmt are required. The generator verifies the downloaded archive's SHA-256 before reading the full catalog. Missing fixtures, missing outputs and duplicate IDs stop generation.
python3 -m pip install -r scripts/requirements.txt
python3 scripts/generate_conformance_tests.py --download
python3 -m unittest discover -s scripts -p 'test_*.py'
# Check that committed tests and coverage match the pinned source:
python3 scripts/generate_conformance_tests.py --download --checkThe ledger records 915 excluded entries:
| Reason | Entries |
|---|---|
| XML 1.1 / Namespaces 1.1 | 274 |
| Tests restricted to older XML editions | 310 |
| Requires external entity contents | 265 |
Byte decoding / encoding-declaration tests outside the decoded String API |
57 |
| Optional errors with no required accept/reject result | 9 |
DTD invalid documents are included: this library does not validate DTD constraints and should accept them. No fixture is excluded because another parser accepts it, rejects it or produces a different result. The explicitly named known failure is separate from these scope exclusions.
Run the benchmarks with:
moon bench --target native --release --filter 'bench *'Inputs are built outside the measured closures; each run constructs a reader and collects all events. The cases cover character-reference queues, deep namespace scopes with 512 inherited bindings, nested replacement markup, and large text documents.
Measurements on 2026-10-10, Apple M3 Max, MoonBit v0.10.14+7d59c7ec9, native release builds. The baseline is commit 19e6070; both versions used the same benchmark cases. Values are local mean timings, not portable performance thresholds.
| Case | Baseline | After optimization |
|---|---|---|
| 1,024 character references | 801 µs | 375 µs |
| 4,096 character references | 11.55 ms | 1.57 ms |
| 16,384 character references | 228.67 ms | 6.30 ms |
| Namespace depth 256 | 7.02 ms | 916 µs |
| Namespace depth 1,024 | 26.68 ms | 1.50 ms |
| 4,096 children from nested entities | 345.00 ms | 8.79 ms |
| 2 MiB of text | 11.14 ms | 10.96 ms |
Pending events are consumed by index, and namespace scopes store only changed bindings. Replacement content is parsed directly while sharing DTD tables, recursion tracking, and the document expansion budget. Attribute values retain their separate XML normalization rules.
- Non-validating - Does not validate against DTD
- UTF-8 only - Other encodings not supported
- XML 1.0 only - XML 1.1 not supported
- Bounded entity expansion - Internal entities are limited to 32 nesting levels and 262,144 expanded characters per document
External entity declarations are parsed, but their contents are not resolved. Referencing an external entity raises an error.
Apache-2.0