Add a ListSyms type (vimwiki's g:vimwiki_listsyms) and thread it through
the lexer so checkbox glyphs are recognized from the configured palette
rather than the hardcoded ' .oOX'. Glyphs map to the existing five-bucket
CheckboxState, so the HTML renderer and AST consumers are unchanged.
VimwikiSyntax::parse_with_listsyms opts into a custom palette; the trait
parse() keeps the default.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hand-rolled recursive-descent parser over the token stream. Resilient
per SPEC §6.7: never aborts the document — anything the dispatcher
can't claim becomes BlockNode::Error and the cursor advances, and
unterminated wikilinks / transclusions / delimiters fall back to
literal text.
Block coverage: every variant from §6.4 — Heading (with inline
content between markers), Paragraph (with soft-break across single
newlines), HorizontalRule, Blockquote (both `>` and 4-space styles),
Preformatted, MathBlock, List (with checkbox + indent-driven
sublists), DefinitionList, Table (header separator promotes the
preceding row, plus colspan / rowspan cells), Comment (single +
multiline), and page-level placeholders threaded into PageMetadata.
Inline pairing: left-to-right scan with "find next matching delim",
recursing on the slice between. `_*x*_` becomes Italic(Bold(...)),
mirroring SPEC §6.7 precedence. URLs inside `[[ ]]` route to
ExternalLinkNode; everything else to WikiLinkNode with full
LinkTarget classification (Wiki / Interwiki numbered + named /
Diary / File / Local / AnchorOnly / directory / root-relative /
filesystem-absolute).
VimwikiSyntax registers as id="vimwiki", extensions=[".wiki"] and
chains lexer + parser inside SyntaxPlugin::parse.
SPEC §4 updated to record the hand-rolled choice with rationale —
span-bearing tokens fit awkwardly into winnow/nom combinator style;
resilience and recovery are explicit in code instead.
Tests (41 new parser cases) cover every AST construct, inline
precedence including nested bold/italic, every LinkKind, transclusion
alt + attrs, resilience on unterminated links, and end-to-end
SyntaxRegistry dispatch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-pass, hand-rolled lexer per SPEC.md §6.6. Block pass walks the
source line by line; recognised constructs emit structural tokens, and
text-bearing line content is fed to the inline pass. Multi-line fences
(`{{{ }}}`, `{{$ }}$`, `%%+ +%%`) flip a `BlockMode` so subsequent
lines are captured raw until the matching closer.
VimwikiToken covers the full §9 feature checklist:
- headings (1–6, centered), horizontal rule, blockquotes (`>` and
4-space), all list-marker variants, checkboxes, definition `::`,
tables (separators, header row, col/row span)
- preformatted fences (with language + key=val attrs), math blocks
(with `%env%`), single- and multi-line comments, all four
page placeholders (%title / %nohtml / %template / %date)
- inline: bold/italic/strike/super/sub delimiters, inline code, inline
math, all six keywords (word-bounded), wikilinks (with description
separator), transclusions (with attr separator), raw URLs across
http(s)/ftp/mailto/file schemes — trailing sentence punctuation
stripped from URL spans
Spans are byte-accurate per SPEC §6.5 (0-indexed line + byte column +
absolute byte offset). The lexer is permissive: every delimiter is
emitted, the parser will pair them and fall back to literal text on
mismatches.
Tests (41 cases) cover one example per construct in §9, plus
word-boundary keyword detection, list-vs-bold disambiguation,
indent-vs-list disambiguation, and a span-correctness check.
README now carries a per-phase status table; SPEC top-line status
advanced to Phase 3 complete.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gffranco