phase 3: vimwiki lexer
CI / cargo fmt --check (push) Successful in 22s
CI / cargo clippy (push) Successful in 53s
CI / cargo test (push) Successful in 1m2s

Two-pass, hand-rolled lexer per SPEC.md §6.6. Block pass walks the
source line by line; recognised constructs emit structural tokens, and
text-bearing line content is fed to the inline pass. Multi-line fences
(`{{{ }}}`, `{{$ }}$`, `%%+ +%%`) flip a `BlockMode` so subsequent
lines are captured raw until the matching closer.

VimwikiToken covers the full §9 feature checklist:

- headings (1–6, centered), horizontal rule, blockquotes (`>` and
  4-space), all list-marker variants, checkboxes, definition `::`,
  tables (separators, header row, col/row span)
- preformatted fences (with language + key=val attrs), math blocks
  (with `%env%`), single- and multi-line comments, all four
  page placeholders (%title / %nohtml / %template / %date)
- inline: bold/italic/strike/super/sub delimiters, inline code, inline
  math, all six keywords (word-bounded), wikilinks (with description
  separator), transclusions (with attr separator), raw URLs across
  http(s)/ftp/mailto/file schemes — trailing sentence punctuation
  stripped from URL spans

Spans are byte-accurate per SPEC §6.5 (0-indexed line + byte column +
absolute byte offset). The lexer is permissive: every delimiter is
emitted, the parser will pair them and fall back to literal text on
mismatches.

Tests (41 cases) cover one example per construct in §9, plus
word-boundary keyword detection, list-vs-bold disambiguation,
indent-vs-list disambiguation, and a span-correctness check.

README now carries a per-phase status table; SPEC top-line status
advanced to Phase 3 complete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-10 17:20:46 +00:00
parent 8f35c0a80c
commit 942dbe2aa8
6 changed files with 1792 additions and 2 deletions
+18 -1
View File
@@ -3,10 +3,27 @@
A Vim/Neovim plugin providing full vimwiki syntax support, implemented as a
Rust-based language server (LSP).
> **Status:** Phase 0 — scaffolding. Not yet functional.
> **Status:** Phase 3 (vimwiki lexer) complete. Parser, renderer, and editor
> glue still pending — not yet functional end-to-end.
See [`SPEC.md`](./SPEC.md) for the full project specification.
## Implementation status
| Phase | Name | Status |
|---|---|---|
| 0 | Scaffolding | ✅ done |
| 1 | Core AST | ✅ done |
| 2 | Syntax Plugin Interface | ✅ done |
| 3 | Vimwiki Lexer | ✅ done |
| 4 | Vimwiki Parser | ⏳ next |
| 5 | Renderer | ⏳ |
| 6 | LSP Foundation | ⏳ |
| 7 | Semantic Tokens | ⏳ |
| 8 | Navigation | ⏳ |
| 9 | Editor Glue | ⏳ |
| 10 | CI/CD release pipeline | ⏳ |
## Repository layout
```