Move SPEC.md to development/ and rewrite as a technical reference
CI / cargo fmt --check (push) Successful in 19s
CI / cargo clippy (push) Successful in 24s
CI / cargo test (push) Successful in 28s
CI / cargo fmt --check (pull_request) Successful in 17s
CI / cargo clippy (pull_request) Successful in 36s
CI / cargo test (pull_request) Successful in 35s
CI / editor keymaps (push) Successful in 1m23s
CI / editor keymaps (pull_request) Successful in 1m35s
CI / cargo fmt --check (push) Successful in 19s
CI / cargo clippy (push) Successful in 24s
CI / cargo test (push) Successful in 28s
CI / cargo fmt --check (pull_request) Successful in 17s
CI / cargo clippy (pull_request) Successful in 36s
CI / cargo test (pull_request) Successful in 35s
CI / editor keymaps (push) Successful in 1m23s
CI / editor keymaps (pull_request) Successful in 1m35s
The old spec was a 1426-line forward-looking planning doc (phases, pending decisions, design sketches for work now shipped). Replace it with a concise 339-line reference describing the implemented system — architecture, AST, LSP capabilities + executeCommand surface, editor integration, config schema, keymaps, syntax support, and CI — verified against the current code. Lives under development/ with the other developer-only docs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -70,6 +70,7 @@ nuwiki/
|
|||||||
│
|
│
|
||||||
├── development/ # Developer-only tooling and docs (not shipped)
|
├── development/ # Developer-only tooling and docs (not shipped)
|
||||||
│ ├── ONBOARDING.md # This file
|
│ ├── ONBOARDING.md # This file
|
||||||
|
│ ├── SPEC.md # Technical reference (architecture, LSP, commands)
|
||||||
│ ├── nuwiki-architecture.html
|
│ ├── nuwiki-architecture.html
|
||||||
│ ├── start-nvim.sh # Launch Neovim against a generated sample wiki
|
│ ├── start-nvim.sh # Launch Neovim against a generated sample wiki
|
||||||
│ ├── start-vim.sh # Launch Vim against a generated sample wiki
|
│ ├── start-vim.sh # Launch Vim against a generated sample wiki
|
||||||
|
|||||||
@@ -0,0 +1,339 @@
|
|||||||
|
# nuwiki — Technical Reference
|
||||||
|
|
||||||
|
A developer-facing description of how nuwiki is built and what it implements.
|
||||||
|
For day-one setup see [ONBOARDING.md](ONBOARDING.md); for user-facing help see
|
||||||
|
`doc/nuwiki.txt` (`:h nuwiki`).
|
||||||
|
|
||||||
|
nuwiki is a vimwiki-compatible Vim/Neovim plugin backed by a Rust language
|
||||||
|
server. The editor layers are thin clients; all parsing, navigation, and
|
||||||
|
editing logic lives in the Rust crates and is reached over LSP.
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|---|---|
|
||||||
|
| License | Dual MIT / Apache-2.0 |
|
||||||
|
| MSRV | Rust 1.83 |
|
||||||
|
| Edition / resolver | 2021 / v2 |
|
||||||
|
| Repo | `https://code.gfran.co/gffranco/nuwiki` |
|
||||||
|
| VCS / CI | Gitea + Gitea Actions |
|
||||||
|
| LSP library | `tower-lsp` (tokio, stdio transport) |
|
||||||
|
| Min editors | Neovim 0.11+, Vim 9.1+ |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Architecture
|
||||||
|
|
||||||
|
### Layer model
|
||||||
|
|
||||||
|
```
|
||||||
|
Editor client (VimL / Lua) — pure wiring, no logic
|
||||||
|
│ LSP over stdio
|
||||||
|
▼
|
||||||
|
nuwiki-lsp — protocol bridge, executeCommand, document store, config
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
nuwiki-core — lexer → parser → AST → renderer (no editor deps)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Crate dependency rules
|
||||||
|
|
||||||
|
```
|
||||||
|
nuwiki-ls → nuwiki-lsp → nuwiki-core
|
||||||
|
```
|
||||||
|
|
||||||
|
- `nuwiki-core` never depends on `nuwiki-lsp` / `nuwiki-ls`.
|
||||||
|
- `nuwiki-lsp` never depends on `nuwiki-ls`.
|
||||||
|
- The VimL and Lua layers contain no logic — every command body issues a
|
||||||
|
`workspace/executeCommand` or a built-in LSP request.
|
||||||
|
|
||||||
|
### Repository layout
|
||||||
|
|
||||||
|
```
|
||||||
|
crates/
|
||||||
|
nuwiki-core/ # parser, AST, renderer — editor-independent
|
||||||
|
src/
|
||||||
|
ast/ # block.rs inline.rs link.rs span.rs visit.rs
|
||||||
|
syntax/ # registry.rs + vimwiki/{lexer,parser}.rs
|
||||||
|
render/ # html.rs
|
||||||
|
date.rs # diary period math (no chrono dependency)
|
||||||
|
nuwiki-lsp/ # LSP backend
|
||||||
|
src/
|
||||||
|
lib.rs # Backend, capabilities, textDocument handlers
|
||||||
|
commands.rs # executeCommand dispatcher + pure edit ops
|
||||||
|
config.rs # Config / WikiConfig / HtmlConfig
|
||||||
|
diagnostics.rs index.rs nav.rs rename.rs semantic_tokens.rs
|
||||||
|
diary.rs export.rs edits.rs folding.rs wiki.rs
|
||||||
|
nuwiki-ls/ # thin binary: starts the stdio server
|
||||||
|
|
||||||
|
plugin/nuwiki.vim # universal entry point (detects Vim vs Neovim)
|
||||||
|
lua/nuwiki/ # Neovim layer: init, config, lsp, keymaps, commands,
|
||||||
|
# folding, install
|
||||||
|
autoload/nuwiki/ # Vim layer: lsp.vim, commands.vim
|
||||||
|
ftdetect/ ftplugin/ syntax/ # filetype detection, buffer setup, fallback HL
|
||||||
|
doc/nuwiki.txt # :help
|
||||||
|
scripts/download_bin.vim # plugin-runtime binary download (build hooks)
|
||||||
|
development/ # dev-only tooling + this document (not shipped)
|
||||||
|
.gitea/workflows/ # ci.yaml, release.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. nuwiki-core
|
||||||
|
|
||||||
|
### AST
|
||||||
|
|
||||||
|
Every node carries a `Span { start, end }` of `Position { line, column, offset }`
|
||||||
|
(0-indexed; column and offset are byte-based). Spans drive diagnostics,
|
||||||
|
semantic tokens, and go-to-definition.
|
||||||
|
|
||||||
|
**Block nodes:** `Heading` (level 1–6, `centered`), `Paragraph`,
|
||||||
|
`HorizontalRule`, `Blockquote` (recursive), `Preformatted` (optional
|
||||||
|
language), `MathBlock` (optional environment), `List`, `DefinitionList`,
|
||||||
|
`Table`, `Comment`, `Tag`, `Error` (resilient parse failure).
|
||||||
|
|
||||||
|
**Inline nodes:** `Text`, `Bold`, `Italic`, `BoldItalic`, `Strikethrough`,
|
||||||
|
`Code`, `Superscript`, `Subscript`, `MathInline`, `Keyword`, `Color`,
|
||||||
|
`WikiLink`, `ExternalLink`, `Transclusion`, `RawUrl`.
|
||||||
|
|
||||||
|
**Enums:**
|
||||||
|
- `ListSymbol` = `Dash | Star | Hash | Numeric | NumericParen | AlphaParen |
|
||||||
|
AlphaUpperParen | RomanParen | RomanUpperParen`
|
||||||
|
- `CheckboxState` = `Empty | Quarter | Half | ThreeQuarters | Done | Rejected`
|
||||||
|
- `Keyword` = `Todo | Done | Started | Fixme | Fixed | Xxx | Stopped`
|
||||||
|
- `LinkKind` = `Wiki | Interwiki | Diary | File | Local | Raw | AnchorOnly`
|
||||||
|
- `TagScope` = `File | Heading(idx) | Standalone`
|
||||||
|
|
||||||
|
`PageMetadata` carries `title`, `nohtml`, `template`, `date`, and aggregated
|
||||||
|
file-level `tags`.
|
||||||
|
|
||||||
|
### Lexer / parser
|
||||||
|
|
||||||
|
- Two-pass lexer (block pass, then inline pass per block); syntax-specific
|
||||||
|
`VimwikiToken`s with byte-offset spans, collected eagerly into a
|
||||||
|
`TokenStream`.
|
||||||
|
- Hand-rolled recursive-descent parser (not a combinator library). Resilient:
|
||||||
|
malformed input yields an `ErrorNode` and parsing continues — a document
|
||||||
|
never fails to parse.
|
||||||
|
- Full re-parse on every `didOpen` / `didChange` (no incremental parsing).
|
||||||
|
|
||||||
|
### Syntax plugin interface
|
||||||
|
|
||||||
|
Syntaxes register against a `SyntaxRegistry` keyed by id and file extension:
|
||||||
|
|
||||||
|
```
|
||||||
|
SyntaxPlugin { id, display_name, file_extensions, lexer, parser }
|
||||||
|
```
|
||||||
|
|
||||||
|
vimwiki (`.wiki`) is the only registered syntax; the registry exists so a
|
||||||
|
markdown plugin can be added without touching the LSP or editor layers.
|
||||||
|
|
||||||
|
### Renderer
|
||||||
|
|
||||||
|
`Renderer` trait writes to a `dyn Write`; `HtmlRenderer` is the implementation.
|
||||||
|
Link resolution is injected as a callback. Template substitution supports
|
||||||
|
`{{title}}`, `{{content}}`, `{{date}}`, `{{root_path}}`, `{{toc}}`, `{{css}}`.
|
||||||
|
`color_dic` maps colour-tag names to CSS values, falling back to
|
||||||
|
`class="color-<name>"`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. LSP server
|
||||||
|
|
||||||
|
### Advertised capabilities
|
||||||
|
|
||||||
|
| Capability | Notes |
|
||||||
|
|---|---|
|
||||||
|
| `textDocumentSync` | full sync (open/close/change/save) |
|
||||||
|
| `semanticTokensProvider` | full + range; custom vimwiki token legend |
|
||||||
|
| `documentSymbolProvider` / `workspaceSymbolProvider` | outline + workspace search (tags included) |
|
||||||
|
| `definitionProvider` / `referencesProvider` | link follow + backlinks |
|
||||||
|
| `hoverProvider` | link preview |
|
||||||
|
| `completionProvider` | trigger `[` |
|
||||||
|
| `renameProvider` | cross-document link rewrite |
|
||||||
|
| `foldingRangeProvider` | heading blocks + top-level lists |
|
||||||
|
| `executeCommandProvider` | see command surface below |
|
||||||
|
| `workspace.fileOperations` | `did_rename` + `did_delete` for `**/*.wiki`, `**/*.md` |
|
||||||
|
| `positionEncoding` | UTF-8 when client supports LSP 3.17+, else UTF-16 |
|
||||||
|
|
||||||
|
Workspace indexing runs as a background tokio task reporting via
|
||||||
|
`window/workDoneProgress`; nav features answer with partial data until the
|
||||||
|
scan completes. The document store is an `Arc<DashMap<Url, DocumentState>>`
|
||||||
|
holding text + AST + version.
|
||||||
|
|
||||||
|
### executeCommand surface
|
||||||
|
|
||||||
|
All editing commands return a `WorkspaceEdit` (applied via
|
||||||
|
`workspace/applyEdit`) built through `edits::WorkspaceEditBuilder`, so the AST
|
||||||
|
stays the source of truth. Navigation commands return a `Location`.
|
||||||
|
|
||||||
|
| Namespace | Commands |
|
||||||
|
|---|---|
|
||||||
|
| `list` | `toggleCheckbox` `cycleCheckbox` `rejectCheckbox` `nextTask` `removeDone` `renumber` `changeSymbol` `changeLevel` |
|
||||||
|
| `table` | `insert` `align` `moveColumn` |
|
||||||
|
| `heading` | `addLevel` `removeLevel` |
|
||||||
|
| `link` | `pasteWikilink` `pasteUrl` |
|
||||||
|
| `toc` / `links` | `toc.generate` · `links.generate` |
|
||||||
|
| `workspace` | `checkLinks` `findOrphans` |
|
||||||
|
| `diary` | `openToday` `openYesterday` `openTomorrow` `openIndex` `generateIndex` `next` `prev` `listEntries` `openForDate` |
|
||||||
|
| `tags` | `search` `generateLinks` `rebuild` |
|
||||||
|
| `export` | `currentToHtml` `allToHtml` `allToHtmlForce` `browse` `rss` |
|
||||||
|
| `wiki` | `listAll` `select` `openIndex` `tabOpenIndex` `gotoPage` |
|
||||||
|
| `file` | `delete` |
|
||||||
|
|
||||||
|
`workspace/rename` emits a `RenameFile` op plus `[[A]]` → `[[B]]` rewrites
|
||||||
|
across every linking page (anchors and descriptions preserved). Closed
|
||||||
|
documents are re-parsed on demand so cross-document edits use current source.
|
||||||
|
|
||||||
|
### Diagnostics
|
||||||
|
|
||||||
|
Composable collector chains parse errors (`ErrorNode` walk) and broken-link
|
||||||
|
checks (wiki target / anchor / `file:` / `local:` existence). Per-source
|
||||||
|
severity is config-driven (`diagnostic.link_severity`: `off|hint|warn|error`);
|
||||||
|
raw and external URLs are never diagnosed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Editor integration
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
Pre-built binaries are published as Gitea release assets
|
||||||
|
(`nuwiki-ls-{version}-{target}.tar.gz`) and fetched at install time by
|
||||||
|
`lua/nuwiki/install.lua` (Neovim) or `scripts/download_bin.vim` (Vim build
|
||||||
|
hooks), installed to `{plugin_dir}/bin/nuwiki-ls`. Falls back to
|
||||||
|
`cargo build --release` when download fails or
|
||||||
|
`g:nuwiki_build_from_source = 1`.
|
||||||
|
|
||||||
|
`plugin/nuwiki.vim` dispatches: Neovim → `require('nuwiki').setup()`; Vim →
|
||||||
|
`nuwiki#lsp#start()` (prefers `vim-lsp`, falls back to `coc.nvim`). On Neovim
|
||||||
|
the server is registered via `vim.lsp.config{}` + `vim.lsp.enable`, with a
|
||||||
|
`vim.lsp.start` autocmd fallback.
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
Single-wiki (v1.0) and multi-wiki forms both work; the scalar form desugars to
|
||||||
|
a one-entry `wikis` list.
|
||||||
|
|
||||||
|
```lua
|
||||||
|
require('nuwiki').setup({
|
||||||
|
-- single-wiki shorthand
|
||||||
|
wiki_root = '~/vimwiki', file_extension = '.wiki', syntax = 'vimwiki',
|
||||||
|
log_level = 'warn',
|
||||||
|
|
||||||
|
-- or multi-wiki
|
||||||
|
wikis = {
|
||||||
|
{ name = 'personal', root = '~/vimwiki', diary_rel_path = 'diary',
|
||||||
|
diary_frequency = 'daily', html = { html_path = '~/vimwiki/_html' } },
|
||||||
|
},
|
||||||
|
|
||||||
|
diagnostic = { link_severity = 'warn' },
|
||||||
|
mappings = { -- all default true except mouse
|
||||||
|
enabled = true, wiki_prefix = true, links = true, lists = true,
|
||||||
|
headers = true, table_editing = true, diary = true, html_export = true,
|
||||||
|
text_objects = true, mouse = false,
|
||||||
|
},
|
||||||
|
folding = 'lsp', -- lsp | expr | off
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
Per-wiki keys include `index`, `diary_rel_path`/`diary_index`/
|
||||||
|
`diary_frequency`/`diary_start_week_day`/`diary_sort`/`diary_caption_level`/
|
||||||
|
`diary_header`, `listsyms`/`listsyms_propagate`/`list_margin`,
|
||||||
|
`links_space_char`, `nested_syntaxes`, `auto_toc`, `maxhi`, and an `html`
|
||||||
|
table (`html_path`, `template_path`/`template_default`/`template_ext`/
|
||||||
|
`template_date_format`, `css_name`, `auto_export`,
|
||||||
|
`html_filename_parameterization`, `exclude_files`, `color_dic`).
|
||||||
|
|
||||||
|
### Command surface
|
||||||
|
|
||||||
|
`ftplugin/vimwiki.vim` (Vim) and `lua/nuwiki/commands.lua` (Neovim) define the
|
||||||
|
vimwiki command set with `:Vimwiki*` names plus `:Nuwiki*` aliases — index/tab
|
||||||
|
index/UI select, diary (note/yesterday/tomorrow/next/prev/index/generate),
|
||||||
|
link follow/backlinks/next/prev, goto/delete/rename, list toggle/reject/
|
||||||
|
remove-done, TOC, generate/check links, find orphans, tags rebuild/search/
|
||||||
|
generate, HTML 2HTML/browse/all/rss, table insert/move/colorize, and paste
|
||||||
|
link/url. Search uses `lvimgrep` and backlinks/follow use built-in LSP
|
||||||
|
requests; the rest route to `executeCommand`.
|
||||||
|
|
||||||
|
### Default keymaps
|
||||||
|
|
||||||
|
Buffer-local, gated per subgroup by `mappings.*`:
|
||||||
|
|
||||||
|
- **Wiki** `<Leader>ww`/`wt`/`ws`/`wi`, `<Leader>w<Leader>w|y|t|m|i`,
|
||||||
|
`<Leader>wn`/`wd`/`wr`/`wc`
|
||||||
|
- **Links** `<CR>` (follow / wrap-word), `<S-CR>`/`<C-CR>`/`<C-S-CR>`
|
||||||
|
(split/vsplit/tab), `<BS>` (back), `<Tab>`/`<S-Tab>` (next/prev link), `+`
|
||||||
|
(wrap as wikilink)
|
||||||
|
- **Lists** `<C-Space>` (toggle), `gnt` (next task), `gln`/`glp` (cycle
|
||||||
|
forward/back), `glx` (reject), `gll`/`glh` + `gLl`/`gLh` (indent/dedent,
|
||||||
|
item vs subtree), `glr`/`gLr` (renumber list/all), `gl<Space>`/`gL<Space>`
|
||||||
|
(remove done, list/whole-doc), `o`/`O` (open with bullet); insert-mode
|
||||||
|
`<C-T>`/`<C-D>`, `<C-L><C-J>`/`<C-L><C-K>`, `<C-L><C-M>`; smart `<CR>`/`<Tab>`
|
||||||
|
- **Headers** `=`/`-` (deeper/shallower), `]]`/`[[` (next/prev), `]=`/`[=`
|
||||||
|
(sibling), `]u`/`[u` (parent) — pure VimL/Lua, no LSP round-trip
|
||||||
|
- **Tables** `gqq`/`gww` (align), `<A-Left>`/`<A-Right>` (move column)
|
||||||
|
- **Diary** `<C-Down>`/`<C-Up>` (next/prev entry)
|
||||||
|
- **HTML** `<Leader>wh`/`whh`/`wha` (export/browse/all)
|
||||||
|
- **Mouse** (opt-in) `<2-LeftMouse>` follow + shift/ctrl variants
|
||||||
|
|
||||||
|
**Text objects:** `ah`/`ih` (heading section), `aH`/`iH` (heading + subtree),
|
||||||
|
`al`/`il` (list item), `a\`/`i\` (table cell), `ac`/`ic` (table column). Pure
|
||||||
|
VimL/Lua.
|
||||||
|
|
||||||
|
### Folding
|
||||||
|
|
||||||
|
Primary: LSP `textDocument/foldingRange` (`folding::folding_ranges`, heading
|
||||||
|
blocks + top-level lists). Fallback: a pure Lua `foldexpr` in
|
||||||
|
`lua/nuwiki/folding.lua` for clients without `foldingRange`.
|
||||||
|
|
||||||
|
### Health check
|
||||||
|
|
||||||
|
`:checkhealth nuwiki` verifies the binary exists and responds to `--version`,
|
||||||
|
the LSP client is attached, `.wiki` filetype detection works, and reports
|
||||||
|
registered commands and per-wiki index state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Vimwiki syntax support
|
||||||
|
|
||||||
|
Parsed and highlighted (custom semantic-token legend; `syntax/nuwiki.vim`
|
||||||
|
provides a no-LSP fallback):
|
||||||
|
|
||||||
|
- **Typefaces** bold `*…*`, italic `_…_`, bold-italic, strikethrough `~~…~~`,
|
||||||
|
inline code, super `^…^` / sub `,,…,,`, keywords (TODO/DONE/STARTED/FIXME/
|
||||||
|
FIXED/XXX/STOPPED)
|
||||||
|
- **Links** plain/described/subdir/root/absolute wikilinks, `[[dir/]]`,
|
||||||
|
interwiki (`wiki1:` / `wn.Name:`), diary, anchor-only and `Page#anchor`,
|
||||||
|
raw URLs, `file:`/`local:`, transclusion `{{url|alt|attrs}}`
|
||||||
|
- **Headers** levels 1–6, centered
|
||||||
|
- **Lists** unordered (`-` `*` `#`), ordered (`1.` `1)` `a)` `A)` `i)` `I)`),
|
||||||
|
nested/mixed, multi-line items, definition lists, all six checkbox states
|
||||||
|
- **Tables** `|`-delimited, header separator, column alignment markers,
|
||||||
|
colspan `>` / rowspan `\/`, inline formatting in cells
|
||||||
|
- **Blocks** preformatted `{{{ … }}}` (optional language), inline math `$…$`,
|
||||||
|
block math `{{$ … }}$` (optional environment), blockquotes, comments
|
||||||
|
(`%% …`), horizontal rule `----`
|
||||||
|
- **Tags** `:tag1:tag2:` (file / heading / standalone scope), tag-as-anchor
|
||||||
|
- **Placeholders** `%title` `%nohtml` `%template` `%date`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. CI/CD
|
||||||
|
|
||||||
|
`.gitea/workflows/ci.yaml` runs on every push and PR:
|
||||||
|
|
||||||
|
| Job | Command |
|
||||||
|
|---|---|
|
||||||
|
| fmt | `cargo fmt --all -- --check` |
|
||||||
|
| clippy | `cargo clippy --workspace --all-targets -- -D warnings` |
|
||||||
|
| test | `cargo test --workspace --all-targets` |
|
||||||
|
| keymaps | `development/tests/test-keymaps.sh` (Neovim 0.11) + `test-keymaps-vim.sh` (Vim) |
|
||||||
|
|
||||||
|
`.gitea/workflows/release.yaml` triggers on `v*` tags: cross-compiles the four
|
||||||
|
Linux targets (gnu/musl × x86_64/aarch64) via `cross`, packages `.tar.gz`, and
|
||||||
|
creates a Gitea release using `RELEASE_TOKEN`. macOS and Windows binaries are
|
||||||
|
built manually and attached to the same release. crates.io publishing is not
|
||||||
|
wired up.
|
||||||
|
|
||||||
|
A change is semver-breaking if it alters `nuwiki-core` AST nodes, the
|
||||||
|
`SyntaxPlugin`/`Renderer` traits, the user-config schema, or removes an LSP
|
||||||
|
capability.
|
||||||
Reference in New Issue
Block a user