gffranco
cb11889e72
Delete all Rust crates (crates/) and Cargo files. The nuwiki-ls binary is now built in the separate nuwiki-rs repository and downloaded at install time from the Gitea release assets, with a cargo build fallback that clones nuwiki-rs. Update all documentation to reflect the split repo layout.
313 lines
13 KiB
Markdown
313 lines
13 KiB
Markdown
# nuwiki — Technical Reference
|
||
|
||
A developer-facing description of how nuwiki is built and what it implements.
|
||
For day-one setup see [ONBOARDING.md](ONBOARDING.md); for user-facing help see
|
||
`doc/nuwiki.txt` (`:h nuwiki`).
|
||
|
||
nuwiki is a vimwiki-compatible Vim/Neovim plugin backed by a Rust language
|
||
server. The editor layers are thin clients; all parsing, navigation, and
|
||
editing logic lives in the [nuwiki-rs](https://code.gfran.co/gffranco/nuwiki-rs)
|
||
repository and is reached over LSP.
|
||
|
||
| Property | Value |
|
||
|---|---|
|
||
| License | Dual MIT / Apache-2.0 |
|
||
| Min editors | Neovim 0.11+, Vim 9.1+ |
|
||
|
||
---
|
||
|
||
## 1. Architecture
|
||
|
||
### Layer model
|
||
|
||
```
|
||
Editor client (VimL / Lua) — pure wiring, no logic
|
||
│ LSP over stdio
|
||
▼
|
||
nuwiki-ls (nuwiki-rs) — Rust LSP server: protocol bridge, executeCommand,
|
||
document store, config, parser, renderer
|
||
```
|
||
|
||
> The Rust LSP server is in the separate [nuwiki-rs](https://code.gfran.co/gffranco/nuwiki-rs)
|
||
> repository. This plugin repo is the Vim/Neovim client layer only.
|
||
>
|
||
> The architecture table below is for reference — the Rust source lives in nuwiki-rs.
|
||
|
||
- The VimL and Lua layers contain no logic — every command body issues a
|
||
`workspace/executeCommand` or a built-in LSP request.
|
||
|
||
### Repository layout
|
||
|
||
```
|
||
plugin/nuwiki.vim # universal entry point (detects Vim vs Neovim)
|
||
lua/nuwiki/ # Neovim layer: init, config, lsp, keymaps, commands,
|
||
# folding, install
|
||
autoload/nuwiki/ # Vim layer: lsp.vim, commands.vim
|
||
ftdetect/ ftplugin/ syntax/ # filetype detection, buffer setup, fallback HL
|
||
doc/nuwiki.txt # :help
|
||
scripts/download_bin.vim # plugin-runtime binary download (build hooks)
|
||
development/ # dev-only tooling + this document (not shipped)
|
||
```
|
||
|
||
---
|
||
|
||
## 2. nuwiki-core (nuwiki-rs)
|
||
|
||
> This section documents the Rust server's internals — the source lives in
|
||
> the [nuwiki-rs](https://code.gfran.co/gffranco/nuwiki-rs) repository. Kept
|
||
> here as a technical reference for the LSP protocol contract.
|
||
|
||
### AST
|
||
|
||
Every node carries a `Span { start, end }` of `Position { line, column, offset }`
|
||
(0-indexed; column and offset are byte-based). Spans drive diagnostics,
|
||
semantic tokens, and go-to-definition.
|
||
|
||
**Block nodes:** `Heading` (level 1–6, `centered`), `Paragraph`,
|
||
`HorizontalRule`, `Blockquote` (recursive), `Preformatted` (optional
|
||
language), `MathBlock` (optional environment), `List`, `DefinitionList`,
|
||
`Table`, `Comment`, `Tag`, `Error` (resilient parse failure).
|
||
|
||
**Inline nodes:** `Text`, `Bold`, `Italic`, `BoldItalic`, `Strikethrough`,
|
||
`Code`, `Superscript`, `Subscript`, `MathInline`, `Keyword`, `Color`,
|
||
`WikiLink`, `ExternalLink`, `Transclusion`, `RawUrl`.
|
||
|
||
**Enums:**
|
||
- `ListSymbol` = `Dash | Star | Hash | Numeric | NumericParen | AlphaParen |
|
||
AlphaUpperParen | RomanParen | RomanUpperParen`
|
||
- `CheckboxState` = `Empty | Quarter | Half | ThreeQuarters | Done | Rejected`
|
||
- `Keyword` = `Todo | Done | Started | Fixme | Fixed | Xxx | Stopped`
|
||
- `LinkKind` = `Wiki | Interwiki | Diary | File | Local | Raw | AnchorOnly`
|
||
- `TagScope` = `File | Heading(idx) | Standalone`
|
||
|
||
`PageMetadata` carries `title`, `nohtml`, `template`, `date`, and aggregated
|
||
file-level `tags`.
|
||
|
||
### Lexer / parser
|
||
|
||
- Two-pass lexer (block pass, then inline pass per block); syntax-specific
|
||
`VimwikiToken`s with byte-offset spans, collected eagerly into a
|
||
`TokenStream`.
|
||
- Hand-rolled recursive-descent parser (not a combinator library). Resilient:
|
||
malformed input yields an `ErrorNode` and parsing continues — a document
|
||
never fails to parse.
|
||
- Full re-parse on every `didOpen` / `didChange` (no incremental parsing).
|
||
|
||
### Syntax plugin interface
|
||
|
||
Syntaxes register against a `SyntaxRegistry` keyed by id and file extension:
|
||
|
||
```
|
||
SyntaxPlugin { id, display_name, file_extensions, lexer, parser }
|
||
```
|
||
|
||
vimwiki (`.wiki`) is the only registered syntax; the registry exists so a
|
||
markdown plugin can be added without touching the LSP or editor layers.
|
||
|
||
### Renderer
|
||
|
||
`Renderer` trait writes to a `dyn Write`; `HtmlRenderer` is the implementation.
|
||
Link resolution is injected as a callback. Template substitution supports
|
||
`{{title}}`, `{{content}}`, `{{date}}`, `{{root_path}}`, `{{toc}}`, `{{css}}`.
|
||
`color_dic` maps colour-tag names to CSS values, falling back to
|
||
`class="color-<name>"`.
|
||
|
||
---
|
||
|
||
## 3. LSP server
|
||
|
||
### Advertised capabilities
|
||
|
||
| Capability | Notes |
|
||
|---|---|
|
||
| `textDocumentSync` | full sync (open/close/change/save) |
|
||
| `semanticTokensProvider` | full + range; custom vimwiki token legend |
|
||
| `documentSymbolProvider` / `workspaceSymbolProvider` | outline + workspace search (tags included) |
|
||
| `definitionProvider` / `referencesProvider` | link follow + backlinks |
|
||
| `hoverProvider` | link preview |
|
||
| `completionProvider` | trigger `[` |
|
||
| `renameProvider` | cross-document link rewrite |
|
||
| `foldingRangeProvider` | heading blocks + top-level lists |
|
||
| `executeCommandProvider` | see command surface below |
|
||
| `workspace.fileOperations` | `did_rename` + `did_delete` for `**/*.wiki`, `**/*.md` |
|
||
| `positionEncoding` | UTF-8 when client supports LSP 3.17+, else UTF-16 |
|
||
|
||
Workspace indexing runs as a background tokio task reporting via
|
||
`window/workDoneProgress`; nav features answer with partial data until the
|
||
scan completes. The document store is an `Arc<DashMap<Url, DocumentState>>`
|
||
holding text + AST + version.
|
||
|
||
### executeCommand surface
|
||
|
||
All editing commands return a `WorkspaceEdit` (applied via
|
||
`workspace/applyEdit`) built through `edits::WorkspaceEditBuilder`, so the AST
|
||
stays the source of truth. Navigation commands return a `Location`.
|
||
|
||
| Namespace | Commands |
|
||
|---|---|
|
||
| `list` | `toggleCheckbox` `cycleCheckbox` `rejectCheckbox` `nextTask` `removeDone` `renumber` `changeSymbol` `changeLevel` |
|
||
| `table` | `insert` `align` `moveColumn` |
|
||
| `heading` | `addLevel` `removeLevel` |
|
||
| `link` | `pasteWikilink` `pasteUrl` |
|
||
| `toc` / `links` | `toc.generate` · `links.generate` |
|
||
| `workspace` | `checkLinks` `findOrphans` |
|
||
| `diary` | `openToday` `openYesterday` `openTomorrow` `openIndex` `generateIndex` `next` `prev` `listEntries` `openForDate` |
|
||
| `tags` | `search` `generateLinks` `rebuild` |
|
||
| `export` | `currentToHtml` `allToHtml` `allToHtmlForce` `browse` `rss` |
|
||
| `wiki` | `listAll` `select` `openIndex` `tabOpenIndex` `gotoPage` |
|
||
| `file` | `delete` |
|
||
|
||
`workspace/rename` emits a `RenameFile` op plus `[[A]]` → `[[B]]` rewrites
|
||
across every linking page (anchors and descriptions preserved). Closed
|
||
documents are re-parsed on demand so cross-document edits use current source.
|
||
|
||
### Diagnostics
|
||
|
||
Composable collector chains parse errors (`ErrorNode` walk) and broken-link
|
||
checks (wiki target / anchor / `file:` / `local:` existence). Per-source
|
||
severity is config-driven (`diagnostic.link_severity`: `off|hint|warn|error`);
|
||
raw and external URLs are never diagnosed.
|
||
|
||
---
|
||
|
||
## 4. Editor integration
|
||
|
||
### Installation
|
||
|
||
Pre-built binaries are published as Gitea release assets
|
||
(`nuwiki-ls-{version}-{target}.tar.gz`) and fetched at install time by
|
||
`lua/nuwiki/install.lua` (Neovim) or `scripts/download_bin.vim` (Vim build
|
||
hooks), installed to `{plugin_dir}/bin/nuwiki-ls`. The binary is downloaded
|
||
from the [nuwiki-rs](https://code.gfran.co/gffranco/nuwiki-rs) releases.
|
||
|
||
`plugin/nuwiki.vim` dispatches: Neovim → `require('nuwiki').setup()`; Vim →
|
||
`nuwiki#lsp#start()` (prefers `vim-lsp`, falls back to `coc.nvim`). On Neovim
|
||
the server is registered via `vim.lsp.config{}` + `vim.lsp.enable`, with a
|
||
`vim.lsp.start` autocmd fallback.
|
||
|
||
### Configuration
|
||
|
||
Single-wiki (v1.0) and multi-wiki forms both work; the scalar form desugars to
|
||
a one-entry `wikis` list.
|
||
|
||
```lua
|
||
require('nuwiki').setup({
|
||
-- single-wiki shorthand
|
||
wiki_root = '~/vimwiki', file_extension = '.wiki', syntax = 'vimwiki',
|
||
log_level = 'warn',
|
||
|
||
-- or multi-wiki
|
||
wikis = {
|
||
{ name = 'personal', root = '~/vimwiki', diary_rel_path = 'diary',
|
||
diary_frequency = 'daily', html = { html_path = '~/vimwiki/_html' } },
|
||
},
|
||
|
||
diagnostic = { link_severity = 'warn' },
|
||
mappings = { -- all default true except mouse
|
||
enabled = true, wiki_prefix = true, links = true, lists = true,
|
||
headers = true, table_editing = true, diary = true, html_export = true,
|
||
text_objects = true, mouse = false,
|
||
},
|
||
folding = 'lsp', -- lsp | expr | off
|
||
})
|
||
```
|
||
|
||
Per-wiki keys include `index`, `diary_rel_path`/`diary_index`/
|
||
`diary_frequency`/`diary_sort`/`diary_caption_level`/
|
||
`diary_header`, `listsyms`/`listsyms_propagate`/`list_margin`,
|
||
`links_space_char`, `auto_toc`, and an `html`
|
||
table (`html_path`, `template_path`/`template_default`/`template_ext`/
|
||
`template_date_format`, `css_name`, `auto_export`,
|
||
`html_filename_parameterization`, `exclude_files`, `color_dic`).
|
||
|
||
### Command surface
|
||
|
||
`ftplugin/vimwiki.vim` (Vim) and `lua/nuwiki/commands.lua` (Neovim) define the
|
||
vimwiki command set with `:Vimwiki*` names plus `:Nuwiki*` aliases — index/tab
|
||
index/UI select, diary (note/yesterday/tomorrow/next/prev/index/generate),
|
||
link follow/backlinks/next/prev, goto/delete/rename, list toggle/reject/
|
||
remove-done, TOC, generate/check links, find orphans, tags rebuild/search/
|
||
generate, HTML 2HTML/browse/all/rss, table insert/move/colorize, and paste
|
||
link/url. Search uses `lvimgrep` and backlinks/follow use built-in LSP
|
||
requests; the rest route to `executeCommand`.
|
||
|
||
### Default keymaps
|
||
|
||
Buffer-local, gated per subgroup by `mappings.*`:
|
||
|
||
- **Wiki** `<Leader>ww`/`wt`/`ws`/`wi`, `<Leader>w<Leader>w|y|t|m|i`,
|
||
`<Leader>wn`/`wd`/`wr`/`wc`
|
||
- **Links** `<CR>` (follow / wrap-word), `<S-CR>`/`<C-CR>`/`<C-S-CR>`
|
||
(split/vsplit/tab), `<BS>` (back), `<Tab>`/`<S-Tab>` (next/prev link), `+`
|
||
(wrap as wikilink)
|
||
- **Lists** `<C-Space>` (toggle), `gnt` (next task), `gln`/`glp` (cycle
|
||
forward/back), `glx` (reject), `gll`/`glh` + `gLl`/`gLh` (indent/dedent,
|
||
item vs subtree), `glr`/`gLr` (renumber list/all), `gl<Space>`/`gL<Space>`
|
||
(remove done, list/whole-doc), `o`/`O` (open with bullet); insert-mode
|
||
`<C-T>`/`<C-D>`, `<C-L><C-J>`/`<C-L><C-K>`, `<C-L><C-M>`; smart `<CR>`/`<Tab>`
|
||
- **Headers** `=`/`-` (deeper/shallower), `]]`/`[[` (next/prev), `]=`/`[=`
|
||
(sibling), `]u`/`[u` (parent) — pure VimL/Lua, no LSP round-trip
|
||
- **Tables** `gqq`/`gww` (align), `<A-Left>`/`<A-Right>` (move column)
|
||
- **Diary** `<C-Down>`/`<C-Up>` (next/prev entry)
|
||
- **HTML** `<Leader>wh`/`whh`/`wha` (export/browse/all)
|
||
- **Mouse** (opt-in) `<2-LeftMouse>` follow + shift/ctrl variants
|
||
|
||
**Text objects:** `ah`/`ih` (heading section), `aH`/`iH` (heading + subtree),
|
||
`al`/`il` (list item), `a\`/`i\` (table cell), `ac`/`ic` (table column). Pure
|
||
VimL/Lua.
|
||
|
||
### Folding
|
||
|
||
Primary: LSP `textDocument/foldingRange` (`folding::folding_ranges`, heading
|
||
blocks + top-level lists). Fallback: a pure Lua `foldexpr` in
|
||
`lua/nuwiki/folding.lua` for clients without `foldingRange`.
|
||
|
||
### Health check
|
||
|
||
`:checkhealth nuwiki` verifies the binary exists and responds to `--version`,
|
||
the LSP client is attached, `.wiki` filetype detection works, and reports
|
||
registered commands and per-wiki index state.
|
||
|
||
---
|
||
|
||
## 5. Vimwiki syntax support
|
||
|
||
Parsed and highlighted (custom semantic-token legend; `syntax/nuwiki.vim`
|
||
provides a no-LSP fallback):
|
||
|
||
- **Typefaces** bold `*…*`, italic `_…_`, bold-italic, strikethrough `~~…~~`,
|
||
inline code, super `^…^` / sub `,,…,,`, keywords (TODO/DONE/STARTED/FIXME/
|
||
FIXED/XXX/STOPPED)
|
||
- **Links** plain/described/subdir/root/absolute wikilinks, `[[dir/]]`,
|
||
interwiki (`wiki1:` / `wn.Name:`), diary, anchor-only and `Page#anchor`,
|
||
raw URLs, `file:`/`local:`, transclusion `{{url|alt|attrs}}`
|
||
- **Headers** levels 1–6, centered
|
||
- **Lists** unordered (`-` `*` `#`), ordered (`1.` `1)` `a)` `A)` `i)` `I)`),
|
||
nested/mixed, multi-line items, definition lists, all six checkbox states
|
||
- **Tables** `|`-delimited, header separator, column alignment markers,
|
||
colspan `>` / rowspan `\/`, inline formatting in cells
|
||
- **Blocks** preformatted `{{{ … }}}` (optional language), inline math `$…$`,
|
||
block math `{{$ … }}$` (optional environment), blockquotes, comments
|
||
(`%% …`), horizontal rule `----`
|
||
- **Tags** `:tag1:tag2:` (file / heading / standalone scope), tag-as-anchor
|
||
- **Placeholders** `%title` `%nohtml` `%template` `%date`
|
||
|
||
---
|
||
|
||
## 6. CI/CD
|
||
|
||
`.gitea/workflows/ci.yaml` runs on every push and PR:
|
||
|
||
| Job | Command |
|
||
|---|---|
|
||
| keymaps | `development/tests/test-keymaps.sh` (Neovim 0.11) + `test-keymaps-vim.sh` (Vim) |
|
||
| calendar | `development/tests/test-calendar.sh` (Neovim) + `test-calendar-vim.sh` (Vim) |
|
||
|
||
`.gitea/workflows/release.yaml` (in [nuwiki-rs](https://code.gfran.co/gffranco/nuwiki-rs))
|
||
triggers on `v*` tags: cross-compiles the four Linux targets (gnu/musl × x86_64/aarch64)
|
||
via `cross`, packages `.tar.gz`, and creates a Gitea release using `RELEASE_TOKEN`.
|
||
|
||
A change is semver-breaking if it alters `nuwiki-core` AST nodes, the
|
||
`SyntaxPlugin`/`Renderer` traits, the user-config schema, or removes an LSP
|
||
capability.
|