nuwiki/development/SPEC.md

# nuwiki — Technical Reference

A developer-facing description of how nuwiki is built and what it implements.
For day-one setup see [ONBOARDING.md](ONBOARDING.md); for user-facing help see
`doc/nuwiki.txt` (`:h nuwiki`).

nuwiki is a vimwiki-compatible Vim/Neovim plugin backed by a Rust language
server. The editor layers are thin clients; all parsing, navigation, and
editing logic lives in the Rust crates and is reached over LSP.

| Property | Value |
|---|---|
| License | Dual MIT / Apache-2.0 |
| MSRV | Rust 1.83 |
| Edition / resolver | 2021 / v2 |
| Repo | `https://code.gfran.co/gffranco/nuwiki` |
| VCS / CI | Gitea + Gitea Actions |
| LSP library | `tower-lsp` (tokio, stdio transport) |
| Min editors | Neovim 0.11+, Vim 9.1+ |

---

## 1. Architecture

### Layer model

```
Editor client (VimL / Lua)  — pure wiring, no logic
        │  LSP over stdio
        ▼
nuwiki-lsp  — protocol bridge, executeCommand, document store, config
        │
        ▼
nuwiki-core — lexer → parser → AST → renderer (no editor deps)
```

### Crate dependency rules

```
nuwiki-ls  →  nuwiki-lsp  →  nuwiki-core
```

- `nuwiki-core` never depends on `nuwiki-lsp` / `nuwiki-ls`.
- `nuwiki-lsp` never depends on `nuwiki-ls`.
- The VimL and Lua layers contain no logic — every command body issues a
  `workspace/executeCommand` or a built-in LSP request.

### Repository layout

```
crates/
  nuwiki-core/        # parser, AST, renderer — editor-independent
    src/
      ast/            # block.rs inline.rs link.rs span.rs visit.rs
      syntax/         # registry.rs + vimwiki/{lexer,parser}.rs
      render/         # html.rs
      date.rs         # diary period math (no chrono dependency)
  nuwiki-lsp/         # LSP backend
    src/
      lib.rs          # Backend, capabilities, textDocument handlers
      commands.rs     # executeCommand dispatcher + pure edit ops
      config.rs       # Config / WikiConfig / HtmlConfig
      diagnostics.rs index.rs nav.rs rename.rs semantic_tokens.rs
      diary.rs export.rs edits.rs folding.rs wiki.rs
  nuwiki-ls/          # thin binary: starts the stdio server

plugin/nuwiki.vim     # universal entry point (detects Vim vs Neovim)
lua/nuwiki/           # Neovim layer: init, config, lsp, keymaps, commands,
                      #   folding, install
autoload/nuwiki/      # Vim layer: lsp.vim, commands.vim
ftdetect/ ftplugin/ syntax/   # filetype detection, buffer setup, fallback HL
doc/nuwiki.txt        # :help
scripts/download_bin.vim      # plugin-runtime binary download (build hooks)
development/          # dev-only tooling + this document (not shipped)
.gitea/workflows/     # ci.yaml, release.yaml
```

---

## 2. nuwiki-core

### AST

Every node carries a `Span { start, end }` of `Position { line, column, offset }`
(0-indexed; column and offset are byte-based). Spans drive diagnostics,
semantic tokens, and go-to-definition.

**Block nodes:** `Heading` (level 1–6, `centered`), `Paragraph`,
`HorizontalRule`, `Blockquote` (recursive), `Preformatted` (optional
language), `MathBlock` (optional environment), `List`, `DefinitionList`,
`Table`, `Comment`, `Tag`, `Error` (resilient parse failure).

**Inline nodes:** `Text`, `Bold`, `Italic`, `BoldItalic`, `Strikethrough`,
`Code`, `Superscript`, `Subscript`, `MathInline`, `Keyword`, `Color`,
`WikiLink`, `ExternalLink`, `Transclusion`, `RawUrl`.

**Enums:**
- `ListSymbol` = `Dash | Star | Hash | Numeric | NumericParen | AlphaParen |
  AlphaUpperParen | RomanParen | RomanUpperParen`
- `CheckboxState` = `Empty | Quarter | Half | ThreeQuarters | Done | Rejected`
- `Keyword` = `Todo | Done | Started | Fixme | Fixed | Xxx | Stopped`
- `LinkKind` = `Wiki | Interwiki | Diary | File | Local | Raw | AnchorOnly`
- `TagScope` = `File | Heading(idx) | Standalone`

`PageMetadata` carries `title`, `nohtml`, `template`, `date`, and aggregated
file-level `tags`.

### Lexer / parser

- Two-pass lexer (block pass, then inline pass per block); syntax-specific
  `VimwikiToken`s with byte-offset spans, collected eagerly into a
  `TokenStream`.
- Hand-rolled recursive-descent parser (not a combinator library). Resilient:
  malformed input yields an `ErrorNode` and parsing continues — a document
  never fails to parse.
- Full re-parse on every `didOpen` / `didChange` (no incremental parsing).

### Syntax plugin interface

Syntaxes register against a `SyntaxRegistry` keyed by id and file extension:

```
SyntaxPlugin { id, display_name, file_extensions, lexer, parser }
```

vimwiki (`.wiki`) is the only registered syntax; the registry exists so a
markdown plugin can be added without touching the LSP or editor layers.

### Renderer

`Renderer` trait writes to a `dyn Write`; `HtmlRenderer` is the implementation.
Link resolution is injected as a callback. Template substitution supports
`{{title}}`, `{{content}}`, `{{date}}`, `{{root_path}}`, `{{toc}}`, `{{css}}`.
`color_dic` maps colour-tag names to CSS values, falling back to
`class="color-<name>"`.

---

## 3. LSP server

### Advertised capabilities

| Capability | Notes |
|---|---|
| `textDocumentSync` | full sync (open/close/change/save) |
| `semanticTokensProvider` | full + range; custom vimwiki token legend |
| `documentSymbolProvider` / `workspaceSymbolProvider` | outline + workspace search (tags included) |
| `definitionProvider` / `referencesProvider` | link follow + backlinks |
| `hoverProvider` | link preview |
| `completionProvider` | trigger `[` |
| `renameProvider` | cross-document link rewrite |
| `foldingRangeProvider` | heading blocks + top-level lists |
| `executeCommandProvider` | see command surface below |
| `workspace.fileOperations` | `did_rename` + `did_delete` for `**/*.wiki`, `**/*.md` |
| `positionEncoding` | UTF-8 when client supports LSP 3.17+, else UTF-16 |

Workspace indexing runs as a background tokio task reporting via
`window/workDoneProgress`; nav features answer with partial data until the
scan completes. The document store is an `Arc<DashMap<Url, DocumentState>>`
holding text + AST + version.

### executeCommand surface

All editing commands return a `WorkspaceEdit` (applied via
`workspace/applyEdit`) built through `edits::WorkspaceEditBuilder`, so the AST
stays the source of truth. Navigation commands return a `Location`.

| Namespace | Commands |
|---|---|
| `list` | `toggleCheckbox` `cycleCheckbox` `rejectCheckbox` `nextTask` `removeDone` `renumber` `changeSymbol` `changeLevel` |
| `table` | `insert` `align` `moveColumn` |
| `heading` | `addLevel` `removeLevel` |
| `link` | `pasteWikilink` `pasteUrl` |
| `toc` / `links` | `toc.generate` · `links.generate` |
| `workspace` | `checkLinks` `findOrphans` |
| `diary` | `openToday` `openYesterday` `openTomorrow` `openIndex` `generateIndex` `next` `prev` `listEntries` `openForDate` |
| `tags` | `search` `generateLinks` `rebuild` |
| `export` | `currentToHtml` `allToHtml` `allToHtmlForce` `browse` `rss` |
| `wiki` | `listAll` `select` `openIndex` `tabOpenIndex` `gotoPage` |
| `file` | `delete` |

`workspace/rename` emits a `RenameFile` op plus `[[A]]` → `[[B]]` rewrites
across every linking page (anchors and descriptions preserved). Closed
documents are re-parsed on demand so cross-document edits use current source.

### Diagnostics

Composable collector chains parse errors (`ErrorNode` walk) and broken-link
checks (wiki target / anchor / `file:` / `local:` existence). Per-source
severity is config-driven (`diagnostic.link_severity`: `off|hint|warn|error`);
raw and external URLs are never diagnosed.

---

## 4. Editor integration

### Installation

Pre-built binaries are published as Gitea release assets
(`nuwiki-ls-{version}-{target}.tar.gz`) and fetched at install time by
`lua/nuwiki/install.lua` (Neovim) or `scripts/download_bin.vim` (Vim build
hooks), installed to `{plugin_dir}/bin/nuwiki-ls`. Falls back to
`cargo build --release` when download fails or
`g:nuwiki_build_from_source = 1`.

`plugin/nuwiki.vim` dispatches: Neovim → `require('nuwiki').setup()`; Vim →
`nuwiki#lsp#start()` (prefers `vim-lsp`, falls back to `coc.nvim`). On Neovim
the server is registered via `vim.lsp.config{}` + `vim.lsp.enable`, with a
`vim.lsp.start` autocmd fallback.

### Configuration

Single-wiki (v1.0) and multi-wiki forms both work; the scalar form desugars to
a one-entry `wikis` list.

```lua
require('nuwiki').setup({
  -- single-wiki shorthand
  wiki_root = '~/vimwiki', file_extension = '.wiki', syntax = 'vimwiki',
  log_level = 'warn',

  -- or multi-wiki
  wikis = {
    { name = 'personal', root = '~/vimwiki', diary_rel_path = 'diary',
      diary_frequency = 'daily', html = { html_path = '~/vimwiki/_html' } },
  },

  diagnostic = { link_severity = 'warn' },
  mappings = {                  -- all default true except mouse
    enabled = true, wiki_prefix = true, links = true, lists = true,
    headers = true, table_editing = true, diary = true, html_export = true,
    text_objects = true, mouse = false,
  },
  folding = 'lsp',              -- lsp | expr | off
})
```

Per-wiki keys include `index`, `diary_rel_path`/`diary_index`/
`diary_frequency`/`diary_start_week_day`/`diary_sort`/`diary_caption_level`/
`diary_header`, `listsyms`/`listsyms_propagate`/`list_margin`,
`links_space_char`, `nested_syntaxes`, `auto_toc`, `maxhi`, and an `html`
table (`html_path`, `template_path`/`template_default`/`template_ext`/
`template_date_format`, `css_name`, `auto_export`,
`html_filename_parameterization`, `exclude_files`, `color_dic`).

### Command surface

`ftplugin/vimwiki.vim` (Vim) and `lua/nuwiki/commands.lua` (Neovim) define the
vimwiki command set with `:Vimwiki*` names plus `:Nuwiki*` aliases — index/tab
index/UI select, diary (note/yesterday/tomorrow/next/prev/index/generate),
link follow/backlinks/next/prev, goto/delete/rename, list toggle/reject/
remove-done, TOC, generate/check links, find orphans, tags rebuild/search/
generate, HTML 2HTML/browse/all/rss, table insert/move/colorize, and paste
link/url. Search uses `lvimgrep` and backlinks/follow use built-in LSP
requests; the rest route to `executeCommand`.

### Default keymaps

Buffer-local, gated per subgroup by `mappings.*`:

- **Wiki** `<Leader>ww`/`wt`/`ws`/`wi`, `<Leader>w<Leader>w|y|t|m|i`,
  `<Leader>wn`/`wd`/`wr`/`wc`
- **Links** `<CR>` (follow / wrap-word), `<S-CR>`/`<C-CR>`/`<C-S-CR>`
  (split/vsplit/tab), `<BS>` (back), `<Tab>`/`<S-Tab>` (next/prev link), `+`
  (wrap as wikilink)
- **Lists** `<C-Space>` (toggle), `gnt` (next task), `gln`/`glp` (cycle
  forward/back), `glx` (reject), `gll`/`glh` + `gLl`/`gLh` (indent/dedent,
  item vs subtree), `glr`/`gLr` (renumber list/all), `gl<Space>`/`gL<Space>`
  (remove done, list/whole-doc), `o`/`O` (open with bullet); insert-mode
  `<C-T>`/`<C-D>`, `<C-L><C-J>`/`<C-L><C-K>`, `<C-L><C-M>`; smart `<CR>`/`<Tab>`
- **Headers** `=`/`-` (deeper/shallower), `]]`/`[[` (next/prev), `]=`/`[=`
  (sibling), `]u`/`[u` (parent) — pure VimL/Lua, no LSP round-trip
- **Tables** `gqq`/`gww` (align), `<A-Left>`/`<A-Right>` (move column)
- **Diary** `<C-Down>`/`<C-Up>` (next/prev entry)
- **HTML** `<Leader>wh`/`whh`/`wha` (export/browse/all)
- **Mouse** (opt-in) `<2-LeftMouse>` follow + shift/ctrl variants

**Text objects:** `ah`/`ih` (heading section), `aH`/`iH` (heading + subtree),
`al`/`il` (list item), `a\`/`i\` (table cell), `ac`/`ic` (table column). Pure
VimL/Lua.

### Folding

Primary: LSP `textDocument/foldingRange` (`folding::folding_ranges`, heading
blocks + top-level lists). Fallback: a pure Lua `foldexpr` in
`lua/nuwiki/folding.lua` for clients without `foldingRange`.

### Health check

`:checkhealth nuwiki` verifies the binary exists and responds to `--version`,
the LSP client is attached, `.wiki` filetype detection works, and reports
registered commands and per-wiki index state.

---

## 5. Vimwiki syntax support

Parsed and highlighted (custom semantic-token legend; `syntax/nuwiki.vim`
provides a no-LSP fallback):

- **Typefaces** bold `*…*`, italic `_…_`, bold-italic, strikethrough `~~…~~`,
  inline code, super `^…^` / sub `,,…,,`, keywords (TODO/DONE/STARTED/FIXME/
  FIXED/XXX/STOPPED)
- **Links** plain/described/subdir/root/absolute wikilinks, `[[dir/]]`,
  interwiki (`wiki1:` / `wn.Name:`), diary, anchor-only and `Page#anchor`,
  raw URLs, `file:`/`local:`, transclusion `{{url|alt|attrs}}`
- **Headers** levels 1–6, centered
- **Lists** unordered (`-` `*` `#`), ordered (`1.` `1)` `a)` `A)` `i)` `I)`),
  nested/mixed, multi-line items, definition lists, all six checkbox states
- **Tables** `|`-delimited, header separator, column alignment markers,
  colspan `>` / rowspan `\/`, inline formatting in cells
- **Blocks** preformatted `{{{ … }}}` (optional language), inline math `$…$`,
  block math `{{$ … }}$` (optional environment), blockquotes, comments
  (`%% …`), horizontal rule `----`
- **Tags** `:tag1:tag2:` (file / heading / standalone scope), tag-as-anchor
- **Placeholders** `%title` `%nohtml` `%template` `%date`

---

## 6. CI/CD

`.gitea/workflows/ci.yaml` runs on every push and PR:

| Job | Command |
|---|---|
| fmt | `cargo fmt --all -- --check` |
| clippy | `cargo clippy --workspace --all-targets -- -D warnings` |
| test | `cargo test --workspace --all-targets` |
| keymaps | `development/tests/test-keymaps.sh` (Neovim 0.11) + `test-keymaps-vim.sh` (Vim) |

`.gitea/workflows/release.yaml` triggers on `v*` tags: cross-compiles the four
Linux targets (gnu/musl × x86_64/aarch64) via `cross`, packages `.tar.gz`, and
creates a Gitea release using `RELEASE_TOKEN`. macOS and Windows binaries are
built manually and attached to the same release. crates.io publishing is not
wired up.

A change is semver-breaking if it alters `nuwiki-core` AST nodes, the
`SyntaxPlugin`/`Renderer` traits, the user-config schema, or removes an LSP
capability.