phase 8: navigation — definition, refs, hover, completion, ws/symbol

P8 resolved (SPEC §11): lazy + background indexing with progress.
Initial scan runs in a tokio task spawned from `initialized`; the
server stays responsive immediately and nav features answer with
partial data until the index is complete.

WorkspaceIndex (`index.rs`):
- Per-URI `IndexedPage` keeps name, title, headings, outgoing links.
- Reverse maps: `pages_by_name`, `backlinks: HashMap<page, …>`.
- `upsert` scrubs stale entries first so re-parses don't leak.
- `resolve` handles Wiki + AnchorOnly link kinds against the index;
  other kinds (Interwiki/Diary/File/Local/Raw) fall through.
- `slugify` for anchors (lowercases, collapses runs, drops trailing
  dashes, preserves Unicode), `page_name_from_uri` for workspace-
  rooted names, `walk_wiki_files` for the background scanner.

Backend wiring (`lib.rs`):
- `initialize` captures workspace root from `workspaceFolders` or the
  deprecated `root_uri`; advertises definition/references/hover/
  completion (trigger `[`)/workspace_symbol providers.
- `initialized` spawns the indexer, wrapped in
  `window/workDoneProgress` begin/end notifications.
- `did_open`/`did_change` synchronously update the index alongside
  publishing diagnostics; `did_close` keeps the indexed projection
  (file may still exist on disk and other pages link to it) and
  only clears diagnostics + drops the live document state.

Position lookup (`nav.rs`):
- `lsp_to_byte_pos` translates between LSP encoding (UTF-8 or
  UTF-16) and our byte columns.
- `find_inline_at` descends blocks → inlines → inline containers,
  returning the most specific inline whose span covers the cursor.

Handlers:
- definition: wikilinks resolve to a target URI; anchored links
  return the range at the matching heading.
- references: backlinks of the cursor's link target, or of the
  current page when the cursor isn't on a link. Ranges fall back
  to byte-column LSP positions when the source doc isn't open.
- hover: markdown preview with page title + outline (first 8
  headings), or a "not in index" fallback.
- completion: fires only inside an unterminated `[[ … ` on the
  current line; suggests every indexed page name.
- workspace/symbol: case-insensitive substring over every indexed
  heading.

Tests (20 new): slugify (basic / punctuation / Unicode / trailing
dash), page-name derivation under various roots, upsert / replace /
remove, resolve for Wiki + AnchorOnly, sorted `page_names`, UTF-8
passthrough + UTF-16 conversion, position lookup hits and misses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

crates/nuwiki-lsp/src/lib.rs

@@ -1,36 +1,52 @@
 //! LSP protocol bridge for nuwiki.
 //!
-//! `tower-lsp` server that maintains a `DocumentStore` (SPEC §6.10) and
-//! exposes the Phase 6 feature set:
+//! `tower-lsp` server that maintains a `DocumentStore` (SPEC §6.10) plus a
+//! workspace-wide index (SPEC §6.10 + P8) and exposes every LSP method
+//! through Phase 8:
 //!
 //!   - `initialize` / `initialized` / `shutdown`
 //!   - `textDocument/didOpen`, `didChange` (full sync), `didClose`
 //!   - `textDocument/publishDiagnostics` from `BlockNode::Error` nodes
 //!   - `textDocument/documentSymbol` — nested outline from headings
+//!   - `textDocument/semanticTokens/full` + `/range`
+//!   - `textDocument/definition`, `references`, `hover`
+//!   - `textDocument/completion` (trigger: `[`)
+//!   - `workspace/symbol` — search across indexed headings
 //!
 //! Position encoding is negotiated as UTF-8 when the client supports LSP
 //! 3.17+ encodings (SPEC §6.11). When the client only supports the legacy
 //! UTF-16 default, positions get translated through a per-line lookup.
 //!
 //! Re-parses are full per change (incremental parsing deferred post-v1).
+//! The workspace scan runs in a background tokio task spawned from
+//! `initialize` (P8) with `window/workDoneProgress` begin/end events.
 
+pub mod index;
+pub mod nav;
 pub mod semantic_tokens;
 
 use std::io;
+use std::path::PathBuf;
 use std::sync::atomic::{AtomicBool, Ordering};
-use std::sync::Arc;
+use std::sync::{Arc, RwLock};
 
 use dashmap::DashMap;
 use tokio::io::{AsyncRead, AsyncWrite};
 use tower_lsp::jsonrpc::Result as LspResult;
 use tower_lsp::lsp_types::{
+    CompletionItem, CompletionItemKind, CompletionOptions, CompletionParams, CompletionResponse,
     Diagnostic, DiagnosticSeverity, DidChangeTextDocumentParams, DidCloseTextDocumentParams,
     DidOpenTextDocumentParams, DocumentSymbol, DocumentSymbolParams, DocumentSymbolResponse,
-    InitializeParams, InitializeResult, InitializedParams, MessageType, OneOf,
-    PositionEncodingKind, SemanticTokens, SemanticTokensFullOptions, SemanticTokensOptions,
-    SemanticTokensParams, SemanticTokensRangeParams, SemanticTokensRangeResult,
-    SemanticTokensResult, SemanticTokensServerCapabilities, ServerCapabilities, ServerInfo,
-    SymbolKind, TextDocumentSyncCapability, TextDocumentSyncKind, Url, WorkDoneProgressOptions,
+    GotoDefinitionParams, GotoDefinitionResponse, Hover, HoverContents, HoverParams,
+    HoverProviderCapability, InitializeParams, InitializeResult, InitializedParams, Location,
+    MarkupContent, MarkupKind, MessageType, OneOf, PositionEncodingKind, ProgressParams,
+    ProgressParamsValue, ProgressToken, Range, ReferenceParams, SemanticTokens,
+    SemanticTokensFullOptions, SemanticTokensOptions, SemanticTokensParams,
+    SemanticTokensRangeParams, SemanticTokensRangeResult, SemanticTokensResult,
+    SemanticTokensServerCapabilities, ServerCapabilities, ServerInfo,
+    SymbolInformation as LspSymbolInformation, SymbolKind, TextDocumentSyncCapability,
+    TextDocumentSyncKind, Url, WorkDoneProgress, WorkDoneProgressBegin, WorkDoneProgressEnd,
+    WorkDoneProgressOptions, WorkspaceSymbolParams,
 };
 use tower_lsp::{Client, LanguageServer, LspService, Server};
 
@@ -41,6 +57,8 @@ use nuwiki_core::ast::{
 use nuwiki_core::syntax::vimwiki::VimwikiSyntax;
 use nuwiki_core::syntax::SyntaxRegistry;
 
+use crate::index::{IndexedPage, WorkspaceIndex};
+
 /// Run the LSP server over the given async reader/writer pair. The binary
 /// (`nuwiki-ls`) just calls this with stdin/stdout.
 pub async fn run<I, O>(reader: I, writer: O)
@@ -78,6 +96,8 @@ struct Backend {
     /// True after `initialize` if the client opted into UTF-8 encoding.
     /// Otherwise we keep the LSP 3.16 default of UTF-16.
     use_utf8: Arc<AtomicBool>,
+    /// Workspace index — populated lazily in the background per P8.
+    index: Arc<RwLock<WorkspaceIndex>>,
 }
 
 impl Backend {
@@ -89,6 +109,7 @@ impl Backend {
             documents: Arc::new(DashMap::new()),
             registry: Arc::new(registry),
             use_utf8: Arc::new(AtomicBool::new(false)),
+            index: Arc::new(RwLock::new(WorkspaceIndex::default())),
         }
     }
 
@@ -106,6 +127,13 @@ impl Backend {
         };
         let ast = plugin.parse(&text);
         let diagnostics = ast_diagnostics(&ast, &text, self.use_utf8.load(Ordering::Relaxed));
+
+        // Keep the workspace index in sync with what the editor is showing.
+        {
+            let mut idx = self.index.write().expect("index lock poisoned");
+            idx.upsert(uri.clone(), &ast);
+        }
+
         self.documents
             .insert(uri.clone(), DocumentState { text, ast, version });
         self.client
@@ -133,6 +161,12 @@ impl LanguageServer for Backend {
             None // server defaults to UTF-16 per LSP 3.16.
         };
 
+        // Capture the workspace root for the background indexer.
+        let root = workspace_root_from_params(&params);
+        if let Some(p) = &root {
+            self.index.write().expect("index lock poisoned").root = Some(p.clone());
+        }
+
         Ok(InitializeResult {
             capabilities: ServerCapabilities {
                 position_encoding,
@@ -140,6 +174,14 @@ impl LanguageServer for Backend {
                     TextDocumentSyncKind::FULL,
                 )),
                 document_symbol_provider: Some(OneOf::Left(true)),
+                workspace_symbol_provider: Some(OneOf::Left(true)),
+                definition_provider: Some(OneOf::Left(true)),
+                references_provider: Some(OneOf::Left(true)),
+                hover_provider: Some(HoverProviderCapability::Simple(true)),
+                completion_provider: Some(CompletionOptions {
+                    trigger_characters: Some(vec!["[".into()]),
+                    ..CompletionOptions::default()
+                }),
                 semantic_tokens_provider: Some(
                     SemanticTokensServerCapabilities::SemanticTokensOptions(
                         SemanticTokensOptions {
@@ -171,6 +213,17 @@ impl LanguageServer for Backend {
                 format!("nuwiki-lsp initialized (positionEncoding={enc})"),
             )
             .await;
+
+        // Kick off the background workspace scan (P8). Skips if no root.
+        let root = self.index.read().expect("index lock poisoned").root.clone();
+        if let Some(root) = root {
+            let index = Arc::clone(&self.index);
+            let client = self.client.clone();
+            let registry = Arc::clone(&self.registry);
+            tokio::spawn(async move {
+                index_workspace(root, index, client, registry).await;
+            });
+        }
     }
 
     async fn shutdown(&self) -> LspResult<()> {
@@ -194,10 +247,13 @@ impl LanguageServer for Backend {
     }
 
     async fn did_close(&self, params: DidCloseTextDocumentParams) {
-        self.documents.remove(&params.text_document.uri);
+        let uri = &params.text_document.uri;
+        self.documents.remove(uri);
+        // Keep the index entry — the file may still exist on disk and other
+        // pages can still link to it. We only drop the live document state.
         // Clear lingering diagnostics so closed buffers don't keep red squiggles.
         self.client
-            .publish_diagnostics(params.text_document.uri, Vec::new(), None)
+            .publish_diagnostics(params.text_document.uri.clone(), Vec::new(), None)
             .await;
     }
 
@@ -245,6 +301,211 @@ impl LanguageServer for Backend {
             data: pack_data(&data),
         })))
     }
+
+    async fn goto_definition(
+        &self,
+        params: GotoDefinitionParams,
+    ) -> LspResult<Option<GotoDefinitionResponse>> {
+        let pos = params.text_document_position_params.position;
+        let uri = &params.text_document_position_params.text_document.uri;
+        let Some(doc) = self.documents.get(uri) else {
+            return Ok(None);
+        };
+        let utf8 = self.use_utf8.load(Ordering::Relaxed);
+        let (line, col) = nav::lsp_to_byte_pos(pos, &doc.text, utf8);
+        let Some(node) = nav::find_inline_at(&doc.ast, line, col) else {
+            return Ok(None);
+        };
+        let location = match node {
+            InlineNode::WikiLink(w) => {
+                let source_page = index::page_name_from_uri(
+                    uri,
+                    self.index
+                        .read()
+                        .expect("index lock poisoned")
+                        .root
+                        .as_deref(),
+                );
+                let idx = self.index.read().expect("index lock poisoned");
+                let target_uri = idx.resolve(&w.target, &source_page).cloned();
+                target_uri.map(|target| {
+                    let target_range = w
+                        .target
+                        .anchor
+                        .as_deref()
+                        .and_then(|a| heading_range_in(&idx, &target, a, utf8))
+                        .unwrap_or_else(zero_range);
+                    Location {
+                        uri: target,
+                        range: target_range,
+                    }
+                })
+            }
+            InlineNode::ExternalLink(_) | InlineNode::RawUrl(_) => None,
+            _ => None,
+        };
+        Ok(location.map(GotoDefinitionResponse::Scalar))
+    }
+
+    async fn references(&self, params: ReferenceParams) -> LspResult<Option<Vec<Location>>> {
+        let pos = params.text_document_position.position;
+        let uri = &params.text_document_position.text_document.uri;
+        let Some(doc) = self.documents.get(uri) else {
+            return Ok(None);
+        };
+        let utf8 = self.use_utf8.load(Ordering::Relaxed);
+        let (line, col) = nav::lsp_to_byte_pos(pos, &doc.text, utf8);
+
+        // Identify the page (and optional anchor) the cursor is "asking
+        // about". Two cases:
+        //   - cursor is on a wikilink → references to the linked page
+        //   - cursor isn't on a link → references to the *current* page
+        let source_page = index::page_name_from_uri(
+            uri,
+            self.index
+                .read()
+                .expect("index lock poisoned")
+                .root
+                .as_deref(),
+        );
+        let target_page = match nav::find_inline_at(&doc.ast, line, col) {
+            Some(InlineNode::WikiLink(w)) => match w.target.kind {
+                nuwiki_core::ast::LinkKind::Wiki => w.target.path.clone(),
+                nuwiki_core::ast::LinkKind::AnchorOnly => Some(source_page.clone()),
+                _ => None,
+            },
+            _ => Some(source_page),
+        };
+        let Some(page) = target_page else {
+            return Ok(None);
+        };
+        let idx = self.index.read().expect("index lock poisoned");
+        let locations: Vec<Location> = idx
+            .backlinks_for(&page)
+            .iter()
+            .filter_map(|b| {
+                self.documents.get(&b.source_uri).map(|src| Location {
+                    uri: b.source_uri.clone(),
+                    range: span_to_lsp_range(&b.source_span, &src.text, utf8),
+                })
+            })
+            .collect();
+        if locations.is_empty() {
+            // Even without open docs we can still produce LSP ranges from
+            // the stored span; the editor opens the file to display it.
+            let locations: Vec<Location> = idx
+                .backlinks_for(&page)
+                .iter()
+                .map(|b| Location {
+                    uri: b.source_uri.clone(),
+                    range: span_to_lsp_range_no_text(&b.source_span),
+                })
+                .collect();
+            return Ok(Some(locations));
+        }
+        Ok(Some(locations))
+    }
+
+    async fn hover(&self, params: HoverParams) -> LspResult<Option<Hover>> {
+        let pos = params.text_document_position_params.position;
+        let uri = &params.text_document_position_params.text_document.uri;
+        let Some(doc) = self.documents.get(uri) else {
+            return Ok(None);
+        };
+        let utf8 = self.use_utf8.load(Ordering::Relaxed);
+        let (line, col) = nav::lsp_to_byte_pos(pos, &doc.text, utf8);
+        let Some(node) = nav::find_inline_at(&doc.ast, line, col) else {
+            return Ok(None);
+        };
+
+        let (markdown, hover_span) = match node {
+            InlineNode::WikiLink(w) => {
+                let source_page = index::page_name_from_uri(
+                    uri,
+                    self.index
+                        .read()
+                        .expect("index lock poisoned")
+                        .root
+                        .as_deref(),
+                );
+                let idx = self.index.read().expect("index lock poisoned");
+                let target_uri = idx.resolve(&w.target, &source_page).cloned();
+                let body = match target_uri.as_ref().and_then(|u| idx.page(u)) {
+                    Some(page) => preview_for(page, w.target.anchor.as_deref()),
+                    None => match &w.target.path {
+                        Some(p) => format!("**{p}** — page not in index"),
+                        None => "(unresolved link)".into(),
+                    },
+                };
+                (body, w.span)
+            }
+            InlineNode::ExternalLink(e) => (format!("[{0}]({0})", e.url), e.span),
+            InlineNode::RawUrl(u) => (format!("<{0}>", u.url), u.span),
+            _ => return Ok(None),
+        };
+
+        Ok(Some(Hover {
+            contents: HoverContents::Markup(MarkupContent {
+                kind: MarkupKind::Markdown,
+                value: markdown,
+            }),
+            range: Some(span_to_lsp_range(&hover_span, &doc.text, utf8)),
+        }))
+    }
+
+    async fn completion(&self, params: CompletionParams) -> LspResult<Option<CompletionResponse>> {
+        let uri = &params.text_document_position.text_document.uri;
+        let Some(doc) = self.documents.get(uri) else {
+            return Ok(None);
+        };
+        let utf8 = self.use_utf8.load(Ordering::Relaxed);
+        let (line, col) =
+            nav::lsp_to_byte_pos(params.text_document_position.position, &doc.text, utf8);
+        // Only fire completion inside an unterminated `[[ ... ` run.
+        if !is_inside_wikilink(&doc.text, line, col) {
+            return Ok(None);
+        }
+        let idx = self.index.read().expect("index lock poisoned");
+        let items: Vec<CompletionItem> = idx
+            .page_names()
+            .into_iter()
+            .map(|name| CompletionItem {
+                label: name.clone(),
+                kind: Some(CompletionItemKind::FILE),
+                insert_text: Some(name),
+                ..CompletionItem::default()
+            })
+            .collect();
+        Ok(Some(CompletionResponse::Array(items)))
+    }
+
+    async fn symbol(
+        &self,
+        params: WorkspaceSymbolParams,
+    ) -> LspResult<Option<Vec<LspSymbolInformation>>> {
+        let q = params.query.to_lowercase();
+        let idx = self.index.read().expect("index lock poisoned");
+        let mut out = Vec::new();
+        for page in idx.pages_by_uri.values() {
+            for h in &page.headings {
+                if q.is_empty() || h.title.to_lowercase().contains(&q) {
+                    #[allow(deprecated)]
+                    out.push(LspSymbolInformation {
+                        name: h.title.clone(),
+                        kind: SymbolKind::STRING,
+                        tags: None,
+                        deprecated: None,
+                        location: Location {
+                            uri: page.uri.clone(),
+                            range: span_to_lsp_range_no_text(&h.span),
+                        },
+                        container_name: Some(page.name.clone()),
+                    });
+                }
+            }
+        }
+        Ok(Some(out))
+    }
 }
 
 fn pack_data(flat: &[u32]) -> Vec<tower_lsp::lsp_types::SemanticToken> {
@@ -469,3 +730,165 @@ fn keyword_str(k: nuwiki_core::ast::Keyword) -> &'static str {
         nuwiki_core::ast::Keyword::Xxx => "XXX",
     }
 }
+
+// ===== Workspace + navigation helpers =====
+
+fn workspace_root_from_params(params: &InitializeParams) -> Option<PathBuf> {
+    if let Some(folders) = &params.workspace_folders {
+        if let Some(first) = folders.first() {
+            if let Ok(p) = first.uri.to_file_path() {
+                return Some(p);
+            }
+        }
+    }
+    #[allow(deprecated)]
+    if let Some(uri) = &params.root_uri {
+        if let Ok(p) = uri.to_file_path() {
+            return Some(p);
+        }
+    }
+    None
+}
+
+/// Background workspace scan (P8). Walks every `.wiki` file under `root`,
+/// parses it, and feeds it into the shared index. Wraps the work in a
+/// `window/workDoneProgress` begin/end pair so editors can show progress.
+async fn index_workspace(
+    root: PathBuf,
+    index: Arc<RwLock<WorkspaceIndex>>,
+    client: Client,
+    registry: Arc<SyntaxRegistry>,
+) {
+    let token = ProgressToken::String("nuwiki-index".into());
+    // Best-effort: if the client doesn't pre-create progress tokens we just
+    // send the begin/end directly. tower-lsp permits this.
+    let _ = client
+        .send_notification::<tower_lsp::lsp_types::notification::Progress>(ProgressParams {
+            token: token.clone(),
+            value: ProgressParamsValue::WorkDone(WorkDoneProgress::Begin(WorkDoneProgressBegin {
+                title: "Indexing wiki".into(),
+                cancellable: Some(false),
+                message: None,
+                percentage: None,
+            })),
+        })
+        .await;
+
+    let plugin = registry.get("vimwiki");
+    let files = index::walk_wiki_files(&root);
+    for path in files {
+        let Ok(text) = std::fs::read_to_string(&path) else {
+            continue;
+        };
+        let Some(plugin) = plugin else { continue };
+        let ast = plugin.parse(&text);
+        let Ok(uri) = Url::from_file_path(&path) else {
+            continue;
+        };
+        if let Ok(mut idx) = index.write() {
+            idx.upsert(uri, &ast);
+        }
+    }
+
+    let _ = client
+        .send_notification::<tower_lsp::lsp_types::notification::Progress>(ProgressParams {
+            token,
+            value: ProgressParamsValue::WorkDone(WorkDoneProgress::End(WorkDoneProgressEnd {
+                message: Some("done".into()),
+            })),
+        })
+        .await;
+}
+
+fn span_to_lsp_range(span: &Span, text: &str, utf8: bool) -> Range {
+    to_lsp_range(span, text, utf8)
+}
+
+fn span_to_lsp_range_no_text(span: &Span) -> Range {
+    Range {
+        start: tower_lsp::lsp_types::Position {
+            line: span.start.line,
+            character: span.start.column,
+        },
+        end: tower_lsp::lsp_types::Position {
+            line: span.end.line,
+            character: span.end.column,
+        },
+    }
+}
+
+fn zero_range() -> Range {
+    Range {
+        start: tower_lsp::lsp_types::Position {
+            line: 0,
+            character: 0,
+        },
+        end: tower_lsp::lsp_types::Position {
+            line: 0,
+            character: 0,
+        },
+    }
+}
+
+fn heading_range_in(idx: &WorkspaceIndex, uri: &Url, anchor: &str, _utf8: bool) -> Option<Range> {
+    let page = idx.page(uri)?;
+    let h = page.headings.iter().find(|h| h.anchor == anchor)?;
+    Some(span_to_lsp_range_no_text(&h.span))
+}
+
+fn is_inside_wikilink(text: &str, line: u32, byte_col: u32) -> bool {
+    // Scan the current line for the most recent unclosed `[[` before the
+    // cursor.
+    let mut cur = 0u32;
+    let mut current_line = 0u32;
+    let bytes = text.as_bytes();
+    while current_line < line && cur < bytes.len() as u32 {
+        if bytes[cur as usize] == b'\n' {
+            current_line += 1;
+        }
+        cur += 1;
+    }
+    let line_start = cur as usize;
+    let limit = (line_start + byte_col as usize).min(bytes.len());
+    let segment = &bytes[line_start..limit];
+    // Find last "[[" and check no "]]" closes it before cursor.
+    let mut open_at: Option<usize> = None;
+    let mut i = 0;
+    while i + 1 < segment.len() {
+        if segment[i] == b'[' && segment[i + 1] == b'[' {
+            open_at = Some(i);
+            i += 2;
+        } else if segment[i] == b']' && segment[i + 1] == b']' {
+            open_at = None;
+            i += 2;
+        } else {
+            i += 1;
+        }
+    }
+    open_at.is_some()
+}
+
+fn preview_for(page: &IndexedPage, anchor: Option<&str>) -> String {
+    let mut buf = String::new();
+    let title = page.title.as_deref().unwrap_or(&page.name);
+    buf.push_str("**");
+    buf.push_str(title);
+    buf.push_str("**\n\n");
+    if let Some(anchor) = anchor {
+        if let Some(h) = page.headings.iter().find(|h| h.anchor == anchor) {
+            buf.push_str(&format!("§ {}\n", h.title));
+        } else {
+            buf.push_str(&format!("(anchor `{anchor}` not found)\n"));
+        }
+    } else if let Some(h) = page.headings.first() {
+        buf.push_str(&format!("§ {}\n", h.title));
+    }
+    if !page.headings.is_empty() {
+        buf.push_str("\nOutline:\n");
+        for h in page.headings.iter().take(8) {
+            let indent = "  ".repeat((h.level.saturating_sub(1)) as usize);
+            buf.push_str(&format!("{indent}- {}\n", h.title));
+        }
+    }
+    buf
+}