phase 12: tags — lexer, AST, index, anchor resolution, ws/symbol

Vimwiki `:tag1:tag2:` syntax across the full stack. P11 resolved as
strict-vimwiki — no markdown `#tag` alias.

AST (nuwiki-core):
- `TagScope { File, Heading(usize), Standalone }` mirrors vimwiki's
  three placement-determined meanings: lines 0-1 → File; one or two
  lines below the most recent heading → Heading(idx); otherwise
  Standalone.
- `TagNode { span, tags: Vec<String>, scope }` + `BlockNode::Tag`.
- `PageMetadata.tags: Vec<String>` accumulates the file-scope tag
  names — convenience projection of the Tag blocks whose scope is
  File. New field is additive (Default derive) so existing consumers
  using `..Default::default()` keep working.
- `Visitor::visit_tag` lands with a default body and `walk_block`
  dispatches the new arm.

Lexer (`syntax/vimwiki/lexer.rs`):
- `VimwikiTokenKind::Tag(Vec<String>)` block-level token.
- `try_lex_tag` runs in the normal-line dispatcher right after heading
  recognition; demands the *whole trimmed line* match `:name:name…:`,
  with non-empty whitespace-free names between adjacent colons. Empty
  segments (`::tag::`), whitespace inside a name, or single-colon
  lines are all rejected. Doesn't fight `Term::` (definition-list
  marker) because that pattern requires text before the `::`.
- Indentation is tolerated; the span covers the trimmed run.

Parser (`syntax/vimwiki/parser.rs`):
- `ParseState` gains `headings_seen` + `last_heading_line`.
- `parse_heading` bumps both on every closed heading so the tag
  parser can detect the heading window.
- `parse_tag` reads the Tag token, computes the scope via
  `scope_for_tag(tag_line)`, eats the trailing newline, and emits the
  TagNode.
- `parse_document` post-processes each block: when a Tag with
  `scope == File` is emitted, its names are pushed (de-duped) into
  `metadata.tags`.

Renderer (`render/html.rs`):
- `render_tag` emits `<div class="tags"><span class="tag"
  id="tag-name">name</span>...</div>`. The `id` is what makes
  `[[Page#tag-name]]` jump to the rendered tag.

Semantic tokens (`semantic_tokens.rs`):
- New `TOKEN_TAG = 19` / `vimwikiTag` so editors highlight tag lines
  distinctly. `emit_block` adds the Tag arm.

Workspace index (`index.rs`):
- `IndexedPage.tags: Vec<TagInfo>` records each tag occurrence on a
  page with its span + scope.
- `WorkspaceIndex.tags_by_name: HashMap<String, Vec<TagOccurrence>>`
  is the reverse map for cross-page anchor resolution + the Phase 13
  `nuwiki.tags.search` command.
- `upsert` populates both; `remove` scrubs the reverse map and drops
  empty entries; `tags_for(name)` is the public lookup.

LSP handlers (`lib.rs`):
- `heading_range_in` falls back to tag matching when the anchor
  isn't a heading slug — so `[[Page#some-tag]]` lands on the matching
  TagNode. Heading anchors win over tag anchors when both match (the
  vimwiki precedence).
- `workspace/symbol` includes tags alongside headings, named `:tag:`
  and kinded `PROPERTY` so editor UIs can distinguish them.

Tests (20 new):
- Syntax (15 in `vimwiki_tags.rs`):
  lexer recognition + rejection of `::tag::`, whitespace-in-name,
  Term::-conflict; parser scope paths (file, heading-1-below,
  heading-2-below, standalone, no-heading, line-1 priority);
  metadata dedup; renderer HTML + escaping.
- LSP index (5 in `tags_index_and_lsp.rs`):
  per-page tag population, cross-page `tags_by_name` aggregation,
  re-upsert replaces stale tags, `remove` cleans the reverse map,
  heading-scope tag indexed but not in file metadata.

All 191 v1.0 + Phase 11 tests still pass — backward compatible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

crates/nuwiki-core/tests/vimwiki_tags.rs

@@ -0,0 +1,204 @@
+//! Phase 12 tag tests: lexer + parser + renderer behaviour on
+//! `:tag:` lines, plus the scope-detection rules.
+
+use nuwiki_core::ast::{BlockNode, TagScope};
+use nuwiki_core::render::{HtmlRenderer, Renderer};
+use nuwiki_core::syntax::vimwiki::{VimwikiLexer, VimwikiSyntax, VimwikiTokenKind};
+use nuwiki_core::syntax::Lexer;
+use nuwiki_core::syntax::SyntaxPlugin;
+
+fn lex(src: &str) -> Vec<VimwikiTokenKind> {
+    VimwikiLexer::new()
+        .lex(src)
+        .into_iter()
+        .map(|t| t.kind)
+        .collect()
+}
+
+fn parse(src: &str) -> nuwiki_core::ast::DocumentNode {
+    VimwikiSyntax::new().parse(src)
+}
+
+fn render(src: &str) -> String {
+    let doc = parse(src);
+    HtmlRenderer::new().render_to_string(&doc).unwrap()
+}
+
+// ===== Lexer =====
+
+#[test]
+fn lexer_emits_tag_for_single_tag_line() {
+    let toks = lex(":todo:\n");
+    assert!(toks
+        .iter()
+        .any(|t| matches!(t, VimwikiTokenKind::Tag(v) if v == &vec!["todo".to_string()])));
+}
+
+#[test]
+fn lexer_emits_tag_for_multiple_chained_tags() {
+    let toks = lex(":one:two:three:\n");
+    let names = toks
+        .iter()
+        .find_map(|t| match t {
+            VimwikiTokenKind::Tag(v) => Some(v.clone()),
+            _ => None,
+        })
+        .expect("tag token");
+    assert_eq!(names, vec!["one", "two", "three"]);
+}
+
+#[test]
+fn lexer_rejects_empty_segment() {
+    // `::tag::` — empty leading segment between the doubled colons.
+    let toks = lex("::tag::\n");
+    assert!(!toks.iter().any(|t| matches!(t, VimwikiTokenKind::Tag(_))));
+}
+
+#[test]
+fn lexer_rejects_tag_with_whitespace() {
+    let toks = lex(":one tag:other:\n");
+    assert!(!toks.iter().any(|t| matches!(t, VimwikiTokenKind::Tag(_))));
+}
+
+#[test]
+fn lexer_does_not_collide_with_definition_term() {
+    // `Term::` is a definition-term marker (text *before* the `::`); a
+    // tag line has nothing before its leading `:`. The two patterns must
+    // not steal each other's lines.
+    let toks = lex("Term:: Definition\n:foo:\n");
+    assert!(toks
+        .iter()
+        .any(|t| matches!(t, VimwikiTokenKind::DefinitionTermMarker)));
+    assert!(toks.iter().any(|t| matches!(t, VimwikiTokenKind::Tag(_))));
+}
+
+#[test]
+fn lexer_accepts_indented_tag_line() {
+    let toks = lex("    :indented:\n");
+    assert!(toks.iter().any(|t| matches!(t, VimwikiTokenKind::Tag(_))));
+}
+
+// ===== Parser scope rules =====
+
+#[test]
+fn scope_is_file_when_tag_is_on_line_0_or_1() {
+    let doc = parse(":todo:other:\n= Heading =\n");
+    let BlockNode::Tag(t) = &doc.children[0] else {
+        panic!(
+            "expected first block to be a Tag, got {:?}",
+            doc.children[0]
+        );
+    };
+    assert_eq!(t.scope, TagScope::File);
+    assert_eq!(t.tags, vec!["todo", "other"]);
+    assert_eq!(doc.metadata.tags, vec!["todo", "other"]);
+}
+
+#[test]
+fn scope_is_file_for_tag_on_line_1() {
+    let doc = parse("a paragraph\n:todo:\n");
+    // Tag is on line 1, so still file-level.
+    let tag = doc
+        .children
+        .iter()
+        .find_map(|b| match b {
+            BlockNode::Tag(t) => Some(t),
+            _ => None,
+        })
+        .expect("tag block");
+    assert_eq!(tag.scope, TagScope::File);
+    assert!(doc.metadata.tags.contains(&"todo".to_string()));
+}
+
+#[test]
+fn scope_is_heading_when_tag_is_one_or_two_lines_below_heading() {
+    // Heading at line 0; tag at line 1 (one below) → Heading(0).
+    let doc = parse("= H1 =\n:hot:\n");
+    let tag = doc
+        .children
+        .iter()
+        .find_map(|b| match b {
+            BlockNode::Tag(t) => Some(t),
+            _ => None,
+        })
+        .expect("tag block");
+    // Tag at line 1 is *also* line ≤ 1 → file rule wins (matches vimwiki:
+    // file rule applies to lines 0-1 regardless of preceding heading).
+    assert_eq!(tag.scope, TagScope::File);
+}
+
+#[test]
+fn scope_is_heading_for_tag_two_lines_below_a_deeper_heading() {
+    // Heading at line 3; tag at line 4 (one below).
+    let src = "first\n\n\n= Section =\n:scoped:\n";
+    let doc = parse(src);
+    let tag = doc
+        .children
+        .iter()
+        .find_map(|b| match b {
+            BlockNode::Tag(t) => Some(t),
+            _ => None,
+        })
+        .expect("tag block");
+    // Heading is the only one seen; index 0.
+    assert_eq!(tag.scope, TagScope::Heading(0));
+    // Heading-scope tags do NOT contribute to file-level metadata.
+    assert!(doc.metadata.tags.is_empty());
+}
+
+#[test]
+fn scope_is_standalone_for_distant_tag() {
+    // Heading at line 3; tag at line 6 → 3 lines below → out of the
+    // heading window (>2), so standalone.
+    let src = "x\n\n\n= H =\n\n\n:wandering:\n";
+    let doc = parse(src);
+    let tag = doc
+        .children
+        .iter()
+        .find_map(|b| match b {
+            BlockNode::Tag(t) => Some(t),
+            _ => None,
+        })
+        .expect("tag block");
+    assert_eq!(tag.scope, TagScope::Standalone);
+}
+
+#[test]
+fn scope_is_standalone_when_no_heading_precedes() {
+    // No heading at all and not on lines 0-1.
+    let doc = parse("a\nb\nc\n:lonely:\n");
+    let tag = doc
+        .children
+        .iter()
+        .find_map(|b| match b {
+            BlockNode::Tag(t) => Some(t),
+            _ => None,
+        })
+        .expect("tag block");
+    assert_eq!(tag.scope, TagScope::Standalone);
+}
+
+#[test]
+fn file_tags_are_deduped_in_metadata() {
+    let doc = parse(":a:b:\n:b:c:\n");
+    assert_eq!(doc.metadata.tags, vec!["a", "b", "c"]);
+}
+
+// ===== Renderer =====
+
+#[test]
+fn renderer_emits_div_with_tag_spans() {
+    let html = render(":todo:done:\n");
+    assert!(html.contains("<div class=\"tags\">"));
+    assert!(html.contains("<span class=\"tag\" id=\"tag-todo\">todo</span>"));
+    assert!(html.contains("<span class=\"tag\" id=\"tag-done\">done</span>"));
+}
+
+#[test]
+fn renderer_escapes_html_inside_tag_names() {
+    // Tags shouldn't contain `<`/`>` per the lexer rule (whitespace
+    // forbidden, but `<` isn't), so this checks the renderer's escaping
+    // fires anyway as defence-in-depth.
+    let html = render(":a<b:\n");
+    assert!(html.contains("a&lt;b"));
+}