phase 12: tags — lexer, AST, index, anchor resolution, ws/symbol
Vimwiki `:tag1:tag2:` syntax across the full stack. P11 resolved as
strict-vimwiki — no markdown `#tag` alias.
AST (nuwiki-core):
- `TagScope { File, Heading(usize), Standalone }` mirrors vimwiki's
three placement-determined meanings: lines 0-1 → File; one or two
lines below the most recent heading → Heading(idx); otherwise
Standalone.
- `TagNode { span, tags: Vec<String>, scope }` + `BlockNode::Tag`.
- `PageMetadata.tags: Vec<String>` accumulates the file-scope tag
names — convenience projection of the Tag blocks whose scope is
File. New field is additive (Default derive) so existing consumers
using `..Default::default()` keep working.
- `Visitor::visit_tag` lands with a default body and `walk_block`
dispatches the new arm.
Lexer (`syntax/vimwiki/lexer.rs`):
- `VimwikiTokenKind::Tag(Vec<String>)` block-level token.
- `try_lex_tag` runs in the normal-line dispatcher right after heading
recognition; demands the *whole trimmed line* match `:name:name…:`,
with non-empty whitespace-free names between adjacent colons. Empty
segments (`::tag::`), whitespace inside a name, or single-colon
lines are all rejected. Doesn't fight `Term::` (definition-list
marker) because that pattern requires text before the `::`.
- Indentation is tolerated; the span covers the trimmed run.
Parser (`syntax/vimwiki/parser.rs`):
- `ParseState` gains `headings_seen` + `last_heading_line`.
- `parse_heading` bumps both on every closed heading so the tag
parser can detect the heading window.
- `parse_tag` reads the Tag token, computes the scope via
`scope_for_tag(tag_line)`, eats the trailing newline, and emits the
TagNode.
- `parse_document` post-processes each block: when a Tag with
`scope == File` is emitted, its names are pushed (de-duped) into
`metadata.tags`.
Renderer (`render/html.rs`):
- `render_tag` emits `<div class="tags"><span class="tag"
id="tag-name">name</span>...</div>`. The `id` is what makes
`[[Page#tag-name]]` jump to the rendered tag.
Semantic tokens (`semantic_tokens.rs`):
- New `TOKEN_TAG = 19` / `vimwikiTag` so editors highlight tag lines
distinctly. `emit_block` adds the Tag arm.
Workspace index (`index.rs`):
- `IndexedPage.tags: Vec<TagInfo>` records each tag occurrence on a
page with its span + scope.
- `WorkspaceIndex.tags_by_name: HashMap<String, Vec<TagOccurrence>>`
is the reverse map for cross-page anchor resolution + the Phase 13
`nuwiki.tags.search` command.
- `upsert` populates both; `remove` scrubs the reverse map and drops
empty entries; `tags_for(name)` is the public lookup.
LSP handlers (`lib.rs`):
- `heading_range_in` falls back to tag matching when the anchor
isn't a heading slug — so `[[Page#some-tag]]` lands on the matching
TagNode. Heading anchors win over tag anchors when both match (the
vimwiki precedence).
- `workspace/symbol` includes tags alongside headings, named `:tag:`
and kinded `PROPERTY` so editor UIs can distinguish them.
Tests (20 new):
- Syntax (15 in `vimwiki_tags.rs`):
lexer recognition + rejection of `::tag::`, whitespace-in-name,
Term::-conflict; parser scope paths (file, heading-1-below,
heading-2-below, standalone, no-heading, line-1 priority);
metadata dedup; renderer HTML + escaping.
- LSP index (5 in `tags_index_and_lsp.rs`):
per-page tag population, cross-page `tags_by_name` aggregation,
re-upsert replaces stale tags, `remove` cleans the reverse map,
heading-scope tag indexed but not in file metadata.
All 191 v1.0 + Phase 11 tests still pass — backward compatible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -15,7 +15,7 @@ use std::path::{Path, PathBuf};
|
||||
|
||||
use nuwiki_core::ast::{
|
||||
BlockNode, BlockquoteNode, DocumentNode, InlineNode, LinkKind, LinkTarget, ListNode, Span,
|
||||
TableNode,
|
||||
TableNode, TagScope,
|
||||
};
|
||||
use tower_lsp::lsp_types::Url;
|
||||
|
||||
@@ -28,6 +28,19 @@ pub struct IndexedPage {
|
||||
pub title: Option<String>,
|
||||
pub headings: Vec<HeadingInfo>,
|
||||
pub outgoing: Vec<OutgoingLink>,
|
||||
/// Phase 12: every `TagNode` on the page. `tags_by_name` on the
|
||||
/// containing `WorkspaceIndex` is the reverse map.
|
||||
pub tags: Vec<TagInfo>,
|
||||
}
|
||||
|
||||
/// One tag occurrence on a page. `name` is the bare tag string (no
|
||||
/// surrounding colons). Multiple tags on the same source line each get
|
||||
/// their own `TagInfo` so anchor lookup can return a precise location.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct TagInfo {
|
||||
pub name: String,
|
||||
pub scope: TagScope,
|
||||
pub span: Span,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
@@ -56,12 +69,25 @@ pub struct Backlink {
|
||||
pub target_anchor: Option<String>,
|
||||
}
|
||||
|
||||
/// A tag's location, used by `WorkspaceIndex::tags_for` for cross-page
|
||||
/// `[[Page#tag]]` resolution and `nuwiki.tags.search` (Phase 13).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct TagOccurrence {
|
||||
pub uri: Url,
|
||||
pub span: Span,
|
||||
pub scope: TagScope,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct WorkspaceIndex {
|
||||
pub root: Option<PathBuf>,
|
||||
pub pages_by_uri: HashMap<Url, IndexedPage>,
|
||||
pub pages_by_name: HashMap<String, Url>,
|
||||
pub backlinks: HashMap<String, Vec<Backlink>>,
|
||||
/// Phase 12: tag name → every occurrence across the workspace. Used by
|
||||
/// the v1.1 nav handlers (tag-as-anchor `definition`, `workspace/symbol`
|
||||
/// inclusion) and the Phase 13 `nuwiki.tags.search` command.
|
||||
pub tags_by_name: HashMap<String, Vec<TagOccurrence>>,
|
||||
}
|
||||
|
||||
impl WorkspaceIndex {
|
||||
@@ -84,6 +110,7 @@ impl WorkspaceIndex {
|
||||
title: ast.metadata.title.clone(),
|
||||
headings: Vec::new(),
|
||||
outgoing: Vec::new(),
|
||||
tags: Vec::new(),
|
||||
};
|
||||
index_blocks(&ast.children, &mut page);
|
||||
|
||||
@@ -100,6 +127,18 @@ impl WorkspaceIndex {
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 12: maintain the reverse tag map alongside the per-page list.
|
||||
for tag in &page.tags {
|
||||
self.tags_by_name
|
||||
.entry(tag.name.clone())
|
||||
.or_default()
|
||||
.push(TagOccurrence {
|
||||
uri: uri.clone(),
|
||||
span: tag.span,
|
||||
scope: tag.scope,
|
||||
});
|
||||
}
|
||||
|
||||
self.pages_by_name.insert(name, uri.clone());
|
||||
self.pages_by_uri.insert(uri, page);
|
||||
}
|
||||
@@ -112,6 +151,19 @@ impl WorkspaceIndex {
|
||||
entries.retain(|b| &b.source_uri != uri);
|
||||
}
|
||||
self.backlinks.retain(|_, v| !v.is_empty());
|
||||
for entries in self.tags_by_name.values_mut() {
|
||||
entries.retain(|t| &t.uri != uri);
|
||||
}
|
||||
self.tags_by_name.retain(|_, v| !v.is_empty());
|
||||
}
|
||||
|
||||
/// All occurrences of a tag across the workspace. Empty slice when the
|
||||
/// tag isn't indexed.
|
||||
pub fn tags_for(&self, name: &str) -> &[TagOccurrence] {
|
||||
self.tags_by_name
|
||||
.get(name)
|
||||
.map(Vec::as_slice)
|
||||
.unwrap_or(&[])
|
||||
}
|
||||
|
||||
/// Resolve a `LinkTarget` to a page URI in this workspace.
|
||||
@@ -258,6 +310,15 @@ fn index_block(block: &BlockNode, page: &mut IndexedPage) {
|
||||
}
|
||||
}
|
||||
}
|
||||
BlockNode::Tag(t) => {
|
||||
for name in &t.tags {
|
||||
page.tags.push(TagInfo {
|
||||
name: name.clone(),
|
||||
scope: t.scope,
|
||||
span: t.span,
|
||||
});
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user