phase 12: tags — lexer, AST, index, anchor resolution, ws/symbol
Vimwiki `:tag1:tag2:` syntax across the full stack. P11 resolved as
strict-vimwiki — no markdown `#tag` alias.
AST (nuwiki-core):
- `TagScope { File, Heading(usize), Standalone }` mirrors vimwiki's
three placement-determined meanings: lines 0-1 → File; one or two
lines below the most recent heading → Heading(idx); otherwise
Standalone.
- `TagNode { span, tags: Vec<String>, scope }` + `BlockNode::Tag`.
- `PageMetadata.tags: Vec<String>` accumulates the file-scope tag
names — convenience projection of the Tag blocks whose scope is
File. New field is additive (Default derive) so existing consumers
using `..Default::default()` keep working.
- `Visitor::visit_tag` lands with a default body and `walk_block`
dispatches the new arm.
Lexer (`syntax/vimwiki/lexer.rs`):
- `VimwikiTokenKind::Tag(Vec<String>)` block-level token.
- `try_lex_tag` runs in the normal-line dispatcher right after heading
recognition; demands the *whole trimmed line* match `:name:name…:`,
with non-empty whitespace-free names between adjacent colons. Empty
segments (`::tag::`), whitespace inside a name, or single-colon
lines are all rejected. Doesn't fight `Term::` (definition-list
marker) because that pattern requires text before the `::`.
- Indentation is tolerated; the span covers the trimmed run.
Parser (`syntax/vimwiki/parser.rs`):
- `ParseState` gains `headings_seen` + `last_heading_line`.
- `parse_heading` bumps both on every closed heading so the tag
parser can detect the heading window.
- `parse_tag` reads the Tag token, computes the scope via
`scope_for_tag(tag_line)`, eats the trailing newline, and emits the
TagNode.
- `parse_document` post-processes each block: when a Tag with
`scope == File` is emitted, its names are pushed (de-duped) into
`metadata.tags`.
Renderer (`render/html.rs`):
- `render_tag` emits `<div class="tags"><span class="tag"
id="tag-name">name</span>...</div>`. The `id` is what makes
`[[Page#tag-name]]` jump to the rendered tag.
Semantic tokens (`semantic_tokens.rs`):
- New `TOKEN_TAG = 19` / `vimwikiTag` so editors highlight tag lines
distinctly. `emit_block` adds the Tag arm.
Workspace index (`index.rs`):
- `IndexedPage.tags: Vec<TagInfo>` records each tag occurrence on a
page with its span + scope.
- `WorkspaceIndex.tags_by_name: HashMap<String, Vec<TagOccurrence>>`
is the reverse map for cross-page anchor resolution + the Phase 13
`nuwiki.tags.search` command.
- `upsert` populates both; `remove` scrubs the reverse map and drops
empty entries; `tags_for(name)` is the public lookup.
LSP handlers (`lib.rs`):
- `heading_range_in` falls back to tag matching when the anchor
isn't a heading slug — so `[[Page#some-tag]]` lands on the matching
TagNode. Heading anchors win over tag anchors when both match (the
vimwiki precedence).
- `workspace/symbol` includes tags alongside headings, named `:tag:`
and kinded `PROPERTY` so editor UIs can distinguish them.
Tests (20 new):
- Syntax (15 in `vimwiki_tags.rs`):
lexer recognition + rejection of `::tag::`, whitespace-in-name,
Term::-conflict; parser scope paths (file, heading-1-below,
heading-2-below, standalone, no-heading, line-1 priority);
metadata dedup; renderer HTML + escaping.
- LSP index (5 in `tags_index_and_lsp.rs`):
per-page tag population, cross-page `tags_by_name` aggregation,
re-upsert replaces stale tags, `remove` cleans the reverse map,
heading-scope tag indexed but not in file metadata.
All 191 v1.0 + Phase 11 tests still pass — backward compatible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -96,6 +96,12 @@ pub enum VimwikiTokenKind {
|
||||
CommentMultiClose,
|
||||
CommentMultiLine(String),
|
||||
|
||||
// ----- Tags (SPEC §12.3) -----
|
||||
/// `:tag1:tag2:` — colon-delimited tag line. The lexer treats this as
|
||||
/// block-level (must be the entire trimmed content of the line).
|
||||
/// Scope (file / heading / standalone) is the parser's job.
|
||||
Tag(Vec<String>),
|
||||
|
||||
// ----- Page placeholders (SPEC §9) -----
|
||||
PlaceholderTitle(Option<String>),
|
||||
PlaceholderNohtml,
|
||||
@@ -318,6 +324,7 @@ impl<'src> LexState<'src> {
|
||||
|| self.try_lex_placeholder(line)
|
||||
|| self.try_lex_horizontal_rule(line)
|
||||
|| self.try_lex_heading(line)
|
||||
|| self.try_lex_tag(line)
|
||||
|| self.try_lex_table_row(line)
|
||||
|| self.try_lex_blockquote(line)
|
||||
|| self.try_lex_list_item(line)
|
||||
@@ -713,6 +720,44 @@ impl<'src> LexState<'src> {
|
||||
true
|
||||
}
|
||||
|
||||
/// `:tag1:tag2:tag3:` — colon-delimited tag line (block-level).
|
||||
///
|
||||
/// The whole trimmed line must match: starts and ends with `:`, with
|
||||
/// non-empty whitespace-free names between adjacent colons. Empty
|
||||
/// segments or whitespace inside tag names disqualify the line.
|
||||
/// Doesn't conflict with `::` (definition list marker) because that
|
||||
/// pattern requires text *before* the `::`.
|
||||
fn try_lex_tag(&mut self, line: &str) -> bool {
|
||||
let trimmed = line.trim();
|
||||
if trimmed.len() < 3 || !trimmed.starts_with(':') || !trimmed.ends_with(':') {
|
||||
return false;
|
||||
}
|
||||
// `:tag1:tag2:` splits as `["", "tag1", "tag2", ""]`; the empty
|
||||
// strings at the ends are expected, everything in between must be
|
||||
// non-empty and contain no whitespace.
|
||||
let parts: Vec<&str> = trimmed.split(':').collect();
|
||||
if parts.first() != Some(&"") || parts.last() != Some(&"") {
|
||||
return false;
|
||||
}
|
||||
let names = &parts[1..parts.len() - 1];
|
||||
if names.is_empty()
|
||||
|| names
|
||||
.iter()
|
||||
.any(|t| t.is_empty() || t.chars().any(char::is_whitespace))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
let indent = (line.len() - line.trim_start().len()) as u32;
|
||||
let end_col = indent + trimmed.len() as u32;
|
||||
self.emit(
|
||||
VimwikiTokenKind::Tag(names.iter().map(|s| (*s).to_owned()).collect()),
|
||||
indent,
|
||||
end_col,
|
||||
);
|
||||
self.emit_newline(line);
|
||||
true
|
||||
}
|
||||
|
||||
fn try_lex_definition_term(&mut self, line: &str) -> bool {
|
||||
// `Term:: Definition` or `Term::` (continuation lines follow).
|
||||
// The `::` must be preceded by non-`:` text and followed by either
|
||||
|
||||
Reference in New Issue
Block a user