feat(render): list_ignore_newline / text_ignore_newline = false → <br>
CI / cargo fmt --check (push) Failing after 22s
CI / cargo clippy (push) Successful in 22s
CI / cargo test (push) Successful in 36s
CI / editor keymaps (push) Successful in 1m26s

Implements the deferred half of these options: a soft line break inside a
paragraph / list item can render as <br /> instead of a space.

- Parser: soft breaks now emit a dedicated InlineNode::SoftBreak node
  (parse_inline_seq K::Newline + the list-continuation synthetic join) instead
  of collapsing to Text(" "). Added SoftBreakNode + the variant.
- All inline consumers treat SoftBreak as whitespace: AST visitor (no-op),
  text extraction (index.rs + lib.rs → space), nav/semantic_tokens span_of_inline,
  semantic emit (skip like Text), parser span_of_inline.
- Renderer: render_inlines_break renders top-level SoftBreaks as the
  context-specific string; render_paragraph uses text_ignore_newline,
  render_list_item uses list_ignore_newline; default render_inline → space.
  with_newline_handling builder; export.rs wires both from HtmlConfig.

Default (true/true = upstream default) behaviour is unchanged (space). Updated
3 core tests + 1 parser test that asserted the old Text(" ") shape to treat
SoftBreak as whitespace. New test render_page_html_ignore_newline_controls_soft_breaks.

Gap doc: P3 Config item closed — only the deferred markdown cluster remains.
Full rust suite + clippy clean; Neovim 307, Vim 301/18/21.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-03 13:06:38 +00:00
parent 1246c99c3c
commit e6e3f70dd2
14 changed files with 122 additions and 26 deletions
+10
View File
@@ -34,6 +34,11 @@ pub enum InlineNode {
ExternalLink(ExternalLinkNode), ExternalLink(ExternalLinkNode),
Transclusion(TransclusionNode), Transclusion(TransclusionNode),
RawUrl(RawUrlNode), RawUrl(RawUrlNode),
/// A soft line break joining two content lines within a paragraph or
/// list item. Consumers treat it as whitespace; the HTML renderer emits
/// a space (vimwiki `*_ignore_newline = 1`, the default) or `<br />`
/// (`= 0`).
SoftBreak(SoftBreakNode),
} }
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
@@ -42,6 +47,11 @@ pub struct TextNode {
pub content: String, pub content: String,
} }
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SoftBreakNode {
pub span: Span,
}
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub struct BoldNode { pub struct BoldNode {
pub span: Span, pub span: Span,
+1 -1
View File
@@ -17,7 +17,7 @@ pub use block::{
}; };
pub use inline::{ pub use inline::{
BoldItalicNode, BoldNode, CodeNode, ColorNode, InlineNode, ItalicNode, Keyword, KeywordNode, BoldItalicNode, BoldNode, CodeNode, ColorNode, InlineNode, ItalicNode, Keyword, KeywordNode,
MathInlineNode, StrikethroughNode, SubscriptNode, SuperscriptNode, TextNode, MathInlineNode, SoftBreakNode, StrikethroughNode, SubscriptNode, SuperscriptNode, TextNode,
}; };
pub use link::{ pub use link::{
ExternalLinkNode, LinkKind, LinkTarget, RawUrlNode, TransclusionNode, WikiLinkNode, ExternalLinkNode, LinkKind, LinkTarget, RawUrlNode, TransclusionNode, WikiLinkNode,
+2
View File
@@ -184,6 +184,8 @@ pub fn walk_inline<V: Visitor + ?Sized>(v: &mut V, node: &InlineNode) {
InlineNode::ExternalLink(n) => v.visit_external_link(n), InlineNode::ExternalLink(n) => v.visit_external_link(n),
InlineNode::Transclusion(n) => v.visit_transclusion(n), InlineNode::Transclusion(n) => v.visit_transclusion(n),
InlineNode::RawUrl(n) => v.visit_raw_url(n), InlineNode::RawUrl(n) => v.visit_raw_url(n),
// A soft break carries no content/children — treated as whitespace.
InlineNode::SoftBreak(_) => {}
} }
} }
+42 -2
View File
@@ -61,6 +61,12 @@ pub struct HtmlRenderer {
/// vimwiki `emoji_enable` (default `false` in the renderer; set from /// vimwiki `emoji_enable` (default `false` in the renderer; set from
/// config): substitute `:alias:` shortcodes with their emoji glyph. /// config): substitute `:alias:` shortcodes with their emoji glyph.
emoji_enable: bool, emoji_enable: bool,
/// vimwiki `text_ignore_newline` (default `true`): a soft break in a
/// paragraph renders as a space; `false` → `<br />`.
text_ignore_newline: bool,
/// vimwiki `list_ignore_newline` (default `true`): a soft break in a list
/// item renders as a space; `false` → `<br />`.
list_ignore_newline: bool,
} }
impl Default for HtmlRenderer { impl Default for HtmlRenderer {
@@ -80,9 +86,20 @@ impl HtmlRenderer {
header_numbering_sym: String::new(), header_numbering_sym: String::new(),
valid_html_tags: Vec::new(), valid_html_tags: Vec::new(),
emoji_enable: false, emoji_enable: false,
text_ignore_newline: true,
list_ignore_newline: true,
} }
} }
/// Set vimwiki `text_ignore_newline` / `list_ignore_newline`. When `false`,
/// a soft line break in a paragraph / list item renders as `<br />`
/// instead of a space.
pub fn with_newline_handling(mut self, text_ignore: bool, list_ignore: bool) -> Self {
self.text_ignore_newline = text_ignore;
self.list_ignore_newline = list_ignore;
self
}
/// Set the inline HTML tags allowed through export unescaped (vimwiki /// Set the inline HTML tags allowed through export unescaped (vimwiki
/// `valid_html_tags`). `span` is always added so colour spans render. /// `valid_html_tags`). `span` is always added so colour spans render.
pub fn with_valid_html_tags(mut self, mut tags: Vec<String>) -> Self { pub fn with_valid_html_tags(mut self, mut tags: Vec<String>) -> Self {
@@ -294,7 +311,8 @@ impl HtmlRenderer {
fn render_paragraph(&self, n: &ParagraphNode, w: &mut dyn Write) -> io::Result<()> { fn render_paragraph(&self, n: &ParagraphNode, w: &mut dyn Write) -> io::Result<()> {
w.write_all(b"<p>")?; w.write_all(b"<p>")?;
self.render_inlines(&n.children, w)?; let sb = if self.text_ignore_newline { " " } else { "<br />" };
self.render_inlines_break(&n.children, sb, w)?;
w.write_all(b"</p>\n") w.write_all(b"</p>\n")
} }
@@ -371,7 +389,8 @@ impl HtmlRenderer {
} }
None => w.write_all(b"<li>")?, None => w.write_all(b"<li>")?,
} }
self.render_inlines(&n.children, w)?; let sb = if self.list_ignore_newline { " " } else { "<br />" };
self.render_inlines_break(&n.children, sb, w)?;
if let Some(sublist) = &n.sublist { if let Some(sublist) = &n.sublist {
w.write_all(b"\n")?; w.write_all(b"\n")?;
self.render_list(sublist, w)?; self.render_list(sublist, w)?;
@@ -526,6 +545,24 @@ impl HtmlRenderer {
Ok(()) Ok(())
} }
/// Like [`render_inlines`] but renders top-level soft breaks as `soft_break`
/// (a space or `<br />`). Used by paragraphs / list items, which are the
/// only blocks that contain `SoftBreak` nodes.
fn render_inlines_break(
&self,
nodes: &[InlineNode],
soft_break: &str,
w: &mut dyn Write,
) -> io::Result<()> {
for n in nodes {
match n {
InlineNode::SoftBreak(_) => w.write_all(soft_break.as_bytes())?,
_ => self.render_inline(n, w)?,
}
}
Ok(())
}
fn render_inline(&self, n: &InlineNode, w: &mut dyn Write) -> io::Result<()> { fn render_inline(&self, n: &InlineNode, w: &mut dyn Write) -> io::Result<()> {
match n { match n {
InlineNode::Text(t) => self.render_text(t, w), InlineNode::Text(t) => self.render_text(t, w),
@@ -543,6 +580,9 @@ impl HtmlRenderer {
InlineNode::ExternalLink(t) => self.render_external_link(t, w), InlineNode::ExternalLink(t) => self.render_external_link(t, w),
InlineNode::Transclusion(t) => self.render_transclusion(t, w), InlineNode::Transclusion(t) => self.render_transclusion(t, w),
InlineNode::RawUrl(t) => self.render_raw_url(t, w), InlineNode::RawUrl(t) => self.render_raw_url(t, w),
// Default rendering (e.g. inside a heading): a space. Paragraphs
// and list items use render_inlines_break for `<br />` support.
InlineNode::SoftBreak(_) => w.write_all(b" "),
} }
} }
+10 -13
View File
@@ -39,7 +39,7 @@ use crate::ast::{
InlineNode, ItalicNode, KeywordNode, LinkKind, LinkTarget, ListItemNode, ListNode, ListSymbol, InlineNode, ItalicNode, KeywordNode, LinkKind, LinkTarget, ListItemNode, ListNode, ListSymbol,
MathBlockNode, MathInlineNode, PageMetadata, ParagraphNode, PreformattedNode, RawUrlNode, Span, MathBlockNode, MathInlineNode, PageMetadata, ParagraphNode, PreformattedNode, RawUrlNode, Span,
StrikethroughNode, SubscriptNode, SuperscriptNode, TableCellNode, TableNode, TableRowNode, StrikethroughNode, SubscriptNode, SuperscriptNode, TableCellNode, TableNode, TableRowNode,
TagNode, TagScope, TextNode, TransclusionNode, WikiLinkNode, SoftBreakNode, TagNode, TagScope, TextNode, TransclusionNode, WikiLinkNode,
}; };
use crate::syntax::{Parser, TokenStream}; use crate::syntax::{Parser, TokenStream};
@@ -596,14 +596,11 @@ impl<'a> ParseState<'a> {
} }
} }
} }
// Insert a synthetic " " between the previous content and // Insert a soft break between the previous content and the
// the continuation so the rendered text flows naturally. // continuation so the rendered text flows naturally (and honours
// `*_ignore_newline`).
if !children.is_empty() { if !children.is_empty() {
let span = first.span; children.push(InlineNode::SoftBreak(SoftBreakNode { span: first.span }));
children.push(InlineNode::Text(TextNode {
span,
content: " ".into(),
}));
} }
let cont = parse_inline_seq(&buf); let cont = parse_inline_seq(&buf);
children.extend(cont); children.extend(cont);
@@ -1008,11 +1005,10 @@ fn parse_inline_seq(toks: &[VimwikiToken]) -> Vec<InlineNode> {
i += 1; i += 1;
} }
K::Newline => { K::Newline => {
// Soft break inside a multi-line block: render as space. // Soft break inside a multi-line block. A dedicated node lets
out.push(InlineNode::Text(TextNode { // the renderer honour `*_ignore_newline` (space vs `<br />`);
span: token.span, // other consumers treat it as whitespace.
content: " ".into(), out.push(InlineNode::SoftBreak(SoftBreakNode { span: token.span }));
}));
i += 1; i += 1;
} }
K::BoldDelim => { K::BoldDelim => {
@@ -1454,5 +1450,6 @@ fn span_of_inline(node: &InlineNode) -> Span {
InlineNode::ExternalLink(n) => n.span, InlineNode::ExternalLink(n) => n.span,
InlineNode::Transclusion(n) => n.span, InlineNode::Transclusion(n) => n.span,
InlineNode::RawUrl(n) => n.span, InlineNode::RawUrl(n) => n.span,
InlineNode::SoftBreak(n) => n.span,
} }
} }
+2
View File
@@ -27,6 +27,8 @@ fn text_of(inlines: &[InlineNode]) -> String {
InlineNode::Text(t) => out.push_str(&t.content), InlineNode::Text(t) => out.push_str(&t.content),
InlineNode::Bold(b) => out.push_str(&text_of(&b.children)), InlineNode::Bold(b) => out.push_str(&text_of(&b.children)),
InlineNode::Italic(i) => out.push_str(&text_of(&i.children)), InlineNode::Italic(i) => out.push_str(&text_of(&i.children)),
// Soft breaks join continuation lines (formerly a Text(" ")).
InlineNode::SoftBreak(_) => out.push(' '),
_ => {} _ => {}
} }
} }
+3 -2
View File
@@ -91,12 +91,13 @@ fn paragraph_spans_multiple_lines() {
let BlockNode::Paragraph(p) = &doc.children[0] else { let BlockNode::Paragraph(p) = &doc.children[0] else {
panic!("expected paragraph"); panic!("expected paragraph");
}; };
// Contents: Text("Hello"), Text(" ") (from soft newline), Text("world") // Contents: Text("Hello"), SoftBreak (from soft newline), Text("world").
let text_concat: String = p let text_concat: String = p
.children .children
.iter() .iter()
.filter_map(|n| match n { .filter_map(|n| match n {
InlineNode::Text(t) => Some(t.content.as_str()), InlineNode::Text(t) => Some(t.content.clone()),
InlineNode::SoftBreak(_) => Some(" ".to_string()),
_ => None, _ => None,
}) })
.collect(); .collect();
+1
View File
@@ -261,6 +261,7 @@ pub fn render_page_html(
r = r.with_valid_html_tags(cfg.valid_html_tags.clone()); r = r.with_valid_html_tags(cfg.valid_html_tags.clone());
} }
r = r.with_emoji(cfg.emoji_enable); r = r.with_emoji(cfg.emoji_enable);
r = r.with_newline_handling(cfg.text_ignore_newline, cfg.list_ignore_newline);
r.render_to_string(doc) r.render_to_string(doc)
} }
+1
View File
@@ -533,6 +533,7 @@ fn inline_to_text_into(inlines: &[InlineNode], out: &mut String) {
InlineNode::Keyword(_) => {} InlineNode::Keyword(_) => {}
InlineNode::Transclusion(t) => out.push_str(t.alt.as_deref().unwrap_or(&t.url)), InlineNode::Transclusion(t) => out.push_str(t.alt.as_deref().unwrap_or(&t.url)),
InlineNode::RawUrl(r) => out.push_str(&r.url), InlineNode::RawUrl(r) => out.push_str(&r.url),
InlineNode::SoftBreak(_) => out.push(' '),
} }
} }
} }
+1
View File
@@ -1464,6 +1464,7 @@ fn inline_to_text_into(inlines: &[InlineNode], out: &mut String) {
}, },
InlineNode::Transclusion(t) => out.push_str(t.alt.as_deref().unwrap_or(&t.url)), InlineNode::Transclusion(t) => out.push_str(t.alt.as_deref().unwrap_or(&t.url)),
InlineNode::RawUrl(r) => out.push_str(&r.url), InlineNode::RawUrl(r) => out.push_str(&r.url),
InlineNode::SoftBreak(_) => out.push(' '),
} }
} }
} }
+1
View File
@@ -145,5 +145,6 @@ pub fn span_of_inline(node: &InlineNode) -> Span {
InlineNode::ExternalLink(n) => n.span, InlineNode::ExternalLink(n) => n.span,
InlineNode::Transclusion(n) => n.span, InlineNode::Transclusion(n) => n.span,
InlineNode::RawUrl(n) => n.span, InlineNode::RawUrl(n) => n.span,
InlineNode::SoftBreak(n) => n.span,
} }
} }
+2 -1
View File
@@ -231,7 +231,7 @@ fn emit_list_item(item: &ListItemNode, text: &str, idx: &LineIndex, out: &mut Ve
fn emit_inlines(inlines: &[InlineNode], text: &str, idx: &LineIndex, out: &mut Vec<RawToken>) { fn emit_inlines(inlines: &[InlineNode], text: &str, idx: &LineIndex, out: &mut Vec<RawToken>) {
for n in inlines { for n in inlines {
let (kind, span) = match n { let (kind, span) = match n {
InlineNode::Text(_) => continue, InlineNode::Text(_) | InlineNode::SoftBreak(_) => continue,
InlineNode::Bold(b) => (TOKEN_BOLD, b.span), InlineNode::Bold(b) => (TOKEN_BOLD, b.span),
InlineNode::Italic(i) => (TOKEN_ITALIC, i.span), InlineNode::Italic(i) => (TOKEN_ITALIC, i.span),
InlineNode::BoldItalic(bi) => (TOKEN_BOLD_ITALIC, bi.span), InlineNode::BoldItalic(bi) => (TOKEN_BOLD_ITALIC, bi.span),
@@ -268,6 +268,7 @@ fn span_of_inline(node: &InlineNode) -> Span {
InlineNode::ExternalLink(n) => n.span, InlineNode::ExternalLink(n) => n.span,
InlineNode::Transclusion(n) => n.span, InlineNode::Transclusion(n) => n.span,
InlineNode::RawUrl(n) => n.span, InlineNode::RawUrl(n) => n.span,
InlineNode::SoftBreak(n) => n.span,
} }
} }
+37
View File
@@ -253,6 +253,43 @@ fn render_page_html_passes_through_valid_html_tags() {
assert!(html.contains("&lt;script&gt;"), "script escaped: {html}"); assert!(html.contains("&lt;script&gt;"), "script escaped: {html}");
} }
#[test]
fn render_page_html_ignore_newline_controls_soft_breaks() {
// Paragraph + list item, each with a soft (single) newline inside.
let ast = parse("alpha\nbeta\n\n- one\n two\n");
// Default (true/true): soft breaks render as spaces, no <br>.
let c = cfg("/tmp/x");
let def = export::render_page_html(
&ast,
Some("{{content}}".into()),
"Home",
DiaryDate::today_utc(),
&c.html,
None,
HashMap::new(),
)
.unwrap();
assert!(!def.contains("<br"), "default joins with spaces: {def}");
assert!(def.contains("alpha beta"), "para space: {def}");
// false/false: soft breaks become <br /> in both paragraph and list item.
let mut c2 = cfg("/tmp/x");
c2.html.text_ignore_newline = false;
c2.html.list_ignore_newline = false;
let br = export::render_page_html(
&ast,
Some("{{content}}".into()),
"Home",
DiaryDate::today_utc(),
&c2.html,
None,
HashMap::new(),
)
.unwrap();
assert!(br.contains("alpha<br />beta"), "para <br>: {br}");
assert!(br.contains("one<br />two") || br.contains("one<br /> two"), "list <br>: {br}");
}
#[test] #[test]
fn render_page_html_substitutes_emoji_when_enabled() { fn render_page_html_substitutes_emoji_when_enabled() {
let ast = parse("ship it :rocket: and :tada:\n"); let ast = parse("ship it :rocket: and :tada:\n");
+9 -7
View File
@@ -594,13 +594,15 @@ fix site.
`[[#anchor]]` (no description) vs `0``[[#anchor|title]]`. **`markdown_link_ext` `[[#anchor]]` (no description) vs `0``[[#anchor|title]]`. **`markdown_link_ext`
stays deferred** — part of the markdown generated-content cluster (see the stays deferred** — part of the markdown generated-content cluster (see the
divergence note below). divergence note below).
- [ ] **`list_ignore_newline` / `text_ignore_newline`** — config keys round-trip - [x] **`list_ignore_newline` / `text_ignore_newline`** _(fixed 2026-06-03)_
(`HtmlConfig`, default `true` = upstream's default, which nuwiki already the parser now emits a dedicated `InlineNode::SoftBreak` for soft line breaks
honours: soft newlines render as spaces). The **`false``<br>`** behaviour (instead of collapsing to `Text(" ")`); every inline consumer (visitor,
is **deferred**: the parser collapses soft breaks to spaces before the renderer, semantic tokens, nav, text extraction) treats it as whitespace. The
renderer sees them, so `=false` needs a parser-level soft-break node (risky; HTML renderer renders it as a space (`= true`, default) or `<br />`
touches every parse consumer). Documented as a parser limitation; the common (`= false`), distinguishing paragraph (`text_ignore_newline`) from list-item
default works. (`list_ignore_newline`) context via `render_inlines_break` +
`with_newline_handling`. Test
`render_page_html_ignore_newline_controls_soft_breaks` (`html_export.rs`).
### Mappings ### Mappings
- [x] **Visual `<CR>` normalize-link** _(fixed 2026-06-03)_ — upstream binds - [x] **Visual `<CR>` normalize-link** _(fixed 2026-06-03)_ — upstream binds