feat(render): list_ignore_newline / text_ignore_newline = false → <br>
Implements the deferred half of these options: a soft line break inside a
paragraph / list item can render as <br /> instead of a space.
- Parser: soft breaks now emit a dedicated InlineNode::SoftBreak node
(parse_inline_seq K::Newline + the list-continuation synthetic join) instead
of collapsing to Text(" "). Added SoftBreakNode + the variant.
- All inline consumers treat SoftBreak as whitespace: AST visitor (no-op),
text extraction (index.rs + lib.rs → space), nav/semantic_tokens span_of_inline,
semantic emit (skip like Text), parser span_of_inline.
- Renderer: render_inlines_break renders top-level SoftBreaks as the
context-specific string; render_paragraph uses text_ignore_newline,
render_list_item uses list_ignore_newline; default render_inline → space.
with_newline_handling builder; export.rs wires both from HtmlConfig.
Default (true/true = upstream default) behaviour is unchanged (space). Updated
3 core tests + 1 parser test that asserted the old Text(" ") shape to treat
SoftBreak as whitespace. New test render_page_html_ignore_newline_controls_soft_breaks.
Gap doc: P3 Config item closed — only the deferred markdown cluster remains.
Full rust suite + clippy clean; Neovim 307, Vim 301/18/21.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -34,6 +34,11 @@ pub enum InlineNode {
|
||||
ExternalLink(ExternalLinkNode),
|
||||
Transclusion(TransclusionNode),
|
||||
RawUrl(RawUrlNode),
|
||||
/// A soft line break joining two content lines within a paragraph or
|
||||
/// list item. Consumers treat it as whitespace; the HTML renderer emits
|
||||
/// a space (vimwiki `*_ignore_newline = 1`, the default) or `<br />`
|
||||
/// (`= 0`).
|
||||
SoftBreak(SoftBreakNode),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
@@ -42,6 +47,11 @@ pub struct TextNode {
|
||||
pub content: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct SoftBreakNode {
|
||||
pub span: Span,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct BoldNode {
|
||||
pub span: Span,
|
||||
|
||||
@@ -17,7 +17,7 @@ pub use block::{
|
||||
};
|
||||
pub use inline::{
|
||||
BoldItalicNode, BoldNode, CodeNode, ColorNode, InlineNode, ItalicNode, Keyword, KeywordNode,
|
||||
MathInlineNode, StrikethroughNode, SubscriptNode, SuperscriptNode, TextNode,
|
||||
MathInlineNode, SoftBreakNode, StrikethroughNode, SubscriptNode, SuperscriptNode, TextNode,
|
||||
};
|
||||
pub use link::{
|
||||
ExternalLinkNode, LinkKind, LinkTarget, RawUrlNode, TransclusionNode, WikiLinkNode,
|
||||
|
||||
@@ -184,6 +184,8 @@ pub fn walk_inline<V: Visitor + ?Sized>(v: &mut V, node: &InlineNode) {
|
||||
InlineNode::ExternalLink(n) => v.visit_external_link(n),
|
||||
InlineNode::Transclusion(n) => v.visit_transclusion(n),
|
||||
InlineNode::RawUrl(n) => v.visit_raw_url(n),
|
||||
// A soft break carries no content/children — treated as whitespace.
|
||||
InlineNode::SoftBreak(_) => {}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -61,6 +61,12 @@ pub struct HtmlRenderer {
|
||||
/// vimwiki `emoji_enable` (default `false` in the renderer; set from
|
||||
/// config): substitute `:alias:` shortcodes with their emoji glyph.
|
||||
emoji_enable: bool,
|
||||
/// vimwiki `text_ignore_newline` (default `true`): a soft break in a
|
||||
/// paragraph renders as a space; `false` → `<br />`.
|
||||
text_ignore_newline: bool,
|
||||
/// vimwiki `list_ignore_newline` (default `true`): a soft break in a list
|
||||
/// item renders as a space; `false` → `<br />`.
|
||||
list_ignore_newline: bool,
|
||||
}
|
||||
|
||||
impl Default for HtmlRenderer {
|
||||
@@ -80,9 +86,20 @@ impl HtmlRenderer {
|
||||
header_numbering_sym: String::new(),
|
||||
valid_html_tags: Vec::new(),
|
||||
emoji_enable: false,
|
||||
text_ignore_newline: true,
|
||||
list_ignore_newline: true,
|
||||
}
|
||||
}
|
||||
|
||||
/// Set vimwiki `text_ignore_newline` / `list_ignore_newline`. When `false`,
|
||||
/// a soft line break in a paragraph / list item renders as `<br />`
|
||||
/// instead of a space.
|
||||
pub fn with_newline_handling(mut self, text_ignore: bool, list_ignore: bool) -> Self {
|
||||
self.text_ignore_newline = text_ignore;
|
||||
self.list_ignore_newline = list_ignore;
|
||||
self
|
||||
}
|
||||
|
||||
/// Set the inline HTML tags allowed through export unescaped (vimwiki
|
||||
/// `valid_html_tags`). `span` is always added so colour spans render.
|
||||
pub fn with_valid_html_tags(mut self, mut tags: Vec<String>) -> Self {
|
||||
@@ -294,7 +311,8 @@ impl HtmlRenderer {
|
||||
|
||||
fn render_paragraph(&self, n: &ParagraphNode, w: &mut dyn Write) -> io::Result<()> {
|
||||
w.write_all(b"<p>")?;
|
||||
self.render_inlines(&n.children, w)?;
|
||||
let sb = if self.text_ignore_newline { " " } else { "<br />" };
|
||||
self.render_inlines_break(&n.children, sb, w)?;
|
||||
w.write_all(b"</p>\n")
|
||||
}
|
||||
|
||||
@@ -371,7 +389,8 @@ impl HtmlRenderer {
|
||||
}
|
||||
None => w.write_all(b"<li>")?,
|
||||
}
|
||||
self.render_inlines(&n.children, w)?;
|
||||
let sb = if self.list_ignore_newline { " " } else { "<br />" };
|
||||
self.render_inlines_break(&n.children, sb, w)?;
|
||||
if let Some(sublist) = &n.sublist {
|
||||
w.write_all(b"\n")?;
|
||||
self.render_list(sublist, w)?;
|
||||
@@ -526,6 +545,24 @@ impl HtmlRenderer {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Like [`render_inlines`] but renders top-level soft breaks as `soft_break`
|
||||
/// (a space or `<br />`). Used by paragraphs / list items, which are the
|
||||
/// only blocks that contain `SoftBreak` nodes.
|
||||
fn render_inlines_break(
|
||||
&self,
|
||||
nodes: &[InlineNode],
|
||||
soft_break: &str,
|
||||
w: &mut dyn Write,
|
||||
) -> io::Result<()> {
|
||||
for n in nodes {
|
||||
match n {
|
||||
InlineNode::SoftBreak(_) => w.write_all(soft_break.as_bytes())?,
|
||||
_ => self.render_inline(n, w)?,
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn render_inline(&self, n: &InlineNode, w: &mut dyn Write) -> io::Result<()> {
|
||||
match n {
|
||||
InlineNode::Text(t) => self.render_text(t, w),
|
||||
@@ -543,6 +580,9 @@ impl HtmlRenderer {
|
||||
InlineNode::ExternalLink(t) => self.render_external_link(t, w),
|
||||
InlineNode::Transclusion(t) => self.render_transclusion(t, w),
|
||||
InlineNode::RawUrl(t) => self.render_raw_url(t, w),
|
||||
// Default rendering (e.g. inside a heading): a space. Paragraphs
|
||||
// and list items use render_inlines_break for `<br />` support.
|
||||
InlineNode::SoftBreak(_) => w.write_all(b" "),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ use crate::ast::{
|
||||
InlineNode, ItalicNode, KeywordNode, LinkKind, LinkTarget, ListItemNode, ListNode, ListSymbol,
|
||||
MathBlockNode, MathInlineNode, PageMetadata, ParagraphNode, PreformattedNode, RawUrlNode, Span,
|
||||
StrikethroughNode, SubscriptNode, SuperscriptNode, TableCellNode, TableNode, TableRowNode,
|
||||
TagNode, TagScope, TextNode, TransclusionNode, WikiLinkNode,
|
||||
SoftBreakNode, TagNode, TagScope, TextNode, TransclusionNode, WikiLinkNode,
|
||||
};
|
||||
use crate::syntax::{Parser, TokenStream};
|
||||
|
||||
@@ -596,14 +596,11 @@ impl<'a> ParseState<'a> {
|
||||
}
|
||||
}
|
||||
}
|
||||
// Insert a synthetic " " between the previous content and
|
||||
// the continuation so the rendered text flows naturally.
|
||||
// Insert a soft break between the previous content and the
|
||||
// continuation so the rendered text flows naturally (and honours
|
||||
// `*_ignore_newline`).
|
||||
if !children.is_empty() {
|
||||
let span = first.span;
|
||||
children.push(InlineNode::Text(TextNode {
|
||||
span,
|
||||
content: " ".into(),
|
||||
}));
|
||||
children.push(InlineNode::SoftBreak(SoftBreakNode { span: first.span }));
|
||||
}
|
||||
let cont = parse_inline_seq(&buf);
|
||||
children.extend(cont);
|
||||
@@ -1008,11 +1005,10 @@ fn parse_inline_seq(toks: &[VimwikiToken]) -> Vec<InlineNode> {
|
||||
i += 1;
|
||||
}
|
||||
K::Newline => {
|
||||
// Soft break inside a multi-line block: render as space.
|
||||
out.push(InlineNode::Text(TextNode {
|
||||
span: token.span,
|
||||
content: " ".into(),
|
||||
}));
|
||||
// Soft break inside a multi-line block. A dedicated node lets
|
||||
// the renderer honour `*_ignore_newline` (space vs `<br />`);
|
||||
// other consumers treat it as whitespace.
|
||||
out.push(InlineNode::SoftBreak(SoftBreakNode { span: token.span }));
|
||||
i += 1;
|
||||
}
|
||||
K::BoldDelim => {
|
||||
@@ -1454,5 +1450,6 @@ fn span_of_inline(node: &InlineNode) -> Span {
|
||||
InlineNode::ExternalLink(n) => n.span,
|
||||
InlineNode::Transclusion(n) => n.span,
|
||||
InlineNode::RawUrl(n) => n.span,
|
||||
InlineNode::SoftBreak(n) => n.span,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,6 +27,8 @@ fn text_of(inlines: &[InlineNode]) -> String {
|
||||
InlineNode::Text(t) => out.push_str(&t.content),
|
||||
InlineNode::Bold(b) => out.push_str(&text_of(&b.children)),
|
||||
InlineNode::Italic(i) => out.push_str(&text_of(&i.children)),
|
||||
// Soft breaks join continuation lines (formerly a Text(" ")).
|
||||
InlineNode::SoftBreak(_) => out.push(' '),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -91,12 +91,13 @@ fn paragraph_spans_multiple_lines() {
|
||||
let BlockNode::Paragraph(p) = &doc.children[0] else {
|
||||
panic!("expected paragraph");
|
||||
};
|
||||
// Contents: Text("Hello"), Text(" ") (from soft newline), Text("world")
|
||||
// Contents: Text("Hello"), SoftBreak (from soft newline), Text("world").
|
||||
let text_concat: String = p
|
||||
.children
|
||||
.iter()
|
||||
.filter_map(|n| match n {
|
||||
InlineNode::Text(t) => Some(t.content.as_str()),
|
||||
InlineNode::Text(t) => Some(t.content.clone()),
|
||||
InlineNode::SoftBreak(_) => Some(" ".to_string()),
|
||||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
|
||||
@@ -261,6 +261,7 @@ pub fn render_page_html(
|
||||
r = r.with_valid_html_tags(cfg.valid_html_tags.clone());
|
||||
}
|
||||
r = r.with_emoji(cfg.emoji_enable);
|
||||
r = r.with_newline_handling(cfg.text_ignore_newline, cfg.list_ignore_newline);
|
||||
r.render_to_string(doc)
|
||||
}
|
||||
|
||||
|
||||
@@ -533,6 +533,7 @@ fn inline_to_text_into(inlines: &[InlineNode], out: &mut String) {
|
||||
InlineNode::Keyword(_) => {}
|
||||
InlineNode::Transclusion(t) => out.push_str(t.alt.as_deref().unwrap_or(&t.url)),
|
||||
InlineNode::RawUrl(r) => out.push_str(&r.url),
|
||||
InlineNode::SoftBreak(_) => out.push(' '),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1464,6 +1464,7 @@ fn inline_to_text_into(inlines: &[InlineNode], out: &mut String) {
|
||||
},
|
||||
InlineNode::Transclusion(t) => out.push_str(t.alt.as_deref().unwrap_or(&t.url)),
|
||||
InlineNode::RawUrl(r) => out.push_str(&r.url),
|
||||
InlineNode::SoftBreak(_) => out.push(' '),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -145,5 +145,6 @@ pub fn span_of_inline(node: &InlineNode) -> Span {
|
||||
InlineNode::ExternalLink(n) => n.span,
|
||||
InlineNode::Transclusion(n) => n.span,
|
||||
InlineNode::RawUrl(n) => n.span,
|
||||
InlineNode::SoftBreak(n) => n.span,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -231,7 +231,7 @@ fn emit_list_item(item: &ListItemNode, text: &str, idx: &LineIndex, out: &mut Ve
|
||||
fn emit_inlines(inlines: &[InlineNode], text: &str, idx: &LineIndex, out: &mut Vec<RawToken>) {
|
||||
for n in inlines {
|
||||
let (kind, span) = match n {
|
||||
InlineNode::Text(_) => continue,
|
||||
InlineNode::Text(_) | InlineNode::SoftBreak(_) => continue,
|
||||
InlineNode::Bold(b) => (TOKEN_BOLD, b.span),
|
||||
InlineNode::Italic(i) => (TOKEN_ITALIC, i.span),
|
||||
InlineNode::BoldItalic(bi) => (TOKEN_BOLD_ITALIC, bi.span),
|
||||
@@ -268,6 +268,7 @@ fn span_of_inline(node: &InlineNode) -> Span {
|
||||
InlineNode::ExternalLink(n) => n.span,
|
||||
InlineNode::Transclusion(n) => n.span,
|
||||
InlineNode::RawUrl(n) => n.span,
|
||||
InlineNode::SoftBreak(n) => n.span,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -253,6 +253,43 @@ fn render_page_html_passes_through_valid_html_tags() {
|
||||
assert!(html.contains("<script>"), "script escaped: {html}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_page_html_ignore_newline_controls_soft_breaks() {
|
||||
// Paragraph + list item, each with a soft (single) newline inside.
|
||||
let ast = parse("alpha\nbeta\n\n- one\n two\n");
|
||||
// Default (true/true): soft breaks render as spaces, no <br>.
|
||||
let c = cfg("/tmp/x");
|
||||
let def = export::render_page_html(
|
||||
&ast,
|
||||
Some("{{content}}".into()),
|
||||
"Home",
|
||||
DiaryDate::today_utc(),
|
||||
&c.html,
|
||||
None,
|
||||
HashMap::new(),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(!def.contains("<br"), "default joins with spaces: {def}");
|
||||
assert!(def.contains("alpha beta"), "para space: {def}");
|
||||
|
||||
// false/false: soft breaks become <br /> in both paragraph and list item.
|
||||
let mut c2 = cfg("/tmp/x");
|
||||
c2.html.text_ignore_newline = false;
|
||||
c2.html.list_ignore_newline = false;
|
||||
let br = export::render_page_html(
|
||||
&ast,
|
||||
Some("{{content}}".into()),
|
||||
"Home",
|
||||
DiaryDate::today_utc(),
|
||||
&c2.html,
|
||||
None,
|
||||
HashMap::new(),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(br.contains("alpha<br />beta"), "para <br>: {br}");
|
||||
assert!(br.contains("one<br />two") || br.contains("one<br /> two"), "list <br>: {br}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_page_html_substitutes_emoji_when_enabled() {
|
||||
let ast = parse("ship it :rocket: and :tada:\n");
|
||||
|
||||
@@ -594,13 +594,15 @@ fix site.
|
||||
`[[#anchor]]` (no description) vs `0` → `[[#anchor|title]]`. **`markdown_link_ext`
|
||||
stays deferred** — part of the markdown generated-content cluster (see the
|
||||
divergence note below).
|
||||
- [ ] **`list_ignore_newline` / `text_ignore_newline`** — config keys round-trip
|
||||
(`HtmlConfig`, default `true` = upstream's default, which nuwiki already
|
||||
honours: soft newlines render as spaces). The **`false` → `<br>`** behaviour
|
||||
is **deferred**: the parser collapses soft breaks to spaces before the
|
||||
renderer sees them, so `=false` needs a parser-level soft-break node (risky;
|
||||
touches every parse consumer). Documented as a parser limitation; the common
|
||||
default works.
|
||||
- [x] **`list_ignore_newline` / `text_ignore_newline`** _(fixed 2026-06-03)_ —
|
||||
the parser now emits a dedicated `InlineNode::SoftBreak` for soft line breaks
|
||||
(instead of collapsing to `Text(" ")`); every inline consumer (visitor,
|
||||
renderer, semantic tokens, nav, text extraction) treats it as whitespace. The
|
||||
HTML renderer renders it as a space (`= true`, default) or `<br />`
|
||||
(`= false`), distinguishing paragraph (`text_ignore_newline`) from list-item
|
||||
(`list_ignore_newline`) context via `render_inlines_break` +
|
||||
`with_newline_handling`. Test
|
||||
`render_page_html_ignore_newline_controls_soft_breaks` (`html_export.rs`).
|
||||
|
||||
### Mappings
|
||||
- [x] **Visual `<CR>` normalize-link** _(fixed 2026-06-03)_ — upstream binds
|
||||
|
||||
Reference in New Issue
Block a user