Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ These are the non-obvious constraints; the rest of the architecture is in `docs/
- **Time comes from `ClockService`.** Pure time utils take `now: Date` as a parameter and callers pass `clock.now()`. Reading `new Date()` inside a `computed()` freezes the value: the computed depends on no signal representing time, so a card shows "4 min ago" forever.
- **CSP is on.** `src-tauri/tauri.conf.json` locks the WebView down; `ipc:` and `http://ipc.localhost` must stay in `connect-src` or `invoke()` is blocked. Anything remote (fonts, images, APIs) needs a deliberate widening — fonts are self-hosted via `@fontsource` for exactly this reason.
- **The Rust model structs are a _contract_.** Don't change the shape of anything in `notes/model.rs`, `notes/view.rs` or `spaces/model.rs` without changing the DTOs on the front; their serde tests will fail if you do.
- **Search matching is Rust, not SQL.** SQLite's `LOWER()` only folds ASCII without ICU, so a `WHERE LOWER(title) LIKE …` would stop matching `Étape` against `étape`. Coarse filters (space, pin, lifecycle, language, tags) stay in SQL where they're indexed; text matching runs on the fetched rows via `to_lowercase()`.
- **Search matching is Rust, not SQL.** SQLite's `LOWER()` only folds ASCII without ICU, so a `WHERE LOWER(title) LIKE …` would stop matching `Étape` against `étape`. Coarse filters (space, pin, lifecycle, language, tags) stay in SQL where they're indexed; text matching runs on the fetched rows through `view::fold`, which lowercases **and** strips the accents — `etape` finds `Étape`, and the needle goes through the same fold, so it works the other way round too. Tag normalisation does not fold accents: two spellings there are two tags.
- **Syntax highlighting is highlight.js, front-side, in exactly one module.** `notes/ui/code-viewer/highlighter.ts` imports grammars **one by one** (`highlight.js/lib/languages/…`), never the default bundle. It colours the whole block — that's what handles multi-line comments and strings — then re-splits the output with `splitHighlightedLines`, which reopens the tag stack across each newline. The `.hljs-*` theme lives in the **global** `src/styles/_code-theme.scss`: injected by `[innerHTML]`, it carries no `_ngcontent` attribute, so a component-scoped rule would never match.
- **Tag normalisation lives in `notes::model::normalize_tags`, and only there.** Trim, strip leading `#`, drop blanks, collapse case-insensitive duplicates. `NoteDraft::into_note` and `NotePatch::apply` call it; `notes::store::replace_tags` receives tags already normalised and is pure SQL. It **re-reads** them after writing rather than sorting: `note_tags.tag` is `COLLATE NOCASE` (migration 2) and a read orders in that collation, which a byte-wise `sort()` does not reproduce — `Urgent` would come back before `auth` on write and after it on reload.
- **A `computed` feeding a `resource` needs an `equal` comparator.** `resource` compares params by identity. `NotesQueryStore.queryParams` returns a fresh object literal and reads `clock.now()`: without `sameQueryParams`, every 30 s tick fired a full `query_notes` round trip, invisible behind the retained view.
Expand Down
11 changes: 11 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -1602,6 +1602,17 @@ installed or shipped alongside the executable. The database file lives in Tauri'
matching is done **in Rust** (`notes::view`), because SQLite's `LOWER()` only folds ASCII
without ICU, so `Étape` would not match `étape`. Grouping is `notes::view`, which
touches no connection and is therefore testable without a database.
- **The search fold drops the accents as well as the case, on both sides.** `view::fold`
lowercases and strips the combining marks, so `etape` finds `Étape` and `Étape` finds
`etape` — the needle goes through the same function as every haystack. It decomposes one
character at a time (`unicode_normalization::char::decompose_canonical`) rather than
streaming the whole string through `nfd()`: on 800 notes of 13 kB of accented text, in
release, that is 15 ms against 76 ms, and against 27 ms for the `to_lowercase()` it
replaces — which folded no accent at all. Pure ASCII, which is what a snippet of code is,
never leaves the fast path. Only what a **canonical** decomposition separates is folded:
`ø` and `ß` are letters in their own right and stay. ⚠️ Tag normalisation
(`notes::model::normalize_tags`) deliberately does **not** fold accents — `Étape` and
`etape` are two tags, and merging them would lose one.
- **Tag normalisation lives in `notes::model::normalize_tags`, and only there.** Trimming,
stripping leading `#`, dropping blanks and collapsing case-insensitive duplicates (first
spelling wins) all happen on write, so the front sends what the user typed. The returned
Expand Down
10 changes: 10 additions & 0 deletions e2e/specs/06-search-and-filters.e2e.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,16 @@ describe('Search, filters and facets', () => {
expect(await canvas.titles()).toEqual(['Étape de migration']);
});

it('folds the accents too, on both sides of the comparison', async () => {
// Nobody reaches for the accent key to search, and the corpus is written with them.
await canvas.search('etape');
expect(await canvas.titles()).toEqual(['Étape de migration']);

// Symmetric, because the needle goes through the same fold as the haystack.
await canvas.search('Dôcker');
expect(await canvas.titles()).toEqual(['Docker compose']);
});

it('collapses to a single flat results section while searching', async () => {
// Searched here rather than inherited from the test above: an `it` that depends on
// what the previous one left cannot be run, reordered or bailed on alone.
Expand Down
10 changes: 10 additions & 0 deletions src-tauri/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions src-tauri/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ tauri-plugin-clipboard-manager = "2"
tauri-plugin-log = "2"
log = "0.4"
thiserror = "2.0.20"
# Search folds accents, which means decomposing before dropping the marks: `etape`
# has to find `Étape` on a French corpus. Hand-rolling a Latin table instead would
# be a list that goes stale the first time someone writes Polish or Vietnamese.
unicode-normalization = "0.1.25"
# Pulled in only by the `e2e` feature, and not optional in practice: the runner
# waits on `window.wdioTauri` before it will open a session, so a binary without
# the plugin hangs the suite rather than failing it. Its Rust half and the npm half
Expand Down
110 changes: 98 additions & 12 deletions src-tauri/src/notes/view.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ use std::collections::{BTreeMap, HashMap};
use chrono::{DateTime, Datelike, FixedOffset, TimeDelta, Utc};
use serde::{Deserialize, Serialize};
use specta::Type;
use unicode_normalization::char::{decompose_canonical, is_combining_mark};

use super::language::Language;
use super::model::{self, DisplayNote, Note};
Expand Down Expand Up @@ -104,7 +105,7 @@ pub fn apply_global_defaults(view: &mut NotesView, globals: &BTreeMap<String, St
}

pub fn build(mut notes: Vec<Note>, facets: Facets, request: &NotesQuery) -> NotesView {
let needle = request.search.trim().to_lowercase();
let needle = fold(request.search.trim());
if !needle.is_empty() {
notes.retain(|note| matches_search(note, &needle));
}
Expand Down Expand Up @@ -136,7 +137,7 @@ pub fn build(mut notes: Vec<Note>, facets: Facets, request: &NotesQuery) -> Note
}
}

/// `needle` is expected **already folded to lowercase and trimmed**.
/// `needle` is expected to have been through [`fold`] and trimmed.
///
/// ⚠️ Folded in Rust and not in SQL: without ICU, SQLite's `LOWER()` only handles
/// ASCII, so `Étape` would not match `étape`. The items count as much as the
Expand All @@ -151,12 +152,52 @@ fn matches_search(note: &Note, needle: &str) -> bool {
.any(|item| contains_folded(&item.text, needle))
}

/// Lowercase **and** accent-free, so `etape` finds `Étape`. Both sides go through
/// here, which makes the match symmetric: an accented needle finds unaccented text
/// too, since a search field is not where anyone wants to be precise about it.
///
/// Decomposing to NFD and dropping the combining marks handles every script rather
/// than the Latin letters someone thought to list. Only what a canonical
/// decomposition separates is folded: `ø` and `ß` are letters of their own and stay.
///
/// ⚠️ Decomposed one character at a time, and not through the `nfd()` iterator over
/// the whole string: on 800 notes of 13 kB of accented text, release, the streaming
/// version cost 76 ms against 15 ms here for the same answer. Its lookahead buffering
/// earns nothing when every mark is dropped anyway — canonical order cannot matter to
/// a fold that keeps none of it.
///
/// The ASCII branches are the common ones, not micro-optimisations: a snippet of code
/// is ASCII from end to end, and a French sentence is ASCII between its accents. That
/// is also what pays for the accents: the same corpus took 27 ms through
/// `to_lowercase()`, which folded no accent at all.
fn fold(text: &str) -> String {
if text.is_ascii() {
return text.to_ascii_lowercase();
}

let mut folded = String::with_capacity(text.len());

for character in text.chars() {
if character.is_ascii() {
folded.push(character.to_ascii_lowercase());
} else {
decompose_canonical(character, |part| {
if !is_combining_mark(part) {
folded.extend(part.to_lowercase());
}
});
}
}

folded
}

/// ⚠️ Do **not** hand-roll a fold-as-you-compare scan to save the copy: measured
/// on 800 notes of 13 kB, a needle matching nothing took 11.0 ms that way against
/// 6.3 ms here. `str::contains` runs Two-Way (O(n+m)); a window scan is O(n·m),
/// and searching is precisely the case where most notes do not match.
fn contains_folded(haystack: &str, needle: &str) -> bool {
haystack.to_lowercase().contains(needle)
fold(haystack).contains(needle)
}

const A_WEEK: TimeDelta = TimeDelta::days(7);
Expand Down Expand Up @@ -320,7 +361,7 @@ mod tests {
notes,
Facets::default(),
&NotesQuery {
search: " DÉPLOI ".to_string(),
search: " DEPLOI ".to_string(),
..request()
},
);
Expand Down Expand Up @@ -419,10 +460,10 @@ mod tests {
..sample()
};

assert!(matches_search(&note, "déploi"));
assert!(matches_search(&note, "kubectl"));
assert!(matches_search(&note, "ops"));
assert!(!matches_search(&note, "terraform"));
assert!(matches_search(&note, &fold("déploi")));
assert!(matches_search(&note, &fold("kubectl")));
assert!(matches_search(&note, &fold("ops")));
assert!(!matches_search(&note, &fold("terraform")));
}

#[test]
Expand All @@ -432,7 +473,52 @@ mod tests {
..sample()
};

assert!(matches_search(&note, "étape"));
assert!(matches_search(&note, &fold("étape")));
assert!(matches_search(&note, &fold("ÉTAPE")));
}

/// The point of the whole fold: nobody reaches for the accent key to search.
#[test]
fn an_unaccented_needle_finds_accented_text() {
let note = Note {
title: "Étape de migration".to_string(),
content: "Prévenir l'équipe".to_string(),
tags: vec!["déploiement".to_string()],
..sample()
};

assert!(matches_search(&note, &fold("etape")));
assert!(matches_search(&note, &fold("equipe")));
assert!(matches_search(&note, &fold("deploiement")));
}

/// Both sides go through `fold`, so it works the other way round too.
#[test]
fn an_accented_needle_finds_unaccented_text() {
let note = Note {
title: "Etape de migration".to_string(),
..sample()
};

assert!(matches_search(&note, &fold("Étape")));
}

/// Not everything an eye reads as an accent is one: only what a canonical
/// decomposition separates is folded away.
#[test]
fn a_letter_of_its_own_is_not_folded_into_another() {
let note = Note {
title: "Størrelse".to_string(),
..sample()
};

assert!(matches_search(&note, &fold("størrelse")));
assert!(!matches_search(&note, &fold("storrelse")));
}

#[test]
fn folding_leaves_an_ascii_needle_alone() {
assert_eq!(fold("Kubectl APPLY"), "kubectl apply");
}

#[test]
Expand All @@ -454,9 +540,9 @@ mod tests {
..sample()
};

assert!(matches_search(&note, "migration"));
assert!(matches_search(&note, "équipe"));
assert!(!matches_search(&note, "terraform"));
assert!(matches_search(&note, &fold("migration")));
assert!(matches_search(&note, &fold("equipe")));
assert!(!matches_search(&note, &fold("terraform")));
}
}

Expand Down
Loading