Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- `matcher::start_of_input()` and `matcher::end_of_input()` zero-width matchers for input-boundary checks in grammars; `end_of_input()` is equivalent to `negative_lookahead(AnyToken)`.
- `matcher::repeat(matcher, bounds)` for bounded greedy repetition with Rust count ranges (`2..5`, `1..`, `2..=4`, `3..=3`, etc.); related to `many` (`0..`), `one_or_more` (`1..`), and `optional` (`0..=1`).

## [0.2.1] - 2026-06-18

### Fixed
Expand Down
1 change: 1 addition & 0 deletions guide/02-core-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ Use lookahead to avoid ambiguous parses and to improve diagnostics.
- `many(x)` for zero or more
- `one_or_more(x)` for one or more
- `optional(x)` for optional segments
- `repeat(x, bounds)` for a bounded number of repetitions (e.g. `repeat(x, 2..5)` for 2–4 times, half-open on the count)

These are core building blocks for lists, whitespace, and token groups.

Expand Down
37 changes: 36 additions & 1 deletion guide/06-parser-matcher-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,17 @@ to commit immediately.
of input.

Use it for catch-all recovery, unknown tokens, or end-of-input checks with
`negative_lookahead(AnyToken)`.
[`end_of_input()`] or [`negative_lookahead(AnyToken)`](crate::matcher::negative_lookahead).

### `start_of_input` and `end_of_input`

[`end_of_input()`] succeeds when no tokens remain — equivalent to
[`negative_lookahead(AnyToken)`](crate::matcher::negative_lookahead). Use it in
grammars for explicit end-of-input checks (including with `.with_label(...)`).

[`start_of_input()`] succeeds when the cursor equals [`Input::start_pos`](crate::input::Input::start_pos)
for the current stream. For a sub-slice of a larger buffer, that is the start of
the slice, not necessarily byte offset zero in the outer source.

### `StringMatcher`, `&str`, and `char`

Expand All @@ -285,6 +295,31 @@ let _digit_run = one_or_more('0'..='9');

Use ranges for character or token classes.

### `repeat`

`repeat(matcher, bounds)` is greedy bounded repetition. The second argument is a
**repetition count** [`RangeBounds<usize>`](https://doc.rust-lang.org/std/ops/trait.RangeBounds.html),
not a token class range.

```rust
use marser::matcher::repeat::repeat;

let _two_to_four = repeat('0'..='9', 2..5); // 2, 3, or 4 digits (half-open count)
let _at_least_two = repeat('a', 2..);
let _exactly_three = repeat('b', 3..=3);
```

Count ranges use the same half-open rules as Rust: `2..5` means 2, 3, or 4
repetitions. Use `*name` binds inside `repeat(...)` when capturing each
occurrence.

Equivalences for unbounded forms:

- `many(m)` ≈ `repeat(m, 0..)`
- `one_or_more(m)` ≈ `repeat(m, 1..)`
- `optional(m)` ≈ `repeat(m, 0..=1)` (same attempt count; `optional` always
reports match success when the inner matcher fails once)

### `many`

`many(matcher)` is greedy zero-or-more repetition. It always succeeds, stops when
Expand Down
6 changes: 3 additions & 3 deletions guide/08-common-patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,11 @@ ignoring it:

```rust
use marser::capture;
use marser::matcher::{any_token::AnyToken, negative_lookahead::negative_lookahead};
use marser::matcher::end_of_input;
use marser::parser::{recursive, DeferredWeak, Parser};

let _value = recursive(|_weak: DeferredWeak<'_, '_, &str, ()>| {
capture!(negative_lookahead(AnyToken) => ())
capture!(end_of_input() => ())
});
```

Expand All @@ -83,7 +83,7 @@ Use the bind form that matches the boundary you want:

## Full-input parsing

`Parser::parse_str` / `parse_whole_input` use the same driver as `marser::parse`: the grammar is wrapped so **no trailing tokens** remain. Use `negative_lookahead(AnyToken)` patterns inside the library’s default wrapper; for sub-parsers that only parse a fragment, use segment-specific rules instead of the whole-input entrypoint.
`Parser::parse_str` / `parse_whole_input` use the same driver as `marser::parse`: the grammar is wrapped so **no trailing tokens** remain (via [`end_of_input()`](crate::matcher::end_of_input)). Use `end_of_input()` in grammars when you need an explicit end-of-input check; for sub-parsers that only parse a fragment, use segment-specific rules instead of the whole-input entrypoint.

## Type size and `erase_types`

Expand Down
17 changes: 17 additions & 0 deletions src/input.rs
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,12 @@ impl<'src, I: Input<'src>> InputStream<'src, I> {
self.pos = pos;
}

/// `true` when the cursor equals [`Input::start_pos`] for this stream.
#[inline]
pub(crate) fn is_at_start(&self) -> bool {
self.pos.clone().into() == self.input.start_pos().into()
}

/// Forwards to [`Input::try_consume_prefix_bytes`] on the underlying input.
#[inline]
pub(crate) fn try_consume_prefix_bytes(&mut self, prefix: &[u8]) -> Option<bool> {
Expand Down Expand Up @@ -191,6 +197,17 @@ impl<'src> InputStream<'src, &'src [u8]> {
mod tests {
use super::{Input, InputStream};

#[test]
fn is_at_start_tracks_cursor_against_input_start_pos() {
let s = "abc";
let mut stream = InputStream::new(s);
assert!(stream.is_at_start());
assert_eq!(stream.next(), Some('a'));
assert!(!stream.is_at_start());
stream.set_pos(Input::start_pos(&s));
assert!(stream.is_at_start());
}

#[test]
fn str_try_consume_prefix_advances_pos() {
let mut stream = InputStream::new("hello");
Expand Down
6 changes: 3 additions & 3 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ use crate::{
error::{FurthestFailError, MatcherRunError, ParserError, error_handler::EmptyErrorHandler},
input::{Input, InputStream},
matcher::{
any_token::AnyToken, commit_matcher::commit_on, negative_lookahead::negative_lookahead,
commit_matcher::commit_on, end_of_input,
},
mode::Emit,
parser::{Parser, internal::ParserImpl},
Expand Down Expand Up @@ -112,7 +112,7 @@ where
let eof_wrapped = capture!(
commit_on((), (
bind!(parser.clone(), result),
negative_lookahead(AnyToken),
end_of_input(),
)) => result
);

Expand Down Expand Up @@ -174,7 +174,7 @@ where
let eof_wrapped = capture!(
commit_on((), (
bind!(parser.clone(), result),
negative_lookahead(AnyToken),
end_of_input(),
)) => result
);

Expand Down
53 changes: 53 additions & 0 deletions src/matcher/input_boundary.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
//! AI assistance: this file was written with AI assistance. The maintainer reviewed it and did not find errors.
//!
//! Zero-width matchers at the start or end of an [`crate::input::Input`] stream.

use crate::{
error::{MatcherRunError, error_handler::ErrorHandler},
input::{Input, InputStream},
matcher::{
MatchRunner, MatcherCombinator, any_token::AnyToken,
negative_lookahead::{NegativeLookahead, negative_lookahead},
},
};

/// Succeeds when no tokens remain (equivalent to [`negative_lookahead`](crate::matcher::negative_lookahead)([`AnyToken`])).
#[inline]
pub fn end_of_input() -> NegativeLookahead<AnyToken> {
negative_lookahead(AnyToken)
}

/// Zero-width matcher: succeeds when the cursor equals [`Input::start_pos`] for this stream.
#[derive(Clone, Debug)]
pub struct StartOfInput;

/// See [`StartOfInput`].
#[inline]
pub fn start_of_input() -> StartOfInput {
StartOfInput
}

impl MatcherCombinator for StartOfInput {}

impl<'src, Inp: Input<'src>, MRes> super::internal::MatcherImpl<'src, Inp, MRes> for StartOfInput
where
Inp: Input<'src>,
{
const CAN_MATCH_DIRECTLY: bool = true;
const HAS_PROPERTY: bool = false;
const CAN_FAIL: bool = true;

#[inline]
fn match_with_runner<'a, Runner, M: crate::mode::Mode>(
&'a self,
_runner: &mut Runner,
_error_handler: &mut impl ErrorHandler,
input: &mut InputStream<'src, Inp>,
) -> Result<bool, MatcherRunError>
where
Runner: MatchRunner<'a, 'src, Inp, MRes = MRes>,
'src: 'a,
{
Ok(input.is_at_start())
}
}
7 changes: 6 additions & 1 deletion src/matcher/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
//! # For users
//!
//! - Matchers describe **structure**: sequences (`(a, b)`), [`crate::one_of::one_of`], repetition
//! ([`multiple::many`], [`one_or_more()`], [`optional()`]), lookahead ([`positive_lookahead()`],
//! ([`repeat()`], [`multiple::many`], [`one_or_more()`], [`optional()`]), lookahead ([`positive_lookahead()`],
//! [`negative_lookahead()`]), and [`commit_on()`] for committed sub-rules.
//! - They are composed with parsers through [`crate::capture`]; see [`crate::guide::capture_and_binds`].
//! - Extend matchers with [`MatcherCombinator`] (`with_label`, `try_insert_if_missing`, `unwanted`, …).
Expand Down Expand Up @@ -32,6 +32,8 @@ pub mod commit_matcher;
pub mod err_if;
pub mod error_contextualizer;
pub mod if_error;
/// Zero-width matchers at the start or end of an input stream.
pub mod input_boundary;
/// Parser-as-matcher adapters that discard parser output.
pub mod ignore_result;
pub mod multiple;
Expand All @@ -42,6 +44,7 @@ pub mod one_or_more;
pub mod optional;
pub mod parser_matcher;
pub mod positive_lookahead;
pub mod repeat;
pub(crate) mod runner;
pub mod sequence;
pub mod string;
Expand All @@ -55,12 +58,14 @@ pub use err_if::{
try_insert_if_missing, unwanted,
};
pub use error_contextualizer::ErrorContextualizer;
pub use input_boundary::{StartOfInput, end_of_input, start_of_input};
pub use multiple::{Multiple, many};
pub use negative_lookahead::{NegativeLookahead, negative_lookahead};
pub use one_or_more::{OneOrMore, one_or_more};
pub use optional::{Optional, optional};
pub use parser_matcher::ParserMatcher;
pub use positive_lookahead::{PositiveLookahead, positive_lookahead};
pub use repeat::{Repeat, repeat};
pub(crate) use runner::{DirectMatchRunner, MatchRunner, NoMemoizeBacktrackingRunner};
pub use string::StringMatcher;
pub use to_parser::ToParser;
Expand Down
21 changes: 9 additions & 12 deletions src/matcher/multiple.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
use crate::{
error::{MatcherRunError, error_handler::ErrorHandler},
input::{Input, InputStream},
matcher::{MatchRunner, Matcher},
matcher::{MatchRunner, Matcher, repeat::run_repeat_loop},
};

/// Greedy `matcher*` at the matcher level (always reports match success after the loop).
Expand Down Expand Up @@ -45,16 +45,13 @@ where
Runner: MatchRunner<'a, 'src, Inp, MRes = MRes>,
'src: 'a,
{
loop {
// TODO: maybe throw an error if an infinite loop is detected.
let before = input.get_pos();
if !runner.run_match::<_, M, _>(&self.matcher, error_handler, input)? {
break;
}
if input.get_pos().into() == before.into() {
break;
}
}
Ok(true)
run_repeat_loop::<Inp, MRes, Runner, M, Match, _>(
&self.matcher,
0,
None,
runner,
error_handler,
input,
)
}
}
25 changes: 9 additions & 16 deletions src/matcher/one_or_more.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
use crate::{
error::{MatcherRunError, error_handler::ErrorHandler},
input::{Input, InputStream},
matcher::{MatchRunner, Matcher},
matcher::{MatchRunner, Matcher, repeat::run_repeat_loop},
};

/// Requires at least one successful `matcher`, then behaves like greedy repetition.
Expand Down Expand Up @@ -47,20 +47,13 @@ where
Runner: MatchRunner<'a, 'src, Inp, MRes = MRes>,
'src: 'a,
{
// First match is mandatory — propagate the error if absent.
if !runner.run_match::<_, M, _>(&self.matcher, error_handler, input)? {
return Ok(false);
}
// Remaining matches are optional (same as Multiple).
loop {
let before = input.get_pos();
if !runner.run_match::<_, M, _>(&self.matcher, error_handler, input)? {
break;
}
if input.get_pos().into() == before.into() {
break;
}
}
Ok(true)
run_repeat_loop::<Inp, MRes, Runner, M, Match, _>(
&self.matcher,
1,
None,
runner,
error_handler,
input,
)
}
}
14 changes: 9 additions & 5 deletions src/matcher/optional.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
use crate::{
error::{MatcherRunError, error_handler::ErrorHandler},
input::{Input, InputStream},
matcher::{MatchRunner, Matcher},
matcher::{MatchRunner, Matcher, repeat::run_repeat_loop},
};

/// `matcher?` at the matcher level.
Expand Down Expand Up @@ -46,9 +46,13 @@ where
Runner: MatchRunner<'a, 'src, Inp, MRes = MRes>,
'src: 'a,
{
if runner.run_match::<_, M, _>(&self.matcher, error_handler, input)? {
return Ok(true);
}
Ok(true)
run_repeat_loop::<Inp, MRes, Runner, M, Match, _>(
&self.matcher,
0,
Some(1),
runner,
error_handler,
input,
)
}
}
Loading