Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ jobs:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Verify vendored data is in sync with engine
# NLPPlus/data is a committed copy of the engine's data/ tree that the
# wheel ships and __init__.py copies into the working folder at runtime.
# If it drifts, compiler passes added to the engine (e.g. rfb/spec/xvar.nlp
# for _xVAR) are missing and analyzers fail to build. Fail fast on drift;
# re-sync with: rsync -a --delete nlp-engine/data/ NLPPlus/data/
run: diff -rq nlp-engine/data NLPPlus/data
- uses: actions/setup-python@v4
with:
python-version: '3.10'
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/update-nlp-engine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,14 @@ jobs:
before=$(git rev-parse HEAD:nlp-engine)
git submodule update --remote --recursive nlp-engine
after=$(git -C nlp-engine rev-parse HEAD)
# Keep the vendored rule-compiler spec (NLPPlus/data) in lockstep with
# the engine's data/ tree. The wheel ships this copy and __init__.py
# copies it into the working folder at runtime; if it drifts, newly
# added compiler passes (e.g. rfb/spec/xvar.nlp, which teaches the
# compiler _xVAR("attr")) are missing and analyzers fail to build even
# though the engine source is correct. Mirror with --delete so removed
# files are dropped too.
rsync -a --delete nlp-engine/data/ NLPPlus/data/
echo "latest=$latest" >> "$GITHUB_OUTPUT"
echo "published=$published" >> "$GITHUB_OUTPUT"
echo "before=$before" >> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -87,7 +95,7 @@ jobs:
if: steps.bump.outputs.changed == 'true'
run: |
set -euo pipefail
git add nlp-engine
git add nlp-engine NLPPlus/data
# The submodule may already be at `after` on main (advanced by another
# path), leaving nothing to commit -- but we still need the tag to fire
# publish.yml. Commit only when there's an actual change.
Expand Down
3 changes: 2 additions & 1 deletion NLPPlus/data/rfb/spec/analyzer.seq
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ tokenize #
line #
pat retok #
pat bigtok #
pat x_white #
pat x_white #
pat xvar # merge _xVAR("attr") into one nonliteral token
pat nlppp # comment here
pat un_mark # comment here
pat list #
Expand Down
2 changes: 1 addition & 1 deletion NLPPlus/data/rfb/spec/bigtok.nlp
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ _soRULES [base layer=(_startMark)] <- \@ RULES [t] @@
rfanonlit(2)
single()
@RULES
_NONLIT [base] <- \_ _xWILD [s one match=(_xALPHA _xEMOJI)] @@
_NONLIT [base] <- \_ _xWILD [s plus match=(_xALPHA _xEMOJI _xNUM)] @@ # 06/11/26 DD. plus + _xNUM: allow digits in names (_B2, _34, _C25g).

@RULES
_ARROW [base] <- \< \- @@
Expand Down
21 changes: 21 additions & 0 deletions NLPPlus/data/rfb/spec/xvar.nlp
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
###############################################
# FILE: XVAR.PAT
# SUBJ: Merge _xVAR("attr") / _xVAR(attr) into a single nonliteral token.
# NOTE: Runs after bigtok forms _NONLIT/_STR and whitespace is zapped,
# and before list collects parenthesized match lists, so the inner
# parens of _xVAR(attr) never reach the list grammar. The merged
# token _xVAR("attr") then flows through as one restricted-wildcard
# match-list entry; Pat::modeMatch1 tests the node attribute at
# runtime. The attribute may be quoted (_STR) or an unquoted bareword
# (_LIT/_NUM); Pat::xvarMatch strips the optional quotes either way.
###############################################

@POST
rfaxvar(1,3)
single()
@RULES
# Quoted attr: _xVAR("attr")
_NONLIT [base] <- _NONLIT \( _STR \) @@
# Unquoted attr: _xVAR(attr) / _xVAR(123)
_NONLIT [base] <- _NONLIT \( _LIT \) @@
_NONLIT [base] <- _NONLIT \( _NUM \) @@
Loading