Release 6.0.0: publish flpq-data to PyPI

.github/ISSUE_TEMPLATE/bug_report.md

@@ -25,7 +25,7 @@ If applicable, add screenshots to help explain your problem.
 
 **Desktop (please complete the following information):**
  - OS: `[e.g. Ubuntu 20.04]`
- - CFPQ_Data: `[e.g. 22]`
+ - FLPQ_Data: `[e.g. 22]`
  - Python: `[e.g. 3.8]`
  - Other important stuff versions
 

.github/ISSUE_TEMPLATE/grammar-add-template.md

@@ -13,7 +13,7 @@ assignees: rustam-azimov, vadyushkins
 |           |                                                                        |
 |-----------|------------------------------------------------------------------------|
 | Full Name | ``<Specify the full name of the grammar template>``                    |
-| Version   | ``<Current version of CFPQ_Data>``                                     |
+| Version   | ``<Current version of FLPQ_Data>``                                     |
 | Class     | ``<Specify the formal grammar class to which the grammar belongs>``    |
 | Kind      | ``<Specify the kind of problem for which this grammar can be used>``   |
 | Origin    | [link](``<Link to download the grammar or link to the grammar info>``) |

.github/ISSUE_TEMPLATE/graph-add-template.md

@@ -1,4 +1,3 @@
----
 name: ADD NEW GRAPH
 about: Suggest new graph to this project
 title: "[NEW GRAPH] <Graph brief description>"
@@ -12,13 +11,45 @@ assignees: rustam-azimov, vadyushkins
 ## Info
 | | |
 |---|---|
-| Full Name | ``<Specify the full name of the graph>`` |
-| Version | ``<Current version of CFPQ_Data>`` |
-| Origin | [link](``<Link to download the graph>``) |
+| Type | ``<new graph (full archive) or new queries for an existing graph (partial archive)>`` |
+| Full Name | ``<The full name of the graph — for a partial archive, the name of the existing graph being extended>`` |
+| Category | ``<Existing category name, or "new: <proposed name>" for a new category (new graphs only)>`` |
+| Version | ``<Current version of FLPQ_Data>`` |
+| Archive | [link](``<Google Drive link to the archive — <name>.tar.gz, full or partial>``) |
 
 ## Data format
 
-The archive must follow the standard layout described in the "File structure" section of the [Graphs page](https://formallanguageconstrainedpathquerying.github.io/CFPQ_Data/graphs/index.html): `<name>.tar.gz` unpacks to `<name>/{README.md, grammar/, graph/}`, where `graph/` holds one Boolean MatrixMarket pattern file per edge label.
+The main way to provide data is a **Google Drive link** to an archive prepared per the structure-validation tool. Two kinds of archives exist (both documented in the "File structure" section of the [Graphs page](https://formallanguageconstrainedpathquerying.github.io/FLPQ_Data/graphs/index.html)):
+
+- **Full archive** — a new graph: `<name>.tar.gz` unpacks to a self-contained directory with `README.md`, `graph/` (one Boolean MatrixMarket pattern file per stored edge label), and `queries/` (one directory per query in `cfpq/`, `rpq/`, or `mcfpq/`; each query directory holds every representation of the query — `.cnf`/`.rsm`, `.re`/`.rsm` (regular only), or `.mcfg` — plus one `results.mtx` with the constrained reachability facts).
+- **Partial archive** — new queries for an existing graph: `<name>.tar.gz` unpacks to a directory containing only `queries/` — the new query directories (same layout) plus a `README.md` fragment with one `## <class>/<query>` section per new query.
+
+Validate the archive before sharing it:
+
+```
+python utils/check_archive_structure.py <name>.tar.gz            # full archive
+python utils/check_archive_structure.py <name>.tar.gz --partial  # partial archive
+```
+
+A partial archive is merged into the existing graph archive by the maintainer (`python utils/merge_archive.py EXISTING.tar.gz PARTIAL.tar.gz -o <name>.tar.gz`), re-validated, and uploaded. The query file formats — including the recursive state machine (`.rsm`) format — are documented on the [FLPQ design page](https://formallanguageconstrainedpathquerying.github.io/FLPQ_Data/flpq.html).
+
+## Description document
+
+**Full archive.** The archive's `README.md` must answer all of these questions (one `##` section each):
+| | |
+|---|---|
+| What is this graph? | ``<A short description of the domain and what the graph models>`` |
+| Source | ``<Where the data comes from (paper, dataset, tool) and the full citation>`` |
+| Construction | ``<How the graph was built, extracted, or generated (method, parameters)>`` |
+| Nodes | ``<What a node represents and the total number of nodes>`` |
+| Edges and labels | ``<What each label family means and the number of edges per label>`` |
+| Query classes | ``<Which of RPQ, CFPQ, MCFPQ apply to this graph and why>`` |
+| License | ``<The license of the data>`` |
+| Caveats | ``<Known limitations or approximations, or "None">`` |
+
+**Partial archive.** Instead of a full description document, the `queries/README.md` fragment carries one `## <class>/<query>` section per new query, each describing the language (in words and/or in LaTeX) and its purpose on this graph.
+
+*New graphs only — a partial archive adds no nodes or edges.*
 
 ## Graph Statistics
 | Num Nodes | Num Edges |
@@ -28,12 +59,20 @@ The archive must follow the standard layout described in the "File structure" se
 ## Edges Statistics
 | Edge Label | Num Edge Label |
 |---:|---:|
-| ``<The type of the edge label of the new graph >`` | ``<The number of edges in the new graph of this type>`` |
+| ``<The type of the edge label of the new graph >`` | ``<The number of edges in the graph of this type>`` |
 
 List the stored labels only (no `_r` reversed-edge rows). A family of indexed labels (e.g. `load_0`, ..., `load_857`) is listed once with a placeholder subscript (e.g. `load_f`) and a note listing the index set.
 
-## Canonical grammars
+## Queries
+
+Canonical grammars are documented on the category page, not on the per-graph page. Pick the case that applies:
+
+1. **Graph for an existing category** — name the grammar(s) from the category's "Canonical grammars" section that apply to this graph; they become the table column(s) filled in for the new row. No new grammar text on the site.
+2. **New grammar for an existing category** — provide the grammar below; it will be added to the category page and a new column to its table.
+3. **Graph for a new category** — provide the canonical grammar(s) below; a new category page will be created with them.
+
+Every query you provide must ship inside the archive (a full archive ships all queries of the graph; a partial archive ships the new ones): one directory per query in `queries/cfpq/`, `queries/rpq/`, or `queries/mcfpq/`, holding every representation of the query (`.cnf`/`.rsm`, `.re`/`.rsm` — regular only, or `.mcfg`) plus one `results.mtx` with the constrained reachability facts (a Boolean pattern matrix with the graph's dimensions; an entry `(i, j)` means some path from node `i` to node `j` satisfies the query). Each query directory is described in a `## <class>/<query>` section of `queries/README.md`. A query may only use labels of this graph.
 
-<LaTeX format of grammar>
+<LaTeX format of grammar (only for cases 2 and 3)>
 
-<Pyformlang CFG format of grammar>
+<File-format text of the grammar — .cnf, .rsm, .re, or .mcfg (only for cases 2 and 3)>

.github/PULL_REQUEST_TEMPLATE/new_grammar.md

@@ -4,7 +4,7 @@
 |           |                                                                        |
 |-----------|------------------------------------------------------------------------|
 | Full Name | ``<Specify the full name of the grammar template>``                    |
-| Version   | ``<Current version of CFPQ_Data>``                                     |
+| Version   | ``<Current version of FLPQ_Data>``                                     |
 | Class     | ``<Specify the formal grammar class to which the grammar belongs>``    |
 | Kind      | ``<Specify the kind of problem for which this grammar can be used>``   |
 | Origin    | [link](``<Link to download the grammar or link to the grammar info>``) |

.github/PULL_REQUEST_TEMPLATE/new_graph.md

@@ -3,13 +3,45 @@
 ## Info
 | | |
 |---|---|
-| Full Name | ``<Specify the full name of the graph>`` |
-| Version | ``<Current version of CFPQ_Data>`` |
-| Origin | [link](``<Link to download the graph>``) |
+| Type | ``<new graph (full archive) or new queries for an existing graph (partial archive)>`` |
+| Full Name | ``<The full name of the graph — for a partial archive, the name of the existing graph being extended>`` |
+| Category | ``<Existing category name, or "new: <proposed name>" for a new category (new graphs only)>`` |
+| Version | ``<Current version of FLPQ_Data>`` |
+| Archive | [link](``<Google Drive link to the archive — <name>.tar.gz, full or partial>``) |
 
 ## Data format
 
-The archive must follow the standard layout described in the "File structure" section of the [Graphs page](https://formallanguageconstrainedpathquerying.github.io/CFPQ_Data/graphs/index.html): `<name>.tar.gz` unpacks to `<name>/{README.md, grammar/, graph/}`, where `graph/` holds one Boolean MatrixMarket pattern file per edge label.
+The main way to provide data is a **Google Drive link** to an archive prepared per the structure-validation tool. Two kinds of archives exist (both documented in the "File structure" section of the [Graphs page](https://formallanguageconstrainedpathquerying.github.io/FLPQ_Data/graphs/index.html)):
+
+- **Full archive** — a new graph: `<name>.tar.gz` unpacks to a self-contained directory with `README.md`, `graph/` (one Boolean MatrixMarket pattern file per stored edge label), and `queries/` (one directory per query in `cfpq/`, `rpq/`, or `mcfpq/`; each query directory holds every representation of the query — `.cnf`/`.rsm`, `.re`/`.rsm` (regular only), or `.mcfg` — plus one `results.mtx` with the constrained reachability facts).
+- **Partial archive** — new queries for an existing graph: `<name>.tar.gz` unpacks to a directory containing only `queries/` — the new query directories (same layout) plus a `README.md` fragment with one `## <class>/<query>` section per new query.
+
+Validate the archive before sharing it:
+
+```
+python utils/check_archive_structure.py <name>.tar.gz            # full archive
+python utils/check_archive_structure.py <name>.tar.gz --partial  # partial archive
+```
+
+A partial archive is merged into the existing graph archive by the maintainer (`python utils/merge_archive.py EXISTING.tar.gz PARTIAL.tar.gz -o <name>.tar.gz`), re-validated, and uploaded. The query file formats — including the recursive state machine (`.rsm`) format — are documented on the [FLPQ design page](https://formallanguageconstrainedpathquerying.github.io/FLPQ_Data/flpq.html).
+
+## Description document
+
+**Full archive.** The archive's `README.md` must answer all of these questions (one `##` section each):
+| | |
+|---|---|
+| What is this graph? | ``<A short description of the domain and what the graph models>`` |
+| Source | ``<Where the data comes from (paper, dataset, tool) and the full citation>`` |
+| Construction | ``<How the graph was built, extracted, or generated (method, parameters)>`` |
+| Nodes | ``<What a node represents and the total number of nodes>`` |
+| Edges and labels | ``<What each label family means and the number of edges per label>`` |
+| Query classes | ``<Which of RPQ, CFPQ, MCFPQ apply to this graph and why>`` |
+| License | ``<The license of the data>`` |
+| Caveats | ``<Known limitations or approximations, or "None">`` |
+
+**Partial archive.** Instead of a full description document, the `queries/README.md` fragment carries one `## <class>/<query>` section per new query, each describing the language (in words and/or in LaTeX) and its purpose on this graph.
+
+*New graphs only — a partial archive adds no nodes or edges.*
 
 ## Graph Statistics
 | Num Nodes | Num Edges |
@@ -19,12 +51,20 @@ The archive must follow the standard layout described in the "File structure" se
 ## Edges Statistics
 | Edge Label | Num Edge Label |
 |---:|---:|
-| ``<The type of the edge label of the new graph >`` | ``<The number of edges in the new graph of this type>`` |
+| ``<The type of the edge label of the new graph >`` | ``<The number of edges in the graph of this type>`` |
 
 List the stored labels only (no `_r` reversed-edge rows). A family of indexed labels (e.g. `load_0`, ..., `load_857`) is listed once with a placeholder subscript (e.g. `load_f`) and a note listing the index set.
 
-## Canonical grammars
+## Queries
+
+Canonical grammars are documented on the category page, not on the per-graph page. Pick the case that applies:
+
+1. **Graph for an existing category** — name the grammar(s) from the category's "Canonical grammars" section that apply to this graph; they become the table column(s) filled in for the new row. No new grammar text on the site.
+2. **New grammar for an existing category** — provide the grammar below; it will be added to the category page and a new column to its table.
+3. **Graph for a new category** — provide the canonical grammar(s) below; a new category page will be created with them.
+
+Every query you provide must ship inside the archive (a full archive ships all queries of the graph; a partial archive ships the new ones): one directory per query in `queries/cfpq/`, `queries/rpq/`, or `queries/mcfpq/`, holding every representation of the query (`.cnf`/`.rsm`, `.re`/`.rsm` — regular only, or `.mcfg`) plus one `results.mtx` with the constrained reachability facts (a Boolean pattern matrix with the graph's dimensions; an entry `(i, j)` means some path from node `i` to node `j` satisfies the query). Each query directory is described in a `## <class>/<query>` section of `queries/README.md`. A query may only use labels of this graph.
 
-<LaTeX format of grammar>
+<LaTeX format of grammar (only for cases 2 and 3)>
 
-<Pyformlang CFG format of grammar>
+<File-format text of the grammar — .cnf, .rsm, .re, or .mcfg (only for cases 2 and 3)>

.github/dependabot.yml

@@ -2,6 +2,7 @@ version: 2
 updates:
   - package-ecosystem: "pip"
     directory: "/"
+    target-branch: "dev"
     schedule:
       interval: "monthly"
     ignore:

.github/workflows/coverage.yml

@@ -11,30 +11,22 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v7
+      - name: Set up uv with Python ${{ matrix.python-version }}
+        uses: astral-sh/setup-uv@v10.1.0
         with:
           python-version: ${{ matrix.python-version }}
 
-      - name: Install Poetry
-        run: |
-          pip install poetry
-
-      - name: Install dependencies with Poetry
-        run: |
-          poetry install --no-interaction --no-ansi --with dev,test
+      # Why these flags: see "Development setup" in docs/developer.rst.
+      - name: Install dependencies with uv
+        run: uv sync --frozen --all-groups
 
       - name: Show installed packages
-        run: poetry show
-
-      - name: Install packages
-        run: |
-          python -m pip install .
-
+        run: uv tree
 
-      - name: Test CFPQ_Data with coverage
-        run: |
-          poetry run pytest --cov=cfpq_data --doctest-modules -vv -s cfpq_data tests
+      # Per-metric gate rationale: see "Test pipeline" in docs/developer.rst.
+      # term-missing keeps the uncovered lines in the CI log.
+      - name: Test FLPQ_Data with coverage (line and branch >= 95%)
+        run: uv run pytest --cov=flpq_data --cov-report=term-missing --cov-report=json && uv run python utils/check_coverage.py
 
       - name: Upload coverage to Codecov
         uses: codecov/codecov-action@v7

.github/workflows/deploy_docs.yml

@@ -15,28 +15,20 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v7
+      - name: Set up uv with Python ${{ matrix.python-version }}
+        uses: astral-sh/setup-uv@v10.1.0
         with:
           python-version: ${{ matrix.python-version }}
 
-      - name: Install Poetry
-        run: |
-          pip install poetry
-
-      - name: Install dependencies with Poetry
-        run: |
-          poetry install --no-interaction --no-ansi --with docs
+      # Why these flags: see "Development setup" in docs/developer.rst.
+      - name: Install dependencies with uv
+        run: uv sync --frozen --all-groups
 
       - name: Show installed packages
-        run: poetry show
-
-      - name: Install packages
-        run: |
-          poetry run pip install .
+        run: uv tree
 
       - name: Build
-        run: poetry run make -C docs/ html
+        run: uv run make -C docs/ html
 
       - name: Deploy
         uses: JamesIves/github-pages-deploy-action@v4

.github/workflows/docs.yml

@@ -11,31 +11,22 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v7
+      - name: Set up uv with Python ${{ matrix.python-version }}
+        uses: astral-sh/setup-uv@v10.1.0
         with:
           python-version: ${{ matrix.python-version }}
 
-      - name: Install Poetry
-        run: |
-          pip install poetry
-
-      - name: Install dependencies with Poetry
-        run: |
-          poetry install --no-interaction --no-ansi --with docs
+      # Why these flags: see "Development setup" in docs/developer.rst.
+      - name: Install dependencies with uv
+        run: uv sync --frozen --all-groups
 
       - name: Show installed packages
-        run: poetry show
-
-      - name: Install packages
-        run: poetry run pip install .
+        run: uv tree
 
-      # No-warnings policy: the build runs with -W --keep-going (see
-      # docs/Makefile), so any warning — including an unresolved
-      # cross-reference under nitpicky — exits non-zero on its own.
+      # No-warnings policy: see "Docs build and deploy" in docs/developer.rst.
       - name: Build
-        run: poetry run make -C docs/ html
+        run: uv run make -C docs/ html
 
-      # The linkcheck builder exits non-zero on broken or timed-out links.
+      # Link check policy: see "Docs build and deploy" in docs/developer.rst.
       - name: Check links
-        run: poetry run sphinx-build -b linkcheck docs docs/_build/linkcheck
+        run: uv run sphinx-build -b linkcheck docs docs/_build/linkcheck

.github/workflows/lint.yml

@@ -7,20 +7,28 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [ 3.9 ]
+        python-version: [ 3.11 ]
     steps:
       - uses: actions/checkout@v7
 
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v7
+      - name: Set up uv with Python ${{ matrix.python-version }}
+        uses: astral-sh/setup-uv@v10.1.0
         with:
           python-version: ${{ matrix.python-version }}
 
-      - name: Install pre-commit
-        run: |
-          python -m pip install pre-commit==3.6.2
-          pre-commit install
+      # Why these flags: see "Development setup" in docs/developer.rst.
+      - name: Install dependencies with uv
+        run: uv sync --frozen --all-groups
 
       - name: Run pre-commit
         run: |
-          pre-commit run --all-files --color always --verbose --show-diff-on-failure
+          uv run pre-commit run --all-files --color always --verbose --show-diff-on-failure
+
+      # The pre-commit pass already runs ruff and ty as hooks; these steps
+      # make the type checks explicit — Pyright is the stricter checker and
+      # only runs here.
+      - name: Type check with ty
+        run: uv run ty check
+
+      - name: Type check with Pyright
+        run: uv run pyright