diff --git a/.github/ISSUE_TEMPLATE/new-data-template.md b/.github/ISSUE_TEMPLATE/new-data-template.md index 5480c612..0d6ad4e4 100644 --- a/.github/ISSUE_TEMPLATE/new-data-template.md +++ b/.github/ISSUE_TEMPLATE/new-data-template.md @@ -58,7 +58,7 @@ Every query directory ships one `results.mtx` with the constrained reachability ## Data format -Two kinds of archives exist (both documented in the "File structure" section of the [Graphs page](https://formallanguageconstrainedpathquerying.github.io/FLPQ_Data/graphs/index.html)): +Two kinds of archives exist (both documented on the [File structure page](https://formallanguageconstrainedpathquerying.github.io/FLPQ_Data/file_structure.html)): - **Full archive** — a new graph: `.tar.gz` unpacks to a self-contained directory with `README.md`, `graph/` (one Boolean MatrixMarket pattern file per stored edge label), and `queries/` (one directory per query in `cfpq/`, `rpq/`, or `mcfpq/`; each query directory holds every representation of the query — `.cnf`/`.rsm`, `.re`/`.rsm` (regular only), or `.mcfg` — plus one `results.mtx`). - **Partial archive** — new queries for an existing graph: `.tar.gz` unpacks to a directory named after that graph containing only `queries/` — the new query directories (same layout) plus a `README.md` fragment with one `## /` section per new query. Partial archives are merged into the existing graph archive by the maintainers. diff --git a/.opencode/skills/add-graph/SKILL.md b/.opencode/skills/add-graph/SKILL.md index 3aacb41e..57aa3874 100644 --- a/.opencode/skills/add-graph/SKILL.md +++ b/.opencode/skills/add-graph/SKILL.md @@ -17,8 +17,8 @@ Fill in every field in the triangle brackets (`<>`) of that template. ## Data format -The archive layout and the label conventions are documented once in the -"File structure" section of `docs/graphs/index.rst` — follow it exactly. +The archive layout and the label conventions are documented once in +`docs/file_structure.rst` — follow it exactly. Pack every query of the graph into the archive per the "Queries and grammars" section of the issue template, then validate with `flpq-check-archive .tar.gz` (the upload tool runs the same check diff --git a/AGENTS.md b/AGENTS.md index 4dd825c0..054edfa0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,7 +18,7 @@ gate → merge, and points to the other workflow skills below. * Documentation is about "What" and "Why". Skills are about "How". * This file is a short entry point for fast cold errors-free start. -* Only one source of truth. No duplicates. Each thing (in both code and documentation) described exactly once. Use generalization (especially for code), cross-references, links, other similar techniques to avoid duplicates and reuse staff. +* Only one source of truth. No duplicates. Each thing (in both code, issues, pull request descriptions, and documentation) described exactly once. Use generalization (especially for code), cross-references, links, other similar techniques to avoid duplicates and reuse staff. Find what you can to reuse before creating new one. * Source-of-truth hierarchy: code, scripts, CI configs > docs. If a fact can be extracted from code or scripts (e.g., CI configs), it is not duplicated in docs. Docs hold only what cannot be unambiguously reconstructed from @@ -27,8 +27,13 @@ gate → merge, and points to the other workflow skills below. * Data archives are self-contained and must align with the checker (`flpq_data/tools/check_archive_structure.py`, console command `flpq-check-archive`) — the source of truth for any data structure. -* Tools, not instructions. If you can do something with existing tool --- do it. No thinking, no long instructions, no manual analysis. You want to analyze code coverage? Just run coverage tool and analyze report. No workaround for regular tasks. If there is a tool for regular task it must be installed and configured appropriately. +* Tools, not instructions. If you can do something with existing tool --- do it. No thinking, no long instructions, no manual analysis. Do you want to analyze code coverage? Just run coverage tool and analyze report. No workaround for regular tasks. If there is a tool for regular task it must be installed and configured appropriately. * Always learn, never forget — encode patterns before session ends +* Issues, pull requests, and their comments are a knowledge base of technical + details. Log task-related findings and inspirations in the descriptions or + comments of the respective issues/PRs. When faced with a problem, check + whether a similar problem was solved or described previously — look for the + solution in existing issues, PRs, and their comments first. ## Package layout diff --git a/CHANGELOG.md b/CHANGELOG.md index ebdfbf0b..1f877a4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [6.1.1] - 2026-10-15 + +### Changed + +- The contribution documentation is consolidated under the Dataset section: + new "Contributing data" and "File structure" pages; both sections are + removed from the Graphs page (now a pure catalog) and from the Dataset root. + ## [6.1.0] - 2026-10-14 ### Added diff --git a/README.rst b/README.rst index 1b1d021a..fe676d13 100644 --- a/README.rst +++ b/README.rst @@ -88,7 +88,7 @@ contribution guidelines — is on the `Developer `_ — one or more validated archives (one graph each), links to them, and the motivation. The maintainers upload and merge the data into the dataset; the full contribution guide is on the `Graphs `_ page. +Create an ``Issue`` corresponding to the `"CONTRIBUTE NEW DATA" issue template `_ — one or more validated archives (one graph each), links to them, and the motivation. The maintainers upload and merge the data into the dataset; the full contribution guide is on the `Contributing data `_ page. Citation ******** diff --git a/docs/contributing_data.rst b/docs/contributing_data.rst new file mode 100644 index 00000000..cc831f44 --- /dev/null +++ b/docs/contributing_data.rst @@ -0,0 +1,67 @@ +.. _contributing_data: + +================= +Contributing data +================= + +.. only:: html + + :Release: |release| + :Date: |today| + +New data — graphs, queries, grammars — enters the dataset through a single +`"CONTRIBUTE NEW DATA" issue `_: +you prepare one or more archives (one graph each), validate them, publish +them (e.g. to Google Drive), and open the issue with the links, the +validation output, the SHA256 of every archive, and the motivation. The +maintainers upload the archives and merge them into the dataset — no pull +request is needed. Every archive must follow the layout documented in +:ref:`file_structure`. + +Contribution cases +------------------ + +The issue template's "Contribution type" section lists the four contribution +cases — new graph(s) for an existing or a new category, with an existing or +a new grammar, and new queries for existing graph(s) — and what each +requires; its remaining sections carry every required field (statistics, +grammar definitions, category description). A set of graphs for a category +goes into one issue as several archives (one graph per archive). + +Validation +---------- + +Every archive must pass the structure checker before it is shared:: + + flpq-check-archive .tar.gz # full archive + flpq-check-archive .tar.gz --partial # partial archive + +The issue carries the pasted output of this command per archive, the +archive's SHA256, and the FLPQ_Data version used; the template has the +recipe — ``pip install flpq-data`` installs the checker. + +Consistency invariants +---------------------- + +- No graph without at least one query. +- No query without reference results (``results.mtx``). +- No item without description and motivation: every graph is described in + its archive ``README.md``, every query in ``queries/README.md``, and the + issue states why each new graph, grammar, or category is useful to other + users. + +Not accepted via this flow +-------------------------- + +The issue template's "Not accepted via this flow" section lists the declined +cases: a grammar or query provided outside an archive (without reference +results), updates to an existing graph, a new representation of an existing +query, and more than one graph inside a single archive. + +Grammars +-------- + +Grammars are added together with the data they apply to — the template's +"Queries and grammars" section carries their definitions. A grammar is +accepted only if it is instantiated as at least one query with reference +results in a contributed archive. diff --git a/docs/dataset.rst b/docs/dataset.rst index 2a372c9b..41b1c2e0 100644 --- a/docs/dataset.rst +++ b/docs/dataset.rst @@ -12,7 +12,9 @@ Dataset The dataset catalog: graphs and queries for experimental analysis of formal-language-constrained path querying. The graph catalog is shared by all query classes; each class has its own section with its query templates and the -list of applicable graphs. +list of applicable graphs. The section also documents how to contribute new +data (:doc:`contributing_data`) and the archive file layout +(:doc:`file_structure`). .. toctree:: :maxdepth: 1 @@ -22,8 +24,5 @@ list of applicable graphs. queries/rpq/index queries/mcfpq/index reachable_pairs - -How to add a new grammar? -------------------------- - -Grammars are added together with the data they apply to — create an ``Issue`` corresponding to the `"CONTRIBUTE NEW DATA" issue template `_ (the "Queries and grammars" section). A grammar is accepted only if it is instantiated as at least one query with reference results in a contributed archive. + contributing_data + file_structure diff --git a/docs/developer.rst b/docs/developer.rst index e06166eb..eaff6a5d 100644 --- a/docs/developer.rst +++ b/docs/developer.rst @@ -265,6 +265,6 @@ linear history, no squash, so every subtask commit keeps its own message. Adding graphs and grammars ~~~~~~~~~~~~~~~~~~~~~~~~~~ -New dataset entries follow the contribution templates in :file:`.github/` -(issue and pull-request templates for graphs and grammars); see the "How to -add a new graph?" section of the README. +New dataset entries — graphs, queries, grammars — enter through a single +CONTRIBUTE NEW DATA issue; the full flow is documented in +:doc:`contributing_data`. diff --git a/docs/file_structure.rst b/docs/file_structure.rst new file mode 100644 index 00000000..27d05de7 --- /dev/null +++ b/docs/file_structure.rst @@ -0,0 +1,94 @@ +.. _file_structure: + +************** +File structure +************** + +.. only:: html + + :Release: |release| + :Date: |today| + +A graph is distributed as a self-contained archive ``.tar.gz`` that +unpacks to a directory named after the graph. Every archive has the same +fixed structure:: + + / + ├── README.md description of the graph and of its files + ├── graph/ one MatrixMarket file per stored edge label + │ └──