diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 8bb2a9e8..8bd73988 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1,4 +1,4 @@ -blank_issues_enabled: true +blank_issues_enabled: false contact_links: - name: Something unclear about Haystack? Just ask on Haystack Discussions 😊 url: https://github.com/deepset-ai/haystack/discussions/new diff --git a/.github/ISSUE_TEMPLATE/new-tutorial-request---.md b/.github/ISSUE_TEMPLATE/new-tutorial-request---.md deleted file mode 100644 index df7c2e29..00000000 --- a/.github/ISSUE_TEMPLATE/new-tutorial-request---.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -name: "New Tutorial Request \U0001F4D3" -about: Suggest a new tutorial for this repository -title: 'New tutorial for: ' -labels: new tutorial -assignees: '' - ---- - -**Describe the tutorial you would like to see here** -A clear and concise description of what the problem is. Ex. I'd like to see a tutorial about [...] - -**Additional context** -Add any other context or screenshots about the feature request here. - -[ ] I've checked the existing tutorials diff --git a/.github/ISSUE_TEMPLATE/new_tutorial.yml b/.github/ISSUE_TEMPLATE/new_tutorial.yml new file mode 100644 index 00000000..a95ffe72 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/new_tutorial.yml @@ -0,0 +1,53 @@ +name: 📓 New tutorial +description: Propose a new tutorial for the Haystack Tutorials repository +labels: ["new tutorial"] +body: + - type: markdown + attributes: + value: | + Thanks for wanting to contribute! Every new tutorial must start as an issue. + A maintainer will review it and, once assigned, you can open a PR referencing this issue. + See [CONTRIBUTING.md](https://github.com/deepset-ai/haystack-tutorials/blob/main/CONTRIBUTING.md) for the full process. + - type: input + id: title + attributes: + label: Proposed tutorial title + description: A descriptive name (include model providers, databases, or technologies used). + validations: + required: true + - type: textarea + id: haystack-feature + attributes: + label: What Haystack feature does this tutorial teach? + description: Name the specific Haystack component(s), pipeline pattern, or concept this tutorial walks through. + validations: + required: true + - type: textarea + id: why + attributes: + label: Why is this a good addition to the tutorials? + description: Explain who this is for and what a reader learns about using Haystack. This is not the place to pitch a third-party tool. + validations: + required: true + - type: dropdown + id: third-party + attributes: + label: Does this tutorial feature a third-party integration (model provider, vector database, tool, etc.)? + options: + - "No" + - "Yes" + validations: + required: true + - type: textarea + id: third-party-details + attributes: + label: If yes, name the third-party product and explain how it's used with Haystack (not as a standalone demo of that product) + validations: + required: false + - type: checkboxes + id: acknowledge + attributes: + label: Acknowledgement + options: + - label: I understand a maintainer must assign this issue to me before I open a PR. + required: true diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 00000000..3a34a1a2 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,14 @@ +## Linked issue + + + + + +## Checklist + +- [ ] If this PR adds a new tutorial, the issue proposing it was assigned to me by a maintainer before I opened this PR. +- [ ] The notebook is added to `/tutorials` and named following the [naming convention](https://github.com/deepset-ai/haystack-tutorials/blob/main/CONTRIBUTING.md#naming-convention-for-file-names). +- [ ] The notebook is registered in `index.toml` with a title, description, level, and weight. diff --git a/.github/workflows/enforce_issue_link.yml b/.github/workflows/enforce_issue_link.yml new file mode 100644 index 00000000..807b1772 --- /dev/null +++ b/.github/workflows/enforce_issue_link.yml @@ -0,0 +1,99 @@ +name: Enforce issue-first contribution for new tutorials + +on: + pull_request_target: + types: [opened, edited, reopened, synchronize] + +permissions: + contents: read + pull-requests: write + issues: read + +jobs: + enforce-issue-link: + runs-on: ubuntu-latest + steps: + - uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + script: | + const pr = context.payload.pull_request; + const author = pr.user.login; + + const files = await github.paginate(github.rest.pulls.listFiles, { + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: pr.number, + }); + + const addsNewTutorial = files.some( + (f) => + f.status === "added" && + f.filename.startsWith("tutorials/") && + f.filename.endsWith(".ipynb") && + f.filename !== "tutorials/template.ipynb" + ); + + if (!addsNewTutorial) { + // Edits, fixes, and other non-new-tutorial PRs aren't gated. + return; + } + + const body = pr.body || ""; + const match = body.match(/\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\s+#(\d+)\b/i); + + const explain = async (message) => { + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: pr.number, + body: message, + }); + await github.rest.pulls.update({ + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: pr.number, + state: "closed", + }); + await github.rest.issues.addLabels({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: pr.number, + labels: ["needs-issue"], + }); + }; + + if (!match) { + await explain( + `Hi @${author} — this PR adds a new tutorial, and new tutorials must start as an issue before a PR is opened. ` + + `Please open an issue with the [new tutorial template](../../issues/new?template=new_tutorial.yml), ` + + `get it assigned to you by a maintainer, then reopen this PR with \`Closes #\` in the description. ` + + `See [CONTRIBUTING.md](../../blob/main/CONTRIBUTING.md).` + ); + return; + } + + const issueNumber = parseInt(match[3], 10); + let issue; + try { + issue = (await github.rest.issues.get({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issueNumber, + })).data; + } catch (e) { + await explain( + `Hi @${author} — issue #${issueNumber} referenced in this PR could not be found. ` + + `Please link a valid, assigned issue. See [CONTRIBUTING.md](../../blob/main/CONTRIBUTING.md).` + ); + return; + } + + const assignees = (issue.assignees || []).map((a) => a.login.toLowerCase()); + if (!assignees.includes(author.toLowerCase())) { + await explain( + `Hi @${author} — issue #${issueNumber} isn't assigned to you. ` + + `A maintainer needs to assign the issue to you before a PR for it is accepted. ` + + `See [CONTRIBUTING.md](../../blob/main/CONTRIBUTING.md).` + ); + return; + } diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..57a41e15 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,41 @@ +# AGENTS.md + +Instructions for AI coding agents (Claude Code, Copilot, Cursor, etc.) working in this repository. + +## What this repo is + +The tutorials shown on the [Haystack website](https://haystack.deepset.ai/tutorials/). Each tutorial is +an interactive `.ipynb` notebook that teaches a Haystack feature, pipeline pattern, or best practice — +not a showcase for a third-party product. + +## Contribution process — read before writing a new tutorial + +New tutorials require an issue **assigned by a maintainer** before a PR is opened. Do not write a new +notebook and open a PR for it speculatively — a CI workflow +(`.github/workflows/enforce_issue_link.yml`) auto-closes PRs that add a new tutorial without +referencing an issue assigned to the PR author. Full process in [CONTRIBUTING.md](CONTRIBUTING.md). + +This gate only applies to PRs that add a **new** tutorial notebook. Fixes, edits, and content +improvements to existing tutorials don't need a pre-assigned issue. + +If you're an agent acting on behalf of a user who wants to add a new tutorial: +1. Check whether an issue for it already exists and is assigned to the user. +2. If not, tell the user to open one via `.github/ISSUE_TEMPLATE/new_tutorial.yml` and wait for + assignment. +3. Only write the notebook and PR once the issue is confirmed assigned. + +## Adding/editing a tutorial + +1. Copy [tutorials/template.ipynb](tutorials/template.ipynb) to start a new tutorial. +2. Name the file following the [naming convention](CONTRIBUTING.md#naming-convention-for-file-names): + number prefix, underscores between words, short descriptive name. +3. Register it in `index.toml` under `[[tutorial]]` with `title`, `description`, `level`, `weight`, + and `notebook`. `weight` controls ordering. Set `colab = false` if it can't run on Google Colab. +4. Update `README.md`'s tutorial table if adding a new tutorial. +5. Install pre-commit hooks (`pre-commit install`) so formatting checks run before you commit. + +## Don't + +- Don't add a tutorial that primarily promotes a third-party tool with Haystack as an afterthought. +- Don't open a PR adding a new tutorial without a linked, assigned issue — it will be auto-closed. +- Don't hardcode API keys or secrets in notebook cells or outputs. diff --git a/Contributing.md b/CONTRIBUTING.md similarity index 82% rename from Contributing.md rename to CONTRIBUTING.md index dd667f96..c1e1341c 100644 --- a/Contributing.md +++ b/CONTRIBUTING.md @@ -8,6 +8,17 @@ To make a request for a new tutorial or to suggest edits and fixes, submit an is - **New Tutorial Request 📓:** To suggest that we create a new tutorial. +## New tutorials start as an issue + +⚠️ **A new tutorial must be proposed as an issue before any PR is opened.** Use the +[new tutorial template](.github/ISSUE_TEMPLATE/new_tutorial.yml) to describe the Haystack feature it +teaches and why it's a useful addition. A maintainer reviews the proposal and, once they assign the +issue to you, you can open your PR referencing it with `Closes #`. + +A CI check (`.github/workflows/enforce_issue_link.yml`) automatically closes PRs that add a new +tutorial without a linked, assigned issue. This does not apply to PRs that only fix or improve an +existing tutorial — those can be opened directly. + ## Contributing Edits or New Tutorials All of the Haystack tutorials live in the `tutorials` folder in this repo. Each tutorial is an interactive `.ipynb` file and we generate a Markdown file to accompany it. @@ -19,10 +30,12 @@ Here's what you need to do to add or edit tutorials 👇: - Install the pre-commit hooks with `pre-commit install`. This utility will run some formatting/checking tasks right before all git commit operations. 2. If you're **creating** a new tutorial: + - Make sure you have an issue assigned to you first (see above). - Create a copy of [tutorial template](/tutorials/template.ipynb) in `/tutorials` folder. - Rename the new `.ipynb` file by following the [naming convention](#naming-convention-for-file-names). - Follow the outline in the template as you create the tutorial. - After the tutorial is complete, add necessary information to [index.toml](/index.toml). Here, `weight` is the order in which your tutorial appears. For example, a tutorial with `weight = 15` comes after a tutorial with `weight = 10` and before `20`. Each tutorial comes with a Google Colab link and `Open in Colab` button on the top of the tutorial by default. If your new tutorial cannot be run on Google Colab, set `colab = false` not to display `Open in Colab` button on top the tutorial. + - Open your PR with `Closes #` in the description. 3. If you're **editing** an existing tutorial: - Make necessary changes in the `.ipynb` file of the tutorial and save them. 4. Create a pull request. diff --git a/README.md b/README.md index 5d4d0cd3..8e5523c5 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ This is the repository where we keep all the Haystack tutorials 📓 👇 These tutorials are also published to the [Haystack Website](https://haystack.deepset.ai/tutorials/). -To contribute to the tutorials, please check out our [Contributing Guidelines](./Contributing.md). +To contribute to the tutorials, please check out our [Contributing Guidelines](./CONTRIBUTING.md). ⚠️ New tutorials must start as an issue and be assigned by a maintainer before a PR is opened — see [CONTRIBUTING.md](./CONTRIBUTING.md#new-tutorials-start-as-an-issue). [![Run Tutorials Nightly](https://github.com/deepset-ai/haystack-tutorials/actions/workflows/nightly.yml/badge.svg)](https://github.com/deepset-ai/haystack-tutorials/actions/workflows/nightly.yml) [![Publish tutorials on Haystack Home](https://github.com/deepset-ai/haystack-tutorials/actions/workflows/publish_tutorials.yml/badge.svg)](https://github.com/deepset-ai/haystack-tutorials/actions/workflows/publish_tutorials.yml)