feat: Make nltk optional

[benglewis]

Fixes #769

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c384a7df09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep nltk available in the default test install

Making nltk optional here means a plain poetry install no longer guarantees it, but the test workflow still installs dependencies without extras and runs tests/custom/converter/test_export_conll.py; those tests call create_tokens_and_tags(), which now raises the new ImportError when nltk is missing. In a clean CI/dev environment this turns the default pytest run into a failure unless the conll extra is explicitly installed or CONLL tests are gated.

Useful? React with 👍 / 👎.

[wilsonwu]

I think upgrade to 3.9.3 is OK to fix this CVE.

[benglewis]

@wilsonwu Unfortunately, that won't fix it anymore:

nltk | 3.9.2 | CVE-2026-33236 | | NLTK has a Downloader Path Traversal Vulnerability (AFO) - Arbitrary File Overwrite
nltk | 3.9.2 | CVE-2026-33230 | | Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting') in nltk
nltk | 3.9.2 | CVE-2026-33231 | | Unauthenticated remote shutdown in nltk.app.wordnet_app
nltk | 3.9.2 | GHSA-rf74-v2fm-23pw | | Natural Language Toolkit (NLTK) has unbounded recursion in JSONTaggedDecoder.decode_obj() may cause DoS

I would really appreciate if there was some way to get this PR merged? 🙏