Status
|
|
| Acknowledged |
Yes — thanks @robvanderveer for filing and for the clear examples |
| Scheduled |
Yes — planned after the GSoC/LLM ETL ingestion pipeline is ready |
| Blocked |
Yes — depends on completion of the cheat-sheet LLM ETL pipeline (see Dependencies) |
Target timeline: late August / September 2026
Scope
Improve OpenCRE Chat so it leverages the full relevant knowledge base — multiple matching resources, not just the single best-scoring one — once those resources are reliably ingested and linked via the new pipeline.
Today, chat retrieves one resource (the best-matching hit) even when several resources could better answer the question. Selecting and surfacing more relevant resources should increase accuracy, richness, and give users more links to explore.
Key design question (from original report): finding the right break-off point for similarity scores when including additional resources.
Currently, openCRE chat retrieves one resource: the best matching one, while there may be several resources to help answer the question - potentially ones with much better or more extensive information. Selecting more relevant resources would definitely increase accuracy, richness, and also it would allow users to have more resource links to go to for more information.
The trick is to find a break-off point when it comes to the calculated similarity.
Example:
"how to pick the right people to join in threat modeling"
In my session this selected the Cheat sheets. Great resource, but not with the answer to the question.
When I do: "how to pick the right people to join in threat modeling according to SAMM"
It will select SAMM and tell me "Product owners, architects, security champions, and security testers. At Maturity Level 1, the goal is to expose teams and stakeholders to threat modeling to increase security awareness and create a shared vision for the system's security."
Dependencies
This issue is blocked until the GSoC/LLM ETL ingestion pipeline is ready:
Chat improvements should proceed once ingested cheat-sheet (and related) resources are available in the graph with reliable CRE links.
Status
Target timeline: late August / September 2026
Scope
Improve OpenCRE Chat so it leverages the full relevant knowledge base — multiple matching resources, not just the single best-scoring one — once those resources are reliably ingested and linked via the new pipeline.
Today, chat retrieves one resource (the best-matching hit) even when several resources could better answer the question. Selecting and surfacing more relevant resources should increase accuracy, richness, and give users more links to explore.
Key design question (from original report): finding the right break-off point for similarity scores when including additional resources.
Original report (@robvanderveer)
Currently, openCRE chat retrieves one resource: the best matching one, while there may be several resources to help answer the question - potentially ones with much better or more extensive information. Selecting more relevant resources would definitely increase accuracy, richness, and also it would allow users to have more resource links to go to for more information.
The trick is to find a break-off point when it comes to the calculated similarity.
Example:
"how to pick the right people to join in threat modeling"
In my session this selected the Cheat sheets. Great resource, but not with the answer to the question.
When I do: "how to pick the right people to join in threat modeling according to SAMM"
It will select SAMM and tell me "Product owners, architects, security champions, and security testers. At Maturity Level 1, the goal is to expose teams and stakeholders to threat modeling to increase security awareness and create a shared vision for the system's security."
Dependencies
This issue is blocked until the GSoC/LLM ETL ingestion pipeline is ready:
docs/rfc/cheatsheets-llm-autonomous-mapping-rfc.md)Chat improvements should proceed once ingested cheat-sheet (and related) resources are available in the graph with reliable CRE links.