FEAT: Add remove_seeds_from_memory_async to memory - #1
Open
blahdeblahde wants to merge 561 commits into
Open
Conversation
…#2047) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… scenario E2E tests (microsoft#2048) Co-authored-by: Behnam Ousat <behnamousat@microsoft.com>
…h 2 updates (microsoft#2052) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: hannahwestra25 <hannahwestra@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: hannahwestra25 <hannahwestra@users.noreply.github.com>
…t#2034) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…patch group across 1 directory (microsoft#2055) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
…ft#1902) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…7 to 0.15.19 (microsoft#2073) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…icrosoft#2076) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#2077) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
…t#2095) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
microsoft#2046) Co-authored-by: Behnam Ousat <behnamousat@microsoft.com>
…icrosoft#2098) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2c45e075-9e85-40b5-8352-fb0bcab7261c
…endabot alerts (microsoft#2323) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9009f005-342b-41a1-8513-74cbf282a9a4
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 647cdf27-3786-40a2-918c-b88f784fbc6d
…atch group across 1 directory (microsoft#2324) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1c85c49c-2d6c-4050-b3e3-ce085c97c5ec
…t and Responses targets) (microsoft#2166) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: varunj-msft <vjoginpalli@microsoft.com> Copilot-Session: 00b73356-ae65-4d83-8220-ebad37cc5850 Copilot-Session: ebba1398-76c6-4659-97c5-a08afa209749
…ft#2330) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… to 0.16.1 (microsoft#2328) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tes (microsoft#2332) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…crosoft#2314) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 13a6d940-c11c-4ba7-a4d4-bc6b9d75c757
…tes (microsoft#2326) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a14e9aad-db5a-46bc-9da4-4018369a8eab
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7a2feb74-e66a-427e-8b5c-c503b91d98d8
blahdeblahde
force-pushed
the
blahdeblahde-remove-seeds-from-memory-api
branch
from
August 6, 2026 17:48
7dbf60f to
1873066
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 88783b89-8853-43e4-8d60-483268ab0579
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f742b92-0427-45a9-924f-3fedeb8b915a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 41c8a28e-3109-4b71-b97a-3d8ec8834136
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 48ec2fc7-3fd7-4188-a3ff-df3cc2a64790
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0d1f97bd-49a5-4a6d-b814-fecef1e2a8cb
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92dc0f48-0fb4-4cbd-866c-ce5f41796439
Adds two memory APIs to remove seed prompts using the same filter parameters as get_seeds (value, exact, value_sha256, dataset_name, dataset_name_pattern, data_types, harm_categories, added_by, authors, groups, source, seed_type, parameters, metadata, prompt_group_ids): - remove_seeds_from_memory removes the individual seeds that match. - remove_seed_groups_from_memory expands any match to its whole prompt_group_id so partial groups are never left behind. Implementation: - Extract filter-building logic from get_seeds into a shared _build_seed_filter_conditions helper so query and removal cannot drift. - value matching is controlled by an exact bool flag (replacing the earlier value_exact string, per review). The remove methods default to exact=True (full-string equality) so a short or common value cannot over-delete; get_seeds keeps substring matching. Pass exact=False to opt into substring deletion. The intentional difference is documented. - Require at least one filter (raises ValueError if none provided). - Single transaction with rollback on SQLAlchemyError. - Group removal selects matching prompt_group_ids with a server-side subquery (not a materialized IN(...) list) so a broad filter cannot exceed the backend bound-parameter limit; NULL group ids are excluded, so ungrouped seeds are skipped. - Methods are synchronous, mirroring get_seeds: they do only DB work and never await I/O. Only database records are removed; file-backed seeds (image_path, audio_path, video_path) leave the serialized file on disk (documented). Unit tests cover every individual filter, exact-by-default vs opt-in substring matching, multi-filter narrowing, no-filter ValueError, zero-match, rollback-on-error, and for groups: whole-group removal, groups spanning datasets, non-matching groups preserved, and the ungrouped (NULL group id) skip behavior. Docs: add a "Removing Seeds from the Database" section to the seed database notebook (8_seed_database.py/.ipynb) demonstrating the preview-then-remove workflow, whole-group removal, exact-by-default matching, the ungrouped-skip, and the file-backed caveat. Resolves AB#5688 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f9e3cc10-5c77-4384-92d9-d27dad9baea2
blahdeblahde
force-pushed
the
blahdeblahde-remove-seeds-from-memory-api
branch
from
August 7, 2026 17:29
1873066 to
c6060d9
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0d1f97bd-49a5-4a6d-b814-fecef1e2a8cb Copilot-Session: 67c023ca-8b50-48de-88f1-0e2cc3025996
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2b3468bc-0737-461b-bb82-18dad97d400e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 058569a0-6218-4cf4-947e-67fa51001d85
…h 2 updates (microsoft#2355) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…icrosoft#2261) Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7b84937f-52e6-42a6-9b35-ff544cfd16eb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PyRIT memory can add seed prompt datasets (via
add_seed_datasets_to_memory_async/add_seeds_to_memory_async) and query them withget_seeds, but there is no counterpart for removing seeds. Today users have to drop into raw SQL, which is error-prone and differs across the supported backends (DuckDB, SQLite, Azure SQL).This PR adds
remove_seeds_from_memory_async, which accepts the same filter parameters asget_seeds(dataset_name,dataset_name_pattern,added_by,harm_categories,authors,groups,source,value_sha256,data_types,seed_type,parameters,metadata,prompt_group_ids) and returns the number of seeds removed.Recommended workflow — preview, then delete with the same filters:
Implementation
get_seedsinto a shared private helper (_build_seed_filter_conditions) so both query and removal build conditions from a single source and cannot drift.remove_seeds_from_memory_asyncdeletes matching rows in a single transaction using the SQLAlchemy ORM (rollback on error), so it behaves consistently across DuckDB, SQLite, and Azure SQL.Safety
ValueErrorto prevent accidentally wiping the entire seed database.Tests and Documentation
Tests —
tests/unit/memory/memory_interface/test_interface_remove_seeds.pycovers:dataset_namematchdataset_name_pattern(SQLLIKE)dataset_name+added_by)ValueError0and deletes nothingharm_categories(list field) andsourcefiltersDocumentation — added a "Removing Seeds from the Database" section to
doc/code/memory/8_seed_database.py(and the synced.ipynb) demonstrating the preview-then-remove workflow.