MAINT: enable strict mypy checking and fix violations

[tejas0077]

Related to #720

The @retry decorator from tenacity lacks type stubs, causing mypy
to report an untyped-decorator error when running strict mode:

pyrit/common/net_utility.py:85: error: Untyped decorator makes function
"make_request_and_raise_if_error_async" untyped [untyped-decorator]

Added # type: ignore[untyped-decorator] comment to suppress the error
until tenacity ships proper type stubs.

@romanlutz — tagging you as you filed the original issue.

Tests and Documentation

No new tests or documentation required for this change.

Verified fix by running:
python -m mypy pyrit/common/ --strict

Result: Success: no issues found in 20 source files

[romanlutz]

Is that the only typing error if you turn on strict typing? I'm not sure if you meant to fix more or want me to review now.

[tejas0077]

Hi @romanlutz, I have fixed all strict mypy errors in both common and models modules.
Changes made:
pyrit/common/net_utility.py Removed unused type: ignore comment as tenacity now has proper type stubs in this environment.
pyrit/models/storage_io.py Fixed two no-any-return errors by casting blob_stream.readall() to bytes and blob_properties.size > 0 to bool.
Verification:
python -m mypy pyrit/common/ --strict → Success: no issues found in 20 source files
python -m mypy pyrit/models/ --strict → Success: no issues found in 26 source files
Ready for review!

[tejas0077]

Hi @romanlutz, I have now gone further and fixed all strict mypy errors across the entire pyrit codebase, not just common and models.
Additional files fixed:
pyrit/auth/azure_auth.py -cast token_provider() return to str, added type: ignore[no-any-return] for get_bearer_token_provider which returns Any from the azure-identity library
pyrit/embedding/openai_text_embedding.py -removed unused type: ignore comment
pyrit/score/printer/console_scorer_printer.py -removed 6 unused type: ignore comments
pyrit/prompt_target/openai/openai_tts_target.py -removed 5 unused type: ignore comments
pyrit/prompt_target/openai/openai_completion_target.py -removed unused type: ignore comment
pyrit/prompt_target/hugging_face/hugging_face_chat_target.py -removed 2 unused type: ignore comments
Final verification:
python -m mypy pyrit/ --strict - Success: no issues found in 422 source files
Ready for review!

[romanlutz]

If it's fixed across the entire codebase you may want to set the strict setting to true in the toml file

[tejas0077]

Hi @romanlutz, all review comments have been addressed:
Enabled strict = true in pyproject.toml as suggested
Fixed all 170 strict mypy errors across 61 files (python -m mypy pyrit/ --strict → no issues found in 422 source files)
Removed the stray YAML files (ai_governance_failure.yaml, ai_supply_chain.yaml, ai_system_transparency.yaml) that were accidentally included from another branch
Pre-commit passing

[Copilot]

Pull request overview

This PR tightens type-safety across PyRIT by enabling global strict mypy checks and applying a broad set of typing-related fixes (optionals, casts, and targeted ignores) across core runtime components (memory, targets, scorers, datasets, UI RPC, and backend routes).

Changes:

• Enable mypy strict mode in pyproject.toml and adjust code to satisfy strict typing across the repository.
• Add runtime/type-narrowing guards (assertions) and fix various Optional-related issues (e.g., Optional[...] annotations, casts, safer MIME/extension handling).
• Refactor OpenAI target call sites to use a unified _client accessor and remove some no-longer-needed type ignores.

Reviewed changes

Copilot reviewed 66 out of 66 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
pyproject.toml	Turns on global strict mypy settings.
pyrit/common/net_utility.py	Tightens httpx client kwargs typing/casts; related to Tenacity decorator typing.
pyrit/common/display_response.py	Normalizes formatting and adds storage IO guards for notebook image display.
pyrit/common/data_url_converter.py	Handles missing file extensions safely.
pyrit/ui/rpc.py	Adds asserts and adjusts wait_for_score to return Optional[Score].
pyrit/ui/rpc_client.py	Adds asserts for RPC client internal fields.
pyrit/prompt_target/rpc_client.py	Adds asserts for RPC client internal fields.
pyrit/prompt_target/text_target.py	Changes CSV import default sequencing behavior for MessagePiece.
pyrit/prompt_target/prompt_shield_target.py	Adds required-value asserts and defaults API version.
pyrit/prompt_target/websocket_copilot_target.py	Hardens MIME type validation when checking image paths.
pyrit/prompt_target/openai/openai_target.py	Adds _client accessor and request_piece asserts; updates typing on helpers.
pyrit/prompt_target/openai/openai_chat_target.py	Uses _client for chat completions.
pyrit/prompt_target/openai/openai_completion_target.py	Uses _client for completions.
pyrit/prompt_target/openai/openai_image_target.py	Uses _client for image generation/edit.
pyrit/prompt_target/openai/openai_response_target.py	Uses _client for Responses API calls.
pyrit/prompt_target/openai/openai_tts_target.py	Uses _client for TTS calls and removes arg-type ignores.
pyrit/prompt_target/openai/openai_video_target.py	Uses _client for video calls; adds assertion for missing text piece.
pyrit/prompt_target/hugging_face/hugging_face_chat_target.py	Removes some ignores, guards Optional model_id usage, and tightens typing.
pyrit/prompt_normalizer/prompt_normalizer.py	Introduces memory property and replaces direct _memory access; adds return-value ignore.
pyrit/prompt_normalizer/normalizer_request.py	Makes converter configuration lists Optional.
pyrit/prompt_converter/template_segment_converter.py	Makes template parameters handling robust to Optional.
pyrit/prompt_converter/denylist_converter.py	Makes denylist optional in signature.
pyrit/prompt_converter/codechameleon_converter.py	Improves Optional check for encrypt_function.
pyrit/prompt_converter/add_text_image_converter.py	Defaults MIME type when detection fails.
pyrit/prompt_converter/add_image_text_converter.py	Defaults MIME type when detection fails.
pyrit/prompt_converter/azure_speech_text_to_audio_converter.py	Avoids returning None for audio path.
pyrit/score/scorer.py	Fixes Optional typing on registry behavior and adds targeted assignment ignores.
pyrit/score/printer/console_scorer_printer.py	Removes no-any-return ignores around Fore.* returns.
pyrit/score/human/human_in_the_loop_gradio.py	Asserts score presence after RPC call.
pyrit/score/true_false/self_ask_true_false_scorer.py	Adds assertion after loading YAML.
pyrit/score/true_false/prompt_shield_scorer.py	Ensures boolean parsing returns list[bool].
pyrit/score/float_scale/self_ask_likert_scorer.py	Adds additional Likert scale enum entries.
pyrit/score/float_scale/azure_content_filter_scorer.py	Makes update_registry_behavior Optional in annotation.
pyrit/setup/initializers/components/scorers.py	Adds arg-type ignores and adjusts registration typing/formatting.
pyrit/setup/initializers/airt.py	Avoids passing Optional model names by defaulting to empty string.
pyrit/scenario/scenarios/foundry/red_team_agent.py	Avoids passing Optional endpoint into auth helper; adds ignore for kwargs typing.
pyrit/scenario/scenarios/airt/scam.py	Uses fallback empty list for seed_groups typing.
pyrit/scenario/scenarios/airt/psychosocial.py	Uses fallback empty list for seed_groups typing.
pyrit/scenario/scenarios/airt/jailbreak.py	Makes jailbreak_names Optional and uses fallback empty list for seed_groups.
pyrit/scenario/scenarios/airt/leakage.py	Avoids passing Optional endpoint into auth helper.
pyrit/scenario/scenarios/airt/cyber.py	Avoids passing Optional endpoint into auth helper.
pyrit/scenario/scenarios/airt/content_harms.py	Avoids passing Optional endpoint into auth helper.
pyrit/models/data_type_serializer.py	Makes internal fields Optional and changes storage IO selection/guards.
pyrit/models/message_piece.py	Adds type ignore when setting id = None.
pyrit/models/storage_io.py	Ensures Azure blob read returns bytes and normalizes boolean return.
pyrit/models/seeds/seed_prompt.py	Adds ignore on data_type assignment typing.
pyrit/models/seeds/seed_dataset.py	Normalizes optional path fields to strings when constructing simulated conversation seeds.
pyrit/memory/memory_interface.py	Makes key attributes Optional; adjusts serialization and schema reflection behavior.
pyrit/memory/memory_models.py	Adds targeted ignores and normalizes sequence defaulting.
pyrit/memory/central_memory.py	Makes singleton memory instance Optional.
pyrit/memory/sqlite_memory.py	Adds engine initialization checks before table operations.
pyrit/memory/azure_sql_memory.py	Adds engine/token guards and optional-expiry handling.
pyrit/backend/routes/media.py	Uses Optional-safe results_path when establishing allowed media root.
pyrit/backend/routes/version.py	Makes engine access optional-safe, but introduces a redundant conditional.
pyrit/cli/frontend_core.py	Tightens typing for dummy termcolor.cprint.
pyrit/auth/azure_auth.py	Normalizes token return types and adds ignore for provider callable typing.
pyrit/embedding/openai_text_embedding.py	Removes an unnecessary arg-type ignore for AsyncOpenAI init.
pyrit/executor/workflow/xpia.py	Adds asserts for callback and memory initialization.
pyrit/executor/promptgen/fuzzer/fuzzer.py	Asserts next_message is present before building requests.
pyrit/executor/attack/printer/markdown_printer.py	Refactors strategy identifier handling for Optional safety.
pyrit/executor/attack/multi_turn/tree_of_attacks.py	Removes a cast for metadata access and adds a scorer assert.
pyrit/analytics/result_analysis.py	Refactors strategy identifier handling for Optional safety.
pyrit/show_versions.py	Tightens deps_info typing and variable naming.
pyrit/datasets/seed_datasets/remote/vlsu_multimodal_dataset.py	Skips incomplete examples; hardens category fields; adjusts cache path building/guards.
pyrit/datasets/seed_datasets/remote/harmbench_multimodal_dataset.py	Adjusts cache path building/guards for image caching.

Comments suppressed due to low confidence (4)

pyrit/prompt_target/text_target.py:83

• When sequence is missing from the CSV, this now forces sequence=0. MessagePiece already has a default (sequence=-1), so using 0 changes ordering semantics (unknown vs first message). Consider omitting the argument when absent or defaulting to -1 to preserve the model's default meaning.
  pyrit/setup/initializers/components/scorers.py:157
• This _try_register call is split across lines in a way that hurts readability and makes the # type: ignore hard to associate with the correct expression. Consider formatting the call with one argument per line (or parentheses) and, instead of # type: ignore[arg-type], narrow gpt4o after the non-None check (e.g., via cast/assert) so the scorer constructors receive a non-optional PromptChatTarget.
  pyrit/backend/routes/media.py:96
• allowed_root = Path(memory.results_path or "").resolve() makes the traversal guard ineffective if results_path is unset (empty string resolves to the current working directory). For safety, treat missing/empty results_path as an initialization error and return 500 instead of defaulting to "".

    try:
        memory = CentralMemory.get_memory_instance()
        allowed_root = Path(memory.results_path or "").resolve()
    except Exception as exc:
        raise HTTPException(status_code=500, detail="Memory not initialized; cannot determine results path.") from exc

    # Path traversal guard
    if not requested.is_relative_to(allowed_root):
        raise HTTPException(status_code=403, detail="Access denied: path is outside the allowed results directory.")

pyrit/prompt_target/openai/openai_target.py:503

• There are two identical asserts for request_piece in the ContentFilterFinishReasonError handler. This is dead duplication and should be reduced to a single assert (or reuse the request_piece variable created just above).

---

You can also share your feedback on Copilot code review. Take the survey.

[tejas0077]

Hi @romanlutz, addressed all Copilot review comments:

• Replaced duplicate assert statements with RuntimeError raises
  (asserts can be stripped under python -O)
• Used DB_DATA_PATH as safe fallback instead of empty string in
  data_type_serializer.py to prevent root path writes
• Validated results_path and results_storage_io before use in
  harmbench and vlsu dataset serializers
• Simplified redundant nested conditional in version.py

mypy still passes: python -m mypy pyrit/ --strict → no issues found in 422 source files

Also need to investigate the pre-commit failure on macOS -will check that next.

[tejas0077]

Hi @romanlutz, I have now fixed all strict mypy errors across the entire pyrit codebase including the cascading errors from the Optional[Message] return type change.
Final files fixed in this update:
pyrit/auth/copilot_authenticator.py - assert username and password not None before page.fill, removed unused type: ignore
pyrit/cli/_banner.py - renamed role variable to char_role to fix type narrowing conflict with loop variable
pyrit/score/audio_transcript_scorer.py - cast bool expression to bool for no-any-return
pyrit/prompt_target/azure_blob_storage_target.py -assert client not None before get_blob_client
pyrit/executor/workflow/xpia.py -assert response not None before get_value
pyrit/executor/attack/single_turn/context_compliance.py - assert response not None before get_value
pyrit/executor/attack/multi_turn/tree_of_attacks.py -assert response not None before get_piece and get_value
pyrit/prompt_normalizer/prompt_normalizer.py -changed return type to Optional[Message]
Final verification:
python -m mypy pyrit/ --strict -Success: no issues found in 425 source files
Ready for review!

[tejas0077]

Good call. I used assert as the quickest way to narrow Optional types for mypy, but you're right that they're unsafe in production since python -O strips them. I'll replace them with proper if x is None: raise ValueError(...) checks throughout. These are all invariant guards that shouldn't trigger in practice, but explicit raises are the right pattern here. Will push shortly

[tejas0077]

Fixed — replaced all 67 assert x is not None guards with explicit if x is None: raise ValueError(...) checks so they're not stripped under python -O. mypy still passes: no issues found in 425 source files.