Skip to content

fix(security): replace pickle with json in noise_dive scripts - #106

Open
ha466 wants to merge 24 commits into
mudler:masterfrom
ha466:master
Open

ha466 wants to merge 24 commits into
mudler:masterfrom
ha466:master

Conversation

@ha466

@ha466 ha466 commented Oct 27, 2026

Copy link
Copy Markdown

No description provided.

google-labs-jules Bot and others added 24 commits October 26, 2026 16:31
Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Replaced pickle serialization and deserialization in scripts/vad_bench/real_recordings/rl.py and eval_frames.py with JSON format to eliminate arbitrary code execution risk.

Assisted-by: Jules

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Replace pickle with json for reading and writing seg evaluation
results in seg_report.py and seg_eval.py to prevent arbitrary code
execution.

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Include expected/actual n_mels, n_frames, and mel.size in the error
message when validation fails in StreamingDiarization::feed_mel, and
log the message using PK_LOG before throwing std::runtime_error.

Assisted-by: Jules:1.0 [Playwright] [Bash]

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Check if Content-Length header is set or if chunked Transfer-Encoding is
used in expect_content.

Assisted-by: Jules:claude-3-5-sonnet

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Assisted-by: Jules

Avoid unnecessary string copy overhead by passing json by const std::string&
in named_profiles_json and slicing off the trailing brace using std::string_view.

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Adds a standalone unit test `test_streaming_diarization_config` that verifies
`StreamingDiarization` throws `std::runtime_error("parakeet: invalid diarization streaming config")`
when constructed with invalid stream configuration parameters.

Assisted-by: Jules:0.1

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Replaces the inline static_cast<SSLClient &>(*this) downcast and TODO comment
in ClientImpl::send_ with a virtual method initialize_ssl override in SSLClient.

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Remove fallback to untiled single-graph execution when causal_ is true.
Tiled subsampling window halo (H=64) covers the receptive field for
both non-causal and causal downsampling.

Assisted-by: Jules

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
- Replace `pickle.load` in `scripts/vad_bench/fusion/analyze.py` with `json.load`.
- Update `scripts/vad_bench/fusion/run.py` and `scripts/vad_bench/fusion/report.py` to write results using `json.dump`.
- Update `scripts/vad_bench/fusion/README.md` documentation.
- Add unit test in `tests/python/test_fusion_json.py` to verify JSON serialization/deserialization.

Assisted-by: AGENT_NAME:MODEL_VERSION

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Refactor Accept-Encoding header parsing to parse comma-separated quality values (q-values) for Accept-Encoding headers instead of naive substring matching. Exclude br when q=0 or q=0.0.

Assisted-by: Jules <jules@agent.dev>

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
Optimize audio loading, linear resampling, and mel spectrogram computation:
- Fast path for mono audio (ch == 1) in decode_pcm_16k_mono avoiding double conversions, zero-accumulation, per-sample division by 1, and initial vector zero-fill.
- Precompute inv_ch in multi-channel decoding and inv_ratio in resample_linear to replace inner-loop floating-point divisions with multiplications.
- Fuse preemphasis directly into zero-padded STFT buffer in MelFrontend::compute, eliminating intermediate std::vector<double> allocation and memory pass.
- Specialized mag_power == 2.0 in MelKernel::frame_logmel to avoid std::pow and std::sqrt calls.

Assisted-by: Bolt:gpt-4o [Jules]

Co-authored-by: ha466 <58391348+ha466@users.noreply.github.com>
⚡ Bolt: optimize audio decoding, resampling, and mel frontend
…366913550101185774

🧹 support causal downsampling in subsampling tiling
…error-11459971171452585072

🧹 improve error reporting for streaming diarization mel shape validation
…84335626

Add Brotli support for 'Accept-Encoding'
…-3812223279566050645

🔒 Fix Insecure Deserialization via pickle in vad_bench fusion
…405533615837973

Refactor SSL initialization in ClientImpl request loop
…config-9897282533287482442

🧪 test: add error path test for invalid streaming diarization config
🔒 Fix insecure deserialization via pickle in noise_dive scripts
…s-12332688500804816982

⚡ pass json by const ref in named_profiles_json
httplib: check Content-Length and Transfer-Encoding in expect_content
…n-pickle-15119314714118661949

🔒 Fix insecure deserialization via pickle in seg_report.py
…8023172497908568

🔒 Fix insecure deserialization vulnerability in vad_bench rl.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant