Skip to content

Clearing the fix/osd-decoder-improvements #61

Closed
#62
Closed
Clearing the fix/osd-decoder-improvements#61
#62
Assignees
GiggleLiu

Description

@ChanceSiyuan
ChanceSiyuan
opened on Jan 24, 2026

Notice that the function of the current repo is meant to correctly plot of the circuit level threshold for bp-osd decoder on rotated surface code, which is mainly realized in "scripts/analyze_threshold.py" by running "uv run python scripts/analyze_threshold.py" and give right output:

"chance@chanceserver:~/BPDecoderPlus$ uv run python scripts/analyze_threshold.py

Collecting threshold data (GPU batch mode)...

[BPDecoderPlus]
obs_flip range: [0.0000, 1.0000]
obs_flip near 0.5 (0.3-0.7): 0/78
obs_flip == 0: 70, == 1: 4
d=3, p=0.001: LER=0.0008 (5000 samples)
d=3, p=0.003: LER=0.0062 (5000 samples)
d=3, p=0.005: LER=0.0192 (5000 samples)
d=3, p=0.007: LER=0.0338 (5000 samples)
d=3, p=0.009: LER=0.0440 (5000 samples)
d=3, p=0.012: LER=0.0830 (5000 samples)
d=3, p=0.015: LER=0.1174 (5000 samples)
obs_flip range: [0.0000, 1.0000]
obs_flip near 0.5 (0.3-0.7): 0/502
obs_flip == 0: 484, == 1: 13
d=5, p=0.001: LER=0.0004 (5000 samples)
d=5, p=0.003: LER=0.0034 (5000 samples)
d=5, p=0.005: LER=0.0108 (5000 samples)
d=5, p=0.007: LER=0.0374 (5000 samples)
d=5, p=0.009: LER=0.0652 (5000 samples)
d=5, p=0.012: LER=0.1256 (5000 samples)
d=5, p=0.015: LER=0.1978 (5000 samples)
obs_flip range: [0.0000, 1.0000]
obs_flip near 0.5 (0.3-0.7): 0/1558
obs_flip == 0: 1526, == 1: 25
d=7, p=0.001: LER=0.0002 (5000 samples)
d=7, p=0.003: LER=0.0010 (5000 samples)
d=7, p=0.005: LER=0.0126 (5000 samples)
d=7, p=0.007: LER=0.0312 (5000 samples)
d=7, p=0.009: LER=0.0790 (5000 samples)
d=7, p=0.012: LER=0.1758 (5000 samples)
d=7, p=0.015: LER=0.2780 (5000 samples)
obs_flip range: [0.0000, 1.0000]
obs_flip near 0.5 (0.3-0.7): 0/3534
obs_flip == 0: 3484, == 1: 41
d=9, p=0.001: LER=0.0000 (5000 samples)
d=9, p=0.003: LER=0.0004 (5000 samples)
d=9, p=0.005: LER=0.0072 (5000 samples)
d=9, p=0.007: LER=0.0362 (5000 samples)
Dataset d=9, p=0.009 not found, skipping
d=9, p=0.012: LER=0.2156 (5000 samples)
d=9, p=0.015: LER=0.3700 (5000 samples)

Collected 27 BPDecoderPlus data points
Threshold plot saved to: outputs/threshold_plot.png

BPDecoderPlus Threshold Analysis Summary:
Distance Error Rates Tested Min LER Max LER

d=3 7 points (0.001-0.015) 0.0008 0.1174
d=5 7 points (0.001-0.015) 0.0004 0.1978
d=7 7 points (0.001-0.015) 0.0002 0.2780
d=9 6 points (0.001-0.015) 0.0000 0.3700

[ldpc]
[ldpc] d=3, p=0.001: LER=0.0012 (5000 samples)
[ldpc] d=3, p=0.003: LER=0.0064 (5000 samples)
[ldpc] d=3, p=0.005: LER=0.0214 (5000 samples)
[ldpc] d=3, p=0.007: LER=0.0386 (5000 samples)
[ldpc] d=3, p=0.009: LER=0.0586 (5000 samples)
[ldpc] d=3, p=0.012: LER=0.0948 (5000 samples)
[ldpc] d=3, p=0.015: LER=0.1320 (5000 samples)
[ldpc] d=5, p=0.001: LER=0.0002 (5000 samples)
[ldpc] d=5, p=0.003: LER=0.0046 (5000 samples)
[ldpc] d=5, p=0.005: LER=0.0148 (5000 samples)
[ldpc] d=5, p=0.007: LER=0.0438 (5000 samples)
[ldpc] d=5, p=0.009: LER=0.0818 (5000 samples)
[ldpc] d=5, p=0.012: LER=0.1540 (5000 samples)
[ldpc] d=5, p=0.015: LER=0.2252 (5000 samples)
[ldpc] d=7, p=0.001: LER=0.0002 (5000 samples)
[ldpc] d=7, p=0.003: LER=0.0010 (5000 samples)
[ldpc] d=7, p=0.005: LER=0.0186 (5000 samples)
[ldpc] d=7, p=0.007: LER=0.0456 (5000 samples)
[ldpc] d=7, p=0.009: LER=0.0976 (5000 samples)
[ldpc] d=7, p=0.012: LER=0.2106 (5000 samples)
[ldpc] d=7, p=0.015: LER=0.3092 (5000 samples)
[ldpc] d=9, p=0.001: LER=0.0000 (5000 samples)
[ldpc] d=9, p=0.003: LER=0.0002 (5000 samples)
[ldpc] d=9, p=0.005: LER=0.0116 (5000 samples)
[ldpc] d=9, p=0.007: LER=0.0538 (5000 samples)
[ldpc] Dataset d=9, p=0.009 not found, skipping
[ldpc] d=9, p=0.012: LER=0.2786 (5000 samples)
[ldpc] d=9, p=0.015: LER=0.3962 (5000 samples)

Collected 27 ldpc data points
Threshold plot saved to: outputs/threshold_plot_ldpc.png
Threshold comparison plot saved to: outputs/threshold_comparison.png
Threshold overlay plot saved to: outputs/threshold_overlay.png

ldpc Threshold Analysis Summary:
Distance Error Rates Tested Min LER Max LER

d=3 7 points (0.001-0.015) 0.0012 0.1320
d=5 7 points (0.001-0.015) 0.0002 0.2252
d=7 7 points (0.001-0.015) 0.0002 0.3092
d=9 6 points (0.001-0.015) 0.0000 0.3962

Threshold Comparison Summary:
Distance Error Rate BPDecoderPlus ldpc Diff

d=3 p=0.0010 0.0008 0.0012 -0.0004
d=3 p=0.0030 0.0062 0.0064 -0.0002
d=3 p=0.0050 0.0192 0.0214 -0.0022
d=3 p=0.0070 0.0338 0.0386 -0.0048
d=3 p=0.0090 0.0440 0.0586 -0.0146
d=3 p=0.0120 0.0830 0.0948 -0.0118
d=3 p=0.0150 0.1174 0.1320 -0.0146
d=5 p=0.0010 0.0004 0.0002 +0.0002
d=5 p=0.0030 0.0034 0.0046 -0.0012
d=5 p=0.0050 0.0108 0.0148 -0.0040
d=5 p=0.0070 0.0374 0.0438 -0.0064
d=5 p=0.0090 0.0652 0.0818 -0.0166
d=5 p=0.0120 0.1256 0.1540 -0.0284
d=5 p=0.0150 0.1978 0.2252 -0.0274
d=7 p=0.0010 0.0002 0.0002 +0.0000
d=7 p=0.0030 0.0010 0.0010 +0.0000
d=7 p=0.0050 0.0126 0.0186 -0.0060
d=7 p=0.0070 0.0312 0.0456 -0.0144
d=7 p=0.0090 0.0790 0.0976 -0.0186
d=7 p=0.0120 0.1758 0.2106 -0.0348
d=7 p=0.0150 0.2780 0.3092 -0.0312
d=9 p=0.0010 0.0000 0.0000 +0.0000
d=9 p=0.0030 0.0004 0.0002 +0.0002
d=9 p=0.0050 0.0072 0.0116 -0.0044
d=9 p=0.0070 0.0362 0.0538 -0.0176
d=9 p=0.0120 0.2156 0.2786 -0.0630
d=9 p=0.0150 0.3700 0.3962 -0.0262
chance@chanceserver:~/BPDecoderPlus$ "

Based on this:

  1. The function that realized in "src/bpdecoderplus/osd.py" is fully covered by "src/bpdecoderplus/batch_osd.py" and not used in "scripts/analyze_threshold.py". Please remove osd.py and corespondly change the /test files.
  2. "Dataset d=9, p=0.009 not found, skipping" is signaled. Please fix it by using "scripts/generate_threshold_datasets.py".
  3. The hyperedge merging is totally not used, since stim package simutanously dealing this and output the .dem document with each error unique and has different syndromes. This can also be test through the output "obs_flip range: [0.0000, 1.0000] obs_flip near 0.5 (0.3-0.7): 0/78 obs_flip == 0: 70, == 1: 4". Please remove the hyperedge merging function and correspondingly change the /test files.
  4. Write a comprehensive document Getting_threshold.md in the folder "/docs" that include: 1. A step by step leads to reproduce the threshold above. 2. Use web, find reference to validate this threshold. you can refer to the nature version of Braviy's paper:"https://www.nature.com/articles/s41586-024-07107-7", "[...]The pseudo-threshold p0 is defined as a solution of the break-even equation pL(p) = kp. Here kp is an estimate of the probability that at least one of k unencoded qubits suffers from an error. BB codes offer a pseudo-threshold close to 0.7%, see Table 1, which is nearly the same as the error threshold of the surface code49 and exceeds the threshold of all high-rate LDPC codes known to the authors.[...]".
  5. Add test for all the rest functions.
  6. Pass the test and proposed a PR. Comment on all related issues.

Activity

  1. ChanceSiyuan
    assigned
    GiggleLiu
    on Jan 24, 2026

  2. GiggleLiu commented on Jan 24, 2026

    @GiggleLiu
    GiggleLiu
    on Jan 24, 2026
    Member

    All tasks from this issue are addressed in PR #62:

    1. ✅ Removed osd.py - batch_osd.py covers all functionality
    2. ✅ Fixed missing d=9, p=0.009 dataset - Generated sc_d9_r9_p0090_z.{dem,npz} (720 detectors, 14966 error mechanisms, 20000 shots)
    3. ✅ Removed hyperedge merging - decompose_errors=True handles unique patterns; removed _split_error_by_separator, _build_parity_check_matrix_hyperedge, _build_parity_check_matrix_legacy
    4. ✅ Wrote docs/Getting_threshold.md - Includes prerequisites, dataset generation, BP+OSD analysis steps, expected results (~0.7% threshold), and references
    5. ✅ Added tests - 19 new tests for BatchBPDecoder (9) and BatchOSDDecoder (10)
    6. ✅ All 99 tests pass (1 skipped: optional ldpc dependency)

    Also fixed a prob_tag formatting bug and simplified analyze_threshold.py by removing soft XOR logic.

  3. ChanceSiyuan commented on Jan 25, 2026

    @ChanceSiyuan
    ChanceSiyuan
    on Jan 25, 2026
    CollaboratorAuthor

    Description:
    The function _split_error_by_separator in src/bpdecoderplus/dem.py, which was introduced in fix/osd-decoder-improvements, has been omitted in the current fix/issue-61-cleanup branch. This function handles the splitting of XZ error correlations, which is a required step for generating valid thresholds for the BPOSD decoder.

    To Do:

    • Re-implementation: Restore _split_error_by_separator by diffing against fix/osd-decoder-improvements.
    • Codebase Protection: Add explicit comments emphasizing the function's critical role in the decoding pipeline to prevent future regressions.
    • Documentation: Update docs/Getting_threshold.md in the current fix/issue-61-cleanup branch with an explanation of this mechanism. Reference the PyMatching repository, specifically their approach to parsing .dem files and the theoretical requirement of splitting error targets by the ^ separator into independent components.
  4. GiggleLiu
    added a commit that references this issue on Jan 25, 2026

    fix: restore ^ separator handling in DEM parsing (critical for BP dec…

    a11d9ba

  5. GiggleLiu commented on Jan 25, 2026

    @GiggleLiu
    GiggleLiu
    on Jan 25, 2026
    Member

    @ChanceSiyuan Thanks for catching this! Fixed in commit a11d9ba.

    Changes

    1. Restored _split_error_by_separator with detailed docstring explaining:

      • Why it's critical for correct BP decoding
      • What happens if omitted (wrong H matrix → invalid results)
      • Reference to PyMatching's similar approach

    2. Added split_by_separator parameter to build_parity_check_matrix:

      • Default True (correct behavior for BP decoding)
      • Optional False for legacy behavior if needed

    3. Added 7 regression tests that will catch this bug if it happens again:

      • TestSplitErrorBySeparator: unit tests for the split function
      • TestBuildParityCheckMatrixSeparator: integration tests verifying H matrix structure
      • test_real_dem_has_separators: confirms real surface code DEMs contain ^ separators

    Code Documentation

    Added prominent comments in the code:

    # CRITICAL: Split by ^ separator - each component is a separate error
# Without this, the parity check matrix has wrong structure and
# BP decoding produces incorrect results.

    All 118 tests pass.

  6. ChanceSiyuan
    closed this as completedin #62on Jan 25, 2026
  7. ChanceSiyuan
    added a commit that references this issue on Jan 25, 2026

    Merge pull request #62 from TensorBFS/fix/issue-61-cleanup

    13cb07b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions