Changes 'next_errors' variable in main decoding loop to a linked list

[dandragona-dev]

• Prevents unnecessary vector copies as we can just rebuild the history when we need to.
• Also use arena allocator to prevent repeated heap allocation.

Tested with:

(venv13) dandragona@kaonashi:~/Documents/tesseract/indep/optimizations$ bazel-8.2.1 test --cache_test_results=no //src:all //src/py:all
INFO: Analyzed 24 targets (0 packages loaded, 23 targets configured).
INFO: Found 16 targets and 8 test targets...
INFO: Elapsed time: 6.390s, Critical Path: 5.97s
INFO: 9 processes: 42 action cache hit, 12 linux-sandbox, 2 local.
INFO: Build completed successfully, 9 total actions
//src:common_tests                                                       PASSED in 0.0s
//src:tesseract_tests                                                    PASSED in 0.7s
//src/py:common_test                                                     PASSED in 1.5s
//src/py:requirements_test                                               PASSED in 5.9s
//src/py:simplex_test                                                    PASSED in 1.6s
//src/py:tesseract_sinter_compat_test                                    PASSED in 4.6s
//src/py:tesseract_test                                                  PASSED in 1.5s
//src/py:utils_test                                                      PASSED in 1.0s

Executed 8 out of 8 tests: 8 tests pass.

Additional accuracy comparisons:

# d=11, p=0.001 Surface Code Transversal CNOT X
# baseline:
num_shots = 5000 num_low_confidence = 0 num_errors = 25 total_time_seconds = 364.1171770000001
# arena allocator + linked list optimization:
num_shots = 5000 num_low_confidence = 0 num_errors = 25 total_time_seconds = 253.3283590000004


# d=9, p=.002 Superdense Color Code X
# baseline:
num_shots = 5000 num_low_confidence = 0 num_errors = 64 total_time_seconds = 604.9724190000003
# arena allocator + linked list optimization:
num_shots = 5000 num_low_confidence = 0 num_errors = 64 total_time_seconds = 585.220854999997


# d=12, p=0.002 Bivariate Bicycle X
# baseline:
num_shots = 5000 num_low_confidence = 0 num_errors = 15 total_time_seconds = 3232.868145999989
# arena allocator + linked list optimization:
num_shots = 5000 num_low_confidence = 0 num_errors = 15 total_time_seconds = 3099.478287000002


# d=23, p=0.008 Surface Code Unrotated Memory Z
# baseline:
num_shots = 5000 num_low_confidence = 0 num_errors = 0 total_time_seconds = 803.0619609999993
# arena allocator + linked list optimization:
num_shots = 5000 num_low_confidence = 0 num_errors = 0 total_time_seconds = 142.359363

[LalehB]

Thank you so much @dandragona-dev, this is such a cool optimization! I added a few questions inline, but could you also add a few of your plots to the PR description showing the performance gains you got? (if you have them available, I really liked those plots.)

[LalehB]

LGTM!
Thank you @dandragona-dev

[noajshu]

This is very cool, thanks! I just have a question about the algorithm. Some error chains may end up dangling because there is no longer any node in the PQ with that chain. This could be due to the beam cutoff, for example. Is this correct? This should be fine in any case because the total amount of memory they consume should be controlled by the pqlimit. Just want to make sure I have the right understanding.

[dandragona-dev]

This is very cool, thanks! I just have a question about the algorithm. Some error chains may end up dangling because there is no longer any node in the PQ with that chain. This could be due to the beam cutoff, for example. Is this correct? This should be fine in any case because the total amount of memory they consume should be controlled by the pqlimit. Just want to make sure I have the right understanding.

I believe that sounds correct, but I will say that this PR should not introduce any change to states being explored by Tesseract. For instance, if the current error chain (out of 10) is {1,2,3} and we are considering adding {1,2,3,4}, {1,2,3,5}, ... {1,2,3,8} with the beam cutoff affecting {1,2,3,9} and {1,2,3,10}, then these last two states will be unexplored permanently IIUC (is this what you mean by dangling?).