Enhanced certification test: now results match analytical solution

[mdmitry1]

Added SMLP certify test for BNH problem

[mdmitry1]

SMLP results are verified by:

1. Analytical solution
2. Wolfram Alpha

[mdmitry1]

Added certify result visualization

[konstantin-korovin]

Good work!

I tried on MacOS the bhn example with certification, have some errors:

tutorial % examples/bnh/smlp/run_certify.sh
examples/bnh/smlp/run_certify.sh: line 14: ${test,,}: bad substitution
=============================================== PROBLEM ======================================================
For function f₁(x₁,x₂) = 4x₁² + 4x₂²                                                                          
Certify stability of analytical Pareto solution for witness f₁(x₁,x₂) = 0.692042, where  x₁ = x₂ = 0.294118 
And specified absolute radius rad-abs                                                                         
----------------------------------------------- Expected result: ---------------------------------------------
From inequality: (f₁(x₁,x₂) - 0.692042)² < 4                                                                  
| x₁ | = | x₂ |  < sqrt((2+0.692042)/8) = 0.580091                                                            
Pass/Fail absolute radius boundary point: 0.580091-0.294118 = 0.285973                                        
Checking two values of rad-abs: 0.285 and 0.286                                                               
Expected result: 0.285 -> PASS, 0.286 -> FAIL                                                                 
==============================================================================================================
=============================================== SMLP CERTIFY SOLUTIONS =======================================
gunzip: can't stat: .csv.gz (.csv.gz.gz): No such file or directory
jq: error: Could not open file /Users/korovin/smlp/tutorial/examples/bnh/smlp/_certify_pass.json: No such file or directory

jq: error: Could not open file BNH_BNH_certify_results.json: No such file or directory
(F1-0.692042)*(F1-0.692042) < 4 
jq: error: Could not open file /Users/korovin/smlp/tutorial/examples/bnh/smlp/_certify_fail.json: No such file or directory

jq: error: Could not open file BNH_BNH_certify_results.json: No such file or directory
(F1-0.692042)*(F1-0.692042) < 4 
=============================================== DONE ========================================================
Saved witness_certify.png

===========
Interestingly the generated png file looks ok.

[konstantin-korovin]

On my Ubuntu 22.04 laptop I have another error:

tutorial$ examples/bnh/smlp/run_certify.sh
=============================================== PROBLEM ======================================================
For function f₁(x₁,x₂) = 4x₁² + 4x₂²                                                                          
Certify stability of analytical Pareto solution for witness f₁(x₁,x₂) = 0.692042, where  x₁ = x₂ = 0.294118
And specified absolute radius rad-abs                                                                        
----------------------------------------------- Expected result: ---------------------------------------------
From inequality: (f₁(x₁,x₂) - 0.692042)² < 4                                                                  
| x₁ | = | x₂ |  < sqrt((2+0.692042)/8) = 0.580091                                                            
Pass/Fail absolute radius boundary point: 0.580091-0.294118 = 0.285973                                        
Checking two values of rad-abs: 0.285 and 0.286                                                              
Expected result: 0.285 -> PASS, 0.286 -> FAIL                                                                
==============================================================================================================
=============================================== SMLP CERTIFY SOLUTIONS =======================================
jq: error: syntax error, unexpected ',', expecting ':' (Unix shell quoting issues?) at <top-level>, line 1:
[.variables[] | select(has("rad-abs")) | {label, "rad-abs"}]                                              
jq: 1 compile error

(F1-0.692042)*(F1-0.692042) < 4 "PASS"
jq: error: syntax error, unexpected ',', expecting ':' (Unix shell quoting issues?) at <top-level>, line 1:
[.variables[] | select(has("rad-abs")) | {label, "rad-abs"}]                                              
jq: 1 compile error

(F1-0.692042)*(F1-0.692042) < 4 "FAIL"
=============================================== DONE ========================================================
Saved witness_certify.png

File witness_certify.png looks ok.

[konstantin-korovin]

Before that:

# Generate dataset
./examples/bnh/smlp/run_poly_pareto/bnh_dataset.py

should be

# Generate dataset
./examples/bnh/smlp/bnh_dataset.py

Which I took into account in above.

[konstantin-korovin]

Updates fixed the issue on both MacOS and Ubuntu for me.
Minor: reduce size of the certification figure in tutorial/README.md

For this or next PR: dump results and data into a results/ data/ directories rather than directly to ./

[mdmitry1]

Merged with master

[mdmitry1]

All requested changes were implemented

[mdmitry1]

@fbrausse : both requested changes are marked as resolved. However, GitHub status is "requested changes". Please, fix

Requests have been addressed.

[fbrausse]

I've run the BNH example modified here and results and plots look nice.