pixel_to_real test

[Methasit-Pun]

Do this

python3 vision/pixel_to_real_world.py --ros-args -p real_hardware:=true

and

ros2 service call /pixel_to_real_world custom_interfaces/srv/PixelToReal "{u: 349, v: 306}"

[Methasit-Pun]

16 Feb Test

(u,v) = (43,160)
currentxy = 0.34 -0.44
correctxy = 0.3 -0.63

(u,v) = (43,292)
currentxy = 0.34 -0.61
correctxy = 0.3 -0.51

(u,v) = (174,292)
currentxy = 0.17 -0.606
correctxy = 0.2 -0.51

(u,v) = (449,268)
currentxy =-0.15 -0.57
correctxy = -0.09 -0.5

(u,v) = (449,268)
currentxy =-0.15 -0.57
correctxy = -0.09 -0.5

(u,v) = (579,162)
currentxy = -0.315 -0.45
correctxy = -0.2 -0.59

Total Samples: 6
Maximum Euclidean Error: 0.194 (at point 1)
Minimum Euclidean Error: 0.092 (at points 4 & 5)

Final RMSE: 0.1349 m

[Methasit-Pun]

test 2

(u,v) = (167,328)
currentxy = 0.188 -0.6451
correctxy = 0.188 -0.46

(u,v) = (289,446)
currentxy = 0.038 -0.79
correctxy = 0.09 -0.35

(u,v) = (445,255)
currentxy =-0.15 -0.52

(u,v) = (204,464)
currentxy = 0.144 -0.82
correctxy =0.16 -0.32

(u,v) = (320,464)
currentxy = 0.0006 -0.82
correctxy = 0.05 -0.32

Total Valid Samples: 5

Maximum Euclidean Error: 0.502 (at point 320, 464)

Minimum Euclidean Error: 0.185 (at point 167, 328)

$\Delta x$ = $-0.0494$
$\Delta y$ = $-0.5000$

[Methasit-Pun]

25 Feb 26

(u,v) = (355,305)
currentxy = 0.022 0.32
correctxy = 0.08 0.46

(u,v) = (479,444)
currentxy = 0.2 0.082
correctxy = 0.19 0.36

(u,v) = (347,222)
currentxy = 0.013 0.48
correctxy = 0.06 0.57

(u,v) = (475,106)
currentxy = 0.186 0.7086
correctxy = 0.16 0.67

(u,v) = (110,122)
currentxy = -0.315 0.67
correctxy = -0.19 0.67

(u,v) = (81,334)
currentxy = -0.366 0.265
correctxy = -0.21 0.35

RMSE_X =0.012292 m
RMSE_Y =0.039288 m
total RMSE to approximately 0.0316 m

[Methasit-Pun]

Better improvement

• Perspective-n-Point (PnP) with ChArUco Boards
• Hand-Eye Calibration (Tsai-Lenz Algorithm)

[Methasit-Pun]

Mar 2

(u,v) = (423,286)
currentxy = 0.109 0.512
correctxy = 0.111 0.495

(u,v) = (549,393)
currentxy = 0.222 0.413
correctxy = 0.221 0.410

(u,v) = (556,182)
currentxy = 0.230 0.615
correctxy = 0.221 0.622

(u,v) = (427,62)
currentxy = 0.107 0.732
correctxy = 0.105 0.730

(u,v) = (190,213)
currentxy = -0.127 0.58
correctxy = -0.135 0.590

(u,v) = (62,97)
currentxy = -0.236 0.691
correctxy = -0.240 0.690

(u,v) = (53,451)
currentxy = -0.241 0.365
correctxy = -0.255 0.356

(u,v) = (181,334)
currentxy = -0.137 0.465
correctxy = -0.156 0.455

(u,v) = (350,480)
currentxy = 0.033 0.322
correctxy = 0.025 0.321

(u,v) = (406,283)
currentxy = 0.091 0.516
correctxy = 0.080 0.515

[Methasit-Pun]

before grasp
matplotlib 3.10.8
matplotlib-inline 0.2.1
numpy 1.26.4
open3d 0.19.0
scikit-learn 1.7.2
scipy 1.15.3
tqdm 4.67.1

[Methasit-Pun]

after grasp
h5py 3.15.1
matplotlib 3.10.8
matplotlib-inline 0.2.1
numpy 1.26.4
open3d 0.19.0
scikit-image 0.25.2
scikit-learn 1.7.2
scipy 1.15.3
tqdm 4.67.1
transforms3d 0.4.2
trimesh 4.11.2

[Methasit-Pun]

height 24 inch
avg_height_cal = 0.169 m

pink TL BR
(u,v) = (75, 299)
currentxy = -0.185 0.511
correctxy = -0.2070.495

(u,v) = (263,462)
currentxy = -0.041 0.380
correctxy = 0.93 0.394

green TL BR
(u,v) = (426,279)
currentxy = 0.090 0.55
correctxy = 0.068 0.51

(u,v) = (573,440)
currentxy = 0.189 0.410
correctxy = 0.170 0.405

pink BL TR
(u,v) = (81,245)
currentxy = -0.180 0.552
correctxy = -0.194 0.563

(u,v) = (266,83)
currentxy = -0.039 0.679
correctxy = -0.081 0.66

green TR
(u,v) = (640,65)
currentxy = 0.225 0.676
correctxy = 0.210 0.680

(u,v) = (344,311)
currentxy = 0.02 0.49
correctxy = -0.01 0.49

BL
(u,v) = (506,389)
currentxy = 0.149 0.439
correctxy = 0.131 0.52

TR
(u,v) = (271,287)
currentxy = -0.037 0.518
correctxy = -0.076 0.52

[Methasit-Pun]

9 Mar Benchmark

Height = 0.66

TR
(u,v) = (623,277)
currentxy = 0.176 0.531
correctxy = 0.249 0.485

---

TR
(u,v) = (635 , 169)
currentxy = 0.07 0.63
correctxy = 0.255 0.650

---

TR
(u,v) = (314,302)
currentxy = -0.11 0.501
correctxy = -0.07 0.471

BL
(u,v) = (197, 415)
currentxy = -0.231 0.390
correctxy = -0.176 0.370

---

TR
(u,v) = (161, 45)
currentxy = -0.23 0.73
correctxy = -0.263 0.75

BL
(u,v) = (36,161)
currentxy =-0.338 0.636
correctxy = -0.379 0.641

--

TR
(u,v) = (436,180)
currentxy = 0.01 0.627
correctxy = 0.067 0.607

BL
(u,v) = (318,289)
currentxy = -0.11 0.515
correctxy = -0.051 0.492

[Methasit-Pun]

New Calibration
Height = 0.56 m

BL
(u,v) = (108,311)
currentxy = -0.27 0.469
correctxy = -0.28 0.47

---

Pink

TR
(u,v) = (606,179)
currentxy =
correctxy = 0.229 0.313

BL
(u,v) = (460,308)
currentxy =
correctxy = 0.091 0.475

---

Green

TR
(u,v) = (175,331)
currentxy = -0.21 0.431
correctxy = -0.207 0.449

BL
(u,v) = (50,445)
currentxy = -0.322 0.34
correctxy = -0.326 0.325

--

PINK

TR
(u,v) = (549,173)
currentxy = 0.168 0.622
correctxy = 0.146 0.6

BL
(u,v) = (419,301)
currentxy = 0.0464 0.482
correctxy = 0.042 0.48

---

#5
TR
(u,v) = (469,144)
currentxy = 0.07 0.633
correctxy = 0.079 0.654

BL
(u,v) = (338,284)
currentxy = -0.035 0.515
correctxy = -0.041 0.501

---

#6
TR
(u,v) = (310,297)
currentxy = -0.073 -0.472
correctxy = -0.069 0.3487

BL
(u,v) = (193,410)
currentxy = -0.18 0.373
correctxy = -0.179 0.363

[Methasit-Pun]

45 cm

Green
TR
(u,v) = (243,270)
currentxy = -0.073 0.472
correctxy = -0.096 0.413

BL
(u,v) = (71,428)
currentxy = -0.18 0.373
correctxy = -0.207 0.420

---

Pink
TR
(u,v) = (577,100)
currentxy = 0.117 0.615
correctxy =0.188 0.702

BL
(u,v) = (400,291)
currentxy = 0.02 0.496
correctxy = 0.0367 0.494

---

Green
TR
(u,v) = (571,62)
currentxy = 0.1776 0.7441
correctxy = 0.13 0.649

BL
(u,v) = (398, 217)
currentxy = 0.014 0.575
correctxy = 0.021 0.554

Pink
TR
(u,v) = (307,269)
currentxy = -0.076 0.517
correctxy = -0.042 0.52

BL
(u,v) = (143,285)
currentxy = -0.247 0.499
correctxy = -0.145 0.40

[Methasit-Pun]

Empirical > Intrinsics-based for your setup because:

✅ Empirical: 0.017m RMS (directly measured)
❌ Intrinsics: 0.085m RMS (theory-dependent)

[Methasit-Pun]

12 Mar 2026

Cam_coordinate = [-0.0386, 0.5303, 0.5238]
Note: We can calibrate based on center pixel

Measure_1
TR
(u,v) = (357,107)
currentxy = -0.006 0.65 -0.019
correctxy = -0.04 0.67

BL
(u,v) = (231,246)
currentxy = -0.117 0.538 -0.021
correctxy = -0.14 0.57

---

Measure_2
TR
(u,v) = (598,173)
currentxy = 0.186 0.596 -0.018
correctxy = 0.16 0.606

BL
(u,v) = (455,305)
currentxy = 0.079 0.487 -0.021
correctxy = 0.065 0.506

---

Measure_3
TR
(u,v) = (431,147)
currentxy = 0.058 0.624 -0.019
correctxy = 0.038 0.611

BL
(u,v) = (301,285)
currentxy = -0.055 0.504 0.020
correctxy = -0.056 0.511

---

Measure_4
TR
(u,v) = (298,138)
currentxy = -0.057 0.632 -0.0181
correctxy = -0.084 0.630

BL
(u,v) = (169,275)
currentxy = -0.169 0.513 -0.019
correctxy = -0.176 0.525

---

Measure_5
TR
(u,v) = (286,318)
currentxy = -0.064 0.474 -0.025
correctxy = -0.096 0.472

BL
(u,v) = (158,460)
currentxy = -0.174 0.358 -0.032
correctxy = -0.186 0.370

---

Measure_6
TR
(u,v) = (170,204)
currentxy = -0.168 0.571-0.021
correctxy = -0.195 0.575

BL
(u,v) = (36,334)
currentxy = -0.269 0.466 -0.027
correctxy = -0.295 0.48

---

Measure_7
TR
(u,v) = (569,209)
currentxy = 0.167 0.568 -0.019
correctxy = 0.153 0.570

BL
(u,v) = (428,348)
currentxy = 0.056 0.449 -0.0242
correctxy = 0.059 0.470

---

Measure_8
TR
(u,v) = (589,92)
currentxy = 0.175 0.659 -0.018
correctxy = 0.161 0.662

BL
(u,v) = (458,233)
currentxy = 0.081 0.549
correctxy = 0.068 0.562

---

Measure_9
TR
(u,v) = (454,188)
currentxy = 0.075 0.5885 -0.018
correctxy = 0.064 0.585

BL
(u,v) = (322,324)
currentxy = -0.036 0.469 -0.018
correctxy = -0.04 0.485

---

Measure_10
TR
(u,v) = (320,196)
currentxy = -0.037 0.583
correctxy = -0.025 0.583

BL
(u,v) = (232,332)
currentxy = -0.115 0.463
correctxy = -0.125 0.48

[Methasit-Pun]

12 Mar
Cam_coordinate = [-0.0386, 0.5303, 0.5238]
branch benchmark-emp

Measure_1
TR
(u,v) = (357,107)
currentxy = -0.033 0.657
correctxy = -0.04 0.67

BL
(u,v) = (231,246)
currentxy = -0.131 0.547
correctxy = -0.14 0.57

---

Measure_2
TR
(u,v) = (598,173)
currentxy = 0.179 0.602
correctxy = 0.16 0.606

BL
(u,v) = (455,305)
currentxy = 0.066 0.497
correctxy = 0.065 0.506

---

Measure_3
TR
(u,v) = (431,147)
currentxy = 0.034 0.624
correctxy = 0.038 0.611

BL
(u,v) = (301,285)
currentxy = -0.067 0.515
correctxy = -0.056 0.511

---

Measure_4
TR
(u,v) = (298,138)
currentxy = -0.0813 0.633
correctxy = -0.084 0.630

BL
(u,v) = (169,275)
currentxy = -0.181 0.524
correctxy = -0.176 0.525

---

Measure_5
TR
(u,v) = (286,318)
currentxy = -0.078 0.489
correctxy = -0.096 0.472

BL
(u,v) = (158,460)
currentxy = -0.177 0.376
correctxy = -0.186 0.370

---

Measure_6
TR
(u,v) = (170,204)
currentxy = -0.186 0.581
correctxy = -0.195 0.575

BL
(u,v) = (36,334)
currentxy = -0.291 0.478
correctxy = -0.295 0.48

---

Measure_7
TR
(u,v) = (569,209)
currentxy = 0.156 0.573
correctxy = 0.153 0.570

BL
(u,v) = (428,348)
currentxy = 0.046 0.464
correctxy = 0.059 0.470

---

Measure_8
TR
(u,v) = (589,92)
currentxy = 0.165 0.667
correctxy = 0.161 0.662

BL
(u,v) = (458,233)
currentxy = 0.063 0.555
correctxy = 0.068 0.562

---

Measure_9
TR
(u,v) = (454,188)
currentxy = 0.056 0.591
correctxy = 0.064 0.585

BL
(u,v) = (322,324)
currentxy = -0.046 0.483
correctxy = -0.040 0.485

---

Measure_10
TR
(u,v) = (320,196)
currentxy = -0.05 0.586
correctxy = -0.025 0.583

BL
(u,v) = (232,332)
currentxy = -0.123 0.478
correctxy = -0.125 0.48