Skip to content

Add automated port conformance status page#5389

Open
shai-almog wants to merge 21 commits into
masterfrom
automated-port-status-page
Open

Add automated port conformance status page#5389
shai-almog wants to merge 21 commits into
masterfrom
automated-port-status-page

Conversation

@shai-almog

Copy link
Copy Markdown
Collaborator

Summary

  • add a public /port-status/ page driven by the HelloCodenameOne conformance suite
  • map every registered test and screenshot golden to 49 user-facing feature rows across 10 portability targets
  • normalize per-port CI results and publish the latest reports to a data-only branch consumed by the website
  • validate the mapping contract whenever tests, goldens, or the contract change
  • exclude JavaSE because it is the simulator/development runtime, not a portability target

Validation

  • python3 scripts/hellocodenameone/conformance/port_status.py validate
  • python3 -m unittest -v test_port_status.py from scripts/hellocodenameone/conformance
  • node --check docs/website/assets/js/cn1-port-status.js
  • shell syntax checks for the affected runners and publisher
  • YAML parsing for all affected workflows
  • production Hugo build
  • rendered-page check: 49 feature rows, 10 port cards, 490 matrix cells
  • git diff --check

Comment thread scripts/hellocodenameone/conformance/port_status.py Fixed
@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 12 screenshots: 12 matched.
✅ JavaSE simulator integration screenshots matched stored baselines.

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

✅ Continuous Quality Report

Test & Coverage

Static Analysis

  • SpotBugs [Report archive]
    • ByteCodeTranslator: 0 findings (no issues)
    • android: 0 findings (no issues)
    • codenameone-maven-plugin: 0 findings (no issues)
    • core-unittests: 0 findings (no issues)
    • ios: 0 findings (no issues)
  • PMD: 0 findings (no issues) [Report archive]
  • Checkstyle: 0 findings (no issues) [Report archive]

Generated automatically by the PR CI workflow.

@github-actions

Copy link
Copy Markdown
Contributor

Cloudflare Preview

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 149 screenshots: 149 matched.

Native Android coverage

  • 📊 Line coverage: 11.35% (11878/104611 lines covered) [HTML preview] (artifact android-coverage-report, jacocoAndroidReport/html/index.html)
    • Other counters: instruction 10.13% (58688/579445), branch 5.23% (2768/52952), complexity 4.99% (2771/55527), method 7.46% (2133/28582), class 11.71% (476/4064)
    • Lowest covered classes
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysKt – 0.00% (0/6327 lines covered)
      • kotlin.collections.unsigned.kotlin.collections.unsigned.UArraysKt___UArraysKt – 0.00% (0/2384 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader – 0.00% (0/1519 lines covered)
      • kotlin.collections.kotlin.collections.CollectionsKt___CollectionsKt – 0.00% (0/1148 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.MethodWriter – 0.00% (0/923 lines covered)
      • kotlin.sequences.kotlin.sequences.SequencesKt___SequencesKt – 0.00% (0/730 lines covered)
      • com.google.common.cache.com.google.common.cache.LocalCache$Segment – 0.00% (0/726 lines covered)
      • kotlin.text.kotlin.text.StringsKt___StringsKt – 0.00% (0/623 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.Frame – 0.00% (0/564 lines covered)
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysJvmKt – 0.00% (0/495 lines covered)

✅ Native Android screenshot tests passed.

Native Android coverage

  • 📊 Line coverage: 11.35% (11878/104611 lines covered) [HTML preview] (artifact android-coverage-report, jacocoAndroidReport/html/index.html)
    • Other counters: instruction 10.13% (58688/579445), branch 5.23% (2768/52952), complexity 4.99% (2771/55527), method 7.46% (2133/28582), class 11.71% (476/4064)
    • Lowest covered classes
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysKt – 0.00% (0/6327 lines covered)
      • kotlin.collections.unsigned.kotlin.collections.unsigned.UArraysKt___UArraysKt – 0.00% (0/2384 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.ClassReader – 0.00% (0/1519 lines covered)
      • kotlin.collections.kotlin.collections.CollectionsKt___CollectionsKt – 0.00% (0/1148 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.MethodWriter – 0.00% (0/923 lines covered)
      • kotlin.sequences.kotlin.sequences.SequencesKt___SequencesKt – 0.00% (0/730 lines covered)
      • com.google.common.cache.com.google.common.cache.LocalCache$Segment – 0.00% (0/726 lines covered)
      • kotlin.text.kotlin.text.StringsKt___StringsKt – 0.00% (0/623 lines covered)
      • org.jacoco.agent.rt.internal_b6258fc.asm.org.jacoco.agent.rt.internal_b6258fc.asm.Frame – 0.00% (0/564 lines covered)
      • kotlin.collections.kotlin.collections.ArraysKt___ArraysJvmKt – 0.00% (0/495 lines covered)

Benchmark Results

Detailed Performance Metrics

Metric Duration
SIMD kernel backend scalar fallback (no native SIMD)
SIMD int-add (64K x300) java 145ms / native 60ms = 2.4x speedup
SIMD float-mul (64K x300) java 120ms / native 48ms = 2.5x speedup
SIMD kernel correctness PASS (native result == scalar reference)
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 SIMD byte path gated to scalar (CPU autovectorizes scalar; explicit SIMD not beneficial here)
Base64 CN1 encode 60.000 ms
Base64 CN1 decode 68.000 ms
Base64 native encode 362.000 ms
Base64 encode ratio (CN1/native) 0.166x (83.4% faster)
Base64 native decode 335.000 ms
Base64 decode ratio (CN1/native) 0.203x (79.7% faster)
Image encode benchmark status skipped (SIMD unsupported)

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 145 screenshots: 145 matched.
Native Windows port, REAL shipping pipeline: the hellocodenameone screenshot suite rendered by a binary CROSS-COMPILED on Linux (clang-cl + xwin, WebView2 linked) and RUN on a Windows x64 runner. Compared against the in-repo baseline in scripts/windows/screenshots.

Benchmark Results

Detailed Performance Metrics

Metric Duration
SIMD kernel backend SSE2 (x64) / NEON (arm64) native kernels
SIMD int-add (64K x300) java 64ms / native 3ms = 21.3x speedup
SIMD float-mul (64K x300) java 73ms / native 4ms = 18.2x speedup
SIMD kernel correctness PASS (native result == scalar reference)

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 146 screenshots: 146 matched.
Native Linux port (x64), GTK3/Cairo/Pango, ParparVM bytecode-to-C (no JVM): the hellocodenameone screenshot suite rendered by a native ELF built + run on the GitHub x64 runner. Baseline: scripts/linux/screenshots.

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 176 screenshots: 176 matched.
✅ JavaScript-port screenshot tests passed.

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 146 screenshots: 146 matched.
Native Linux port (arm64), GTK3/Cairo/Pango, ParparVM bytecode-to-C (no JVM): the hellocodenameone screenshot suite rendered by a native ELF built + run on the GitHub arm64 runner. Baseline: scripts/linux/screenshots-arm.

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 147 screenshots: 147 matched.
✅ Native Mac screenshot tests passed.

Benchmark Results

  • VM Translation Time: 0 seconds
  • Compilation Time: 286 seconds

Detailed Performance Metrics

Metric Duration
SIMD kernel backend SSE2 (x64) / NEON (arm64) native kernels
SIMD int-add (64K x300) java 56ms / native 3ms = 18.6x speedup
SIMD float-mul (64K x300) java 55ms / native 3ms = 18.3x speedup
SIMD kernel correctness PASS (native result == scalar reference)
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 SIMD byte path active (NEON-accelerated)
Base64 CN1 encode 172.000 ms
Base64 CN1 decode 117.000 ms
Base64 native encode 527.000 ms
Base64 encode ratio (CN1/native) 0.326x (67.4% faster)
Base64 native decode 208.000 ms
Base64 decode ratio (CN1/native) 0.563x (43.8% faster)
Base64 SIMD encode 48.000 ms
Base64 encode ratio (SIMD/CN1) 0.279x (72.1% faster)
Base64 SIMD decode 44.000 ms
Base64 decode ratio (SIMD/CN1) 0.376x (62.4% faster)
Base64 encode ratio (SIMD/native) 0.091x (90.9% faster)
Base64 decode ratio (SIMD/native) 0.212x (78.8% faster)
Image encode benchmark iterations 100
Image createMask (SIMD off) 7.000 ms
Image createMask (SIMD on) 3.000 ms
Image createMask ratio (SIMD on/off) 0.429x (57.1% faster)
Image applyMask (SIMD off) 53.000 ms
Image applyMask (SIMD on) 45.000 ms
Image applyMask ratio (SIMD on/off) 0.849x (15.1% faster)
Image modifyAlpha (SIMD off) 39.000 ms
Image modifyAlpha (SIMD on) 29.000 ms
Image modifyAlpha ratio (SIMD on/off) 0.744x (25.6% faster)
Image modifyAlpha removeColor (SIMD off) 39.000 ms
Image modifyAlpha removeColor (SIMD on) 51.000 ms
Image modifyAlpha removeColor ratio (SIMD on/off) 1.308x (30.8% slower)

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Apple TV (tvOS / Metal)

Compared 143 screenshots: 141 matched, 2 missing actuals.

  • DesktopMode — missing actual screenshot. Actual screenshot missing (test did not produce output).

    No preview available for this screenshot.

  • Media360Panorama — missing actual screenshot. Actual screenshot missing (test did not produce output).

    No preview available for this screenshot.

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Apple Watch (watchOS / Core Graphics)

Compared 216 screenshots: 215 matched, 1 missing actual.

  • DesktopMode — missing actual screenshot. Actual screenshot missing (test did not produce output).

    No preview available for this screenshot.

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 142 screenshots: 142 matched.
✅ Native iOS screenshot tests passed.

Benchmark Results

  • VM Translation Time: 0 seconds
  • Compilation Time: 391 seconds

Build and Run Timing

Metric Duration
Simulator Boot 95000 ms
Simulator Boot (Run) 1000 ms
App Install 13000 ms
App Launch 3000 ms
Test Execution 1068000 ms

Detailed Performance Metrics

Metric Duration
SIMD kernel backend SSE2 (x64) / NEON (arm64) native kernels
SIMD int-add (64K x300) java 161ms / native 5ms = 32.2x speedup
SIMD float-mul (64K x300) java 172ms / native 2ms = 86.0x speedup
SIMD kernel correctness PASS (native result == scalar reference)
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 SIMD byte path active (NEON-accelerated)
Base64 CN1 encode 689.000 ms
Base64 CN1 decode 269.000 ms
Base64 native encode 2348.000 ms
Base64 encode ratio (CN1/native) 0.293x (70.7% faster)
Base64 native decode 924.000 ms
Base64 decode ratio (CN1/native) 0.291x (70.9% faster)
Base64 SIMD encode 64.000 ms
Base64 encode ratio (SIMD/CN1) 0.093x (90.7% faster)
Base64 SIMD decode 95.000 ms
Base64 decode ratio (SIMD/CN1) 0.353x (64.7% faster)
Base64 encode ratio (SIMD/native) 0.027x (97.3% faster)
Base64 decode ratio (SIMD/native) 0.103x (89.7% faster)
Image encode benchmark iterations 100
Image createMask (SIMD off) 11.000 ms
Image createMask (SIMD on) 12.000 ms
Image createMask ratio (SIMD on/off) 1.091x (9.1% slower)
Image applyMask (SIMD off) 128.000 ms
Image applyMask (SIMD on) 57.000 ms
Image applyMask ratio (SIMD on/off) 0.445x (55.5% faster)
Image modifyAlpha (SIMD off) 80.000 ms
Image modifyAlpha (SIMD on) 78.000 ms
Image modifyAlpha ratio (SIMD on/off) 0.975x (2.5% faster)
Image modifyAlpha removeColor (SIMD off) 59.000 ms
Image modifyAlpha removeColor (SIMD on) 44.000 ms
Image modifyAlpha removeColor ratio (SIMD on/off) 0.746x (25.4% faster)

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 148 screenshots: 148 matched.
✅ Native iOS Metal screenshot tests passed.

Benchmark Results

  • VM Translation Time: 0 seconds
  • Compilation Time: 409 seconds

Build and Run Timing

Metric Duration
Simulator Boot 107000 ms
Simulator Boot (Run) 0 ms
App Install 15000 ms
App Launch 1000 ms
Test Execution 733000 ms

Detailed Performance Metrics

Metric Duration
SIMD kernel backend SSE2 (x64) / NEON (arm64) native kernels
SIMD int-add (64K x300) java 59ms / native 3ms = 19.6x speedup
SIMD float-mul (64K x300) java 67ms / native 3ms = 22.3x speedup
SIMD kernel correctness PASS (native result == scalar reference)
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 SIMD byte path active (NEON-accelerated)
Base64 CN1 encode 157.000 ms
Base64 CN1 decode 153.000 ms
Base64 native encode 269.000 ms
Base64 encode ratio (CN1/native) 0.584x (41.6% faster)
Base64 native decode 360.000 ms
Base64 decode ratio (CN1/native) 0.425x (57.5% faster)
Base64 SIMD encode 51.000 ms
Base64 encode ratio (SIMD/CN1) 0.325x (67.5% faster)
Base64 SIMD decode 49.000 ms
Base64 decode ratio (SIMD/CN1) 0.320x (68.0% faster)
Base64 encode ratio (SIMD/native) 0.190x (81.0% faster)
Base64 decode ratio (SIMD/native) 0.136x (86.4% faster)
Image encode benchmark iterations 100
Image createMask (SIMD off) 7.000 ms
Image createMask (SIMD on) 2.000 ms
Image createMask ratio (SIMD on/off) 0.286x (71.4% faster)
Image applyMask (SIMD off) 44.000 ms
Image applyMask (SIMD on) 33.000 ms
Image applyMask ratio (SIMD on/off) 0.750x (25.0% faster)
Image modifyAlpha (SIMD off) 35.000 ms
Image modifyAlpha (SIMD on) 30.000 ms
Image modifyAlpha ratio (SIMD on/off) 0.857x (14.3% faster)
Image modifyAlpha removeColor (SIMD off) 36.000 ms
Image modifyAlpha removeColor (SIMD on) 30.000 ms
Image modifyAlpha removeColor ratio (SIMD on/off) 0.833x (16.7% faster)

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 145 screenshots: 145 matched.
Native Windows port (x64 / Intel-AMD): full hellocodenameone screenshot suite rendered offscreen with Direct2D/DirectWrite, plus the real benchmarks (base64 native/CN1/SIMD, image createMask/applyMask/modifyAlpha/PNG/JPEG, SSE2 SIMD kernels). Compared against the in-repo baseline in scripts/windows/screenshots.

Benchmark Results

Detailed Performance Metrics

Metric Duration
SIMD kernel backend SSE2 (x64) / NEON (arm64) native kernels
SIMD int-add (64K x300) java 60ms / native 4ms = 15.0x speedup
SIMD float-mul (64K x300) java 68ms / native 4ms = 17.0x speedup
SIMD kernel correctness PASS (native result == scalar reference)

@shai-almog

shai-almog commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Compared 145 screenshots: 145 matched.
Native Windows port (arm64 / Apple Silicon - Arm): full hellocodenameone screenshot suite rendered offscreen with Direct2D/DirectWrite, plus the real benchmarks (base64 native/CN1/SIMD, image createMask/applyMask/modifyAlpha/PNG/JPEG, NEON SIMD kernels). Compared against the in-repo baseline in scripts/windows/screenshots.

Benchmark Results

Detailed Performance Metrics

Metric Duration
SIMD kernel backend SSE2 (x64) / NEON (arm64) native kernels
SIMD int-add (64K x300) java 56ms / native 3ms = 18.6x speedup
SIMD float-mul (64K x300) java 55ms / native 3ms = 18.3x speedup
SIMD kernel correctness PASS (native result == scalar reference)

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

✅ ByteCodeTranslator Quality Report

Test & Coverage

  • Tests: 395 total, 0 failed, 14 skipped

Benchmark Results

  • Execution Time: 20599 ms

  • Hotspots (Top 20 sampled methods):

    • 19.91% java.util.ArrayList.indexOf (367 samples)
    • 9.17% com.codename1.tools.translator.BytecodeMethod.addToConstantPool (169 samples)
    • 3.64% com.codename1.tools.translator.ByteCodeClass.hasDeclaredMethod (67 samples)
    • 3.58% java.lang.StringBuilder.append (66 samples)
    • 3.20% com.codename1.tools.translator.Parser.generateClassAndMethodIndexHeader (59 samples)
    • 2.93% com.codename1.tools.translator.Parser.cn1EnsureSubclassIndex (54 samples)
    • 2.22% com.codename1.tools.translator.BytecodeMethod.equals (41 samples)
    • 2.17% com.codename1.tools.translator.Parser.classIndex (40 samples)
    • 2.12% org.objectweb.asm.tree.analysis.Analyzer.analyze (39 samples)
    • 2.06% org.objectweb.asm.tree.analysis.Analyzer.findSubroutine (38 samples)
    • 1.79% com.codename1.tools.translator.bytecodes.Invoke.resolveDirectTarget (33 samples)
    • 1.41% java.util.HashMap.hash (26 samples)
    • 1.36% java.lang.Object.hashCode (25 samples)
    • 1.36% com.codename1.tools.translator.BytecodeMethod.optimize (25 samples)
    • 1.36% org.objectweb.asm.ClassReader.readCode (25 samples)
    • 1.30% java.lang.String.equals (24 samples)
    • 1.09% java.lang.System.identityHashCode (20 samples)
    • 1.09% com.codename1.tools.translator.BytecodeMethod.appendCMethodPrefix (20 samples)
    • 0.98% sun.nio.fs.UnixNativeDispatcher.open0 (18 samples)
    • 0.92% com.codename1.tools.translator.BytecodeMethod.addInstruction (17 samples)
  • ⚠️ Coverage report not generated.

Static Analysis

  • ✅ SpotBugs: no findings (report was not generated by the build).
  • ⚠️ PMD report not generated.
  • ⚠️ Checkstyle report not generated.

Generated automatically by the PR CI workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant