Skip to content

Real-time threads: periodic SCHED_FIFO loops the collector never stops - #6138

Open
shai-almog wants to merge 8 commits into
masterfrom
realtime-threads
Open

shai-almog wants to merge 8 commits into
masterfrom
realtime-threads

Conversation

@shai-almog

Copy link
Copy Markdown
Collaborator

Adds com.codename1.realtime: periodic real-time threads for control loops on (PREEMPT_RT) Linux, the embedded port and the backend.

  • Scheduling: SCHED_FIFO priority, CPU pinning plus isolation (the core is given back when the thread ends), mlockall(MCL_ONFAULT), and priority-inheriting VM mutexes. Release is on an absolute clock, with missed-period reporting and latency/execution statistics plus a histogram.
  • Collector isolation: a real-time thread is released as soon as its stack is scanned (sound because SATB is armed first), and no stop-the-world cycle runs while one is alive.
  • No allocation in a cycle:
    • Build time: RealtimeAllocationVerifier fails the build on allocation reachable from cycle()/periodsMissed(), unless it is marked @AllocationAllowed.
    • Run time: the thread's page cache is swapped out, so an allocation the build could not see throws RealtimeAllocationError naming the class. Per-period budgets are supported, and the check is re-armed each period, so it stays on even if the cycle catches the error itself.
  • setMemoryLimit: ordinary threads get OutOfMemoryError instead of a stalled collector. The limit is measured on resident memory, after collections and on the allocation slow path.
  • Other platforms: the simulator and other ports run the loop best-effort.
  • Docs: the developer guide gets a new chapter, Realtime.asciidoc, linked from Embedded Linux.
  • CI: vm/backend/demo/rtcheck runs in embedded-linux.yml on the arm64 runner, as root. It covers latency under GC churn, the hidden-allocation trap, budgets, a cycle that catches the error, and the memory limit. The latency bound is loose (20ms) because runners are not PREEMPT_RT.

Verified locally on macOS and in an arm64 Linux container: rtcheck passes. The SpotBugs, PMD, cast-semantics, header, ASCII and control-character gates pass, and RealtimeAllocationVerifierTest passes 7/7.

🤖 Generated with Claude Code

shai-almog and others added 4 commits October 30, 2026 06:21
com.codename1.realtime runs a RealtimeTask on a thread of its own, released
on an absolute clock, on PREEMPT_RT (and any) Linux and on the backend:

- SCHED_FIFO priority, CPU pinning and isolation, mlockall(MCL_ONFAULT),
  priority-inheriting VM mutexes; statistics with a latency histogram.
- The collector releases a real-time thread as soon as its stack is
  scanned (SATB keeps it sound) and never enters stop-the-world phases
  while one runs.
- No allocation in a cycle: RealtimeAllocationVerifier fails the build on
  allocation reachable from cycle()/periodsMissed(); at run time the
  thread's page cache is swapped out, so an allocation the build could not
  see throws RealtimeAllocationError naming the class.
- setMemoryLimit: ordinary threads get OutOfMemoryError instead of a
  struggling collector once resident memory is over the limit.
- The simulator and other ports run the loop best-effort.

vm/backend/demo/rtcheck is the check, run on the arm64 runner by
embedded-linux.yml; developer guide chapter Realtime.asciidoc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e sections

Red-team round 1: an allocation inside a non-zero budget refilled the budget,
so only a budget of 0 was enforced (rtcheck now checks budget 2). Isolation now
returns the core to the moved threads when the real-time thread ends; a virtual
thread resumed from a trapped carrier gets the real page array; the RSS sample
counter no longer drops bytes. Guide: parameters, life cycle, features.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rs statistics and porting

Red-team round 2: the over-budget path lifted the trap to build the error and
nothing re-armed it, so a cycle catching RealtimeAllocationError allocated
unchecked from then on. The trap is re-armed every period (rtcheck: catcher).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Cloudflare Preview

shai-almog and others added 2 commits October 30, 2026 18:16
…e and IDEs

- cn1:buildLinuxLocal / buildLinuxLocal, cn1:runOnDevice / cn1RunOnDevice and
  cn1:device-logs / cn1DeviceLogs; -Dcn1.board names the board and turns on
  the embedded build.
- The executable survives the builder's temp dir (linux/target/linux-native).
- A cross build with no cross compiler fails instead of using the host cc.
- The device refuses a binary for another architecture before installing it;
  the service gets LimitRTPRIO/LimitMEMLOCK for real-time threads.
- Archetype: build.sh/build.bat commands and IntelliJ, VS Code, NetBeans and
  Eclipse entries for Linux Local Build, Run on Device and Device Logs.
- CI: the real-time step installs the backend; the guide's motor loop is an
  included, compiled snippet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r commands

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Developer Guide build artifacts are available for download from this workflow run:

Developer Guide quality checks:

  • AsciiDoc linter: No issues found (report)
  • Vale: 22 alert(s) (17 errors, 5 warnings, 0 suggestions) (exit code 1) (report)
  • Paragraph capitalization: No paragraph capitalization issues (report)
  • LanguageTool: 11 advisory match(es) — top: MORFOLOGIK_RULE_EN_US (4), CROSS_COMPOUNDS (3), EN_A_VS_AN (1) (report)
  • Image references: No unused images detected (report)

shai-almog and others added 2 commits October 30, 2026 21:43
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ Continuous Quality Report

Test & Coverage

Static Analysis

  • SpotBugs [Report archive]
    • ✅ ByteCodeTranslator: 0 findings (no issues)
    • ✅ android: 0 findings (no issues)
    • ✅ android-compat: 0 findings (no issues)
    • ✅ android-res-compiler: 0 findings (no issues)
    • ✅ backend: 0 findings (no issues)
    • ✅ backend-test: 0 findings (no issues)
    • ✅ build-engine: 0 findings (no issues)
    • ✅ build-hint-catalog: 0 findings (no issues)
    • ✅ build-hint-tools: 0 findings (no issues)
    • ✅ codenameone-gradle-plugin: 0 findings (no issues)
    • ✅ codenameone-maven-plugin: 0 findings (no issues)
    • ✅ compose-compat: 0 findings (no issues)
    • ✅ core-unittests: 0 findings (no issues)
    • ✅ desktop-compat-runtime: 0 findings (no issues)
    • ✅ fxml-compiler: 0 findings (no issues)
    • ✅ ios: 0 findings (no issues)
    • ✅ javac: 0 findings (no issues)
    • ✅ javafx-compat: 0 findings (no issues)
    • ✅ project-model: 0 findings (no issues)
    • ✅ swing-compat: 0 findings (no issues)
  • ✅ PMD: 0 findings (no issues) [Report archive]
  • ✅ Checkstyle: 0 findings (no issues) [Report archive]

Generated automatically by the PR CI workflow.

@github-actions

Copy link
Copy Markdown
Contributor

✅ ByteCodeTranslator Quality Report

Test & Coverage

  • ✅ Tests: 908 total, 0 failed, 55 skipped

Benchmark Results

  • Execution Time: 9790 ms

  • Hotspots (Top 20 sampled methods):

    • 9.15% com.codename1.tools.translator.IteratorEscape.mangle (136 samples)
    • 8.14% java.util.ArrayList.indexOf (121 samples)
    • 5.99% java.lang.System.identityHashCode (89 samples)
    • 5.38% java.lang.Object.hashCode (80 samples)
    • 4.91% com.codename1.tools.translator.BytecodeMethod.equals (73 samples)
    • 2.69% java.lang.StringBuilder.append (40 samples)
    • 2.42% java.util.HashMap.hash (36 samples)
    • 2.02% com.codename1.tools.translator.bytecodes.Invoke.resolveDirectTarget (30 samples)
    • 1.62% com.codename1.tools.translator.JavascriptReachability.enqueueResolved (24 samples)
    • 1.55% com.codename1.tools.translator.IteratorEscape.walk (23 samples)
    • 1.55% java.util.AbstractCollection.containsAll (23 samples)
    • 1.48% java.lang.String.equals (22 samples)
    • 1.35% com.codename1.tools.translator.ByteCodeClass.hasDeclaredMethod (20 samples)
    • 1.28% java.io.FileOutputStream.writeBytes (19 samples)
    • 1.08% com.codename1.tools.translator.bytecodes.Invoke.findMethodUp (16 samples)
    • 1.01% com.codename1.tools.translator.IteratorEscape.ctorOnlyStoresParamsIntoThis (15 samples)
    • 0.94% java.util.HashMap.putVal (14 samples)
    • 0.94% com.codename1.tools.translator.Parser.cullMethods (14 samples)
    • 0.94% java.lang.StringCoding.encode (14 samples)
    • 0.87% com.codename1.tools.translator.NativeSymbolIndex.<init> (13 samples)
  • ⚠️ Coverage report not generated.

Static Analysis

  • ✅ SpotBugs: no findings (report was not generated by the build).
  • ⚠️ PMD report not generated.
  • ⚠️ Checkstyle report not generated.

Generated automatically by the PR CI workflow.

@github-actions

Copy link
Copy Markdown
Contributor

Flutter vs Codename One

Same application, built both ways: Flutter's AOT compile of the gallery, and the identical Dart source transpiled to Java on Codename One. Lower is better for every metric, so a ratio above 1.00x means Codename One is ahead by that factor.

android

9 interleaved rounds; host load 0.85, 2.03, 1.78

Metric Codename One Flutter Ratio Winner
Installed size 78.3 MB 95.8 MB 1.22x Codename One
Executable code 4.7 MB 20.6 MB 4.35x Codename One
Download size (zipped) 78.3 MB 95.8 MB 1.22x Codename One
Cold start to first frame on screen 371 ms 1670 ms 4.54x Codename One
Memory at rest 46.8 MB 88.5 MB 1.89x Codename One

Start-up and memory at rest are judged round by round: the ratio is the median of the per-round ratios, each from two launches made back to back, and the times and sizes shown are each side's median, so the ratio need not equal their quotient. Codename One was ahead in 9 of 9 rounds on cold start to first frame on screen; 9 of 9 rounds on memory at rest.

Compute -- the VM workloads (vm/benchmarks), run inside each app; time is the best of the app's own repetitions, higher ratio means Codename One is faster.

Workload Codename One Flutter Ratio
intArithmetic 96 ms 189 ms 1.97x
longArithmetic 57 ms 58 ms 1.02x
mathTranscendental 227 ms 354 ms 1.56x
arraySequential 27 ms 36 ms 1.33x
arrayRandom 262 ms 409 ms 1.56x
objectAllocation 67 ms 49 ms 0.73x
valueEscape 8 ms 7 ms 0.88x
hashMapChurn 164 ms 310 ms 1.89x
stringBuilding 101 ms 131 ms 1.30x
recursion 82 ms 272 ms 3.32x
quicksortBench 129 ms 152 ms 1.18x
Geometric mean 1.40x

Gate: within tolerance against scripts/flutter-bench/baseline.

Android sizes are for the arm64-v8a slice a phone is delivered, on both sides; Flutter's release apk carries three ABIs and would otherwise count its engine about three times.
Android start-up and memory come from an x86_64 emulator with a software GPU. They compare the two runtimes on one machine; they are not phone timings, and a renderer that leans on the GPU is penalised more there.
This platform's adapter has not been exercised end to end before; treat the first published run as the thing under review.

ios

9 interleaved rounds; host load 31.84, 26.32, 22.27

Metric Codename One Flutter Ratio Winner
Installed size 108.5 MB 110.5 MB 1.02x Codename One
Executable code 17.4 MB 19.1 MB 1.09x Codename One
Download size (zipped) 80.0 MB 81.9 MB 1.02x Codename One
Cold start to first frame on screen -- -- -- not measured
Memory at rest -- -- -- not measured

Compute -- the VM workloads (vm/benchmarks), run inside each app; time is the best of the app's own repetitions, higher ratio means Codename One is faster.

Not measured: iOS compute is not measured: a device build needs signed hardware, and on the simulator Flutter runs its JIT debug engine rather than AOT code. The macOS row runs the same ParparVM and Dart AOT compilers on the same Apple silicon.

Gate: within tolerance against scripts/flutter-bench/baseline.

iOS start-up and memory are not measured here: Flutter cannot AOT-compile for the simulator, so a simulator run would time a JIT debug build against a release build. Sizes come from release device bundles; timings need signed hardware.
This platform's adapter has not been exercised end to end before; treat the first published run as the thing under review.

javascript

9 interleaved rounds; host load 2.51, 1.34, 0.56

Metric Codename One Flutter Ratio Winner
Installed size 104.4 MB 122.5 MB 1.17x Codename One
Executable code 13.3 MB 25.2 MB 1.89x Codename One
Download size (zipped) 76.4 MB 84.1 MB 1.10x Codename One
Cold start to first frame on screen -- -- -- not measured
Memory at rest -- -- -- not measured

Compute -- the VM workloads (vm/benchmarks), run inside each app; time is the best of the app's own repetitions, higher ratio means Codename One is faster.

Workload Codename One Flutter Ratio
intArithmetic 552 ms wrong result Flutter: wrong result
longArithmetic 3024 ms did not run Flutter: did not run
mathTranscendental 274 ms did not run Flutter: did not run
arraySequential 139 ms 61 ms 0.44x
arrayRandom 740 ms wrong result Flutter: wrong result
objectAllocation 23 ms 16 ms 0.70x
valueEscape 88 ms 5 ms 0.06x
hashMapChurn 552 ms 170 ms 0.31x
stringBuilding 296 ms 70 ms 0.24x
recursion 296 ms 271 ms 0.92x
quicksortBench 290 ms wrong result Flutter: wrong result
Geometric mean 0.32x

Gate: within tolerance against scripts/flutter-bench/baseline; BEHIND FLUTTER on Compute (geomean of the VM workloads) (reported, not gated: a loss to Flutter on the web is currently accepted).

the JavaScript adapter needs a Chrome DevTools driver; sizes are reported and timings are not
This platform's adapter has not been exercised end to end before; treat the first published run as the thing under review.

linux

9 interleaved rounds; host load 0.65, 3.44, 3.0

Metric Codename One Flutter Ratio Winner
Installed size 113.5 MB 143.8 MB 1.27x Codename One
Executable code 15.8 MB 16.1 MB 1.02x Codename One
Download size (zipped) 81.7 MB 92.9 MB 1.14x Codename One
Cold start to first frame on screen 184 ms 254 ms 1.38x Codename One
Memory at rest 93.3 MB 226.1 MB 2.42x Codename One

Start-up and memory at rest are judged round by round: the ratio is the median of the per-round ratios, each from two launches made back to back, and the times and sizes shown are each side's median, so the ratio need not equal their quotient. Codename One was ahead in 9 of 9 rounds on cold start to first frame on screen; 9 of 9 rounds on memory at rest.

Compute -- the VM workloads (vm/benchmarks), run inside each app; time is the best of the app's own repetitions, higher ratio means Codename One is faster.

Workload Codename One Flutter Ratio
intArithmetic 75 ms 183 ms 2.44x
longArithmetic 56 ms 56 ms 1.00x
mathTranscendental 239 ms 272 ms 1.14x
arraySequential 17 ms 35 ms 2.06x
arrayRandom 278 ms 404 ms 1.45x
objectAllocation 24 ms 37 ms 1.54x
valueEscape 5 ms 7 ms 1.40x
hashMapChurn 45 ms 339 ms 7.53x
stringBuilding 38 ms 109 ms 2.87x
recursion 92 ms 257 ms 2.79x
quicksortBench 140 ms 150 ms 1.07x
Geometric mean 1.90x

Gate: within tolerance against scripts/flutter-bench/baseline.

This platform's adapter has not been exercised end to end before; treat the first published run as the thing under review.

macos

9 interleaved rounds; host load 5.7, 19.16, 17.57

Metric Codename One Flutter Ratio Winner
Installed size 124.7 MB 139.4 MB 1.12x Codename One
Executable code 33.6 MB 47.9 MB 1.43x Codename One
Download size (zipped) 85.6 MB 93.3 MB 1.09x Codename One
Cold start to first frame on screen 194 ms 248 ms 1.28x Codename One
Memory at rest 34.4 MB 51.9 MB 1.51x Codename One

Start-up and memory at rest are judged round by round: the ratio is the median of the per-round ratios, each from two launches made back to back, and the times and sizes shown are each side's median, so the ratio need not equal their quotient. Codename One was ahead in 9 of 9 rounds on cold start to first frame on screen; 9 of 9 rounds on memory at rest.

Compute -- the VM workloads (vm/benchmarks), run inside each app; time is the best of the app's own repetitions, higher ratio means Codename One is faster.

Workload Codename One Flutter Ratio
intArithmetic 105 ms 242 ms 2.31x
longArithmetic 66 ms 66 ms 1.00x
mathTranscendental 263 ms 308 ms 1.17x
arraySequential 10 ms 43 ms 4.30x
arrayRandom 260 ms 302 ms 1.16x
objectAllocation 52 ms 28 ms 0.54x
valueEscape 5 ms 13 ms 2.60x
hashMapChurn 30 ms 159 ms 5.30x
stringBuilding 28 ms 101 ms 3.61x
recursion 131 ms 258 ms 1.97x
quicksortBench 113 ms 135 ms 1.20x
Geometric mean 1.85x

Gate: within tolerance against scripts/flutter-bench/baseline.

This platform's adapter has not been exercised end to end before; treat the first published run as the thing under review.

windows

9 interleaved rounds

Metric Codename One Flutter Ratio Winner
Installed size 108.4 MB 121.1 MB 1.12x Codename One
Executable code 11.8 MB 14.1 MB 1.20x Codename One
Download size (zipped) 78.1 MB 86.4 MB 1.11x Codename One
Cold start to first frame on screen 157 ms 252 ms 1.60x Codename One
Memory at rest 57.3 MB 93.0 MB 1.62x Codename One

Start-up and memory at rest are judged round by round: the ratio is the median of the per-round ratios, each from two launches made back to back, and the times and sizes shown are each side's median, so the ratio need not equal their quotient. Codename One was ahead in 9 of 9 rounds on cold start to first frame on screen; 9 of 9 rounds on memory at rest.

Compute -- the VM workloads (vm/benchmarks), run inside each app; time is the best of the app's own repetitions, higher ratio means Codename One is faster.

Workload Codename One Flutter Ratio
intArithmetic 80 ms 184 ms 2.30x
longArithmetic 61 ms 58 ms 0.95x
mathTranscendental 197 ms 280 ms 1.42x
arraySequential 11 ms 36 ms 3.27x
arrayRandom 265 ms 295 ms 1.11x
objectAllocation 25 ms 40 ms 1.60x
valueEscape 1 ms 7 ms 7.00x
hashMapChurn 50 ms 321 ms 6.42x
stringBuilding 36 ms 121 ms 3.36x
recursion 189 ms 263 ms 1.39x
quicksortBench 147 ms 148 ms 1.01x
Geometric mean 2.12x

Gate: within tolerance against scripts/flutter-bench/baseline.

This platform's adapter has not been exercised end to end before; treat the first published run as the thing under review.

Codename One wins 26 of 26 measured metrics across 6 platform(s).

@shai-almog

shai-almog commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

Compared 195 screenshots: 195 matched.

Native Android coverage

  • 📊 Line coverage: 19.18% (56011/292047 lines covered) [HTML preview] (artifact android-coverage-report, jacocoAndroidReport/html/index.html)
    • Other counters: instruction 18.13% (269804/1488151), branch 12.40% (19628/158314), complexity 10.36% (15804/152567), method 15.71% (11373/72383), class 27.88% (2235/8017)
    • Lowest covered classes
      • kotlin.collections.unsigned.kotlin.collections.unsigned.UArraysKt___UArraysKt – 0.00% (0/2396 lines covered)
      • org.jacoco.agent.rt.internal_0e20598.asm.org.jacoco.agent.rt.internal_0e20598.asm.ClassReader – 0.00% (0/1524 lines covered)
      • androidx.compose.runtime.androidx.compose.runtime.LinkComposer – 0.00% (0/1430 lines covered)
      • org.jacoco.agent.rt.internal_0e20598.asm.org.jacoco.agent.rt.internal_0e20598.asm.MethodWriter – 0.00% (0/922 lines covered)
      • kotlinx.coroutines.channels.kotlinx.coroutines.channels.BufferedChannel – 0.00% (0/863 lines covered)
      • kotlin.sequences.kotlin.sequences.SequencesKt___SequencesKt – 0.00% (0/744 lines covered)
      • com.google.common.cache.com.google.common.cache.LocalCache$Segment – 0.00% (0/726 lines covered)
      • okio.okio.Buffer – 0.00% (0/687 lines covered)
      • kotlin.text.kotlin.text.StringsKt___StringsKt – 0.00% (0/630 lines covered)
      • com.codename1.androidcompat.rt.com.codename1.androidcompat.rt.TextXmlPullParser – 0.00% (0/623 lines covered)

✅ Native Android screenshot tests passed.

Native Android coverage

  • 📊 Line coverage: 19.18% (56011/292047 lines covered) [HTML preview] (artifact android-coverage-report, jacocoAndroidReport/html/index.html)
    • Other counters: instruction 18.13% (269804/1488151), branch 12.40% (19628/158314), complexity 10.36% (15804/152567), method 15.71% (11373/72383), class 27.88% (2235/8017)
    • Lowest covered classes
      • kotlin.collections.unsigned.kotlin.collections.unsigned.UArraysKt___UArraysKt – 0.00% (0/2396 lines covered)
      • org.jacoco.agent.rt.internal_0e20598.asm.org.jacoco.agent.rt.internal_0e20598.asm.ClassReader – 0.00% (0/1524 lines covered)
      • androidx.compose.runtime.androidx.compose.runtime.LinkComposer – 0.00% (0/1430 lines covered)
      • org.jacoco.agent.rt.internal_0e20598.asm.org.jacoco.agent.rt.internal_0e20598.asm.MethodWriter – 0.00% (0/922 lines covered)
      • kotlinx.coroutines.channels.kotlinx.coroutines.channels.BufferedChannel – 0.00% (0/863 lines covered)
      • kotlin.sequences.kotlin.sequences.SequencesKt___SequencesKt – 0.00% (0/744 lines covered)
      • com.google.common.cache.com.google.common.cache.LocalCache$Segment – 0.00% (0/726 lines covered)
      • okio.okio.Buffer – 0.00% (0/687 lines covered)
      • kotlin.text.kotlin.text.StringsKt___StringsKt – 0.00% (0/630 lines covered)
      • com.codename1.androidcompat.rt.com.codename1.androidcompat.rt.TextXmlPullParser – 0.00% (0/623 lines covered)

Benchmark Results

Detailed Performance Metrics

Metric Duration
SIMD kernel backend scalar fallback (no native SIMD)
SIMD int-add (64K x300) java 347ms / native 231ms = 1.5x speedup
SIMD float-mul (64K x300) java 217ms / native 262ms = 0.8x speedup
SIMD kernel correctness PASS (native result == scalar reference)
Base64 payload size 8192 bytes
Base64 benchmark iterations 6000
Base64 SIMD byte path gated to scalar (CPU autovectorizes scalar; explicit SIMD not beneficial here)
Base64 CN1 encode 78.000 ms
Base64 CN1 decode 86.000 ms
Base64 native encode 474.000 ms
Base64 encode ratio (CN1/native) 0.165x (83.5% faster)
Base64 native decode 303.000 ms
Base64 decode ratio (CN1/native) 0.284x (71.6% faster)
Image encode benchmark status skipped (SIMD unsupported)

@shai-almog

shai-almog commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator Author

Compared 231 screenshots: 231 matched.
✅ JavaScript-port screenshot tests passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant