Stick Gen5: CVE Fix Regression Test Report
Second batch: kernel CVEs (dbg1371, MR !541 open)
| Build | dbg1371 · BUILD_ID 20261018T170642Z-1363530621-1d54a11a7 · main-amlinux 499fd567 + kernel CVE batch (earlier build of the same batch: dbg1367) |
|---|---|
| CVE fixes included | 33 kernel CVEs in 5 modules (k-other, k-netproto, k-fs, k-nocommit, k-config), all from official 6.12.y stable plus their prerequisites · MR !541 (open) |
| Rig | optistick-linux-wilber-01 |
Smoke and full (P0 E2E + CVE cases) passed on both builds with 0 failures. The 4-hour dev-soak matched the dbg1323 baseline on stability. The Bluetooth remote worked hands-on. One watch point: lower minimum available memory in the soak (below).
1. Tests
| Test | Build | Result |
|---|---|---|
| Smoke (T-001) | dbg1367 | Passed |
| Full: P0 E2E + CVE cases (T-002) | dbg1367 | 80 / 80 passed |
| Smoke (T-007) | dbg1371 | 47 / 47 + 5 / 5 passed |
| Full: P0 E2E + CVE cases (T-008) | dbg1371 | 79 / 81 passed, 0 failed |
| 4-hour dev-soak (TO run 74ea7213, 10-19 00:51 → 04:55) | dbg1371 | No stability regression vs dbg1323 |
| Bluetooth remote, hands-on (10-19 morning) | dbg1371 | BT06G paired via remoted, all 7 keys OK, Home opened the hub, no BT errors in dmesg |
The 2 cases not counted in T-008 are OS-KERN-001 (first run and its rerun): a known dmesg allowlist case, ignored on Stick Gen5.
2. 4-hour dev-soak vs dbg1323 baseline
| Metric | dbg1371 | dbg1323 baseline |
|---|---|---|
| Reboots / player restarts / kernel panic / OOM | 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 |
| Unusable screen time | 0.08% | 0.06% |
| Coverage | 99.5% | 97.7% |
| Frozen spans | 18 | 37 |
| Blank / dead screen | 0.6% | 0.1% (limit 1%) |
| Min available memory | 69 MB | 122 MB |
Result: no stability regression found against the dbg1323 baseline in this soak. TO marked the run "fail" because of 52 watch alerts; these are alerts, not the verdict (dbg1323 had 65).
Watch point: min available memory is 53 MB lower than the baseline. It cannot be attributed to the kernel CVEs yet: this image is about 90 commits newer than dbg1323 (mediad/player 4K changes, telemetryd), and memory is now read from telemetryd. A soak on unpatched 499fd567 would separate the two.
Shorter-than-scheduled playback (web designer ~40 s vs 120 s) is the same as the baseline.
First batch: dbg1323 (merged !465 + !482)
| Build | stk5-x5m-os5-dbg1323-1g · BUILD_ID 20261016T112935Z-1923215985-6852b8ab9 · main-amlinux 6852b8ab9 |
|---|---|
| CVE fixes included | fc5450f0a (!465: curl 8.22.0, libxml2, openssl, expat) · 05f047ad9 (!482: 29 kernel networking CVEs, 11 kernel Wi-Fi/Ethernet CVEs) |
| Test window | Flashed 10-16 19:37 → all runs finished 10-18 02:16 |
Performance: the 2-hour ROM benchmark PASSED. No reboot, crash or OOM during the 4-hour dev-soak or the 2-hour memory soak. After 8+ hours of soak, the P0 recheck matched the first run exactly.
Function: every remaining failure was traced to a cause outside the CVE-changed modules. All cases on the CVE-changed paths (networking, Wi-Fi, Bluetooth, downloads, TLS) passed.
1. Performance and stability
| Item | Result | Key numbers |
|---|---|---|
| 2-hour ROM benchmark (profile 2h-v2, judged by score.json) | PASS (4/4 legs) | Aquarium 1000 33 fps · rAF 60 fps · MotionMark Arcs 322 · Power BI load 8.9 s · Dev-Soak leg min available memory 67 MB, PSI full peak 16.7% · OOM 0 · player restarts 0 |
| 4-hour dev-soak (TO run 805c4292) | Stability green | Reboots 0 · player restarts 0 · OOM 0 · kernel panic 0 · unusable screen time 0.06% · coverage 97.7% |
| 2-hour memory soak (SYS-REL-008) | PASS | Heavy playlist ran continuously; player PID unchanged throughout; memory peak stayed under 2× the settled baseline |
| Post-soak P0 recheck | Same as first run | 40 passed / 5 failed / 1 skipped; the same 5 failures as the first run, no new ones |
Compared with before the fixes: the dbg1275 nightly (pre-merge, evt3 rig) passed all 3 benchmark legs with a dev-soak PSI peak of 13.95% (dbg1241 was 28%). dbg1323 is 16.7%, the same range. Different rigs, so compare the range only.
Soak observation (not CVE related): some items played shorter than scheduled — web-designer-standard ~41 s vs 120 s, Power BI 29 s vs 60 s, FHD JPG 9.5 s vs 15 s. Video durations were correct. This is player scheduling; the CVE fixes did not touch the player.
2. E2E functional tests
Each band ran once, failures were rerun once, then remaining failures were rerun again with test-side corrections only (no timeouts or thresholds loosened).
| Band | Passed | Failed | Skipped / xfail |
|---|---|---|---|
| P0 | 40 | 5 (OS-KERN-001 excluded by agreement) | 1 |
| P1 | 101 | 22 | 28 |
| P2 | 9 | 5 | 5 |
Turned to pass on rerun: GIF, 6 CNN cases, SYS-FN-015, Bluetooth Wi-Fi scan, device log collection, offline dashboard, MDM reboot, PL-011 live update, DEV-LC-017 unpair menu.
3. Remaining failures
| Area | Cases | Cause |
|---|---|---|
| 4K transforms | CP-IMG-001 · CP-IMG-008 · CP-SS-010 · CP-DES-001 · DEV-CTL-025 ×3 · DEV-CTL-012 | Known issue, already failing before the fixes (dbg1265). Fixed in mediad/player after this build. |
| Remote view | DEV-CTL-017 · SYS-FN-037 | Known agent issue (dbg1275); tunnel itself works. Fixed in a newer mdm-agent. |
| OS-UI hub | DEV-UI-001 · DEV-UI-004 · DEV-UI-012 | Hub key handling; DEV-UI-001/004 known since dbg1275. |
| Weather web in playlist | CP-DES-002 · CP-PL-002 · CP-SS-006 · CP-PL-014 | Weather shows on screen, but the web page attaches too slowly for the test window. Standalone weather case passes. |
| Split screen | CP-PL-013 · CP-SS-007 | Player scheduling / test timing. |
| Offline | CP-WEB-011 · CP-WEB-012 · CP-WEB-013 | Player does not skip online-only items when offline. |
| RTSP | CP-STR-001 | Known RTSP decoding issue. |
| Certificate install | DEV-UI-013 | Test helper cannot run on the image's minimal Python. |
| Player / settingsd | DEV-CTL-009 · DEV-UI-022 · DEV-UI-014 · SYS-FN-013 | Volume and portal rotation not applied to the system; toast over time limit; heartbeat already stale before the memory test. |
4. CVE-changed paths in this test
| CVE change | Product function | Evidence |
|---|---|---|
| curl / openssl / libxml2 / expat | Asset download, HTTPS, TLS, XML/JSON parsing | All image, video and web assets downloaded and played; YouTube HW decode, Power BI, CNN, weather and device log collection passed; HTTPS portal commands (MDM screenshot, timezone, CEC, device info, MDM reboot) passed |
| Kernel networking (29 CVEs) | TCP/IP, tunnels, firewall | SecureLink Live, wg0 healthy; heartbeat and reporting steady through the 4-hour soak; network recovered after the offline tests |
| Kernel Wi-Fi / Ethernet (11 CVEs) | Wi-Fi, Ethernet, Bluetooth | Wi-Fi connect (DEV-UI-021), Bluetooth Wi-Fi provisioning scan (DEV-LC-011) and Bluetooth remote (DEV-CTL-014) passed |
Appendix: raw records
E2E logs and junit: ~/optisigns/.local/notes/stick-run/dbg1323-* · Benchmark: ~/.optisigns-harness/reports/rom-benchmark/20261016T112935Z-1923215985-6852b8ab9-2h-v2-20261017T120242Z/ (TO campaign 216f0a7a) · Dev-soak: TO campaign 192aa777 · dbg1371 dev-soak: benchcc job-20261018-175104-b3 (TO run 74ea7213)