Skip to content

Execute from flash: the game cores in a PSRAM overlay, LVGL and the GUI in PSRAM (18 MB PSRAM free) - #29

Merged
finger563 merged 24 commits into
mainfrom
feat/flash-xip
Nov 20, 2026
Merged

finger563 merged 24 commits into
mainfrom
feat/flash-xip

Conversation

@finger563

@finger563 finger563 commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The firmware executes from flash instead of a copy of all code in PSRAM (CONFIG_SPIRAM_XIP_FROM_PSRAM, which costs 12.7 MB of PSRAM; after #23 that left too little for Jedi Knight's first level: "Could not load level!"). What needs PSRAM's speed runs from PSRAM:

  • The game cores share one PSRAM code overlay. Only one game runs at a time, so every core's code and tables are linked at the same PSRAM address, and the running game's core is copied there from flash when it launches (1.1 MB, the largest core).
  • LVGL and the GUI (they draw in software; from flash the launcher redraws visibly) run from PSRAM for good (0.7 MB).

PSRAM free at the launcher: 18.1 MB (was 7.3 MB). Stacked on #28; it targets main once #26 and #28 are merged.

core fps / presented PSRAM XIP (before) from flash this PR
Genesis 99 / 59.7 77 / 43.5 91 / 59.6
GBA 95 / 59.7 82 90 / 59.7
SNES 70 58 67
Doom 99 78 94.5
MGS 47 / 9.3 45 / 7.8 9.8 presented
JK level 1 does not load 12.5 presented 14.2 presented

NES, GBC, PICO-8, Dark Forces, TPM and ScummVM are within a few percent. (Core fps is the firmware's frame statistic, which counts busy time.)

How

  • sdkconfig.defaults: no XIP from PSRAM; chip rev v0.1+ (for 80 MHz flash); the noinit PSRAM range (to keep the heap off the new sections).
  • The cores' initialised data, formerly in the PSRAM-XIP .rodata (read-only in flash), is in a PSRAM section, .ext_ram.data, filled from flash at boot.
  • tools/bss_to_psram.py:
    • --overlay=<core> --overlay-rodata renames a core archive's code and tables to .tab5_ovl.<core>* and its static constructors to .tab5_ovl_ctors.<core>.
    • Shared template code stays in flash (std::, fmt::, espp::, __gnu_cxx::: the linker keeps one copy for the whole firmware, and it may be a core's). So do Dark Forces' allocator (--wrap=free,realloc) and mgs::release_arena.
    • --psram-code does LVGL, emu-ui, gui and menu.
  • components/tab5-emu/overlay.ld: .tab5_res, the OVERLAY (NOCROSSREFS) and their slots in a flash store right after .flash.tdata (the image's rodata segment, covered by the boot SHA).
  • tools/overlay_image.py (post-link): stores the sections in their slots and marks the PSRAM copies NOBITS. It fails the build if code or data outside a core reaches into the overlay; only the carts, the loader and a core's own code may.
  • tab5_overlay_load() copies a core in, syncs both cores' caches, and runs its constructors on its first load. make_cart calls it before the cart's constructor.
  • The video task's per-frame path is IRAM_ATTR (about 3 KB), and the display rotation is read once per game instead of every frame.
  • docs/PLAN.md describes the layout and what is not handled.

Box art through the P4's JPEG decoder (components/gui)

  • Covers were decoded by LVGL's software decoder (~75 ms each) and rescaled in software on every redraw (most covers are 100 px wide, shown in a 360 px square).
  • Now the hardware decoder decodes them to RGB565 and each cover is scaled once to 360 px, so LVGL just copies it. Anything the decoder cannot take, such as progressive JPEGs, still goes through LVGL's decoder.
  • The decoder refuses pictures whose width × height is not a multiple of 8 (49 of the 125 covers on the test card), so the frame header is given the MCU-rounded size and the result cropped.
  • Over a 20 s scripted scroll of the launcher: a cover takes ~30 ms decoded and scaled; redraws average 75 ms (were 120–130 ms); the GUI's share of core 0 is ~40% (was ~60%). The PSRAM-XIP and from-flash launchers profiled the same before this change.

Not handled

  • A core task that outlives its game would run the next core's code. MGS's logger and JK's music decoder leak theirs only if they fail to stop at quit, which did not happen in testing.
  • In a panic backtrace inside the overlay, the loaded core's name (tab5_overlay_loaded()) tells which core it was.

Testing

On a Tab5:

  • Two-round soaks, 15 games across every core: the overlay switched between all of them 30 times; no panics; internal RAM flat.
  • JK's first level loads and plays.
  • Benchmarks above (12 games, presented frames timed by the device's uptime).
  • Launcher with the hardware decoder: covers render correctly (upscaled and downscaled); GBA and MGS launch and quit cleanly afterwards; internal RAM at the launcher unchanged.
  • A review of this PR found that EXCLUDE_FILE covered only the first pattern, which put MGS's config and SD-root strings inside the range MGS saves in save states. That is fixed here, along with gaps in the post-link checker.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY

finger563 and others added 23 commits November 18, 2026 22:59
The launcher and the pause menu are now emu-ui's (components/emu-ui, a
new submodule shared with esp-box-emu); gui/ and menu/ keep their APIs
and supply the games, the images and the settings:

- consoles as tabs along the top, the list with the console's name over
  each group, the selected game's cover, console, title and Play, over
  the console's background (an accent, colors and a pattern per console)
- settings in a panel from the right (B or the gear), the console picker
  as a grid of tiles (X), button hints along the bottom
- the pause menu: the paused frame, the slot with its screenshot, the
  actions and the settings in one list, over the game's console
- rom_info's ConsoleInfo keeps name, short name and key; the colors and
  the icon are emu-ui's (console_style)
- the keypad: an object flagged LV_OBJ_FLAG_USER_1 gets up / down as keys
  too (emu-ui's list, rows and picker), not only left / right

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…flowed 6 KB

Opening the settings panel builds its rows inside LVGL's key event, on
the Gui task's timer (the esp_timer task), and overflowed its 6 KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
The Gui counts the save states in Cart::save_dir() (emu_ui::count_saves)
on each reload and each return from a game, rebuilding the list when
the counts changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
The pause menu held 1.2 MB of PSRAM for its two screenshots while the
game ran; Jedi Knight's level load needs every MB. They are allocated
(heap_caps_malloc: no throw, no screenshot if there is no room) when the
menu opens and freed when it closes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
ALWAYSINTERNAL=256 sent every allocation of 256 bytes or less to internal
RAM first (the comment said the opposite): the emu-ui launcher's game
titles, every small std::string and map node. With the new launcher that
left Metal Gear Solid ~500 bytes of DMA memory and a black screen, and
drifted internal RAM down per console shown. As on the box, code that
needs internal or DMA memory asks for it explicitly.

Launcher: internal free 47 -> 82 KB, DMA-capable 8 -> 43 KB; in MGS 53 /
14 KB (was 17.6 / 0.5 KB); a 19-game soak keeps internal RAM flat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
A function-local static vector kept 112 KB (up to 478 KB in hi-res) in the
middle of the PSRAM heap after the game, cutting its largest free block
from 6.4 to 3.7 MB for good.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
nofrendo registered its pads again at every NES start and wrote past its
32-entry input table on the 17th game since boot (a load fault in
mmc_peek / retrieve_type).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…he status

Dark Forces' fallback allocations (and the std::string / map template
copies other code links from its library) asked for MALLOC_CAP_DEFAULT,
internal RAM first. After the fix three DF launches leave internal RAM
flat (only the first launch's two mutexes stay, as on the box).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…keys

fmsx feat/start-key:
- Start was fMSX's own menu, which halts emulation. It is now the MSX's F1
  (pause in Konami's games), and Y opens fMSX's menu. Start + select (the
  pause-menu chord) no longer also toggles the on-screen keyboard.
- A frame is an emulated frame, not a drawn one (every other with -skip
  50): muted, MSX games ran at twice their speed, and the fps was per pair
  of frames.
- The on-screen keyboard's F1-F8, Insert, Delete and Stop indexed past
  fMSX's key table; they press the right keys now.

The README describes the MSX controls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
The fixes the soak tests found that are not about the launcher and pause
menu (nofrendo's input table, SNES's screenshot buffer, Dark Forces'
allocator fallback, emulink's fps status, the MSX controls) go to a separate
PR. This one keeps the fixes for what the new UI itself caused: the pause
menu's screenshot buffers, plain malloc in PSRAM first (with the new
launcher, MGS had no DMA memory left) and emu-ui's tile table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
A function-local static vector kept 112 KB (up to 478 KB in hi-res) in the
middle of the PSRAM heap after the game, cutting its largest free block
from 6.4 to 3.7 MB for good.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
nofrendo fix/input-register: input_register ran at every NES start with no
bounds check, so the 17th NES launch since boot wrote past nes_input[32]
and crashed. It now registers each input once and stops at the table's size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…he status

Dark Forces' fallback allocations (and the std::string / map template
copies other code links from its library) asked for MALLOC_CAP_DEFAULT,
internal RAM first. After the fix three DF launches leave internal RAM
flat (only the first launch's two mutexes stay, as on the box).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…keys

fmsx feat/start-key:
- Start was fMSX's own menu, which halts emulation. It is now the MSX's F1
  (pause in Konami's games), and Y opens fMSX's menu. Start + select (the
  pause-menu chord) no longer also toggles the on-screen keyboard.
- A frame is an emulated frame, not a drawn one (every other with -skip
  50): muted, MSX games ran at twice their speed, and the fps was per pair
  of frames.
- The on-screen keyboard's F1-F8, Insert, Delete and Stop indexed past
  fMSX's key table; they press the right keys now.

The README describes the MSX controls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
emulink fix/devlink-priority: devlink's task ran at priority 0, unpinned
(espp::Task's default, the idle tasks' level), so it got the CPU only when
both cores had nothing else to do: while SNES, or GBA's Zelda in play,
kept core 0 busy, status and the other requests went unanswered for 15-35 s
(the game itself ran on). Priority 5 on core 1, as TinyUSB's task: replies
in 0.1-0.2 s during both, and Metal Gear Solid, whose threads are priority
5 on core 0, is unaffected (9.7 fps).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
emulink fix/devlink-priority: the status's fps field reported get_fps(),
frames per second of busy time (on the Tab5 counting the video task's
blits too), which reads 100+ for a game paced at 60 Hz. It is now the
frames shown per second over at least a second, restarted for each game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…rame

fmsx feat/start-key: a start + select chord that did not land in one poll
briefly sent F1 or toggled the on-screen keyboard; select now acts on
release unless start joined it, and start is F1 only after 3 polls without
select. The pause image and slot thumbnails come from the last frame handed
to the display (a run now ends in the middle of drawing the next).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
With XIP from PSRAM all of .text and .rodata (12.7 MB) is copied into PSRAM
at boot. After the emu-ui launcher and #23 that left too little PSRAM for
Jedi Knight's first level ("Could not load level!"). Executing from flash
instead costs speed (Genesis 59.7 -> 43.5 presented, SNES core below 60), so
the game cores' code and tables, which need it, share one PSRAM overlay:
each core is linked at the same PSRAM address, and the running game's core
is copied there from flash when it launches. PSRAM free at the launcher goes
from 7.3 to 18.8 MB; the overlay is 1.1 MB (the largest core, MGS).

Measured on the Tab5 (core fps / presented; PSRAM XIP -> flash -> overlay):
Genesis 99/59.7 -> 77/43.5 -> 90/59.6, GBA 95 -> 82 -> 89 (59.7 presented),
SNES 70 -> 58 -> 67, Doom 99 -> 78 -> 94, MGS 9.3 -> 7.8 -> 8.9 presented,
JK level 1 12.5 (flash) -> 14.2 presented (it no longer loads with PSRAM XIP).
NES, GBC, PICO-8, Dark Forces, TPM and ScummVM are within a few percent.

- sdkconfig.defaults: no CONFIG_SPIRAM_XIP_FROM_PSRAM; chip rev v0.1+ for
  80 MHz flash; CONFIG_SPIRAM_ALLOW_NOINIT_SEG_EXTERNAL_MEMORY.
- The cores' initialised data (formerly renamed into the PSRAM-XIP .rodata,
  read-only from flash) goes into a PSRAM section, .ext_ram.data
  (components/tab5-emu/ext_ram_data.ld): tools/ext_ram_data_image.py stores
  its linked bytes in flash after linking, and src/ext_ram_data.c copies
  them in before the constructors. MGS's .data likewise (linker.lf).
- tools/bss_to_psram.py --overlay=<core> --overlay-rodata renames a core's
  .text* and .rodata.<symbol> to .tab5_ovl.<core>* and its .init_array to
  .tab5_ovl_ctors.<core> (tools/overlay.cmake gives each core's
  CMakeLists.txt the arguments). Shared template code (std::, fmt::,
  espp::, __gnu_cxx::) stays in flash, since the linker keeps one copy for
  the whole firmware and it may be a core's, and so do Dark Forces'
  allocator (--wrap=free,realloc) and mgs::release_arena.
- components/tab5-emu/overlay.ld: the OVERLAY (NOCROSSREFS) after
  .ext_ram.data, and a slot per core in .flash.tab5_ovl_store right after
  .flash.tdata (the image's rodata segment, covered by the boot SHA).
- tools/overlay_image.py (post-link): copies each member into its slot,
  marks the PSRAM sections NOBITS, and fails the build if anything outside a
  core (except the carts and the loader) reaches into the overlay.
- tab5_overlay_load() (src/overlay.c): copies a core in, syncs the caches
  and runs its constructors on its first load; make_cart calls it before
  the cart's constructor, app_main before scummvm_prepare().
- docs/PLAN.md: the code placement and what is not handled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Executing from flash, the video task's code (frame conversion, PPA tiling,
the frame queue) competed with the cores for the flash cache on every
frame, and layout() called into LVGL each frame for the display rotation.
The per-frame functions are IRAM_ATTR now (about 3 KB of internal RAM) and
the rotation is read in native_size(), when a game sets its frame size.
Measured with the code overlay: GBA 89 -> 90, Genesis 90 -> 91.4, Doom
93.7 -> 94.5, MGS 46 -> 47 core fps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Executing from flash, the launcher got slow and redrew visibly: LVGL,
emu-ui, the GUI and the menu draw in software. Their code and read-only data
(683 KB) now run from PSRAM for good: bss_to_psram.py --psram-code renames
them .tab5_res* (on the archives just before the link, from the root
CMakeLists.txt: a POST_BUILD step cannot be added from another directory,
and LVGL is a managed component), overlay.ld places .tab5_res before the
overlay with a slot in the flash store, overlay_image.py stores it (and
still checks it as resident code), and the constructor that fills
.ext_ram.data copies it in. 18.1 MB of PSRAM free at the launcher.

From a review:
- ext_ram_data.ld: inside a section list EXCLUDE_FILE covers only the next
  pattern, so MGS's glue (g_config, g_sd_root: std::strings) was inside the
  _mgs_data range that MGS restores from a snapshot and saves in save
  states; a loaded state put stale string pointers back. Excluded from both
  patterns now.
- overlay_image.py: an address's lui / auipc and its use may be up to 256
  bytes apart (16 missed one in this build); writable data the linker marks
  executable (internal RAM with code in it) is checked as data too.
- bss_to_psram.py --overlay-rodata leaves the per-function string and
  constant pools (.rodata.<fn>.str1.4, .cst8) in flash, as documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…s size

The launcher decoded each selected game's cover with LVGL's TJPGD (~75 ms
a cover, in software, block by block into RGB888), then showed it with
LV_IMAGE_ALIGN_CONTAIN in the 360 px cover square, so LVGL rescaled it in
software on every redraw of that area (~25 ms; most covers are 100 px wide).

Now the P4's JPEG decoder decodes it straight to RGB565, and the cover is
scaled once (separable bilinear) so its longer side is 360 px: CONTAIN then
works out to scale 1 and LVGL copies it as it is. The decoder refuses a
picture whose width x height is not a multiple of 8 (two in five covers),
though it decodes whole MCUs anyway, so the frame header is given the
MCU-rounded size and the result cropped back. Anything it cannot take
(progressive JPEGs) still goes through TJPGD.

Measured over a 20 s scripted scroll of the launcher (emulink presses,
PC sampling of the GUI task): a cover now takes ~30 ms decoded and scaled
(was ~75 ms for the decode), redraws average 75 ms (were 120-130 ms), and
the GUI's share of core 0 is ~40% (was ~60%).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Base automatically changed from fix/soak-fixes to main November 20, 2026 19:24
main's tree is 91b1772's, which this branch already contains, so the
result is this branch's tree; the gui conflicts were the squashed copies
of #26's changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
@finger563
finger563 merged commit 2ff9079 into main Nov 20, 2026
1 check passed
@finger563
finger563 deleted the feat/flash-xip branch November 20, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant