Skip to content

fuel: the SGI Fuel (IP35) machine - #368

Draft
jamesbraid wants to merge 31 commits into
techomancer:mainfrom
jamesbraid:fuel-ip35
Draft

jamesbraid wants to merge 31 commits into
techomancer:mainfrom
jamesbraid:fuel-ip35

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

This adds the SGI Fuel: a 500 MHz R14000 on a Bedrock node with 512 MiB, the dual-head XBridge, the IOC3 (Ethernet, serial, parallel, PS/2), the two-channel ISP12160, the L1 system controller, AMD boot flash and OHCI USB. PROM 6.210 passes its diagnostics and boots IRIX 6.5.30 to the 4Dwm desktop, and with a V12 board OpenGL works: SGI Quake, SGI Doom and Performer all draw.

I ported most of the devices from my sgi-origami QEMU branch, and their tests follow its qtests. HACKING.md and docs/fuel-ip34.md cover the machine.

This sits on #362 (the VPro boards) and the three R10000-family CPU PRs, and lands after them. Those bring the R14000 model, the 44-bit addresses IRIX's n64 stack needs, the cache diagnostic arrays and tag mask the PROM's cache test walks, and refused-store handling. What's left here for the CPU is Fuel-only:

  • R14000 uncached SB/SH go out as write64_masked. The PROM's ECC memory test reads the other byte lanes and fails without it.
  • R10000-family CPUs keep the uncached attribute from EntryLo 63:62 on mapped pages. Bedrock decodes it as the address space. The other R10000 machines drop those bits before the bus, so it doesn't change them.

The PROM's startup stack uses the primary data cache tag-store hook from #366: the Fuel PROM keeps its stack in the cache before any memory is set up, so Bedrock answers that 32 KiB window from a private page while the tags are valid.

PS/2 through the IOC3 is the default keyboard and mouse, as the Fuel manual shows. USB HID is there too with [fuel] usb_input. There's no audio yet: the Fuel had none built in, and SGI's option was a PCI card.

fuel-final-desktop fuel-hinv-quake fuel-final-doom

Testing

  • PROM 6.210 runs its diagnostics clean ("ALL DIAGS PASSED") and boots IRIX 6.5.30 to login; hinv shows the R14000 rev 2.3, both QL12160s, the V12 and OHCI.
  • 4Dwm desktop, SGI Quake in a window, SGI Doom's timedemo and Performer's truck demo all draw correctly on the V12.
  • cargo test --release --lib --features jitv2: 3191 passed, 0 failed at 093f29b.

Performance

The existing machines' CPU speed, since this PR changes shared CPU and JIT code. Measured with the iris bench suite on bare metal: the Indy (IP24) profile with the R10000 model, ship jitv2 builds, interleaved runs, medians of guest MIPS, Mann-Whitney on the medians. The baseline is upstream main 28fa516. The measured build is head a25d1fbf, the same Rust sources as the pushed 093f29b (upstream moved by changes outside src/cpu). It sits on the three CPU PRs, so these numbers include them.

The Fuel's uncached-fetch key (jitv2_track_pcp) and the uncached attribute carried through a TLB write are computed with a mask instead of a branch, so a machine without them pays an AND.

With every function 64-byte aligned in both builds (n=30), one kernel is slower than main: int/branch -1.4% (p 3e-7). It is -2.3% at n=15 and within noise on the three CPU PRs below this one (-0.2% on the last). A profile of the whole suite shows the extra samples in JIT-compiled code rather than in the interpreter or the code this PR adds, and I haven't found the cause. Faster: sys/cache_flush +2.9%, sys/tlb_miss +1.7%, sys/uncached +1.5%, mem/unaligned +1.5%, sys/exception +1.4%, codec/rle +0.7%.

Plain build, n=30. Slower than main: int/dhrystone -1.7%, sys/cache_flush -1.9%, img/dct8x8 -0.5%. cache_flush is +2.9% when alignment is equal, and the same dips are in the plain build of the first CPU PR, so I read the plain numbers as placement. Faster: mem/stream_triad +3.1%, mem/unaligned +1.4%.

EntryHi, the TLB match and the xuseg bounds check all stopped at VA bit 39,
but the R10000 family implements 44 bits: VPN2 is 43:13 and the user segment
is 16 TiB (R10000 UM pp. 219, 288). A 64-bit kernel can put an n64 user
stack above 40 bits (0x0fff_ffff_aee4, for one). EntryHi dropped VA[43:40]
and such an address took an Address Error.

The width now comes from the part's VA_BITS in each place: the EntryHi write
mask, the TLB's 64-bit compare (Tlb::set_va_bits, installed by the executor
beside set_cache_attrs), the xuseg bounds check, the micro-TLB's cacheable
segment test and the EntryHi value a TLB exception loads. The mask was
spelled three times and is now one const fn, mips_tlb::vpn2_mask. All of it
is a const of the CPU model or set once at construction; the TLB lookup is
unchanged.

This reverses a comment in the executor (b7d2a18) that said widening the EntryHi mask
breaks the IP28 and Octane kernels. Those two machines and an O2 set to the
R10000 now see 44 bits. The R4400 and R5000 (Indy, Indigo2, O2 with an
R5000) stay at 40.

One test maps a page at 0x123_4000_0000 on the R10000 and the R4400: the
R10000 keeps EntryHi and reads the page, the R4400 masks it and takes
Address Error.
A PROM's cache walk reads and writes the primary caches with Index Load
Data and Index Store Data. Only the secondary cache had a data array, so
an Index Load Data on a primary cache always returned zero and an Index
Store Data was dropped.

Give the primary caches data arrays and mask each store to its
diagnostic field (R10000 UM pp. 262, 271-272): 36 data bits and 1 parity
bit for the instruction cache, 32 and 4 for the data cache, 64 and 10 for
the secondary. An Index Store Data to the secondary cache writes a
quadword, so the other doubleword becomes zero and takes the same check
bits (p. 91). A mapped primary Index Load Data or Index Store Data
still takes TLB Refill and Invalid, so CACHE now translates the operand
for those two; the other index operations still do not.

The secondary data slot is now the address >> 3 where it was >> 4, so
the walk gets one slot per doubleword instead of two doublewords
sharing one. This affects the Octane, the IP28 and an O2 with an R10000.

Tests cover a primary round trip and the quadword store.

Index Store Data is a diagnostic walk's operation, so it runs out of line
in a cold function, and CACHE tests the primary-cache pair against a bit
mask. Written inline they grew cache_op and exec_cache in every CPU
model and moved the interpreter's hot loops, which cost sys/llsc about
2% on a ship build.
Fuel uses two independent SCSI buses behind one ISP12160 function. Reuse
the ISP1040 command engine and let the part trait own the firmware,
register and channel differences.

A bus reset reaches every attached target as unit attention rather than
a media change. Keep the reset attention on the shared target interface
so a controller can notify each device through the same bus. The
ISP12160 is its only caller: the WD33C93A, AIC-7880 and ISP1040 models do
not call it, so their targets behave as before. A bus reset restores each
disk's write cache bit per channel: a reset of one channel leaves the other's
disks alone, and a chip or soft reset restores both.

The engine changes reach the Octane's ISP1040: mailbox commands now run
with dispatch held, and the target interface gains the bus reset. The
Octane's IRIX 6.5.30 SCSI disk (Octane2, R10000, 256 MiB, V12, jitv2,
release build, headless, NAT) boots to the login prompt and runs commands
on this branch's tip and on main c6dea8c: login after 126 s on the
branch and 109 s on main in one run each (host load makes the times
indicative only), the same `hinv` listing of both integral controllers
and the disk, and the same command output.
Fuel fits a DP83843 at MII management address 1, where the Octane has an
ICS1890. The IOC3 Ethernet controller needs the same packet and link
machinery with either PHY, so keep that machinery shared: move the
Octane's ICS1890 register model behind a PHY interface, and supply the
DP83843's register and negotiation behavior through it. The MAC keeps
its existing packet path.

The interface carries the snapshot hooks too: the ICS1890 keeps the
register save and load it had in the IOC3 model, and the DP83843 saves
its registers and negotiation timers the same way, so the shared Ethernet
block saves whichever PHY it holds.
Fuel stores its clock and nonvolatile configuration in an M48T35 on the
IOC3 byte bus. Keep the register overlay and battery-backed bytes
together so latching and persistence share one owner.
Fuel attaches an M48T35 and has different PCI header policy from the
Octane. Carry those choices in the population configuration and wire the
timekeeper once so byte-bus accesses need no mutable attachment slot.

The Fuel's onboard IOC3 also carries the PS/2 keyboard and mouse
controller: OpenBSD configures a keyboard controller through the
onboard IOC3 (pci/ioc.c, dev/iockbc.c), and the controller core is the
one the Octane already has. Turn the controller on in the Fuel's
population, as the Octane's is, and test it by sending a host key through the IOC3
registers: it latches the keyboard interrupt and arrives in the K_RX
register, and a mouse command answers from its own port, both before and
after a chip reset.
Fuel boot flash uses AMD unlock, program and erase commands. Keep that
state machine separate from array storage and bus decode so its command
handling can serve each fitted flash array.

The Intel and AMD command sets need the same rules for seeding and
writing flash images, so share the file storage while each command engine
keeps its own command state.
Fuel PROM 6.210's memory test fails when uncached SB and SH reach the bus
carrying only their low bits. Its L1 console shows:

  mtest_bank_main failed: Walking Address
  RSLT mtest_bank_mai FAIL                diag_rc = 43  Walking Address

In both memory banks the expected halfwords 0xa55a, 0x5aa5, 0x9999 and
0x6666 read back from the ECC check-bit view as 0x0000. The PROM stores
each check word with SH from a register whose low 16 bits are zero; the
word sits higher in the register, at a position that depends on the
tested address. With the whole register visible to the ECC view and the
same PROM configuration it reports ALL DIAGS PASSED and enables all
512 MiB.

Add BusDevice::write_narrow_from, whose default stores the architectural
byte or halfword exactly as write8/write16 do, and call it for every
uncached SB and SH. Ordinary devices therefore still see the low bits of
the register, and only a device that models a path observing the other
lanes overrides it; Bedrock's ECC view does, in its own commit. Which
lanes real hardware drives for a narrow store is undefined. Debug writes
keep the ordinary-width path, and the JIT callouts pass the same source
register. The default adds one virtual call to a path that already makes
one.
I/O (bedrock_io):
Fuel needs Bedrock I/O to route programmed PIO windows to PCI widgets,
check inbound DMA permissions and run its two block-transfer engines.
Fault reporting and interrupt delivery stay with the block that owns the
state, and unsupported accesses are reported.

Register map and behaviour are ported from the QEMU model.

Node routing (bedrock_ni):
Fuel runs as one node with no NUMAlink peer, but the guest still reads
and programs these local control words during machine setup. Retain the
writes and keep the network fabric out of the model.

Crossbar (bedrock_xb):
Record the fabric error queues and let software service them, with the
outputs available to the Bedrock processor interface. Unsupported queue
reads return zero and writes are logged and ignored.

Local bus (bedrock_lb):
Add the node and region registers, atomic scratch aliases, error and
vector status, and the MicroLAN master that board identity parts sit on.
The tests drive it together with the I/O, node-interface and crossbar
blocks it reports through, which land in this same commit.
The Fuel CPU receives interrupts and timing events through Bedrock PI.
Model the fitted processor slot and its node callbacks while keeping
unconnected graphics control explicit.
Fuel reaches its AMD flash through executable PROM aliases and
programming windows. Share the array bytes and persistent store while
preserving the command and erase state of each flash part.

The configuration block's CPU type word is 1, the R14000, as the IP35
PROM's own configuration editor lists it (0 for the R10000 and R12000, 2
for the R16000); a test pins it.
Fuel routes PCI configuration, DMA and interrupt packets through two
XBridge heads. Keep small sparse PCI decode in window lists so the
bridge can share the machine memory map without allocating more Physical
tables.

A board attached to one of the crossbar's external ports raises its
interrupts as crosstalk packets, the same way the PCI heads do. Route
them through the same sender, so the Fuel's graphics board reaches the
Bedrock's processor interface with no path of its own, and say whether
the destination took the packet.

The crossbar answers with the Crossbow's widget 0 register map, so it
takes its offsets from the Octane's Crossbow model instead of restating
them. What stays separate is behavior that differs between the two: the
XBridge crossbar's status register carries the requesting port, wide
and malformed accesses set the access-error status and answer zero, its
interrupts go out as crosstalk packets to any destination, and it
resets the heads and ports its fences select.
The Fuel console first passes through the L1 system controller. Reuse
the shared 16550 core and keep controller framing, power commands and
identity records in its protocol model.
Fuel enumerates a Lucent OHCI controller on PCI. Provide its registers
and descriptor scheduling through the USB function interface while
leaving devices unattached in the machine configuration.

The frame counter follows the machine clock the board hands the
controller, and every register read and write first delivers the frame
boundaries that clock has passed. Advancing it only from the board's
poll, which runs between CPU batches, left HcFmNumber at zero after the
guest's own 1 ms wait had expired, and the Fuel PROM reported that SOF
did not start within 2 ms. The sync holds one lock from reading the
clock to applying the frames, so a timer thread cannot leave a
concurrent reader behind it.

Entering UsbOperational from Reset or Suspend re-anchors the frame:
it loads HcFmRemaining from HcFmInterval, toggle included, and
increments HcFmNumber, with the first SOF one frame later (OpenHCI 1.0a
6.2.1, 7.3.2, 7.3.3). The tests cover that entry with frame numbers at
and around the overflow flag.

Add two low-speed USB functions for the OHCI root ports: a HID boot
keyboard and a HID boot mouse, each with one configuration, one boot
interface and one interrupt-IN endpoint. They answer the standard
requests (descriptors, SET_ADDRESS taking effect after its status
stage, SET_CONFIGURATION, status and interface queries) and the HID
class requests (GET/SET_REPORT, GET/SET_IDLE, GET/SET_PROTOCOL), and
report host keys and pointer moves as boot reports. Reports queue one
per change so a key tapped between two polls still shows both edges.

Written from the USB 1.1 and HID 1.11 specifications (boot descriptors
from appendix B) and the HID Usage Tables, with no code from another
emulator. The wheel, remote wakeup and the host layout translation that
the PS/2 path has are left out. Ohci::attach loses its test-only gate so
a machine can plug the devices in.
Build the Fuel IP34 board around its R14000, Bedrock, XBridge, IOC3 and
dual-channel ISP12160. Use one Physical memory table with sparse
processor and peripheral views, and retain the CPU memory fast paths
after the PROM startup store retires.

Uncached byte and halfword stores reach a device through
BusDevice::write_narrow_from, which each window forwards at the
device-local address; only the memory controller's ECC view reads more
than the low bits of the register.

With the jitv2 feature, start the compile pool after the memory windows
are published, as the Octane and O2 do. Each worker copies the cache
geometry once as it starts, and without the windows that geometry has no
inline load or store path. Workers read code through the cached plane, which
serves plain physical addresses; the uncached views would decode them as
node registers.

The machine describes where its memory can answer to the CPU: the four
1 GiB slots Bedrock's controller places its two 256 MiB banks in, and the
reset PROM. Its disks on both ISP12160 channels are flushed at power-off,
quit and exit like the other machines', so a copy-on-write overlay keeps
its record of held sectors after a guest `halt -p`.

The processor is R14000FuelShadowCache, the Octane's R14000 at revision 2.3
(PRId 0x0f23), the revision a public software inventory of a Fuel prints,
where the Octane's part reports 2.4. The raw CP0 word, the FIR and the
128-byte lines are taken from the Octane part, not read from a Fuel. Like
the Octane's R14000 it reports "R12000" as its name. The machine loads
its PROM through prom_image.

Feed the IOC3 PS/2 controller from the window, VNC and CI input, and let
a USB keyboard on root port 0 and a USB mouse on port 1 share it. The
Fuel's manual shows PS/2 as the standard keyboard and mouse, so
ps2_input under [fuel] defaults on and usb_input defaults off; setting
usb_input fits the USB pair, and ps2_input = false leaves the PS/2 pair
out. The monitor's ps2 command reaches the keyboard and mouse too.

PROM loader (prom_image):
Add a device-independent loader for PROM images. It accepts a raw
payload or a version-six stored or gzip container, decodes the
recognized containers explicitly and reports malformed inputs with their
path before any bytes reach guest memory.

Config profile (config):
The Fuel is a new machine profile with two memory banks, wide SCSI buses,
no Ethernet adapter choice beyond its built-in one, and no graphics board
by default. The [fuel] section selects the serial console (the L1
controller or either IOC3 port), the L1 TCP port and log, and the
persistent boot flash store. l1_revision and brick_serial replace
the L1's reported firmware revision (default 1.48.1) and brick serial
(default EMUN00). ps2_input defaults on and usb_input off, as
the Fuel's manual shows PS/2 as the standard keyboard and mouse; setting
usb_input fits a USB keyboard and mouse, and ps2_input = false leaves the
PS/2 pair out.

The key names the part: machine.cpu = "r14000" (r10000, the family, is
still accepted), and a Fuel config cannot claim another processor. The
Octane's R14000 is revision 2.4; the Fuel's is 2.3.
PCI functions need different writable and write-one-to-clear header
fields. Describe those fields in one construction policy while keeping
the current header defaults for existing functions.

Forward narrow Bridge configuration writes with their byte enables so
write-one-to-clear bits are handled correctly when a function supplies
them. Existing Octane functions do not use that status policy, so their
behavior stays unchanged.
Index Store Tag stored all 64 bits of a secondary-cache tag, so a
walking-1s test over the tag read back bits the R10000 defines as
reserved. The manual says they return zero (R10000 UM p. 270,
figure 11-31).

Keep MRU, the physical tag, state, virtual index and ECC, which
assembled is 0x8000_000f_ffff_cdff, and drop the rest. The mask is
CpuModel::SECONDARY_TAG_MASK, all ones except on the R10000 family. MRU stays
because the PROM writes it as TagHi bit 31 and reads it back.

The later parts share the tag layout (the R12000 manual agrees; the R14000
is taken to match), and a PROM's tag test on the R14000 expects
the reserved bits to read as zero. Until the R12000 and later models exist
here, the family is just the R10000.

Tests: an all-ones store on the R10000 and R10000 O2 reads back the
mask, an R4400 control keeps TagLo bits 12 and 13, and the walking-1s
round trip expects the reserved bits as zero.
The Octane shipped with an R12000 or R14000 and the O2 with an R12000,
but `cpu` took only r4400, r5000 and r10000. An Octane hinv could not
show the later parts, and CP0 Config reported 1 MB of secondary cache
for every R10000-family part, so a 2 MB model printed "1 Mbyte".

Add r12000 and r14000. They share the R10000's programming model
(44-bit virtual, 40-bit physical, the R10000 cache operations) through
model::r10k_family. Config SS is now log2(secondary size / 512 KB). The
Octane's R14000 is the R12000's 2 MB type with its PRId set at run time,
as the Octane R10000 already does, so only two new executors are
compiled.

  Octane  r10000, r12000, r14000 (195, 360, 600 MHz by default)
  O2      r5000, r10000, r12000 (400 MHz; 64-byte secondary lines,
          because with 128 the O2 PROM panics before its banner)
  IP28    r10000 only

iris-bench gets r12000 and r14000 cells. The bench harness names an
implementation 0x0f CPU an R16000 above major revision 2, and the
prebuilt guest image is refreshed for it.

On the R12000 and later, Index Writeback Invalidate (D) also clears the
line's primary data tag, as Index Load Tag then shows; the R10000 keeps it.

The new parts also keep the upper word on a partial LWR, as the R10000 does;
for the R12000 and later that is a model choice.
Every Fuel has a VPro graphics slot: a Fuel IRIX inventory lists its
board under xtalk/13 and OpenBSD finds Odyssey at crossbar widget 13.
Build the board there when the machine is given a V12 or a V10, through
the widget-offset constructor, and keep none, the serial machine, as the
default.

The XBridge now reports link 13 alive, the board's DMA enters Bedrock's
inbound path as widget 13, and a link reset of port 13 resets it. Its
interrupt packets cross the crossbar to Bedrock like the PCI heads'. The
status bar's activity (SCSI, IOC3 and Ethernet) and the CPU's Compare
ticks reach its LEDs as on the Octane. IRIX's VPro driver writes each
processor interface's graphics flow-control registers at attach and tears
the board down when the credit counter reads zero, so the interfaces keep
those registers; they count no stores and return no credits, so no
processor interface is linked to the board's credit accounting.

With PROM 6.210 the board passes the graphics diagnostic, and IRIX
6.5.30 with the VPro driver lists Graphics board: V12 and draws the X
login screen on it. Input comes from the PS/2 pair the Fuel already has.
Configuration accepts the board on the Fuel too, see the config note below.

Tests build a Fuel board with each board and check that the processor
reaches the board at widget 13 and finds its link alive, that its
interrupt packets set and clear a processor interface's pending bit, that
a link reset of its port resets it, that a headless Fuel has nothing
there, and that the processor interfaces keep their graphics registers.

Config (config):
A Fuel is reported with both the V12 and the V10, so take either with
[graphics] board; the other boards stay refused, with a message that
names the Fuel's choices, and none, the serial machine, stays the
default. The doc comments for the two boards now say they fit the Fuel as
well as the Octane.

The Fuel's validation case now accepts both boards and refuses the V6.
Document the supported serial configuration, image inputs, the host
input options and the explicit limits for graphics, USB devices and
snapshots, in the feature list, hacking notes, debug notes, the sample
config and a Fuel page. Startup storage remains a PROM-specific
compatibility approximation.
FrameMask (CP0 register 21) is new with the R10000. Bits 15:0 clear
EntryLo bits 33:18 as a TLB write passes them on, and the rest of the
register reads as zero (R10000 UM v2.0 p. 260). iris had no register 21: mfc0
read zero and mtc0 was dropped, so a kernel that sets FrameMask to keep
software flags out of the physical address got no effect.

The core holds the register; the executor sets which bits are writable,
all 16 on the R10000 and none on the R4400 and R5000. TLBWI and TLBWR pass
the masked EntryLo values, and TLBR returns the masked entry. The register
is saved with the rest of CP0 and loads as zero from an older save.
CpuModel::R10K_FAMILY says a part is in the R10000 family; for now that is
the R10000.

Nothing changes while FrameMask is zero, its reset value, and the write
path runs only on a TLB write.

Tests write FrameMask on the R10000 and the R4400, map a page and check the
physical address and the readback; a second writes with TLBWR as well as
TLBWI and reads the entry back with TLBR.
In 64-bit mode a partial LWR (offset 0 to 2) sign-extended the merged
word on every CPU. The manual calls the upper word of a partial LWR
implementation specific (NEC U12754E pp. 125-126), and the reference
results observed for the R10000 keep the old bits 63:32 at all
three partial offsets. With sign extension a test that compares those
results miscompares.

The interpreter and the JIT now merge the old upper word on the R10000; a
full word (offset 3) still sign-extends. The upper word is kept only when
64-bit operations are enabled for the current mode, matching QEMU;
otherwise it sign-extends. The later parts are not covered, since nothing
shows what they do. The choice is a JitConsts field set from
CpuModel::PARTIAL_LWR_KEEPS_UPPER, so compile workers see it. Whether
64-bit operations are on is a byte in the core that the executor sets
wherever it re-derives the privilege state, so a compiled LWR spends a
byte load and a compare on it rather than decoding Status; the merge
runs on the whole register and needs no more work than the old
sign-extending sequence. The code-cache
fingerprint gains lwr_upper=1 only when it is set, so the R4400 and
R5000 machines keep their warm caches.

Behaviour changes on the Octane, IP28 and an O2 with the R10000. The
R4400 and R5000 are unchanged.

Tests run LWR at every offset in the interpreter on the R4400, R5000
and R10000 in kernel, supervisor and user mode with the mode's own
64-bit bit set and clear, run the same offsets as compiled code on the
R10000 and R5000 and compare with the interpreter, and check the
fingerprint.
On the R10000 EntryLo holds a 28-bit PFN in bits 33:6, a 40-bit physical
address, and the two-bit uncached attribute in bits 63:62; 61:34 are
reserved (R10000 UM v2.0 pp. 239-240). iris kept the R4400's 30 bits, so the
attribute and PFN bits 33:30 read back as zero and never reached the TLB.

The core now has an EntryLo mask that the executor sets per part: the R4400
and R5000 keep 0x3FFF_FFFF, the R10000 gets 0xC000_0003_FFFF_FFFF. The
uncached attribute is stored and read back only; it is not carried to the
bus. Bits 33:32 are physical 39:38, where the Octane's guest keeps software
flags; FrameMask (previous commit) clears them on a TLB write, so the page
lands at its real address.

A TLB write also clears the PFN bits below the page size, so TLBR does not
return what was written to them (R10000 UM v2.0 pp. 299-300). The manual
documents this for the R10000 only; the later parts follow it as a model
choice. The R4400 and R5000 keep the whole PFN. Translation already
ignored those bits, so only TLBR changes. The Octane PROM's tlb diagnostic
writes 0xa5a5a5a4 and 0x5a5a5a58 with 16 KB pages and expects 0xa5a5a524
and 0x5a5a5a18 back; it reported 320 compare errors.

On the Octane, the IP28 and an O2 set to the R10000, mfc0 of EntryLo now
returns bits 63:62 and PFN bits 33:30 if the guest wrote them. The R4400
and R5000 are unchanged.

Tests check the readback on the R10000 and the R4400, that a page with PFN
bits 33:32 set maps to physical 39:38 and is brought back by FrameMask, and
the diagnostic's patterns through TLBWI and TLBWR.
…sses

The R10000 drives a two-bit uncached attribute on the system bus. For an
unmapped xkphys access it is address bits 58:57; for a mapped reference
it comes from the TLB's uncached-attribute field, which EntryLo0/1 bits
63:62 load (R10000 User's Manual 007-2490-001). EntryLo already keeps
those bits on the family, and a TLB entry holds them, but the entry's
physical base dropped them, so a mapped uncached page always reached
attribute zero.

Carry them to physical bits 58:57 only for pages the model's C table
decodes as uncached, so cached mappings are unchanged. The TLB already
holds that table for its shadow entries; reading it there, and
recomputing each entry's base when a table is installed, avoids a second
decode of C.

An entry with no attribute bits set, the usual case, skips the C table.

The attribute reaches the bus only on a board that decodes physical
bits 58:57. The Octane, Indigo2 and O2 mask the physical address to 40,
32 and 32 bits before the bus, which folds it away, and a test shows
that at the executor. The R4400 and R5000 never store the bits.

Tests cover the attribute and the board masks, and the C table (a value
the model decodes as cached carries nothing; a table installed after the
entry was written decides).

The uncached-attribute comments cite NEC U10278EJ4V0UM, the fourth edition
of the VR10000/VR12000 user's manual, section 11.3, by document number;
the manual itself is not added to the repository.
Bedrock memory configuration changes the RAM decode. Keep ordinary CPU
and DMA memory on Physical directly, divert diagnostic pages through MD,
and update their generation state when memory configuration or ECC
changes.

The ECC view overrides write_narrow_from to take an uncached SH's check
word from the whole source register, at the position Fuel PROM 6.210's
memory test uses. Every other view stores the architectural low bits.
@jamesbraid
jamesbraid marked this pull request as draft October 31, 2026 05:28
@jamesbraid

Copy link
Copy Markdown
Collaborator Author

wip, doing some rework and cleanup

@jamesbraid
jamesbraid force-pushed the fuel-ip35 branch 2 times, most recently from 1b77e64 to 687bc0e Compare October 31, 2026 09:49
The IP35 PROM fetches its own code uncached at plain physical 0x1fcxxxxx.
Bedrock serves that from the reset PROM, but the compile pool read through
the cached view, and once a memory bank covers the window the cached view
answers RAM there. The memory test zeroes it, so the JIT compiled pages of
NOPs in place of PROM routines and the PROM faulted (an instruction fetch
bus error, or endless refills). Code pages are keyed by physical frame, so
the cached and uncached fetches of that address were one page.

Make them two addresses, the way the board decodes them. An uncached
instruction fetch of the reset window is now presented to the bus at the
same PROM at an address of its own (MipsExecutor::set_uncached_fetch_alias, called
by the Fuel; RESET_WINDOW_FETCH, above every RAM slot). The JIT stays as it
is: it keys, reads and tracks that page by address like any other, the
flash device's generation counter invalidates it, and a store to RAM over
the window never does. The alias wraps the
translation the CPU installs when it is set, so a cached fetch, a data
access and every other machine's fetch path are untouched.

The evidence is partial. Which space a request reaches follows the
processor's uncached attribute (ip35-address-map.md, processor-side
address spaces, established), so an uncached and a cached access of one
address are different requests. What attribute 0 returns at 0x1fc00000 is
not established: no public source says how the reset request reaches
flash, and the cached side there is open too. The split rests on what
PROM 6.210's memory test needs, and the code and CHANGELOG say so.

The compile pool restarted after a page-pool flush used the CPU's uncached
bus instead of the cached plane it was started with; the executor now keeps
the compile bus (set_jitv2_code_bus) and passes it to every restart and to
the inline compile. The monitor uses it too: `j2 inline off`, `dumppcp`,
`corpus` and `j2 html` read pages through it.

Tests: an uncached reset-vector fetch compiles the PROM's ADDIU and the
cached frame compiles RAM's boundary word, and a flash write invalidates
only the PROM's page; a pool restart after a flush keeps the compile bus; a
cached fetch and an unaliased executor key the frame they name; uncached
attributes 1 to 3 and addresses past the end of memory get no code page.
The R10000 picks the TLB refill vector from the address space the
missing address lies in and that space's Status bit: user addresses
follow UX, xsseg SX, and kseg3, sseg/ksseg and xkseg KX. The current
mode only decides which space an address is in (R10000 UM v2.0 pp.
333-334, 342). The manual contrasts this with the R4400, which picks by
the current mode's bit. iris did the R4400 thing on every model.

So a 64-bit kernel (KX=1, UX=0) missing on a kuseg address went to the
XTLB vector at 0x080 on the R10000, where the part uses 0x000. The
Octane PROM's utlb diagnostic loads VA 0 in exactly that state and
waits at the 32-bit vector. It got "Read TLB miss exception" from the
XTLB vector instead and failed.

On the R10000 family a refill miss now reads the region of the address,
sign-extended from 32 bits in 32-bit mode, and tests UX, SX or KX. Only
the miss path changes. The TLB compare width still follows the mode.

The test misses in each space with each bit on the R10000, and checks
the R4400 keeps choosing by mode.
@jamesbraid
jamesbraid force-pushed the fuel-ip35 branch 3 times, most recently from 2604eef to 9225aaa Compare November 2, 2026 05:22
The R10000 and R12000 do not shut down on a multiple match. A TLB write
that would let one lookup match two entries first invalidates every
entry it overlaps, the wired ones included, then writes the new entry
and sets Status.TS (NEC U10278EJ4V0UM, Status register TS bit p.223 and
"Avoiding TLB Conflict" p.300). iris stored the entry and left the
duplicates matchable, so which one answered depended on how the lookup
map happened to be rebuilt. On the two-processor Tezro the firmware
flushes the TLB with 64 invalid entries on the kernel's page number,
and the kernel then writes the valid global 16 MiB entry over them;
a later write elsewhere left an invalid duplicate answering, and the
fetch took TLB Invalid.

The R14000 gets the same rule as a model choice; no manual for it was
read. The R4400 and R5000 keep their behaviour.

An overlap is two entries a lookup could match together: same region
and page number under the wider of the two page masks, and equal ASIDs
or either entry global. Invalid entries count. The removed entry reads
back as an unwritten slot except for its PageMask: the kernel TLBRs its
wired sentinel entries and reuses PageMask for the next write, and a
zeroed mask made it write a per-CPU mapping as 4 KiB.

The cost is confined to TLB writes on these models. An entry that
overlaps the new one covers a lookup-map slot the new one covers, so a
write whose slots are all empty or its own skips the scan. A PageMask
that is not a run of ones, which software can write, defeats the map's
ranges and turns the shortcut off. A refill that replaces a one-page entry nothing else covers skips the walk over the other entries. TLBWI and TLBWR are never compiled by
jitv2, so both reach the same write and set TS in the interpreter.

Tests cover the Tezro sequence, wired entries, PageMask kept on a
read-back, entries that cannot match together, and TS from TLBWI and
TLBWR. They fail with the rule off, and the R4400 and R5000 test fails
if the rule is enabled for them.
A store that nothing answers raised a synchronous Data Bus Error with a
precise PC on every CPU. The R10000 cannot do that. It sends an uncached
store as a write request, and a write request gets no response; Bus Error
comes only from an error response to a read or upgrade request, or from
uncorrectable read data (R10000 UM pp. 104, 316). The processor never
learns that a store failed.

The R12000 manual documents the same no-response write path, and the later
parts are taken to match, so the test is the R10000 family and not the R10000
alone.

Add CpuModel::POSTS_UNCACHED_STORES, true for the R10000 family's caches. On such a
model a refused store logs and completes in write_data_impl and in the
masked-store path (SWL, SWR, SDL, SDR), which the interpreter and the
compiled code both go through. An uncached SC or SCD that is refused
completes and succeeds (rt = 1); that is a model choice, since the manual
says nothing about uncached store-conditional. Loads, instruction fetches,
cached accesses and the R4400 and R5000 keep the exception, and device
return values are untouched because BUS_ERR also tells non-CPU callers
that a request was declined.

The error goes no further. Nothing in iris raises an interrupt or sets an
error register for a store the Octane's bus refuses. The IP22/IP28 bus
already completes a write to an unanswered address, latching MC
CPU_ERROR/GIO_ERROR and raising no interrupt, so the stores that change
here are the ones a device refuses itself.

The log line stays visible ("Unsupported uncached write ... posted,
completes") but a probe loop must not flood stderr: it prints the first
four refused stores and then each power of two, with the running count.

The Octane's ide diagnostics store 0x591204 and then 0x4 to physical
0x1e130400, where nothing answers. Before, the first store took a Data Bus
Error; now both print one line and the run carries on.

Tests: the interpreter on the R10000 (full and shadow caches) and the
R5000 for SW, SWL and SC plus the load that still faults; the JIT against
the interpreter for SW and SWL on the R10000 and for the load and the
R5000 store that fault; the report schedule. cargo test --release
--features jitv2 --lib: 2422 passed, 0 failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant