Raspberry PI 5 support

[Lenael]

Few days ago a new Raspberry PI 5 was introduced. It's build based on new Broadcom chip: BCM2712 (full specification: https://www.raspberrypi.com/products/raspberry-pi-5/#specification). It looks really interesting considering its price (80$ for 8GB version). Is there any chance Talos would support this new SBC?

[smira]

We are excited about RPi5!

Same answer as for any other SBC: we need upstream Linux support (6.1) and u-boot support. If that lands, we can start accepting patch for RPi5.

[RobReus]

@smira is there any update on this? It seems RPi 5 is not yet supported and issue #7978 does not see much activity.

[frezbo]

#8065 will address all this

[iamasmith]

I had a week off and wanted to look at this, there's still a little work to do before sending in a PR (somehow) but I've written drivers for the 2 reset controllers used for brcm,brcmstb-reset and brcm,bcm7216-pcie-sata-rescal (which seem solid) alongside, additionally, a pretty experimental split off driver for brcm,bcm2712-pcie which is the main client driver.
At about 17:00 hours today I was finally able to enum devices on the PCIE lanes off the board.. thankfully the NVME isn't hung off the RP1 southbridge which will need more work but I might give it a go to unlock Ethernet in u-boot/PXE boot etc.
Anyhow here's the device list currently..

U-Boot 2024.07-gd655a075 (Aug 30 2024 - 16:56:49 +0100)

DRAM:  1020 MiB (effective 8 GiB)
RPI 5 Model B (0xd04170)
Core:  24 devices, 12 uclasses, devicetree: board
MMC:   mmc@fff000: 0
Loading Environment from FAT... Unable to read "uboot.env" from mmc0:1...
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
mbox: Header response code invalid
bcm2835: Could not query MAC address
Net:   No ethernet found.

reset-controller@119500 SATA/PCIe rescal success
PCIe BRCM2712: link up, 5.0 Gbps x1 (!SSC)
reset-controller@119500 SATA/PCIe rescal success
PCIe BRCM2712: link up, 5.0 Gbps x4 (!SSC)
starting USB...
No USB controllers found
Hit any key to stop autoboot:  0
U-Boot> pci
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x14e4     0x2712     Bridge device           0x04
01.00.00   0x2646     0x501c     Does not fit any class  0xff
02.01.05   0xffff     0xffff     Does not fit any class  0xff
U-Boot>

As you can see the devices show but at this stage aren't reporting a subclass but at this stage after quite a few hours I chose to retire to the 'purple zone' and will look at it again and try to get it nailed before I go back to work on Monday (doing less interesting Cloud Stuff :p).
BTW, yes I based my fork off 2024.07 which is the latest in the Talos beta but I'll ensure the diffs merge and work on current of course.
(btw: I did do some other stuff on my week off :D)
The next step is to work out why the NVME storage is currently showing a sub-class of 0xff (unknown) vs 0x01 (storage) and I think getting that netted will allow the nvme driver to hook up and all may be well, so maybe close to the gate. I suspect that as both devices show this though (including RP1 on bus 02) that it's a matter of enabling some additional clocking. There is some wierdness around a clock called 'sw_pcie' which isn't enabled in the device tree and I had it in code, took it out, added it, took it out.. it may be important but the associated vector looks fishy - but there's code in u-boot to generate safe tries on these things for invalid gets for various things like clocks and resets. It wouldn't surprise me though that the devices on the bus not being clocked could report their IDs but not their class... hmm.

[iamasmith]

Thoughts on why this wasn't working were incorrect, I had actually updated the wrong constant based on device specific information in the Linux driver and it proved hard to track down until I re-read the code several times.

I now have correct device information for regular devices attached to the bus, you can ignore 02.1b.05, I know that's the RP1 controller and it needs a u-class driver if/when I/somebody want to get it to work but as you can see the nvme drive is popping up but cannot yet be scanned in a stable fashion.

U-Boot> pci
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
00.00.00   0x14e4     0x2712     Bridge device           0x04
01.00.00   0x2646     0x501c     Mass storage controller 0x08
02.1b.05   0xffff     0xffff     Does not fit any class  0xff
U-Boot> pci header 01.00.00
  vendor ID =                   0x2646
  device ID =                   0x501c
  command register ID =         0x0006
  status register =             0x0010
  revision ID =                 0x01
  class code =                  0x01 (Mass storage controller)
  sub class code =              0x08
  programming interface =       0x02
  cache line =                  0x08
  latency time =                0x00
  header type =                 0x00
  BIST =                        0x00
  base address 0 =              0x00100004
  base address 1 =              0x00000000
  base address 2 =              0x00000000
  base address 3 =              0x00000000
  base address 4 =              0x00000000
  base address 5 =              0x00000000
  cardBus CIS pointer =         0x00000000
  sub system vendor ID =        0x2646
  sub system ID =               0x501c
  expansion ROM base address =  0x00000000
  interrupt line =              0x00
  interrupt pin =               0x01
  min Grant =                   0x00
  max Latency =                 0x00
U-Boot> nvme scan
"Synchronous Abort" handler, esr 0x96000005, far 0x1b0010001c
elr: 000000000003ada0 lr : 000000000003ad90 (reloc)
elr: 000000003fb7fda0 lr : 000000003fb7fd90
x0 : 0000001b00100000 x1 : 0000000000100000
x2 : 0000001000118000 x3 : 0000000000003e0b
x4 : 0000000000001e01 x5 : 0000000000000001
x6 : 000000003fb7cd98 x7 : 0000000000000028
x8 : 000000003fb7cb60 x9 : 0000000000000000
x10: 0000000000000006 x11: 000000000001869f
x12: 000000003f72dfe8 x13: 000000003f72e140
x14: 0000000000000000 x15: 000000003f74af22
x16: 000000003fb7cb44 x17: 0000000000000000
x18: 000000003f740d60 x19: 000000003f74aec0
x20: 000000003f74a870 x21: 000000003f74a870
x22: 000000003fbc4fc8 x23: 0000000000000002
x24: 000000003fbeb68c x25: 0000000000000000
x26: 000000003f74add0 x27: 0000000000000000
x28: 0000000000000000 x29: 000000003f72dd00

Code: aa0003f3 f8050415 a9000000 f9402660 (b9401c00) 
Resetting CPU ...

resetting ...

[joyshmitz]

Hi @iamasmith!
Perhaps this repo Raspberry Pi PCI Express device compatibility database will help you, unfortunately I'm not familiar with such subtleties.

The guys seem to know the ins and outs of raspberries.

[iamasmith]

@joyshmitz thanks for the post, I'm afraid it's a bit lower level at this point and really about getting the basics working for the bus. As this is the bootloader I don't think there is much value beyond nvme and possibly at some point the peripherals that hang off the RP1 south bridge which is attached to the PCIE bus as well.
I have been looking at old datasheets for other Broadcom devices as much of what is on the PI5 is based on other Broadcom products but it is challenging and there seem to be quite a few unexplained pieces to me at this point and this board is a bit of a challenge in that the only UART available on basic initialisation is the debug port meaning that you have to get the GPIO fired up to have both a serial console off that and an active debug connection using the debug probe which means getting PCIE up so that you can get RP1 going and then enable the UARTs off that - so it's either UART or Debug Port off that one little connector, not both at the same time sadly.