[HELP] USB /dev/console open failure

[stbenn]

Description

I am struggling to get USB Device driver working on stm32h5. The goal is to get NSH over USB, similar to example in the tutorial video 010 on the NuttX YouTube channel (https://www.youtube.com/watch?v=rgZJPZh_po8)

I would love help figuring out if this is a configuration issue or a driver issue.

H5 USB Support Additions (WIP)

The USB peripheral IP for the H5 is almost identical to the G0. So I have ported the usbdev files into the STM32H5 directory and added necessary supporting files... I think... This is the commit if it helps: e222afd

What I've Tried

I am trying to follow something similar to the video as well as maple:usbnsh config. Here is the defconfig defconfig.txt

I have narrowed it down to something failing when opening /dev/console during initialization:

• In group_setupidlefiles.c line 77 (in function group_setupidlefiles), fd = nx_open("/dev/console", O_RDWR); returns -2
• The lower level error occurs in nx_vopen(...) where the call to file_vopen(...) has a return code -2.

Could this be a configuration issue, or is it more likely something wrong with the driver?

Verification

• I have verified before submitting the report.

[raiden00pl]

try to disable CONFIG_DEV_CONSOLE. I don't see # CONFIG_DEV_CONSOLE is not set in your defconfig

[stbenn]

@raiden00pl That fixed the crash from nx_open("/dev/console", O_RDWR);, thank you.

Hypothetically, if everything is set up correctly and working on the NuttX side, should windows pick up the dev board as a USB device? (I have been using PowerShell usbipd list to check) In other words, is it safe to say the USB driver is not working if usbipd list does not show it?

[raiden00pl]

@stbenn I'm not familiar with windows so I can't help.
I highly recommend using a USB analyzer to debug USB issues. They're not cheap, but they save a lot of time when working with USB.