/sys/bus/coresight not part of device tree

[imgod2u]

Hi all,

On an older device (Snapdragon Wear) running Android, I could see coresight funnels in /sys/bus/coresight/devices.

I'm running a device now with Snapdragon 865 and Android 10. I wanna be able to collect (using simpleperf) some ETM traces. However, it seems /sys/bus/coresight is not there. This can mean coresight drivers were disabled in the kernel? Any advice on how to re-enable it?

Thanks in advance!

[enh-google]

unless you're building your own kernel, you're out of luck. tbh, we've found that you probably need to be able to make changes to the bootloader to get ETM working, so unless you work for an SoC manufacturer (in which case, file this bug in your internal bug tracker :-) ) you're probably out of luck...

[imgod2u]

Thanks. That's unfortunate. It's a Qualcomm part that we're using for a product. I was hoping on getting some ETM traces out of it. We have other Qualcomm parts and they have Coresight available. Not sure what's special about the 865.

[imgod2u]

Follow-up: I was able to recompile the kernel with a different config. Coresight seems enabled now but this is what I see (Snapdragon 865):

# ls /sys/bus/coresight/devices/                                                                       
coresight-csr              coresight-funnel-dl-center  coresight-funnel-in0   coresight-funnel-swao   
coresight-funnel-apss      coresight-funnel-dl-compute coresight-funnel-in1   coresight-funnel-turing 
coresight-funnel-apss-merg coresight-funnel-dl-mm      coresight-funnel-lpass coresight-funnel-venus  
coresight-funnel-ddr-0     coresight-funnel-dl-north   coresight-funnel-merg  coresight-hwevent       
coresight-funnel-ddr-ch02  coresight-funnel-dl-south   coresight-funnel-npu   coresight-stm           
coresight-funnel-ddr-ch13  coresight-funnel-gpu        coresight-funnel-qatb  coresight-swao-csr

I don't see any ETM point though, which I believe simpleperf needs? Any recommendations here on how to get simpleperf to work with these coresight devices?

[enh-google]

@yabinc might know more...

[imgod2u]

Additional follow-up: I found the following in the simpleperf ETMRecorder.cpp code:

static const std::string ETM_DIR = "/sys/bus/event_source/devices/cs_etm/";

bool success = ReadValueInEtmDir(name + "/trcidr/trcidr0", &cpu_info.trcidr0) &&
--
  | ReadValueInEtmDir(name + "/trcidr/trcidr1", &cpu_info.trcidr1) &&
  | ReadValueInEtmDir(name + "/trcidr/trcidr2", &cpu_info.trcidr2) &&
  | ReadValueInEtmDir(name + "/trcidr/trcidr4", &cpu_info.trcidr4) &&
  | ReadValueInEtmDir(name + "/trcidr/trcidr8", &cpu_info.trcidr8) &&
  | ReadValueInEtmDir(name + "/mgmt/trcauthstatus", &cpu_info.trcauthstatus);

So it appears to be looking for /trcidr/trcidr[*] for each CPU. When I look at that device tree:

ls /sys/bus/event_source/devices/cs_etm                                                              
format nr_addr_filters perf_event_mux_interval_ms power subsystem type uevent

It seems there's no trcidr directory...
Any idea whether there's a way to check whether the hardware exists and I just didn't enable it in the kernel?

Thanks!

[stephenhines]

Our expert on this won't be able to comment until August, but I think you need to follow up with your hardware vendor on whether even having kernel support is going to be sufficient. There are additional layers that may prevent you from using ETM (up to and including your hardware vendor physically disabling ETM access on production devices). This isn't just a matter of Coresight. ETM is one of the most invasive components of that setup, and thus, it usually ends up being turned off.

[imgod2u]

Another follow-up on this. I was able to get the device provisioned with a custom bootloader that programs the "override" bits of the SoC. CoreSight ETM is now enabled. I also added the following KConfigs:

       CONFIG_CORESIGHT=y
       CONFIG_CORESIGHT_FUSE=y
       CONFIG_CORESIGHT_TMC=y
       CONFIG_CORESIGHT_TPIU=y
       CONFIG_CORESIGHT_FUNNEL=y
       CONFIG_CORESIGHT_REPLICATOR=y
       CONFIG_CORESIGHT_STM=y
       CONFIG_CORESIGHT_HWEVENT=y
       CONFIG_CORESIGHT_ETM=y
       CONFIG_CORESIGHT_ETM_PCSAVE_DEFAULT_ENABLE=y
       CONFIG_CORESIGHT_SOURCE_ETM4X=y
       CONFIG_CORESIGHT_CATU=y
       CONFIG_CORESIGHT_SINK_ETBV10=y
       CONFIG_CORESIGHT_QCOM_REPLICATOR=y
       CONFIG_CORESIGHT_CTI=y
       CONFIG_CORESIGHT_EVENT=y
       CONFIG_CORESIGHT_LINKS_AND_SINKS=y
       CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y

To the kernel as well as included the device tree mapping. Built the kernel image and now I seem to have ETM on each CPU core enabled. Which is great! But...I'm missing a sink point for ETR data to be read out:

# /data/simpleperf record -e cs-etm --duration 2 -a                                                                                                 
simpleperf E ETMRecorder.cpp:130] can't find etr device, which moves etm data to memory
# ls /sys/devices/cs_etm
cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7 format nr_addr_filters perf_event_mux_interval_ms power subsystem type uevent 
hollywood:/sdcard # ls /sys/devices/cs_etm/format                                                                                                                       
cycacc retstack timestamp

There's no "sink" point for the cs_etm and no "sinkid" in cs_etm/format, which according to:
https://github.com/Linaro/OpenCSD/blob/master/HOWTO.md
is needed. I think this is a driver in the kernel that's not being enabled maybe? But I enabled CONFIG_CORESIGHT_LINK_AND_SINK_TMC in the KConfig already. Perhaps the resident expert in CoreSight is back and can answer? :)