Feature: Jlink plugin #150

embeddedteam103 · 2019-03-26T13:57:45Z

Description

Adding an htrun-plugin that flashes, erases and resets using the JLINK commander software.
The plugin enables both JTAG and SWD interfaces (but no 'network JLINK').

Touches on ARMmbed/mbed-cli#475

Pull request type

[x] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[ ] Test update
[ ] Breaking change

Reviewers

bridadan

Thanks very much for the new plugin! I have a few suggestions below:

bridadan · 2019-03-26T18:52:02Z

src/mbed_os_tools/test/host_tests_plugins/module_copy_jlink.py

+        with open(path, "w") as jlink_script:
+            jlink_script.write("connect\n")
+            jlink_script.write("erase\n")
+            jlink_script.write("loadbin {0} 0x08000000\n".format(image_path))


What is the 0x08000000 address here representing?

I believe it is the internal flash address of STM32 microcontrollers. However, there are microcontrollers with different flash addresses.

You are right. This plugin is what we use internally, and we only use ST. We definitely need feedback from non-ST future users of the plugin before we can precede.

bridadan · 2019-03-26T18:57:36Z

src/mbed_os_tools/test/host_tests_plugins/module_copy_jlink.py

+    # Plugin interface
+    name = 'HostTestPluginCopyMethod_Jlink'
+    type = 'CopyMethod'
+    capabilities = ['JTAG', 'SWD']


Since this plugin is specific to JLink, could you prefix these as follows? We also stick with lower case capabilities across the plugins:

Suggested change

capabilities = ['JTAG', 'SWD']

capabilities = ['jlink_jtag', 'jlink_swd']

bridadan · 2019-03-26T18:57:59Z

src/mbed_os_tools/test/host_tests_plugins/module_reset_jlink.py

+    # Plugin interface
+    name = 'HostTestPluginResetMethod_Jlink'
+    type = 'ResetMethod'
+    capabilities = ['JTAG', 'SWD']


Since this plugin is specific to JLink, could you prefix these as follows? We also stick with lower case capabilities across the plugins:

Suggested change

capabilities = ['JTAG', 'SWD']

capabilities = ['jlink_jtag', 'jlink_swd']

bridadan · 2019-03-26T18:59:36Z

packages/mbed-host-tests/mbed_host_tests/host_tests_plugins/module_reset_jlink.py

@@ -0,0 +1,23 @@
+"""
+mbed SDK
+Copyright (c) 2011-2015 ARM Limited


Would you mind updating the copyright year here?

Suggested change

Copyright (c) 2011-2015 ARM Limited

Copyright (c) 2019 ARM Limited

bridadan · 2019-03-26T19:00:04Z

src/mbed_os_tools/test/host_tests_plugins/module_copy_jlink.py

@@ -0,0 +1,113 @@
+# Copyright (c) 2018, Arm Limited and affiliates.


Would you mind updating the copyright year here?

Suggested change

# Copyright (c) 2018, Arm Limited and affiliates.

# Copyright (c) 2019, Arm Limited and affiliates.

bridadan · 2019-03-26T19:00:35Z

src/mbed_os_tools/test/host_tests_plugins/module_reset_jlink.py

@@ -0,0 +1,108 @@
+# Copyright (c) 2018, Arm Limited and affiliates.


Would you mind updating the copyright year here?

Suggested change

# Copyright (c) 2018, Arm Limited and affiliates.

# Copyright (c) 2019, Arm Limited and affiliates.

bridadan · 2019-03-26T19:03:25Z

packages/mbed-host-tests/mbed_host_tests/host_tests_plugins/module_copy_jlink.py

@@ -0,0 +1,23 @@
+"""
+mbed SDK
+Copyright (c) 2011-2015 ARM Limited


Would you mind updating the copyright year here?

Suggested change

Copyright (c) 2011-2015 ARM Limited

Copyright (c) 2019 ARM Limited

Also line 17 to be removed. Plus files should have SPDX identifier (any new file should).

bridadan · 2019-03-26T19:04:15Z

src/mbed_os_tools/test/host_tests_plugins/module_reset_jlink.py

+    required_parameters = ['mcu']
+
+    def is_os_supported(self, os_name=None):
+        """! In this implementation this plugin only is supporeted under Windows machines


Little spelling error here:

Suggested change

"""! In this implementation this plugin only is supporeted under Windows machines

"""! In this implementation this plugin only is supported under Windows machines

bridadan · 2019-03-26T19:07:09Z

src/mbed_os_tools/test/host_tests_plugins/module_copy_jlink.py

+            mcu = os.path.normpath(kwargs['mcu'])
+            image_path = os.path.normpath(kwargs['image_path'])
+
+            jlink_file_path = os.path.join(tempfile.gettempdir(), COPY_FILE)


If there are multiple processes running at once, I believe this it is possible for multiple processes trying to write and read to the same file. I think an alternative to this would be to use tempfile.mkstemp() instead.

Sure. We used the same tempfile methods in the ST-LINK plugin bugfix. Should I open a separate PR for changing to the STLINK plugin to use mkstemp as well?

That would be great, thanks!

bridadan · 2019-03-26T19:07:34Z

src/mbed_os_tools/test/host_tests_plugins/module_reset_jlink.py

+
+            mcu = os.path.normpath(kwargs['mcu'])
+
+            jlink_file_path = os.path.join(tempfile.gettempdir(), RESET_FILE)


If there are multiple processes running at once, I believe this has the opportunity for multiple processes trying to write and read to the same file. I think an alternative to this would be to use tempfile.mkstemp() instead.

desowin

Please consider supporting .hex files containing multiple flash sections.

desowin · 2019-03-27T05:54:52Z

src/mbed_os_tools/test/host_tests_plugins/module_copy_jlink.py

+        with open(path, "w") as jlink_script:
+            jlink_script.write("connect\n")
+            jlink_script.write("erase\n")
+            jlink_script.write("loadbin {0} 0x08000000\n".format(image_path))


You should check the file format of the file under image_path. This line simply assumes target that internal flash starts at 0x08000000 and the image format is .bin. The .bin format is quite problematic as it doesn't include the loading address information - if the .bin format is preferred, then the elf->bin converter should somehow provide the loading address to be used here.

Another issue with .bin format is that it doesn't really support disjoint flash sections. For example, if user provides his own linker file which includes the QSPI section on STM32F746 target, then the resulting bin file is over 2 gigabytes (MCU flash startings at 0x08000000 while the QSPI starts at 0x90000000).

The much better alternative in such situations is .hex. See ARMmbed/mbed-os#9789 (comment) for example.

If the image format is .hex, you should use jlink_script.write("loadfile {}\n".format(image_path)) instead. .hex format includes all the addresses, so there's no need for build_jlink_script() to know them.

If the .hex file contains QSPI flash, it might be necessary to enable flashing external memories before calling loadfile/loadbin. You can do so using exec EnableEraseAllFlashBanks.

Below is fully working jlink script I use to program 32F746GDISCOVERY board (onboard ST-Link upgraded to JLink using https://www.segger.com/products/debug-probes/j-link/models/other-j-links/st-link-on-board/) with the .hex file that contains both internal flash and QSPI flash data.

device STM32F746NG speed auto si 1 exec EnableEraseAllFlashBanks r st loadfile target.hex r g qc

Note that the r and st commands after exec EnableEraseAllFlashBanks are needed. If you call loadfile directly after exec EnableEraseAllFlashBanks, then SEGGER J-Link Commander V6.42b (Compiled Feb 5 2019 17:33:07) fails to flash the QSPI memory on 32F746GDISCOVERY discovery board. I didn't find any mention of that in the JLink documentation, I discovered it by trial and error.

Some questions:

If I want to support .hex files, do I need to pass some configuration to the build system - to output hex files? Should the plugin somehow require this from the mbed_base.py? Is that even part of htrun?

If I want to support .bin files, I need to get the start address of the internal flash from the MCU. Is there any such mapping? I don't want to use mbed-ls here only to detect the target information. Is there a simple way to do this?

In my opinion .hex files are really simple to support here (because .hex files contain the loading address information), while .bin are pretty hard. I guess the reason why .bin is commonly used in mbed, is that the mbed drive does accept it (and it doesn't like .hex). The whole burden of knowing the load address is shifted onto the mbed drive.

There isn't any simple way to determine the load address of .bin file after it is generated. The addresses and what should go into .bin and/or .hex file is essentially determined by the linker script. Finding and parsing the linker script in htrun is in my opinion completely out of question.

To generate .hex file, the target should contain "OUTPUT_EXT": "hex". It is trivial for the user to create custom target based on any currently supported target and change the output format to .hex. Example custom_targets.json that changes the output format to .hex:

{ "CUSTOM_DISCO_F746NG": { "inherits": [ "DISCO_F746NG" ], "OUTPUT_EXT": "hex" } }

You can check which targets do output .hex by default by looking at targets.json. One such example is: LPC55S69_NS.

A bit different approach to the problem, would be to pass the .elf file to tests plugin, and the plugin could then parse the elf header and generate .bin and/or .hex files as needed. Whoever generates the .bin file out of .elf, knows the correct loading address.

Hi everyone, good discussion going on here!

Addressing your questions @embeddedteam103:

If I want to support .hex files, do I need to pass some configuration to the build system - to output hex files? Should the plugin somehow require this from the mbed_base.py? Is that even part of htrun?

Htrun doesn't know anything about the build system really, it just knows about targets and binaries. I think just checking the file extension of image_path would be sufficient to make the behavior conditional (as @desowin suggested).

If I want to support .bin files, I need to get the start address of the internal flash from the MCU. Is there any such mapping? I don't want to use mbed-ls here only to detect the target information. Is there a simple way to do this?

I don't really have much of an idea here unfortunately. Htrun doesn't have access to the start address of ROM at the moment. It only initially supported flashing via the Mbed drive (USB MSD) as @desowin mentioned, so this information was never integrated into this tool.

One idea we've had for the future is to consolidate around pyocd as our flashing and reset mechanism. It has growing target support. At the moment I believe it supports CMSIS-DAP and ST-Link debuggers. Still no mainline support for JLink, but I think that's being worked on.

madchutney · 2019-12-04T14:24:37Z

No response to code review for 9 months, closing PR.

0xc0170 · 2019-12-04T14:47:49Z

Adding an htrun-plugin that flashes, erases and resets using the JLINK commander software.

@embeddedteam103 It might be useful to update this via a new PR

CoolUsername added 4 commits March 25, 2019 19:38

add initial implementation of the copy method for the jlink plugin

d0dea3f

Add initial implementation of the reset method for the jlink plugin

a92ca20

add plugin to htrun package and to the list of supported plugins

934aeff

Fix TODOs

e10eb1d

embeddedteam103 mentioned this pull request Mar 26, 2019

How to run tests over SWD? ARMmbed/mbed-cli#475

Closed

bridadan suggested changes Mar 26, 2019

View reviewed changes

desowin suggested changes Mar 27, 2019

View reviewed changes

madchutney closed this Dec 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: Jlink plugin #150

Feature: Jlink plugin #150

embeddedteam103 commented Mar 26, 2019

bridadan left a comment

bridadan Mar 26, 2019

desowin Mar 27, 2019

embeddedteam103 Mar 27, 2019

bridadan Mar 26, 2019

bridadan Mar 26, 2019

bridadan Mar 26, 2019

bridadan Mar 26, 2019

bridadan Mar 26, 2019

bridadan Mar 26, 2019

0xc0170 Apr 30, 2019

bridadan Mar 26, 2019

bridadan Mar 26, 2019

embeddedteam103 Mar 27, 2019

bridadan Mar 28, 2019

bridadan Mar 26, 2019

desowin left a comment

desowin Mar 27, 2019

embeddedteam103 Mar 28, 2019

desowin Mar 28, 2019 •

edited

Loading

bridadan Mar 28, 2019

madchutney commented Dec 4, 2019

0xc0170 commented Dec 4, 2019

	capabilities = ['JTAG', 'SWD']
	capabilities = ['jlink_jtag', 'jlink_swd']

	Copyright (c) 2011-2015 ARM Limited
	Copyright (c) 2019 ARM Limited

		@@ -0,0 +1,113 @@
		# Copyright (c) 2018, Arm Limited and affiliates.

	# Copyright (c) 2018, Arm Limited and affiliates.
	# Copyright (c) 2019, Arm Limited and affiliates.

		@@ -0,0 +1,108 @@
		# Copyright (c) 2018, Arm Limited and affiliates.

	"""! In this implementation this plugin only is supporeted under Windows machines
	"""! In this implementation this plugin only is supported under Windows machines


		mcu = os.path.normpath(kwargs['mcu'])

		jlink_file_path = os.path.join(tempfile.gettempdir(), RESET_FILE)

Feature: Jlink plugin #150

Feature: Jlink plugin #150

Conversation

embeddedteam103 commented Mar 26, 2019

Description

Pull request type

Reviewers

bridadan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

desowin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

desowin Mar 28, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

madchutney commented Dec 4, 2019

0xc0170 commented Dec 4, 2019

desowin Mar 28, 2019 •

edited

Loading