Streamline the build of golang-related libraries #304

nfeske · 2022-09-05T10:02:48Z

Currently, the golang-related libraries (libatomic, libffi, libbacktrace) are built via target.mk files hosted under src/lib/.

With the upcoming change of genodelabs/genode#4599, those target.mk files will no longer be reachable for triggering the manual build of these targets.

I see two possible ways to go forward.

We could move the target.mk files to a location outside of src/lib/, such as src/golang/lib/.
We could move the rules of the target.mk into lib/mk/ files. The CUSTOM_TARGET_DEPS mechanism of Genode's build system might become handy (https://genode.org/documentation/genode-foundations/22.05/development/Build_system.html#Custom_targets_accompanying_a_library_or_program) to invoke the third-party build system from the library-description file. Naturally, the build results will reside somewhere under var/libcache/<libname>/. So the symlinks installed at bin/ must point to this place.

To me, the second option looks preferable because it has the potential to eliminate the manual steps of using golang (as described at https://github.com/genodelabs/genode-world/tree/master/src/test/go_app). The go application could then specify the required libs as LIBS dependency, which would trigger the build automatically.

Whichever way we chose, we'l need to adjust the documentation and perform a basic functional test.

@tor-m6 What do you think? Would you be willing to implement either of both options? Or do you see a simpler alternative?

The text was updated successfully, but these errors were encountered:

nfeske · 2023-01-11T11:13:31Z

Thank you very much @tor-m6 for attending this issue via pull request #312. Your commit is very much appreciated. I merged it to staging just now.

Now that you have moved the library builds to lib/mk/ files, do you see the potential to simplify the build instructions (src/test/go_app/README.golang) by adding lib/libatomic, lib/libbacktrace, libffi, and libgo to the build_components list of the run scripts (and probably also adding those to the LIBS of the target.mk files for go applications)?

tor-m6 · 2023-01-11T13:32:23Z

Thank you @nfeske for prompt review.

I still had some issues related to the fact that every lib here is build via go configure script. AFAIK I can't build somehow "natively" set of 4 libs, only one-by-one, and I use CUSTOM_DEPENDENCES to glue gnu_build.mk
Now all of them do build over sequence of "make lib/libatomic" calls, for a set of platforms each.

Anyway, I don't want to run them fully as a part of final application build. The reason is a latest changes (not sure where) - all these libs, together with separate libgcc.a/etc somehow added by libgo build script (Makefile generated after configure run) to inside libgo.a by complete scatter of my 3 libs into object files!

as a result, if I add these libs into application target.mk like this
LIBS += libatomic

it fail during linking process because of 2 kind of errors:

libgcc.a (and some others .a files) "are not an object" - yes, but why they linked by libtools into libgo.a? (small investigation show that potentially this is because it appears inside lib.mk file in LIBS and later in LIB_A, and I cant get rid of it)
duplicated symbols from files stored both inside libgo.a and libatiomc.a/etc. I spend a day trying to find a reason and fail - own gcc build is tricky.

So, after some try I found stupid solution for first problem - just remove added .a files manually from inside my libgo.mk/etc file during installation (see finished.tag rule)

For second problem I made another tricky way. I removed direct reference from LIBS variable in target.mk and libgo.mk - due to this these libraries do not appears in build script for libgo.a and later for any go_app, content of these 3 libs stored inside libgo.a and sufficient to link applications.

At least I don't know how to NOT to append libatomic.a/etc to the list of libraries in normal way. While, they also had some .h files (generated/copied from distro) which is placed in the way how libgo configure/Makefile assume (outside of var/libcache/libgo/ build dir -dirty hack).
As I result, I can't add them to a list of dependencies while have to be sure that it build before libgo (I add a check for finished.tag for every lib into libgo.mk, see checked.tag rule in libgo.mk)

May be you can suggest some other ways to work with these supplemental libs (I also suspect that they could be need for some other gcc compilers in the future, may be).
In short, e.g., I want that libatomic.mk do generate libatomic.a and some .h files, but if I add them to target.mk file (or libgo.mk), libatoimic.a should not appears in the LIB_A variable and other places. My analyses of current make build system show that this is impossible - may be I am wrong.

I suspect that this should be done inside file like import-libatomic.mk, but I can't cut libatomic.a from the internal variables of lib.mk / included files.

Any suggestions?

nfeske · 2023-01-18T11:05:20Z

Thank you very much for the additional details, and for exploring the direction I suggested.

The symptoms remind me of past situations where make variables were passed as environment variables to sub makes, which then ended up hard overriding variables in the sub make. The most recent example is genodelabs/genode#4725. When grepping the Genode source tree for unexport one can see several places where we had to explicitly prevent this effect.

In short, e.g., I want that libatomic.mk do generate libatomic.a and some .h files, but if I add them to target.mk file (or libgo.mk), libatoimic.a should not appears in the LIB_A variable and other places. My analyses of current make build system show that this is impossible - may be I am wrong.

Your analysis is right. But I think that the idea to side step the use of the LIB_A variable is not the best way to go. We should better try to prevent the LIB_A variable from interfering with the sub make in the first place.

Maybe @cproc's experience with porting the tool chain to Genode can be of help?

tor-m6 · 2023-01-18T14:59:31Z

I do not use LIB_A - it just filled from LIBS which is modified inside target.mk... Now I get rid of it - anyway, this is still not the best solution

cproc · 2023-01-27T10:37:56Z

Issue genodelabs/genode#4743 has a fix for the problem that libgcc.a and other existing static libraries got added to the newly created go-related static libraries. I'm currently looking for a solution to the other problem with the duplicated symbols.

tor-m6 · 2023-01-27T14:25:51Z

thanks, applied.

About duplicated symbols... during assembling of main libgo.a libtool completely disassemble related static libraries (libatomic.a/etc) into object files and add them "one by one" into libgo.a during "link" operation.
they had some unclear comments about why they do this, I still can't find any options not to do this (may be it is really needed? - I doubt so).

…ete) Issue genodelabs#304

Issue genodelabs#304

cproc · 2023-01-30T09:11:46Z

One way to solve the duplicated symbols problem could be to let lib/mk/libbacktrace.mk etc. not create libbacktrace.lib.a from libbacktrace.a and to have a separate lib/mk/backtrace.mk for use by non-go applications, which creates backtrace.lib.a from libbacktrace.a. Then go applications would link the empty libbacktrace.lib.a which is created by default and other applications would link backtrace.lib.a which contains the real implementation. Commit 107b0cb demonstrates this idea for libbacktrace in particular and commit 188f4f4 has a test application for the backtrace library which appears to work well, although it needs access to binary files with debug information.

tor-m6 · 2023-01-30T13:11:20Z

something wrong with this patch. I removed build dirs, and try
make -C build/x86_64/ KERNEL=linux BOARD=linux lib/libbacktrace VERBOSE= VERBOSE_MK=
or
make -C build/x86_64/ KERNEL=linux BOARD=linux lib/libatomic VERBOSE= VERBOSE_MK=
it hangs and generate 600+ make instances after
checking library dependencies...

when I press CtrlC it says something like

admin@genode_tap:~/gen/22.11$ make -C build/x86_64/ KERNEL=linux BOARD=linux lib/libbacktrace VERBOSE= VERBOSE_MK=
make: Entering directory '/var/services/homes/admin/gen/22.11/build/x86_64'
checking library dependencies...

for lib in libbacktrace; do
make -j4 SHELL=/usr/bin/bash --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk
REP_DIR=$rep LIB=$lib
BUILD_BASE_DIR=/var/services/homes/admin/gen/22.11/build/x86_64
DARK_COL="\033[00;33m" DEFAULT_COL="\033[0m";
echo "all: $lib.lib" >> var/libdeps;
done;
for target in ; do
for rep in /var/services/homes/admin/gen/22.11/repos/base-linux /var/services/homes/admin/gen/22.11/repos/base /var/services/homes/admin/gen/22.11/repos/os /var/services/homes/admin/gen/22.11/repos/demo /var/services/homes/admin/gen/22.11/repos/libports /var/services/homes/admin/gen/22.11/repos/ports /var/services/homes/admin/gen/22.11/repos/dde_linux /var/services/homes/admin/gen/22.11/repos/gems /var/services/homes/admin/gen/22.11/repos/world /var/services/homes/admin/gen/22.11/repos/pc /var/services/homes/admin/gen/22.11/repos/dde_bsd /var/services/homes/admin/gen/22.11/repos/dde_ipxe; do
test -f $rep/src/$target || continue;
make -j4 SHELL=/usr/bin/bash --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_prg.mk
REP_DIR=$rep TARGET_MK=$rep/src/$target
BUILD_BASE_DIR=/var/services/homes/admin/gen/22.11/build/x86_64
DARK_COL="\033[00;33m" DEFAULT_COL="\033[0m" || result=false;
break;
done;
done; $result;
^Cmake[287]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt
make[286]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt
make[285]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt
make[284]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt
make[283]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt
make[282]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt
make[281]: *** [/var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk:138: generate_lib_rule] Interrupt

cproc · 2023-01-30T14:10:39Z

Hm, I cannot reproduce this problem, unfortunately. Do you get some helpful information when you remove the @ character from the for loop in dep_lib.mk:138?

tor-m6 · 2023-01-30T20:11:25Z

a bit more.
I add also before this line
echo LLL: $(LIBS_TO_VISIT) AAA $(LIBS)

for lib in libatomic; do
make -j4 SHELL=/usr/bin/bash --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk
REP_DIR=$rep LIB=$lib
BUILD_BASE_DIR=/var/services/homes/admin/gen/22.11/build/x86_64
DARK_COL="\033[00;33m" DEFAULT_COL="\033[0m";
echo "all: $lib.lib" >> var/libdeps;
done;
for target in ; do
for rep in /var/services/homes/admin/gen/22.11/repos/base-linux /var/services/homes/admin/gen/22.11/repos/base /var/services/homes/admin/gen/22.11/repos/os /var/services/homes/ad
test -f $rep/src/$target || continue;
make -j4 SHELL=/usr/bin/bash --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_prg.mk
REP_DIR=$rep TARGET_MK=$rep/src/$target
BUILD_BASE_DIR=/var/services/homes/admin/gen/22.11/build/x86_64
DARK_COL="\033[00;33m" DEFAULT_COL="\033[0m" || result=false;
break;
done;
done; $result;
echo LLL: libatomic libc posix AAA libatomic libc posix
LLL: libatomic libc posix AAA libatomic libc posix
for i in libatomic libc posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic libc posix libc posix AAA libatomic libc posix libc posix
LLL: libatomic libc posix libc posix AAA libatomic libc posix libc posix
for i in libatomic libc posix libc posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic libc posix libc posix libc posix AAA libatomic libc posix libc posix libc posix
LLL: libatomic libc posix libc posix libc posix AAA libatomic libc posix libc posix libc posix
for i in libatomic libc posix libc posix libc posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic libc posix libc posix libc posix libc posix AAA libatomic libc posix libc posix libc posix libc posix
LLL: libatomic libc posix libc posix libc posix libc posix AAA libatomic libc posix libc posix libc posix libc posix
for i in libatomic libc posix libc posix libc posix libc posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic libc posix libc posix libc posix libc posix libc posix AAA libatomic libc posix libc posix libc posix libc posix libc posix
LLL: libatomic libc posix libc posix libc posix libc posix libc posix AAA libatomic libc posix libc posix libc posix libc posix libc posix
for i in libatomic libc posix libc posix libc posix libc posix libc posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libc-string libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser lit
LLL: libc-string libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser libc-net
for i in libc-string libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser lib
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/libports LIB=$i; done
echo LLL: libc-string libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser lit
LLL: libc-string libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser libc-net
for i in libc-string libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser lib
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/libports LIB=$i; done
echo LLL: libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser libc-net libc-t
LLL: libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser libc-net libc-rpc lt
for i in libc-locale libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser libc-net libc-r
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/libports LIB=$i; done
echo LLL: libc-stdlib libc-stdio libc-gen libc-gdtoa libc-inet libc-stdtime libc-regex libc-compat libc-setjmp libc-mem libc-resolv libc-isc libc-nameser libc-net libc-rpc libc-tzct

tor-m6 · 2023-01-30T20:12:58Z

also I found that progress.log is empty - not filled by libs, probable key of the problem is here
LIBS_TO_VISIT = $(filter-out $(LIBS_READY),$(LIBS))

I removed it and comment out this string in libatomic.mk
#LIBS += libc
then I had in progress.log

Library build progress log - generated by dep_prg.mk and dep_lib.mk

Build artifact posix.lib.so

LIBS_READY += posix

and in the output
...
echo LLL: libatomic posix AAA libatomic posix
LLL: libatomic posix AAA libatomic posix
for i in libatomic posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic posix posix AAA libatomic posix posix
LLL: libatomic posix posix AAA libatomic posix posix
for i in libatomic posix posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic posix posix posix AAA libatomic posix posix posix
LLL: libatomic posix posix posix AAA libatomic posix posix posix
for i in libatomic posix posix posix; do
make --no-print-directory -f /var/services/homes/admin/gen/22.11/repos/base/mk/dep_lib.mk REP_DIR=/var/services/homes/admin/gen/22.11/repos/world LIB=$i; done
echo LLL: libatomic posix posix posix posix AAA libatomic posix posix posix posix
LLL: libatomic posix posix posix posix AAA libatomic posix posix posix posix
...

cproc · 2023-01-31T07:22:52Z

When I add the LLL/AAA output, the first message I see is

echo LLL: libc posix AAA libc posix

It does not contain libatomic here.

chelmuth · 2023-01-31T07:45:52Z

As we're currently addressing issues with different versions of GNU Make in genodelabs/genode#4725, may I ask which version do you both use currently?

cproc · 2023-01-31T07:48:23Z

As we're currently addressing issues with different versions of GNU Make in genodelabs/genode#4725, may I ask which version do you both use currently?

4.1

tor-m6 · 2023-01-31T10:43:16Z

4.2.1

tor-m6 · 2023-01-31T18:15:12Z

now I try to kill some files (libdeps and progress.log) from build dir and found circular rule in build/x86_64/var/libdeps

libatomic.lib: check_ports libatomic.lib posix.lib

tor-m6 · 2023-01-31T19:59:41Z

I set make to 4.1

now, when I try to build run/rm_sub (no relation to libatomic/etc)
I fail again:
var/libdeps:21: *** Recursive variable 'DEP_A_cxx' references itself (eventually). Stop.

in libdep I found

#
# Library dependencies for build 'core init test/sub_rm'
#

export SPEC_FILES := \
  /var/services/homes/admin/gen/22.11/repos/base/mk/spec/x86_64.mk \

LIB_CACHE_DIR = /var/services/homes/admin/gen/22.11/build/x86_64/var/libcache
BASE_DIR      = /var/services/homes/admin/gen/22.11/repos/base
VERBOSE      ?= @
VERBOSE_MK   ?= @
VERBOSE_DIR  ?= --no-print-directory
INSTALL_DIR  ?= /var/services/homes/admin/gen/22.11/build/x86_64/bin
DEBUG_DIR    ?= /var/services/homes/admin/gen/22.11/build/x86_64/debug
SHELL        ?= /usr/bin/bash
MKDIR        ?= mkdir

all:
        @true # prevent nothing-to-be-done message

DEP_A_cxx  = ${ARCHIVE_NAME(cxx)} $(DEP_A_cxx) ${ARCHIVE_NAME(base-linux-common)} $(DEP_A_base-linux-common) ${ARCHIVE_NAME(startup-linux)} $(DEP_A_startup-linux) ${ARCHIVE_NAME(ce)
...

tor-m6 · 2023-01-31T20:14:52Z

I applied patch below before your - may be this is the reason?

gnu_build.mk: pass static libraries in '-l:' format

Fixes #4743

cproc · 2023-02-01T06:37:14Z

I use the following branches:

https://github.com/genodelabs/genode/tree/staging (last commit 0bc124a)
https://github.com/cproc/genode-world/tree/issue304

When I build run/sub_rm, my var/libdeps does not contain a DEP_A_cxx line and the first line related to cxx is cxx.lib: check_ports.

tor-m6 · 2023-02-01T10:18:07Z

I made a clean copy based on staged - everything works ok (probably my working copy was somehow damaged), except that:

when I build and run go_app, then removed var/libcache/libatomic and try to rebuild application - it rebuild libatomic but do not update libgo (while it is explicitly use their includes and take .o files and include them into libgo.a).

something not correct in dependence handling...
Also, when I built run/backtrace - it create another copy of libbacktrace in build/x86_64/var/libcache/backtrace/.libs/ , is it as intended?

cproc · 2023-02-02T07:38:23Z

when I build and run go_app, then removed var/libcache/libatomic and try to rebuild application - it rebuild libatomic but do not update libgo (while it is explicitly use their includes and take .o files and include them into libgo.a).

When I build go for Linux (with configure and make) and remove the libatomic build directory and then run make again, libatomic gets rebuilt, but libgo does not get rebuilt as well. So, not sure if anything can be done about it in the Genode build system.

Also, when I built run/backtrace - it create another copy of libbacktrace in build/x86_64/var/libcache/backtrace/.libs/ , is it as intended?

A solution with LIBS=libbacktrace in backtrace.mk would propably be better, but when I experimented with that, I did not find a suitable dependency that makes sure that the empty default backtrace.lib.a exists before it gets replaced by the symlink to libbacktrace.a.

tor-m6 · 2023-02-02T08:16:51Z

probably we need to be sure that user is aware that if libatomic/etc was build/rebuild then libgo should be updated as well, may be manually. Like message in the end of build.

may be it worth to make the same dirty hack to go outside library build environmnt? We know that both libbacktrace.lib.a and backtrace.a are in var/libcache/libbacktrace and .../backtrace accordingly.
so, using target like ../libbacktrace.a allow us to be sure about that?
also to check that we can use target related to file inside library as I remember, like
backtrace.lib.a(backtrace.o) :
...
https://www.gnu.org/software/make/manual/html_node/Archive-Members.html#Archive-Members

we can point to some known object from libbacktrace as above to be sure that it will hit if library not exist or empty

…ete) Issue genodelabs#304

Issue genodelabs#304

cproc · 2023-02-08T10:16:36Z

Commit f6402e8 builds the backtrace library with LIBS=libbacktrace. The missing key was the ifeq ($(called_from_lib_mk),yes) check. When this condition is met, the rule for creating the symlink can depend on backtrace.lib.a.

cproc · 2023-02-28T11:31:31Z

It seems like this issue got automatically closed when the original commit went to the master branch.

mewmew · 2024-04-15T13:03:50Z

I tried to run the go_app example application on the upcoming release of Sculpt 24.04 (rev genodelabs/genode@dc0e78c and 67fcb0e), following the instructions presented in README.golang.

The libgo target builds successfully.

However, it seems there's some issue when running go_app, namely Error: deadlock ahead. See detailed output below.

Output from `make -C ... run/go_app` command

  Booting `Genode on NOVA'

Bender Version 0.9-beta7-51-g28ba2ad-dirty
Patching BDA with I/O port 0x3f8.
Bender: Hello World.
hwp config: eeo=na irq=na hwp=na epp=na epb=na

NOVA Microhypervisor v9-3e34fa6 (x86_64): [gcc 12.3.0] [MBI2]

[ 0] TSC:2939135 kHz BUS:1012009 kHz (measured)
[ 0] CORE:00:00:0 6:1a:3:0 [1] Intel Core i7 9xx (Nehalem Core i7, IBRS update)
[ 3] CORE:00:03:0 6:1a:3:0 [1] Intel Core i7 9xx (Nehalem Core i7, IBRS update)
[ 2] CORE:00:02:0 6:1a:3:0 [1] Intel Core i7 9xx (Nehalem Core i7, IBRS update)
[ 1] CORE:00:01:0 6:1a:3:0 [1] Intel Core i7 9xx (Nehalem Core i7, IBRS update)

Hypervisor NOVA (API v9)
core     image  [0000000000100000,00000000017a3000)
binaries region [000000000027a000,00000000017a3000) free for reuse
detected physical memory: 0x0000000000000000 - size: 0x000000000009fc00
use      physical memory: 0x0000000000000000 - size: 0x000000000009f000
detected physical memory: 0x0000000000100000 - size: 0x000000001fedf000
use      physical memory: 0x0000000000100000 - size: 0x000000001fedf000
reserved memory: 0x1e93a000 - size: 0x16a4338 type=-2
reserved memory: 0x9fc00 - size: 0x400 type=2
reserved memory: 0xf0000 - size: 0x10000 type=2
reserved memory: 0x1ffdf000 - size: 0x21000 type=2
reserved memory: 0xb0000000 - size: 0x10000000 type=2
reserved memory: 0xfed1c000 - size: 0x4000 type=2
reserved memory: 0xfffc0000 - size: 0x40000 type=2
reserved memory: 0xfd000000 - size: 0x3e8000 type=-5
reserved memory: 0x200000 - size: 0x4000 type=-1
reserved memory: 0x1000 - size: 0x1000 type=-1
reserved memory: 0x1000000 - size: 0x800000 type=-1
reserved memory: 0x1800000 - size: 0x1c00000 type=-1
reserved memory: 0x1ffe2413 - size: 0x0 type=-3
Hypervisor reports 4x1 CPUs
Warning: CPU has no invariant TSC.
mapping: affinity space -> kernel cpu id - package:core:thread
 remap (0x0) -> 0 - 0:0:0 boot cpu
 remap (1x0) -> 1 - 0:1:0
 remap (2x0) -> 2 - 0:2:0
 remap (3x0) -> 3 - 0:3:0
ROM modules:
 ROM: [000000001faf3000,000000001faf3559) config
 ROM: [000000000000e000,0000000000012000) core_log
 ROM: [000000001eacb000,000000001f927750) go_app
 ROM: [000000001ff65000,000000001ffd4670) init
 ROM: [000000001fb10000,000000001fbfcc48) ld.lib.so
 ROM: [000000001f928000,000000001faf2858) libc.lib.so
 ROM: [000000001ff1d000,000000001ff64608) libm.lib.so
 ROM: [000000000000a000,000000000000d000) platform_info
 ROM: [000000001ffd5000,000000001ffddec0) posix.lib.so
 ROM: [000000001fbfd000,000000001fe78690) stdcxx.lib.so
 ROM: [000000001faf4000,000000001fb0f098) timer
 ROM: [000000001fe79000,000000001ff1c9f8) vfs.lib.so
 ROM: [000000001eab5000,000000001eaca940) vfs_pipe.lib.so

36884K kernel memory

Genode 24.02-175-gdc0e78cdf8
452 MiB RAM and 63249 caps assigned to init
[init -> go_app]   0x1000000 .. 0x10ffffff: linker area
[init -> go_app]   0x40000000 .. 0x4fffffff: stack area
[init -> go_app]   0x30000 .. 0x14dfff: ld.lib.so
[init -> go_app]   0x10ff8000 .. 0x10ffffff: posix.lib.so
[init -> go_app]   0x10e15000 .. 0x10ff7fff: libc.lib.so
[init -> go_app]   0x10d71000 .. 0x10e14fff: vfs.lib.so
[init -> go_app]   0x10d2f000 .. 0x10d70fff: libm.lib.so
[init -> go_app]   0x1a9c000 .. 0x1d19fff: stdcxx.lib.so
[init -> go_app]   0x10d1a000 .. 0x10d2efff: vfs_pipe.lib.so
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: int sigaltstack(const stack_t*, stack_t*) called, not implemented
[init -> go_app] Warning: sched_yield: sched_yield not implemented
[init -> go_app] Checking waiting room: 0
[init -> go_app] Warning: sched_yield: sched_yield not implemented
[init -> go_app] fatal error: clock_gettime
[init -> go_app] Warning: sched_yield: sched_yield not implemented
[init -> go_app] 
[init -> go_app] runtime stack:
[init -> go_app] Error: deadlock ahead, mutex=0x14dc10, return ip=0xdd927
Expect: 'interact' received 'strg+c' and was cancelled
make: Leaving directory '/home/u/Desktop/genode_24.04/build/x86_64'

tor-m6 · 2024-04-15T20:50:33Z

I check this recently and still not have a time to fix

Problem that after latest updates, seems that naming of binaries and access to them via run file changed

To work successfully, I need to provide symbolic names for every binary to be able to unwind stack as golang requires.

“Deadlock ahead” is a result of panic being hit during symlink processing, relate to the fact what we did not find a binary name inside run file.
This is not provided “by default” by Genode

I emulate it inside the code , as stated in 23.11 branch of Genode-world from my repo.
https://github.com/tor-m6/genode-world/tree/23.11
Last version of genode change something here (works in 23.05 at least, and 23.08 as I remember) and this is a bit unclear for me, how to fix. My first attempt was failed.

@cproc may be there are easier way to have a binary names in the way that libbacktrace from gcc will work correctly?

Stack unwinding is a normal part of golang support code work on every goroutine context change, not "an exception"...

This is what I typically use to generate in run file all names in vfs section (and I am not that happy - this is complex and error-prune).

Like for each binary which can appears in the stack

# build section
set build_components {
	core init timer lib/ld lib/libc lib/libm lib/posix lib/stdcxx lib/vfs
}

lappend build_components "$executable_place/$executable"

append config {
	<start name="} $executable {" caps="} $executable_caps {">
		<config verbose="} $conf_verbose {"  ld_verbose="} $conf_ld_verbose {">
			<arg value="} $executable {" />
			<vfs>
				<dir name="dev"> <log/> </dir>
}

# add every library/binaries (mandatory for golang stack unwind)
foreach lib $modules {
	append config "<rom name=\"$lib\" label=\"$lib\"/>"
}

append config {
				<rom name="binary" label="} $executable {"/>
				<dir name="proc">
					<dir name="self">
						<rom name="exe" label="} $executable {"/>
					</dir>
				</dir>
}

tor-m6 · 2024-04-28T17:21:23Z

However, it seems there's some issue when running go_app, namely Error: deadlock ahead. See detailed output below.

Seems that now rtc driver became mandatory to clock_gettime() in libc.

fixed in commit 7f5eeeef84787cf07dfebbd1fac224e22f6c0333

Bug related to "deadlock ahead" fixed temporary in tor-m6/genode@658283c

cproc added a commit to cproc/genode-world that referenced this issue Jan 30, 2023

go: build libgo dependencies automatically via LIBS variable (incompl…

107b0cb

…ete) Issue genodelabs#304

cproc added a commit to cproc/genode-world that referenced this issue Jan 30, 2023

Add backtrace test

188f4f4

Issue genodelabs#304

tor-m6 pushed a commit to tor-m6/genode-world that referenced this issue Feb 2, 2023

go: build libgo dependencies automatically via LIBS variable (incompl…

a1f0384

…ete) Issue genodelabs#304

tor-m6 pushed a commit to tor-m6/genode-world that referenced this issue Feb 2, 2023

Add backtrace test

3993c6c

Issue genodelabs#304

cproc added a commit to cproc/genode-world that referenced this issue Feb 8, 2023

go: build libgo dependencies automatically via LIBS variable (backtrace)

f6402e8

Issue genodelabs#304

chelmuth closed this as completed in 10fb4d7 Feb 28, 2023

cproc reopened this Feb 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Streamline the build of golang-related libraries #304

Streamline the build of golang-related libraries #304

nfeske commented Sep 5, 2022

nfeske commented Jan 11, 2023

tor-m6 commented Jan 11, 2023

nfeske commented Jan 18, 2023

tor-m6 commented Jan 18, 2023

cproc commented Jan 27, 2023

tor-m6 commented Jan 27, 2023

cproc commented Jan 30, 2023

tor-m6 commented Jan 30, 2023

cproc commented Jan 30, 2023

tor-m6 commented Jan 30, 2023

tor-m6 commented Jan 30, 2023 •

edited

Loading

cproc commented Jan 31, 2023

chelmuth commented Jan 31, 2023

cproc commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

cproc commented Feb 1, 2023

tor-m6 commented Feb 1, 2023 •

edited

Loading

cproc commented Feb 2, 2023

tor-m6 commented Feb 2, 2023

cproc commented Feb 8, 2023

cproc commented Feb 28, 2023

mewmew commented Apr 15, 2024

tor-m6 commented Apr 15, 2024 •

edited

Loading

tor-m6 commented Apr 28, 2024 •

edited

Loading

Streamline the build of golang-related libraries #304

Streamline the build of golang-related libraries #304

Comments

nfeske commented Sep 5, 2022

nfeske commented Jan 11, 2023

tor-m6 commented Jan 11, 2023

nfeske commented Jan 18, 2023

tor-m6 commented Jan 18, 2023

cproc commented Jan 27, 2023

tor-m6 commented Jan 27, 2023

cproc commented Jan 30, 2023

tor-m6 commented Jan 30, 2023

cproc commented Jan 30, 2023

tor-m6 commented Jan 30, 2023

tor-m6 commented Jan 30, 2023 • edited Loading

Library build progress log - generated by dep_prg.mk and dep_lib.mk

Build artifact posix.lib.so

cproc commented Jan 31, 2023

chelmuth commented Jan 31, 2023

cproc commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

tor-m6 commented Jan 31, 2023

cproc commented Feb 1, 2023

tor-m6 commented Feb 1, 2023 • edited Loading

cproc commented Feb 2, 2023

tor-m6 commented Feb 2, 2023

cproc commented Feb 8, 2023

cproc commented Feb 28, 2023

mewmew commented Apr 15, 2024

tor-m6 commented Apr 15, 2024 • edited Loading

tor-m6 commented Apr 28, 2024 • edited Loading

tor-m6 commented Jan 30, 2023 •

edited

Loading

tor-m6 commented Feb 1, 2023 •

edited

Loading

tor-m6 commented Apr 15, 2024 •

edited

Loading

tor-m6 commented Apr 28, 2024 •

edited

Loading