Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch build system to Meson #304

Merged
merged 10 commits into from
Apr 22, 2024
Merged

Switch build system to Meson #304

merged 10 commits into from
Apr 22, 2024

Conversation

ueno
Copy link
Collaborator

@ueno ueno commented Nov 6, 2023

This adds a basic support for building and installing the provider
with Meson. To test:

meson setup build
meson compile -C build
meson test -C build --num-processes 1

Note that parallel execution of tests are currently not working, as
the same port number is used in multiple tests.

Here is some benchmark:

$ time sh -c 'autoreconf -fi && ./configure && make && make check'
18.18s user 2.97s system 73% cpu 28.639 total

$ time sh -c 'meson setup build && meson compile -C build && meson test -C build --num-processes 1'
10.61s user 1.82s system 81% cpu 15.173 total

@simo5
Copy link
Member

simo5 commented Nov 6, 2023

ok so I think a switch to meson is something we can consider, but it would have to be a full switch, definitely not going to maintain the same stuff in parallel in two places

FYI ignore debian, macos test failures for now they got the broken NSS 3.94;
but they should only fail tests and nothing else of course

Anyway for a change like this we'd have to wait until debian and macos upgrade to 3.95 in a month because we definitely need to ensure all the platforms work end-to-end

@ueno ueno force-pushed the wip/dueno/meson branch 7 times, most recently from b6ffc75 to 6d17aed Compare November 7, 2023 07:58
@ueno ueno force-pushed the wip/dueno/meson branch 4 times, most recently from 489db39 to e971bff Compare April 9, 2024 07:23
@ueno ueno changed the title Support Meson as an alternative build system Switch build system to Meson Apr 9, 2024
@ueno ueno force-pushed the wip/dueno/meson branch 11 times, most recently from 04b88dd to 0da508b Compare April 9, 2024 11:06
@ueno ueno marked this pull request as ready for review April 9, 2024 11:13
@ueno
Copy link
Collaborator Author

ueno commented Apr 9, 2024

@simo5 I think this is almost ready, though I would need to take a closer look at the current CI failures (macOS build, address sanitizer with Debian, scan-build, and check style).

After the xz backdoor, it seems to be more relevant to switch to a simpler build system that doesn't allow extensions in a implicit/obfuscated way.

@simo5
Copy link
Member

simo5 commented Apr 9, 2024

Given you are removing the old build system, can you provide a Makefile with targets that correspond to today high level targets that just turn around and run meson commands?

At least:

  • make
  • make check
  • make check-stile
  • make check-style-show
  • make check-style-fix
  • make generate-code
  • make generate-docs

Copy link
Member

@simo5 simo5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good, just some nitpicks

packaging/pkcs11-provider.spec Show resolved Hide resolved
.github/workflows/scan-build.yml Outdated Show resolved Hide resolved
.github/workflows/scan-build.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
@ueno ueno force-pushed the wip/dueno/meson branch from 257b48c to 9980750 Compare April 16, 2024 13:35
src/signature.c Show resolved Hide resolved
src/signature.c Outdated Show resolved Hide resolved
tests/test-wrapper Outdated Show resolved Hide resolved
This removes checks for <string.h> and strpbrk as they are used
unconditionally and unlikely to be absent on any supported platforms.

This also makes it error out when <dlfcn.h> is not found.

Signed-off-by: Daiki Ueno <[email protected]>
@ueno ueno force-pushed the wip/dueno/meson branch from 53b839c to 27b7544 Compare April 17, 2024 22:27
ueno added 3 commits April 18, 2024 07:32
For better support for out-of-tree builds, this removes assumption in
which directory the test scripts are running.

Signed-off-by: Daiki Ueno <[email protected]>
This adds a basic support for building and installing the provider
with Meson.  To test:

```console
meson setup build
meson compile -C build
meson test -C build
```

Note that parallel execution of tests are currently disabled, as
the same port number is used in multiple tests.

Signed-off-by: Daiki Ueno <[email protected]>
@ueno ueno force-pushed the wip/dueno/meson branch 2 times, most recently from 8cfc386 to de08d11 Compare April 17, 2024 22:37
Copy link
Member

@simo5 simo5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I think we are really close.
The bind test failure is a regression in bind and we will waive it.

However I just realized that this PR will break the coverity scan.
Coverity relies on the command "make" to generate a build, it may even pass variables there.

Unfortunately it cannot be easily tested on a PR as the coverity scan can only run against an action already merged in the main tree.

But I can attempt a test in a local checkout to confirm that just providing a make :all target that call meson compile will work

@simo5
Copy link
Member

simo5 commented Apr 19, 2024

So I think coverity depends only on a PATH change, injecting a different gcc in a coverity/bin path or similar.

How will meson interact with that ?

When I run meson compile -vC builddir it gives me what looks like a pseudo-command line as it identifies the compiler as just 'cc', so iut is not clear to me if meson executes the compiler via an abslute path or will allow interposition by calling the compiler with it's relative name.

@simo5
Copy link
Member

simo5 commented Apr 19, 2024

I did find his post though:
https://community.synopsys.com/s/question/0D5Hr00006VdMFsKAN/in-ninja-build-environment-covbuild-cannot-emit-c-files

Which means you may need to tweak the .github/workflows/coverity-scan.yml steps before we can merge this PR, sorry for not realizing this earlier.

ueno and others added 5 commits April 20, 2024 07:37
Signed-off-by: Daiki Ueno <[email protected]>
Loadable modules on macOS should use .so as extension, but both NSS
softokn and OpenSSL providers do not follow this rule. Modify shlext
accordingly so that NSS softokn can be found for the tests.

Note that p11-kit correctly names its loadable modules with .so, so this
variable should be used carefully as it may be incorrect depending on
your use case.

Signed-off-by: Clemens Lang <[email protected]>
This suppresses a couple of dead assignments and unlikely null pointer
dereferences.

Signed-off-by: Daiki Ueno <[email protected]>
@ueno ueno force-pushed the wip/dueno/meson branch from de08d11 to 0583524 Compare April 19, 2024 22:38
@ueno
Copy link
Collaborator Author

ueno commented Apr 19, 2024

I did find his post though: https://community.synopsys.com/s/question/0D5Hr00006VdMFsKAN/in-ninja-build-environment-covbuild-cannot-emit-c-files

Which means you may need to tweak the .github/workflows/coverity-scan.yml steps before we can merge this PR, sorry for not realizing this earlier.

Thank you for the pointer. I added env.CC: gcc in the setup step, following this example linked from the coverity-scan-action README. Hopefully it should work.

@simo5
Copy link
Member

simo5 commented Apr 22, 2024

I did find his post though: https://community.synopsys.com/s/question/0D5Hr00006VdMFsKAN/in-ninja-build-environment-covbuild-cannot-emit-c-files
Which means you may need to tweak the .github/workflows/coverity-scan.yml steps before we can merge this PR, sorry for not realizing this earlier.

Thank you for the pointer. I added env.CC: gcc in the setup step, following this example linked from the coverity-scan-action README. Hopefully it should work.

Ok, I will merge this, once it is merged I will know soon enough if the covbuild fails. If it does I will experiment with it on a in tree branch until we get it right.

@simo5
Copy link
Member

simo5 commented Apr 22, 2024

@dueno we sill need a Makefile all target to call meson compile as the covscan build just calls make with no arguments

@ueno
Copy link
Collaborator Author

ueno commented Apr 22, 2024

@dueno we sill need a Makefile all target to call meson compile as the covscan build just calls make with no arguments

I can add such make rule, but doesn't coverity-scan-action take care of that already, given the openrc example mentioned above doesn't have Makefile at all?

While coverity-scan-action supports meson/ninja directly, as we have a
maintainer Makefile in the top-level, it requires the standard make
rules (make all, etc.).

Signed-off-by: Daiki Ueno <[email protected]>
@simo5
Copy link
Member

simo5 commented Apr 22, 2024

@dueno we sill need a Makefile all target to call meson compile as the covscan build just calls make with no arguments

I can add such make rule, but doesn't coverity-scan-action take care of that already, given the openrc example mentioned above doesn't have Makefile at all?

Sure we can avoid that if you want, but then you should add a command: item in the yaml file to invoke the correct build command I guess.
See line 23 of the file you linked.

@simo5
Copy link
Member

simo5 commented Apr 22, 2024

I am going to ignore the MacOS failure as it seems a homebrew screwup, as all of this was working until today.
Hopefully it will get fixed soon, and if not we'll handle it separately anyway.

@simo5 simo5 added the covscan Triggers Coverity Scanner label Apr 22, 2024
@github-actions github-actions bot removed the covscan Triggers Coverity Scanner label Apr 22, 2024
@simo5 simo5 merged commit c822c11 into latchset:main Apr 22, 2024
26 of 29 checks passed
@neverpanic
Copy link
Collaborator

The macOS CI failure is a Homebrew screw-up, see Homebrew/homebrew-core#169728.

I can locally reproduce:

$ podman pull ghcr.io/homebrew/core/openssl/3:3.3.0@sha256:636c1743241379d29fb964fcccce1ea86f46fa66a85319c1ac59c9698e4ac809  # this is the pre-compiled OpenSSL for Homebrew
$ podman image save 1f09999b3994 | tar xO f35bb52aadbdae43149bf5eb3a3a4948c1149879d13d2041e0139cbd80a08535.tar | tar tv | grep libcrypto.dylib
lrwxrwxrwx  0 0      0           0 Apr  9 14:12 openssl@3/3.3.0/lib/libcrypto.dylib -> libcrypto.3.dylib
$ podman image save 1f09999b3994 | tar xO f35bb52aadbdae43149bf5eb3a3a4948c1149879d13d2041e0139cbd80a08535.tar | tar xO openssl@3/3.3.0/lib/pkgconfig/libcrypto.pc | head -1
libdir=@@HOMEBREW_CELLAR@@/openssl@3/3.3.0

^ there's a /lib missing at the end of that path in the pkg-config file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants