p11-kit (current master) somehow fails token operations (???) #501

mouse07410 · 2023-05-02T02:36:03Z

MacOS Ventura 13.3.1, Macports-installed OpenSSL-3.1.0, pkcs11-provider (main), OpenSC (master, with commit 1d5c81e85baaa8 reverted).

In short, when pkcs11-provider for OpenSSL is configured to use p11-kit-proxy.dylib to select the module to load - operations fail. When I force the specific driver, like /Library/OpenSC/lib/opensc-pkcs11.so or /usr/local/lib/libykcs11.dylib (in other words, take p11-kit out of the picture) - operations succeed.

openssl.cnf config:

[pkcs11_sect]
 module = /opt/local/libexec/openssl3/lib/ossl-modules/pkcs11.dylib
 pkcs11-module-quirks = no-deinit
 pkcs11-module-login-behavior = auto
 pkcs11-module-cache-pins = cache
 #pkcs11-module-path = /Library/OpenSC/lib/opensc-pkcs11.so
 #pkcs11-module-path = /usr/local/lib/libykcs11.dylib
 #pkcs11-module-path = /Library/OpenSC/lib/pkcs11-spy.so
 pkcs11-module-path = /opt/local/lib/p11-kit-proxy.dylib
 activate = 1

Failures:

$ openssl dgst  -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:48 -out /tmp/pt.txt.sig2 /tmp/pt.txt
Enter pass phrase for PKCS#11 Token (Slot 17 - Yubico YubiKey OTP+FIDO+CCID):
Error setting context
405BA0FA01000000:error:03000096:digital envelope routines:do_sigver_init:operation not supported for this keytype:crypto/evp/m_sigver.c:298:
$ 
$ openssl dgst  -sign "pkcs11:model=YubiKey%20YK4;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:48 -out /tmp/pt.txt.sig3 /tmp/pt.txt
Enter pass phrase for PKCS#11 Token (Slot 21 - Yubico YubiKey OTP+FIDO+CCID):
Error setting context
405BA0FA01000000:error:03000096:digital envelope routines:do_sigver_init:operation not supported for this keytype:crypto/evp/m_sigver.c:298:
$

Successes:

$ PKCS11_PROVIDER_MODULE=/Library/OpenSC/lib/opensc-pkcs11.so openssl dgst  -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:48 -out /tmp/pt.txt.sig2 /tmp/pt.txt
Enter pass phrase for PKCS#11 Token (Slot 0 - Yubico YubiKey OTP+FIDO+CCID):
$ ll /tmp/pt.txt.sig2
-rw-r--r--  1 ur20980  wheel  256 May  1 22:28 /tmp/pt.txt.sig2
$ PKCS11_PROVIDER_MODULE=/usr/local/lib/libykcs11.dylib openssl dgst  -sign "pkcs11:model=YubiKey%20YK4;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:48 -out /tmp/pt.txt.sig3 /tmp/pt.txt
Enter pass phrase for PKCS#11 Token (Slot 0 - Yubico YubiKey OTP+FIDO+CCID):
$ ll /tmp/pt.txt.sig3
-rw-r--r--  1 ur20980  wheel  256 May  1 22:29 /tmp/pt.txt.sig3
$

@simo5 It looks like the problem manifests itself when pkcs11-provider is used, because with libp11 engine it seems to work:

$ openssl dgst -engine pkcs11 -keyform engine  -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:48 -out /tmp/pt.txt.sig2 /tmp/pt.txt
Engine "pkcs11" set.
Enter PKCS#11 token PIN for XXXXXXXXXXXX:
$ ll /tmp/pt.txt.sig2
-rw-r--r--  1 ur20980  wheel  256 May  1 22:30 /tmp/pt.txt.sig2
$ openssl dgst -engine pkcs11 -keyform engine  -sign "pkcs11:model=YubiKey%20YK4;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:48 -out /tmp/pt.txt.sig3 /tmp/pt.txt
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiKey PIV #xxxxxxxxx:
Enter PKCS#11 key PIN for Private key for Digital Signature:
$ ll /tmp/pt.txt.sig3
-rw-r--r--  1 ur20980  wheel  256 May  1 22:31 /tmp/pt.txt.sig3
$

Also, see latchset/pkcs11-provider#234

The text was updated successfully, but these errors were encountered:

dengert · 2023-05-06T13:16:52Z

Two other tests would be to use p11-kit but with only one module: opensc-pkcs11.so Then with only libykcs11.dylib

ueno · 2023-05-07T01:19:44Z

Going through this and latchset/pkcs11-provider#234, I still fail to see why this is relevant to p11-kit-proxy, as it simply passes through the calls to the underlying modules, without any modification except slot mappings.

The only thing I can suggest is to try collecting logs with either log-calls: yes in the module configuration or the P11_KIT_DEBUG envvar (set it to proxy or all).

mouse07410 · 2023-05-07T02:16:11Z

I still fail to see why this is relevant to p11-kit-proxy, as it simply passes through the calls to the underlying modules, without any modification except slot mappings

I hear you. However, the summary of situation is this:

when only a single module is present in ~/.config/pkcs11/modules/, everything works - except that of course in this case it cannot access multions tokens;
with mode than one module present (i.e., for YubiKey and for HSM), I'm getting OpenSSL error (as you saw described in Problem interacting with OpenSC, maybe with p11-kit too latchset/pkcs11-provider#234);
when pkcs11-provider is pointed at the correct module/driver directly (bypassing p11-kit), everything works.

I had a few runs with log-calls: yes (if there's a way to collect logs into a file, I'd like to know how), and I'd be happy to do more of such runs if it's helpful.

Update

Two other tests would be to use p11-kit but with only one module: opensc-pkcs11.so. Then with only libykcs11.dylib

With only one module - it seems to work.

ueno · 2023-05-07T05:09:09Z

when only a single module is present in ~/.config/pkcs11/modules/, everything works - except that of course in this case it cannot access multions tokens

I might have missed, but does this work also for YubiHSM2? latchset/pkcs11-provider#234 (comment) only mentions OpenSC so I can't figure this is really caused by having multiple tokens or sorely by the YubiHSM2 module.

mouse07410 · 2023-05-07T15:14:26Z

I might have missed, but does this work also for YubiHSM2? latchset/pkcs11-provider#234 (comment) only mentions OpenSC so I can't figure this is really caused by having multiple tokens or sorely by the YubiHSM2 module.

It does, with OpenSSL-3.2.0-dev:

$ openssl3 dgst -sign "pkcs11:model=YubiHSM;id=%04%01;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 17 - YubiHSM Connector 0.0.0.0):
Segmentation fault: 11
$ openssl3 dgst -verify "pkcs11:model=YubiHSM;id=%04%01;type=public" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -signature /tmp/pt.txt.sig /tmp/pt.txt
Enter pass phrase for PKCS#11 Token (Slot 17 - YubiHSM Connector 0.0.0.0):
Verified OK
Segmentation fault: 11
$

Seg-fault is because somehow provider or p11-kit get into an infinite loop on CloseSession:

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Codes:       0x0000000000000001, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [33710]

VM Region Info: 0 is not in any region.  Bytes before following region: 4536389632
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      10e63d000-10e6d5000    [  608K] r-x/r-x SM=COW  ...SER/*/openssl

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   ???                           	               0x0 ???
1   libp11-kit.0.dylib            	       0x10eb52f70 binding_C_CloseSession + 48
2   libffi.8.dylib                	       0x10e7ed942 ffi_closure_unix64_inner + 545
3   libffi.8.dylib                	       0x10e7edf4c ffi_closure_unix64 + 72
4   pkcs11.dylib                  	       0x10e81e220 p11prov_CloseSession + 288 (interface.gen.c:235)
5   pkcs11.dylib                  	       0x10e81e220 p11prov_CloseSession + 288 (interface.gen.c:235)
6   pkcs11.dylib                  	       0x10e81e220 p11prov_CloseSession + 288 (interface.gen.c:235)
.  .  .
509 pkcs11.dylib                  	       0x10e81e220 p11prov_CloseSession + 288 (interface.gen.c:235)
510 pkcs11.dylib                  	       0x10e81e220 p11prov_CloseSession + 288 (interface.gen.c:235)
511 pkcs11.dylib                  	       0x10e81e220 p11prov_CloseSession + 288 (interface.gen.c:235)

How can I build p11-kit in debugging mode, so symbols and such are preserved? Apparently, CFLAGS=-g is not sufficient - I'm still getting these ffi_closure_unix64 + 72 etc.

With OpenSSL-3.1.0, even when I have only one module YubiHSM, I'm getting an error from OpenSSL:

$ openssl dgst -sign "pkcs11:model=YubiHSM;id=%04%01;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 17 - YubiHSM Connector 0.0.0.0):
Error setting context
4083FE52F87F0000:error:03000096:digital envelope routines:do_sigver_init:operation not supported for this keytype:crypto/evp/m_sigver.c:298:
$

With only YKCS11 module present, it works:

$ openssl dgst -sign "pkcs11:model=YubiKey%20YK4;id=%01;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
$ openssl dgst -verify "pkcs11:model=YubiKey%20YK4;id=%01;type=public" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -signature /tmp/pt.txt.sig /tmp/pt.txt 
Verified OK
$

Likewise, with both OpenSC and YKCS11 modules present:

$ openssl dgst -sign "pkcs11:manufacturer=piv_II;id=%01;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
$ openssl dgst -verify "pkcs11:model=YubiKey%20YK4;id=%01;type=public" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -signature /tmp/pt.txt.sig /tmp/pt.txt 
Verified OK
$ openssl dgst -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
$ openssl dgst -verify "pkcs11:model=YubiKey%20YK4;id=%02;type=public" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -signature /tmp/pt.txt.sig /tmp/pt.txt 
Verified OK
$ 
$ pwd
/Users/ur20980/.config/pkcs11/modules
$ ll
total 16
drwxr-xr-x  4 ur20980  staff  128 May  7 11:12 ./
drwxr-xr-x  4 ur20980  staff  128 May  3 15:39 ../
-rw-r--r--  1 ur20980  staff  157 Nov 14  2021 pkcs11.module
-rw-r--r--  1 ur20980  staff  151 Nov 19  2021 ykcs11.module
$

Update

When I added yhsm2.module for HSM (so, three modules in total), I got this:

$ openssl dgst -verify "pkcs11:model=YubiKey%20YK4;id=%02;type=public" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -signature /tmp/pt.txt.sig /tmp/pt.txt 
Verified OK
$ openssl dgst -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
Error setting context
4083FE52F87F0000:error:03000096:digital envelope routines:do_sigver_init:operation not supported for this keytype:crypto/evp/m_sigver.c:298:
$ openssl3 dgst -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
Segmentation fault: 11
$ ll /tmp/pt.txt*
-rw-r--r--  1 ur20980  wheel  102 May  2 23:18 /tmp/pt.txt
-rw-r--r--  1 ur20980  wheel  256 May  7 11:18 /tmp/pt.txt.sig
-rw-r--r--  1 ur20980  wheel  384 May  6 22:31 /tmp/pt.txt.sighsm
$

"openssl" = OpenSSL-3.1.0, "openssl3" = OpenSSL-3.2.0-dev.

Update 2

Replacing yhsm2.module with softhsm.module led to a working setup:

$ openssl3 dgst -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
Segmentation fault: 11
$ openssl dgst -sign "pkcs11:manufacturer=piv_II;id=%02;type=private" -sha384 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:digest -out /tmp/pt.txt.sig /tmp/pt.txt 
Enter pass phrase for PKCS#11 Token (Slot 18 - Yubico YubiKey OTP+FIDO+CCID):
$

ueno · 2023-05-07T22:34:26Z

How can I build p11-kit in debugging mode, so symbols and such are preserved? Apparently, CFLAGS=-g is not sufficient

With Meson, you could use -Dbuildtype=debug, though it should be enabled by default:
https://mesonbuild.com/Builtin-options.html#core-options

I'm still getting these ffi_closure_unix64 + 72 etc.

This is runtime generated code, so might not even have debugging information.

mouse07410 · 2023-05-08T02:18:34Z

This is runtime generated code, so might not even have debugging information.

In some runs I get infinite loop there (ffi_closure_unix64 + <whatever>). Is there any way to figure out what's going on there? Also, it seems to be happening only on x86_64 platform, not on Apple Silicon (aarch64).

dengert · 2023-05-08T11:11:15Z

Sounds like libffi comes from: https://github.com/libffi/libffi

dengert · 2023-05-08T11:14:05Z

If building autoconfig check if configure has any options for debugging. If not try CFLAGS=-g LDFLAGS=-g

dengert · 2023-05-08T11:32:00Z

Best I can tell on MacOS you are mixing .dylib and .so files so libffi is needed.
https://stackoverflow.com/questions/2339679/what-are-the-differences-between-so-and-dylib-on-macos

Is there anyway to simplify you test environment? Using code still under development when building from any github master; .dylib vs .so; multiple build systems: autoconfig, Clang, Mason; multiple compilers: Clang,gcc; multiple arch: aarch64, x86_64

mouse07410 · 2023-05-08T13:45:44Z

Best I can tell on MacOS you are mixing .dylib and .so files so libffi is needed. https://stackoverflow.com/questions/2339679/what-are-the-differences-between-so-and-dylib-on-macos

Is there anyway to simplify you test environment?

I doubt it.

Using code still under development when building from any github master;

Well, that's OpenSSL-3.2.0-dev that works where the released OpenSSL-3.1.0 doesn't.
p11-kit master causes infinite loop with provider on OpenSSL-3.2.0-dev, but otherwise seems to work fine (except for that HSM thing), and it doesn't have those obnoxious assertion messages that plague Macports (released) p11-kit...

.dylib vs .so;

I don't think this is a problem. MacOS seems to be handling that quit nicely. Not to mention that if OpenSC insists on producing .so, and other packages like p11-kit produce .dylib - what would you expect me to do about it?

multiple build systems: autoconfig, Clang, Mason;

Again, if authors of some packages insist on "non-traditional" build systems - what would you expect me to do about it? Rewrite those packages myself for something more mature, like make/autoconf?

multiple compilers: Clang,gcc; multiple arch: aarch64, x86_64

That's not a hindrance, but a benefit. E.g., I can compare how the same code behaves (works vs. doesn't work) on x86_64 (doesn't work) vs. aarch64 (works, to my pleasant surprise). GCC did not seem to make an appreciable/positive difference, so I'm sticking with Clang. Macports Clang seems to have bugs different from those of Apple Clang... But again, what would you expect me to do here?

dengert · 2023-05-08T18:48:38Z

But again, what would you expect me to do here?

Try running the failing test in a linux system?

mouse07410 · 2023-05-08T20:00:14Z

Try running the failing test in a linux system?

I don't have a Linux system anymore. On MacOS:

With all modules present including HSM
- Signature works on aarch64 with OpenSSL-3.2.0-dev, fails with OpenSSL-3.1.0.
- Signature works on x86_64 with OpenSSL-3.2.0-dev but openssl crashes after the operation is complete, fails with OpenSSL-3.1.0.
Without HSM module present, signature seems to work.

dengert · 2023-05-10T00:20:06Z

I never had a MacOS system. And don't have a YubiHSM 2.

For wat ever its worth...
I did get a clone of [email protected]:Yubico/yubihsm-shell.git to build on Ubuntu with GCC debugging.

 #!/bin/bash
OSSL=/opt/ossl-3.1.0

export LD_LIBRARY_PATH=$OSSL/lib/
export PKG_CONFIG_PATH=$OSSL/lib/pkgconfig
export CFLAGS="-g"
export LDFLAGS="-g -Wl,-rpath,$OSSL/lib"

cmake -DCMAKE_INSTALL_PREFIX=$OSSL \
      ../yubihsm-shell

The build dir is at the same level as the source.

ueno added question category: proxy module labels May 7, 2023

dengert mentioned this issue May 7, 2023

Problem interacting with OpenSC, maybe with p11-kit too latchset/pkcs11-provider#234

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

p11-kit (current master) somehow fails token operations (???) #501

p11-kit (current master) somehow fails token operations (???) #501

mouse07410 commented May 2, 2023

dengert commented May 6, 2023

ueno commented May 7, 2023 •

edited

Loading

mouse07410 commented May 7, 2023 •

edited

Loading

ueno commented May 7, 2023

mouse07410 commented May 7, 2023 •

edited

Loading

ueno commented May 7, 2023

mouse07410 commented May 8, 2023

dengert commented May 8, 2023

dengert commented May 8, 2023

dengert commented May 8, 2023

mouse07410 commented May 8, 2023

dengert commented May 8, 2023

mouse07410 commented May 8, 2023

dengert commented May 10, 2023

p11-kit (current master) somehow fails token operations (???) #501

p11-kit (current master) somehow fails token operations (???) #501

Comments

mouse07410 commented May 2, 2023

dengert commented May 6, 2023

ueno commented May 7, 2023 • edited Loading

mouse07410 commented May 7, 2023 • edited Loading

Update

ueno commented May 7, 2023

mouse07410 commented May 7, 2023 • edited Loading

Update

Update 2

ueno commented May 7, 2023

mouse07410 commented May 8, 2023

dengert commented May 8, 2023

dengert commented May 8, 2023

dengert commented May 8, 2023

mouse07410 commented May 8, 2023

dengert commented May 8, 2023

mouse07410 commented May 8, 2023

dengert commented May 10, 2023

ueno commented May 7, 2023 •

edited

Loading

mouse07410 commented May 7, 2023 •

edited

Loading

mouse07410 commented May 7, 2023 •

edited

Loading