Update XMSS secret key object APIs, sync with LMS #1588

ducnguyen-sb · 2023-10-20T02:51:17Z

XMSS changes

Changelogs

Made code compatible with MSVC in order to merge into main branch: Remove variable length array in XMSS/external/ by converting them to malloc/calloc and OQS_insecure_free()
Cleaned up unused files: sign.c and sign.h.
Fixed all segfaults.
Added katfile to test that takes too long to generate a key pair.
Removed a lot of LOC, so the code base of XMSS parameters are shorter, and easier to edit.
Changed in memory allocation: sk->secret_key_data is allocated by default. The OQS_SECRET_KEY_XMSS_deserialize_key is simply copy memory, instead of allocation new memory and copy. I think this should also be applied to LMS as well. I see it’s error-prone and inefficient to allocate memory each time the deserialize happens.

Tests

This code will work with either OQS_USE_PTHREADS_IN_TESTS on or off. I tested both cases.

fix astyle fixed all memory errors

…signatures

…_SECRET_KEY initialization

src/sig_stfl/sig_stfl.h

…s/sig_stfl_lms.c src/sig_stfl/xmss/sig_stfl_xmss_secret_key_functions.c tests/test_sig_stfl.c

…nvention

ducnguyen-sb · 2023-11-01T03:46:02Z

I don't know why the test duration increase by 50%.
I think our case is simple enough to use recursive mutex and it's nicer to do so.
I guess the major time execution is thread state management. We can improve the our test with thread to see if it's better.

I'll think about it, if we are short on time then increase runtime is another solution.

ducnguyen-sb · 2023-11-01T15:59:03Z

@ashman-p , I removed the recursive mutex, it works right away.
To do so, I create an inner_serialize function, which is identical to serialize except it does not have the acquire_lock and release_lock functions.

Other than that I simplify the logic code in test_sig_stfl.c. I think you would want to take a look at it.
For short, let's just use a the simple mutex.

ashman-p · 2023-11-01T16:12:09Z

@ashman-p , I removed the recursive mutex, it works right away. To do so, I create an inner_serialize function, which is identical to serialize except it does not have the acquire_lock and release_lock functions.

Other than that I simplify the logic code in test_sig_stfl.c. I think you would want to take a look at it. For short, let's just use a the simple mutex.

Yep, this sounds good. Thank you.

ashman-p

Looks good. Thanks.

* Init * convert all variable length array to malloc/free fix astyle fixed all memory errors * refactor XMSS and XMSS^MT, shorten LOC * clean up unused function * TODO: restore core_hash.c later * Add activate_lock and activate_unlock functions * Add `bool is_locked` to retain lock information, and adjust function signatures * cleanup test_sig_stfl.c * remove const in LMS_serialize_key and add `is_locked` to OQS_SIG_STFL_SECRET_KEY initialization * fix astyle error * fix astyle. I have to update local astyle to 3.4.10 * remove incorrect comments * remove unsued variables * fix if guard * fix const warnings * fix namespace error. revert core_hash.c to original namespace separation * move XMSS_free to internal of XMSS * Fix memory leaks * fix astyle format * fix typo * improve readablity * Update OID comment. * Trim the space * Remove mutex status bool * Remove use of mutex status bool. Use recursive mutex” src/sig_stfl/lms/sig_stfl_lms.c src/sig_stfl/xmss/sig_stfl_xmss_secret_key_functions.c tests/test_sig_stfl.c * rename lock function * simplify the check with 0 * Fix grammar * add `const` back to serialize. Reorder parameters to follow liboqs convention * use inner_serialize to avoid recursive lock * add return code in case pthread API has errors * fix scan_build NULL error --------- Co-authored-by: Norman Ashley <[email protected]>

commit 244288f Add XMSS parameter xmss_sha256_h10 (#1482) commit a7e26d9 Add 12 XMSS and 16 XMSSMT parameters. (#1489) commit 4694fc3 Add secret key object to XMSS (#1530) commit 99067be Add XMSS Serialize/Deserialize (#1542) commit 2dbfc40 Update XMSS secret key object APIs, sync with LMS (#1588) commit 47740ad Enforce idx from unsigned int to uint32_t. (#1611) commit 9610576 Fix windows-x86 and arm compiling error. (#1634) commit bb658b7 Address stateful-sigs comments in #1650 (#1656) commit 7db8ddf Update `sig_stfl.h` document for #1650 (#1655) commit c3e5750 Add Apache 2.0 and MIT License to XMSS (#1662) commit e1f02b2 Change XMSS License from `(Apache 2.0 AND MIT)` to `(Apache 2.0 OR MIT) AND CC0-1.0` (#1697) commit 17c12c3 Add return status for XMSS lock/unlock functions. (#1712) commit 1941636 Add return check for lock/unlock function (#1727) commit b45415c Use `abort()` instead of exit to get the trace log. (#1728) commit ba63672 Reduce number of `malloc/free` call in `XMSS/external` (#1724) Signed-off-by: Duc Tri Nguyen <[email protected]>

ducnguyen-sb requested a review from dstebila as a code owner October 20, 2023 02:51

ducnguyen-sb requested review from ashman-p and removed request for dstebila October 20, 2023 02:51

ducnguyen-sb added 2 commits October 20, 2023 15:03

Init

b05776c

convert all variable length array to malloc/free

77668b9

fix astyle fixed all memory errors

ducnguyen-sb force-pushed the add_secret_function_calls_to_xmss branch from 6a8fca0 to 77668b9 Compare October 20, 2023 19:03

ducnguyen-sb added 5 commits October 21, 2023 21:30

refactor XMSS and XMSS^MT, shorten LOC

ac3456a

clean up unused function

03ebc65

TODO: restore core_hash.c later

54fa829

Add activate_lock and activate_unlock functions

aff21eb

Add bool is_locked to retain lock information, and adjust function …

6178748

…signatures

ducnguyen-sb mentioned this pull request Oct 22, 2023

Stateful sigs XMSS updates #1590

Merged

2 tasks

ducnguyen-sb added 14 commits October 22, 2023 12:52

cleanup test_sig_stfl.c

4ad3f42

remove const in LMS_serialize_key and add is_locked to OQS_SIG_STFL…

c04295f

…_SECRET_KEY initialization

fix astyle error

269ada6

fix astyle. I have to update local astyle to 3.4.10

0a924bf

remove incorrect comments

b23e898

remove unsued variables

7a0945b

fix if guard

a36c288

fix const warnings

521afc7

fix namespace error. revert core_hash.c to original namespace separation

8d47af8

move XMSS_free to internal of XMSS

b4c7ca0

Fix memory leaks

5a49b6b

fix astyle format

1afc35c

fix typo

2d817e1

improve readablity

ef1bb9a

ducnguyen-sb requested review from dstebila, SWilson4 and baentsch October 22, 2023 19:46

ashman-p reviewed Oct 23, 2023

View reviewed changes

src/sig_stfl/sig_stfl.h Show resolved Hide resolved

baentsch reviewed Oct 24, 2023

View reviewed changes

src/sig_stfl/sig_stfl.h Outdated Show resolved Hide resolved

ashman-p self-requested a review October 24, 2023 14:07

ducnguyen-sb and others added 6 commits October 24, 2023 09:38

Update OID comment.

8e422a5

Trim the space

f5b86ab

Remove mutex status bool

fc0a7f4

Remove use of mutex status bool. Use recursive mutex” src/sig_stfl/lm…

28e8773

…s/sig_stfl_lms.c src/sig_stfl/xmss/sig_stfl_xmss_secret_key_functions.c tests/test_sig_stfl.c

rename lock function

760d274

simplify the check with 0

1c71662

ashman-p approved these changes Nov 1, 2023

View reviewed changes

ducnguyen-sb added 2 commits October 31, 2023 22:58

Fix grammar

efbeec9

add const back to serialize. Reorder parameters to follow liboqs co…

60e947c

…nvention

ducnguyen-sb added 2 commits November 1, 2023 11:29

use inner_serialize to avoid recursive lock

b2d5670

add return code in case pthread API has errors

9fbb5c3

ducnguyen-sb requested a review from ashman-p November 1, 2023 15:55

ashman-p closed this Nov 1, 2023

ducnguyen-sb reopened this Nov 1, 2023

ashman-p reviewed Nov 1, 2023

View reviewed changes

fix scan_build NULL error

2dab954

ashman-p approved these changes Nov 1, 2023

View reviewed changes

ducnguyen-sb merged commit 3953cbf into stateful-sigs Nov 1, 2023
11 checks passed

ducnguyen-sb deleted the add_secret_function_calls_to_xmss branch November 1, 2023 17:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update XMSS secret key object APIs, sync with LMS #1588

Update XMSS secret key object APIs, sync with LMS #1588

ducnguyen-sb commented Oct 20, 2023 •

edited

Loading

ducnguyen-sb commented Nov 1, 2023

ducnguyen-sb commented Nov 1, 2023 •

edited

Loading

ashman-p commented Nov 1, 2023

ashman-p left a comment

Update XMSS secret key object APIs, sync with LMS #1588

Update XMSS secret key object APIs, sync with LMS #1588

Conversation

ducnguyen-sb commented Oct 20, 2023 • edited Loading

XMSS changes

Changelogs

Tests

ducnguyen-sb commented Nov 1, 2023

ducnguyen-sb commented Nov 1, 2023 • edited Loading

ashman-p commented Nov 1, 2023

ashman-p left a comment

Choose a reason for hiding this comment

ducnguyen-sb commented Oct 20, 2023 •

edited

Loading

ducnguyen-sb commented Nov 1, 2023 •

edited

Loading