Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acvp auto testing #129

Open
wants to merge 74 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
bb5b032
Create EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
8dc2ef2
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
e77389a
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
0d0df6d
Create test_certs_r3_with_acvp.sh
ounsworth Jul 22, 2024
1814bfb
Create setup_acvp_env.sh
ounsworth Jul 22, 2024
86c7d44
Update test_certs_r3_with_acvp.sh
ounsworth Jul 22, 2024
7896eb0
Rename setup_acvp_env.sh to setup_acvts_env.sh
ounsworth Jul 22, 2024
9c7a4c2
Update test_certs_r3_with_acvp.sh
ounsworth Jul 22, 2024
241ce90
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
9fe985d
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
9137d9c
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
97b9ab6
debugging
ounsworth Jul 22, 2024
d214429
Update test_certs_r3_with_acvp.sh
ounsworth Jul 22, 2024
09f81d9
Update test_certs_r3_with_acvp.sh
ounsworth Jul 22, 2024
4486211
Update setup_acvts_env.sh
ounsworth Jul 22, 2024
b68049e
Update test_certs_r3_with_acvp.sh
ounsworth Jul 22, 2024
27769a9
debugging
ounsworth Jul 22, 2024
ab04623
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
da3df92
debugging
ounsworth Jul 22, 2024
5716136
Update EXPERIMENTAL_artifact_validation_acvp.yaml
ounsworth Jul 22, 2024
5d0920d
debugging
ounsworth Jul 23, 2024
8e7274c
Update setup_acvts_env.sh
ounsworth Jul 23, 2024
5624c89
Wrote framework script for ACVP tests
ounsworth Jul 23, 2024
9665ffb
Setting working dir
ounsworth Jul 23, 2024
d9346df
debug
ounsworth Jul 23, 2024
3129772
Debug
ounsworth Jul 23, 2024
a3778ce
Debug
ounsworth Jul 23, 2024
5c73a89
Debug
ounsworth Jul 23, 2024
26801fd
Debug
ounsworth Jul 23, 2024
3f88f55
Debug
ounsworth Jul 23, 2024
0991547
Debug
ounsworth Jul 23, 2024
36f0ca4
Debug
ounsworth Jul 23, 2024
7defef3
Debug
ounsworth Jul 23, 2024
8f9fd15
Debug
ounsworth Jul 23, 2024
73312e0
Debug
ounsworth Jul 23, 2024
1c077d1
Test oids json in external file
ounsworth Jul 23, 2024
e3ae9f4
Test oids json in external file
ounsworth Jul 23, 2024
b7ba3a5
Debug
ounsworth Jul 23, 2024
7043bcf
Debug
ounsworth Jul 23, 2024
694ee08
Debug
ounsworth Jul 23, 2024
b2455bc
Starting to integrate into workflow
ounsworth Jul 23, 2024
2d5e0be
Starting to integrate into workflow
ounsworth Jul 23, 2024
ccb89ab
Starting to integrate into workflow
ounsworth Jul 23, 2024
4c5d1f6
Starting to integrate into workflow
ounsworth Jul 23, 2024
e2c58f2
Starting to integrate into workflow
ounsworth Jul 23, 2024
6946211
Starting to integrate into workflow
ounsworth Jul 23, 2024
309a5e1
Starting to integrate into workflow
ounsworth Jul 23, 2024
05c50aa
Don't need this anymore
ounsworth Jul 23, 2024
7f9f911
Copy acvp logs
ounsworth Jul 23, 2024
f00263b
Debugging
ounsworth Jul 23, 2024
0983c48
Bug
ounsworth Jul 23, 2024
610ed34
|tee logfile
ounsworth Jul 23, 2024
a59fdd4
WIP basic C# program, working toward verifying DER using NIST algorithms
Oct 12, 2024
42334e6
WIP slhdsa certs, starting mldsa certs
Oct 13, 2024
e948b84
mldsa generatoin & verification work; I think that's about it
Oct 13, 2024
7738ae3
redo artifacts, one with pem, one with der
Oct 13, 2024
75bea02
WIP test...r3.sh and test...r4.sh work in my container; first draft g…
Oct 13, 2024
e1b8d78
path issues in github action?
Oct 13, 2024
054de27
provider renamed; verification scripts distinguish between verificati…
Oct 13, 2024
87e1ef6
readme; clean up old artifacts
Oct 13, 2024
1e8ccd8
a bit more in the readme
Oct 13, 2024
86220b5
Merge branch 'acvp-auto-testing' into jwl/nist-acvts-verifier
ounsworth Oct 13, 2024
24e33ee
Merge pull request #140 from IETF-Hackathon/jwl/nist-acvts-verifier
ounsworth Oct 13, 2024
e68748f
Update artifact_validation.yaml
ounsworth Oct 13, 2024
4c94af0
Update artifact_validation.yaml
ounsworth Oct 13, 2024
8ee1694
fix image reference "lost in translation"
Oct 14, 2024
ad4d313
small typo fix in image reference
Oct 14, 2024
cd98cfc
trying code checkout fixes
Oct 14, 2024
784a303
trying other code checkout paths
Oct 14, 2024
cbd258d
more path issues
Oct 14, 2024
f1c265d
trying remove superfluous steps
Oct 14, 2024
8f29111
I think one more path change
Oct 14, 2024
d125b9e
path issues
Oct 14, 2024
4f07fff
Merge branch 'master' into acvp-auto-testing
josephlukefahr Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 22 additions & 9 deletions .github/workflows/artifact_validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,35 @@ jobs:
with:
name: Compatibility_oqs_csv
path: ./output/
bc_validation:

acvts_validation:
runs-on: ubuntu-latest
container: bcdocker2000/bc_hackathon_pqc
# Is this an appropriate dotnet image?
container: mcr.microsoft.com/dotnet/sdk:6.0
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Test artifacts with BC
run: ./src/test_certs_r3_with_bc.sh
with:
path: ./pqc-certificates
- name: Checkout acvts code
uses: actions/checkout@v4
with:
repository: josephlukefahr/ACVP-Server
ref: jwl/versions-in-some-packages
path: ./ACVP-Server
- run: apt update && apt install zip unzip
- run: mkdir ./output
- run: /bin/bash ./pqc-certificates/src/test_certs_r3_with_acvts.sh ./pqc-certificates
- name: Save artifacts
uses: actions/upload-artifact@v4
with:
name: Compatibility_bc_csv
path: ./output/
name: Compatibility_acvts_csv
path: ./output

build_results_html:
runs-on: ubuntu-latest
container: ubuntu:latest
needs: [oqs_validation, bc_validation]
needs: [oqs_validation, acvts_validation]
steps:
- name: Checkout code
uses: actions/checkout@v4
Expand All @@ -59,10 +71,10 @@ jobs:
with:
name: Compatibility_oqs_csv
path: output/
- name: Get BC results from previous job
- name: Get acvts results from previous job
uses: actions/download-artifact@v4
with:
name: Compatibility_bc_csv
name: Compatibility_acvts_csv
path: output/
- name: Build compat matrix (r3)
run: ./src/rebuild_results_certs_r3.sh
Expand All @@ -74,6 +86,7 @@ jobs:
uses: actions/upload-pages-artifact@v3
with:
path: ./docs/gh-pages

publish_to_gh_pages:
needs: build_results_html
if: github.ref == 'refs/heads/master'
Expand Down
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,11 @@ providers/entrust/default/artifacts
providers/corey-digicert/default/artifacts
providers/bc/java-artifacts
providers/botan/default/artifacts
**/artifacts*/
**/output/

# C-sharp artifacts
src/nist-acvts-generator/bin/
src/nist-acvts-generator/obj/
src/nist-acvts-verifier/bin/
src/nist-acvts-verifier/obj/
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,27 @@ dotnet test NIST.CVP.ACVTS.Libraries.Crypto.SLHDSA.Tests.csproj > ~/Downloads/pq
- Unit test logs included as dilithium.log and slh-dsa.log
- NIST code seems to interchange "ML-DSA" and "Dilithium"; not sure which algorithm is actually implemented; could this be causing the issues with those certificates?
- Big disadvantage to this approach -- I cannot assert any claims related to the NIST code (not even sure which algorithm is implemented!); a NIST engineer who is familiar with the code should generate and contribute the artifacts

### 20241013

- Lots of work!
- Refactored X.509 generation (based on unit test) into a separate program
- Found that writing a verification tool is feasible
- Generation tool located at `pqc-certificates/src/nist-acvts-generator`
- Verification tool located at `pqc-certificates/src/nist-acvts-verifier`
- Tools require ACVP-Server code to exist "adjacent" to pqc-certificates repo, i.e...
```
/
= opt/
====== ACVP-Server/
====== pqc-certificates/
```
- Tools include relevant ACVP-Server projects as source packages
- Tools support PEM (for _r3 scripts) and DER (for _r4 scripts) files
- New X.509 certificates generated and zippd manually into `nist-acvts-generator` provider
- Tools accomodate OIDs for FIPS 204 & FIPS 205 algorithms
- **Still cannot assert any claims to the NIST code**; still would be good to have NIST engineer verify that we're using the NIST code properly
- Drafted an updated GitHub Actions file; not familiar with GitHub Actions, though, so need help with that part also
- Also, I still need to amend the ACVP-Server code; relevant source packages do not specify version bounds for dependencies,
which results in inconsistent version, and for me, that means compilation errors! I added minimum versions to some dependencies,
which fixes my problem; again, would be good to discuss with NIST POCs about merging these changes (or something similar)
Binary file removed providers/nist-acvts-test/artifacts_certs_r3.zip
Binary file not shown.
464 changes: 0 additions & 464 deletions providers/nist-acvts-test/dilithium.log

This file was deleted.

13 changes: 13 additions & 0 deletions providers/nist-acvts-test/setup_acvts_env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/sh

# if no input was provided, then working dir is the current dir
if [ -z "$1" ]; then
workingdir="./"
else
workingdir=$1
fi

rm $workingdir/Directory.Build.props 2>&1 > /dev/null
rm $workingdir/Directory.Packages.props 2>&1 > /dev/null
ln -s $workingdir/_config/Directory.Build.props
ln -s $workingdir/_config/Directory.Packages.props
Loading
Loading