WASM compatibillity #1199
Comments
|
We haven't tried this before and don't have experience with WASM. Is this something you've tried? If you have specific CMakeFile changes that have to be made to make it work, feel free to submit that as a pull request. |
|
I'm going to leave this here for now that way I can look into making liboqs compatible with WASM: https://stackoverflow.com/questions/70311540/compiling-c-with-external-library-to-wasm-using-wasmer |
|
If you'd like us to look into this, please point to information how to install all required tooling for this sample build script to work? Are there docker images containing such tooling? A quick install on my local machine yields --> Where does this reference to "clang-9" come from (I've got 14 installed on the machine)? Why is the architecture reported to be "x86"? It's x86_64: Here's the result of running the "simple" wasienv install script: Pretty unhelpful output... |
|
I notice there is 36% of assembly code in this project. The design goal of WASM is not to handle any hardware vendor-specific native assembly code. Are there any configuration switches to bypass the assembly code and replace it with its C equivalent code (at the expense of performance, most likely)? If the native assembly code is mandatory, WASM build will be impossible. |
No, a single such liboqs build option does not exist "out of the box". Arguably, all code/components do exist for most (i.e, the PQClean) algorithms that would allow adding such switch to the You could get very far already using the build option combination OQS_DIST_BUILD=OFF OQS_OPT_TARGET=generic OQS_USE_OPENSSL=OFF as that combination should disable including all optimization code parts (i.e., assembly code). So, please give the above a try, feel free to do a PR if you need this or provide us with information why we should spend work cycles on this: I personally have doubts anyone would want to use slow-running cryptography... If you know of concrete use case(s), please let us know. |
|
Thanks to the advice from @baentsch, the
By the way, The Keep up with the great work. Thanks. Just a friendly reminder. I hope you keep in mind that WASM, representing the web, should be considered a legitimate target. If it is considered "generic", that is fine, as long as it contains only pure (free from assembly code, inline or not) C/C++ code that is not contaminated by the vendor-dependent features such as Linux/Windows system calls. |
Valid point. Three questions to this: |
|
@DevelopDaily When trying out your steps above, the following error appears: Any idea what's going on here? To reproduce: |
Yes for a) |
|
@baentsch I only tried the procedure on a full-blown Ubuntu 20.04 Desktop and it worked perfectly. I will try to do it in a docker container and let you know my testing results. |
Thanks. I also tried on a desktop (VM) but got the same error messages. If you simply run the commands above the error message is output straight away. What really is surprising is that the pretty strange error message "a function declaration without a prototype is deprecated in all versions of C" is output: This simply cannot be the case as |
Actually, that is a legitimate error in the In the header file
In the
The The WASM build succeeded on my local machine because it used a slightly older compiler, which does not enforce the function prototype. The docker image always uses the latest compiler, which is supposed to fail the build. |
|
Thanks for the analysis, @DevelopDaily ! |
|
After the issue #1204 is resolved, you may build a local docker image to build the WASM. I use Ubuntu. I've made a very simple Here is the procedure.
|
|
Thanks for the walk-through, @DevelopDaily . Based on this, it should be possible to build a fully working WASM environment in the above even before #1204 is resolved by simply removing one line: liboqs/.CMake/compiler_opts.cmake Line 68 in 9dea043 |
|
@baentsch You are correct. After removing that line, I just built the WASM in the container successfully. You may polish the image however you see fit. That pretty much fulfills your earlier request:
If necessary, I may add a a little description later for the demo. Basically, all the out-of-the-box For example, you can create a test file
You should see the hash of that file. |
|
Thanks for the confirmation, @DevelopDaily ! As I have 0 knowledge of WASM, (where) are there specific crypto tests (beyond hashing as per your suggestion) that we could run/trigger QSC? Or is it as simple as running "our" speed_kem etc. tests? Do they become "WASM executables" or is there a JS-wrapper required to trigger them? |
|
@baentsch
Under the hood, the |
|
@baentsch This may help visualize how it is run. By the way, some results look unusual. But, that would be a different story worth investigating later. |
|
Thanks for the explanations and test-run, @DevelopDaily ! At first glance this looks like 5-15% of native performance: Is that expected for WASM code? Some papers claim the gap should be smaller. Any idea how to compare this with "classic crypto" used via WASM? e.g., is there a way to compile openssl (3) for WASM? We could then run classic and QS crypto "head-to-head"... |
Yes, but it is more difficult because openssl is quite an old-school project. I'll see what I can do... |
I have built the openssl perfectly. Unfortunately, however, I am afraid a "head-to-head" comparison won't be useful because the WASM
Performance benchmark is a very complex issue, on which I am not an expert. Here is my understanding. The near-native performance claim of WASM is based on pure C/C++ code, which must be well-written and straightforward enough to give a modern compiler a chance to apply its optimization strategy to produce high quality assembly code. It may never beat hand-crafted assembly code in some occasions, but, in most cases, it should outperform its hand-crafted counterparts. I usually refine my C/C++ as much as possible, unless I have to write assembly, as a last resort. For pure C/C++, I did some casual benchmarking on this code: Native build and run: WASM build and run: As you can see, the WASM run is actually much faster. |
No question, if it runs through all algorithms. What I meant to ask is: What is the relative WASM performance of functionally comparable algorithms, say for signatures, i.e.,
If you use the commands above, it should allow you to compare non-assembly crypto implementations (classic vs. QS) with the same testing overhead in both cases.
It might -- but this is highly improbable as this is pretty "seasoned" code as you also point out. It surely has not been written with WASM in mind -- but so did none of the QS algorithms.
Agreed -- but then again this sample basically runs two, probably differently optimized library functions against each other, not real, external code as the two commands suggested above would do. |
WASM run: Native run: Both tests failed on the WASM openssl. If you have some other specific commands to run, I can do it. I update the docker file and you can build your own as well. Dockerfile In the docker image, follow this procedure to build: |
SWilson4
added
the
platform-specific
SWilson4
added
enhancement
|
Performance aside, I've created a small demo of using the library in the browser: https://github.com/x0wllaar/liboqs-wasm. I did not test with Node/Bun, but I think it should work there as well. In general, unmodified library code works, but I had to write some very boilerplate accessor methods, and then write a more idiomatic wrapper in JS. On the performance side, for me everything I tested except SPHINCS+ worked smoothly enough to not notice it blocking the event loop. For client side, even with SPHINCS+, it can be mitigated by moving it into a worker thread. I did not test McEliece. PS: Is it possible to enable BIKE in WASM and to add MAYO signatures? Testing BIKE was kind of a half of my inspiration for doing all this and MAYO just seems cool. |
Thanks, looks interesting. MAYO is being worked on in #1707 and will hopefully land in liboqs soon. BIKE is already available. We have two BIKE implementation versions -- a generic C for 64-bit little endian, and an x86_64 optimized implementation. I'm not sure how that would interact with the constraints of WASM. |
Yeah that's the problem as by default Emscripten compiles for 32-bit WASM. The 64-bit version is not really supported right now (see https://webassembly.org/features/, specifically Memory64 there). Emscripten supports (https://emscripten.org/docs/tools_reference/settings_reference.html?highlight=environment#memory64) "faking it" by compiling to 64-bit and then lowering to 32, I'll see if/how well it works for liboqs. From the JS side, the only change would be to add a way to determine the size of I'll try to do it by the end of this week. Among other things to address is the 16mb stack size, which is kinda necessary for McEliece to work, but I'll see how much I can lower it and still have it working. Is the large stack still required when using heap-allocated versions of KEMs/Signatures? |
|
After some fighting with CMake, I managed to get BIKE to work in browsers without any experimental options, thank you! |
Thanks for letting us know @x0wllaar . If there'd be something you learned that may benefit other users of |
Ideally, this project should be compilable via emscripten to WASM. Adding compatibility and documentation to make this possible should exist.
The text was updated successfully, but these errors were encountered: