Bugs in build instructions and build scripts

[kenballus]

Merry Christmas and Happy Hanukkah!

Building from source, as described in the build instructions, doesn't work.

Here's a set of steps to reproduce the numerous issues:

0. Start a fresh Debian 12 container:

docker run --workdir /root --rm -it debian:bookworm

1. Update it and install wget:

apt -y update && apt -y upgrade && apt -y install wget

3. wget the source code:

wget https://github.com/litespeedtech/openlitespeed/archive/refs/tags/v1.8.1.tar.gz

4. The instructions then say to run the following:

tar -zxvf openlitespeed-*.tgz

This won't work because the source tarball's filename doesn't start with openlitespeed- and doesn't end with .tgz. The command should be this instead:

tar -zxvf v1.8.1.tar.gz

5. cd into the new directory:

cd openlitespeed*

6. Run the build script:

./build.sh |& tee build_log.txt

The full build log is too long to attach here (or even for pastebin), but I can go over some highlights.

i. Error during build of libbcrypt:

Cloning into 'libbcrypt'...
gcc  -W -Wall -Wbad-function-cast -Wcast-align -Wcast-qual -Wmissing-prototypes -Wstrict-prototypes -Wshadow -Wundef -Wpointer-arith -O2 -fomit-frame-pointer -funroll-loops -c bcrypt.c
make -C crypt_blowfish
make[1]: Entering directory '/root/third-party/src/libbcrypt/crypt_blowfish'
gcc -c -W -Wall -Wbad-function-cast -Wcast-align -Wcast-qual -Wmissing-prototypes -Wstrict-prototypes -Wshadow -Wundef -Wpointer-arith -O2 -fomit-frame-pointer -funroll-loops crypt_blowfish.c
gcc -c x86.S
gcc -c -W -Wall -Wbad-function-cast -Wcast-align -Wcast-qual -Wmissing-prototypes -Wstrict-prototypes -Wshadow -Wundef -Wpointer-arith -O2 -fomit-frame-pointer -funroll-loops crypt_gensalt.c
gcc -c -W -Wall -Wbad-function-cast -Wcast-align -Wcast-qual -Wmissing-prototypes -Wstrict-prototypes -Wshadow -Wundef -Wpointer-arith -O2 -fomit-frame-pointer -funroll-loops wrapper.c
echo '.so man3/crypt.3' > crypt_r.3
echo '.so man3/crypt.3' > crypt_rn.3
echo '.so man3/crypt.3' > crypt_ra.3
echo '.so man3/crypt.3' > crypt_gensalt.3
echo '.so man3/crypt.3' > crypt_gensalt_rn.3
echo '.so man3/crypt.3' > crypt_gensalt_ra.3
make[1]: Leaving directory '/root/third-party/src/libbcrypt/crypt_blowfish'
ar r bcrypt.a bcrypt.o crypt_blowfish/*.o
ar: creating bcrypt.a
fatal: destination path 'libexpat' already exists and is not an empty directory.

ii. ModSecurity build fails due to missing header file

Make mod_security
-- Performing Test HAS_NO_UNUSED_FUNCTION
-- Performing Test HAS_NO_UNUSED_FUNCTION - Success
-- Performing Test HAS_NO_SIGN_COMPARE
-- Performing Test HAS_NO_SIGN_COMPARE - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /root/openlitespeed-1.8.1/build
[  0%] Generating lsquic_versions_to_string.c
[  0%] Building CXX object src/adns/CMakeFiles/adns.dir/adns.cpp.o
[  0%] Building CXX object src/h2/CMakeFiles/h2.dir/h2connection.cpp.o
[  0%] Building CXX object src/h2/CMakeFiles/h2.dir/h2stream.cpp.o
[  1%] Building CXX object src/h2/CMakeFiles/h2.dir/h2protocol.cpp.o
[  1%] Building CXX object src/extensions/CMakeFiles/extensions.dir/loadbalancer.cpp.o
[  1%] Building CXX object src/extensions/fcgi/CMakeFiles/fcgi.dir/fcgienv.cpp.o
[  1%] Building CXX object src/lsiapi/CMakeFiles/lsiapi.dir/ediohandler.cpp.o
[  1%] Building CXX object src/extensions/jk/CMakeFiles/jk.dir/jkajp13.cpp.o
[  1%] Building CXX object src/extensions/cgi/CMakeFiles/cgi.dir/suexec.cpp.o
[  1%] Building CXX object src/extensions/proxy/CMakeFiles/proxy.dir/proxyconfig.cpp.o
[  1%] Building CXX object src/edio/CMakeFiles/edio.dir/reactorindex.cpp.o
[  1%] Building CXX object src/extensions/registry/CMakeFiles/registry.dir/appconfig.cpp.o
[  1%] Building CXX object src/extensions/lsapi/CMakeFiles/lsapi.dir/lsapiworker.cpp.o
[  1%] Building CXX object src/main/CMakeFiles/main.dir/mainserverconfig.cpp.o
[  1%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/appendermanager.cpp.o
[  2%] Building CXX object src/modules/modgzip/CMakeFiles/modgzip.dir/modgzip.cpp.o
[  2%] Building CXX object src/modules/cache/CMakeFiles/cache.dir/cachemanager.cpp.o
[  3%] Building CXX object src/quic/CMakeFiles/quic.dir/quicshm.cpp.o
[  3%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_aho.c.o
[  3%] Building CXX object src/http/CMakeFiles/http.dir/moduserdir.cpp.o
[  3%] Building CXX object src/spdy/CMakeFiles/spdy.dir/spdyprotocol.cpp.o
[  3%] Building CXX object src/shm/CMakeFiles/lsshm.dir/addrmap.cpp.o
[  3%] Building CXX object src/ssi/CMakeFiles/ssi.dir/ssiengine.cpp.o
[  3%] Building CXX object src/sslpp/CMakeFiles/sslpp.dir/sslcontextconfig.cpp.o
[  3%] Building CXX object src/socket/CMakeFiles/socket.dir/reuseport.cpp.o
[  3%] Building CXX object src/thread/CMakeFiles/thread.dir/mtnotifier.cpp.o
[  3%] Building CXX object src/modules/pagespeed/CMakeFiles/modpagespeed.dir/pagespeed.cpp.o
[  3%] Building CXX object src/modules/lua/CMakeFiles/mod_lua.dir/lsluaengine.cpp.o
[  3%] Building CXX object src/modules/modsecurity-ls/CMakeFiles/mod_security.dir/mod_security.cpp.o
[  3%] Building C object support/unmount_ns/CMakeFiles/unmount_ns.dir/unmount_ns.c.o
[  3%] Building CXX object src/util/CMakeFiles/util.dir/httpfetchdriver.cpp.o
[  4%] Building C object src/liblsquic/CMakeFiles/lsquic.dir/ls-qpack/lsqpack.c.o
[  4%] Building CXX object src/lsiapi/CMakeFiles/lsiapi.dir/envmanager.cpp.o
[  5%] Building CXX object src/spdy/CMakeFiles/spdy.dir/spdyconnection.cpp.o
In file included from /root/openlitespeed-1.8.1/src/modules/pagespeed/pagespeed.h:25,
                 from /root/openlitespeed-1.8.1/src/modules/pagespeed/pagespeed.cpp:21:
/root/openlitespeed-1.8.1/src/modules/pagespeed/ls_rewrite_driver_factory.h:25:10: fatal error: pagespeed/kernel/base/md5_hasher.h: No such file or directory
   25 | #include "pagespeed/kernel/base/md5_hasher.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [src/modules/pagespeed/CMakeFiles/modpagespeed.dir/build.make:76: src/modules/pagespeed/CMakeFiles/modpagespeed.dir/pagespeed.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1223: src/modules/pagespeed/CMakeFiles/modpagespeed.dir/all] Error 2

iii. The same build also fails due to use of std::size without the proper includes:

make[1]: *** Waiting for unfinished jobs....
[  5%] Building CXX object src/util/CMakeFiles/util.dir/httpfetch.cpp.o
[  5%] Building CXX object src/extensions/lsapi/CMakeFiles/lsapi.dir/lsapireq.cpp.o
[  5%] Building CXX object src/thread/CMakeFiles/thread.dir/pthreadcond.cpp.o
[  5%] Building CXX object src/extensions/proxy/CMakeFiles/proxy.dir/proxyworker.cpp.o
[  5%] Building CXX object src/main/CMakeFiles/main.dir/lshttpdmain.cpp.o
[  5%] Building CXX object src/extensions/fcgi/CMakeFiles/fcgi.dir/fcgiappconfig.cpp.o
[  5%] Building CXX object src/sslpp/CMakeFiles/sslpp.dir/sslengine.cpp.o
[  5%] Building C object support/unmount_ns/CMakeFiles/unmount_ns.dir/nsopts.c.o
[  5%] Building CXX object src/edio/CMakeFiles/edio.dir/fdindex.cpp.o
[  5%] Building CXX object src/thread/CMakeFiles/thread.dir/pthreadmutex.cpp.o
[  6%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/fileappender.cpp.o
[  6%] Building CXX object src/http/CMakeFiles/http.dir/contextnode.cpp.o
[  6%] Building CXX object src/extensions/registry/CMakeFiles/registry.dir/extappregistry.cpp.o
[  6%] Building CXX object src/modules/cache/CMakeFiles/cache.dir/shmcachemanager.cpp.o
[  7%] Building CXX object src/extensions/CMakeFiles/extensions.dir/localworkerconfig.cpp.o
[  7%] Building CXX object src/socket/CMakeFiles/socket.dir/gsockaddr.cpp.o
[  7%] Building CXX object src/extensions/proxy/CMakeFiles/proxy.dir/proxyconn.cpp.o
[  7%] Building CXX object src/thread/CMakeFiles/thread.dir/pthreadworkqueue.cpp.o
[  8%] Building CXX object src/extensions/fcgi/CMakeFiles/fcgi.dir/fcgiapp.cpp.o
[  9%] Building CXX object src/edio/CMakeFiles/edio.dir/kqueuer.cpp.o
[  9%] Building CXX object src/edio/CMakeFiles/edio.dir/epoll.cpp.o
[ 10%] Building CXX object src/thread/CMakeFiles/thread.dir/thread.cpp.o
[ 10%] Building CXX object src/lsiapi/CMakeFiles/lsiapi.dir/lsimoduledata.cpp.o
[ 11%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_base64.c.o
[ 11%] Linking CXX static library libmodgzip.a
[ 11%] Building CXX object src/shm/CMakeFiles/lsshm.dir/lsshm.cpp.o
[ 11%] Building CXX object src/thread/CMakeFiles/thread.dir/workcrew.cpp.o
[ 11%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_buf.c.o
[ 12%] Building CXX object src/lsiapi/CMakeFiles/lsiapi.dir/lsiapihooks.cpp.o
[ 12%] Built target modgzip
[ 13%] Building CXX object src/http/CMakeFiles/http.dir/phpconfig.cpp.o
[ 13%] Building CXX object src/edio/CMakeFiles/edio.dir/rtsigio.cpp.o
[ 13%] Building CXX object src/sslpp/CMakeFiles/sslpp.dir/sslcert.cpp.o
[ 13%] Building CXX object src/extensions/fcgi/CMakeFiles/fcgi.dir/fcginamevaluepair.cpp.o
[ 14%] Building CXX object src/extensions/cgi/CMakeFiles/cgi.dir/cgidreq.cpp.o
[ 15%] Building CXX object src/sslpp/CMakeFiles/sslpp.dir/sslcertcomp.cpp.o
[ 15%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/logrotate.cpp.o
[ 16%] Building CXX object src/modules/lua/CMakeFiles/mod_lua.dir/edluastream.cpp.o
[ 16%] Building CXX object src/socket/CMakeFiles/socket.dir/hostinfo.cpp.o
[ 16%] Building CXX object src/extensions/fcgi/CMakeFiles/fcgi.dir/fcgiconnection.cpp.o
[ 16%] Building CXX object src/extensions/lsapi/CMakeFiles/lsapi.dir/lsapiconn.cpp.o
[ 16%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_confparser.c.o
[ 16%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_crc64.c.o
[ 16%] Building CXX object src/extensions/CMakeFiles/extensions.dir/localworker.cpp.o
[ 16%] Building C object support/unmount_ns/CMakeFiles/unmount_ns.dir/nspersist.c.o
[ 16%] Building CXX object src/edio/CMakeFiles/edio.dir/ediostream.cpp.o
[ 16%] Building CXX object src/sslpp/CMakeFiles/sslpp.dir/sslerror.cpp.o
[ 16%] Building CXX object src/socket/CMakeFiles/socket.dir/tcpsockopt.cpp.o
[ 16%] Building CXX object src/http/CMakeFiles/http.dir/pipeappender.cpp.o
[ 16%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/patternlayout.cpp.o
[ 16%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_dlinkq.c.o
[ 16%] Building CXX object src/lsiapi/CMakeFiles/lsiapi.dir/modulehandler.cpp.o
[ 16%] Building CXX object src/socket/CMakeFiles/socket.dir/tcpserversocket.cpp.o
[ 17%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_fileio.c.o
[ 17%] Building CXX object src/socket/CMakeFiles/socket.dir/tcpconnection.cpp.o
[ 17%] Linking CXX static library libadns.a
[ 18%] Building CXX object src/main/CMakeFiles/main.dir/serverinfo.cpp.o
[ 18%] Building CXX object src/ssi/CMakeFiles/ssi.dir/ssiconfig.cpp.o
[ 19%] Building CXX object src/socket/CMakeFiles/socket.dir/streamsocket.cpp.o
[ 19%] Building C object src/lsr/CMakeFiles/lsr.dir/ls_hash.c.o
[ 19%] Built target adns
[ 19%] Building CXX object src/modules/lua/CMakeFiles/mod_lua.dir/lsluaapi.cpp.o
[ 19%] Building CXX object src/extensions/cgi/CMakeFiles/cgi.dir/cgidconfig.cpp.o
[ 19%] Building CXX object src/h2/CMakeFiles/h2.dir/h2streampool.cpp.o
[ 19%] Building CXX object src/thread/CMakeFiles/thread.dir/worker.cpp.o
[ 19%] Building CXX object src/modules/lua/CMakeFiles/mod_lua.dir/lsluasession.cpp.o
[ 19%] Building CXX object src/socket/CMakeFiles/socket.dir/serversocket.cpp.o
[ 19%] Building CXX object src/ssi/CMakeFiles/ssi.dir/ssiruntime.cpp.o
[ 19%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/loggingevent.cpp.o
[ 19%] Building CXX object src/http/CMakeFiles/http.dir/awstats.cpp.o
[ 19%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/ilog.cpp.o
[ 20%] Building CXX object src/log4cxx/CMakeFiles/log4cxx.dir/bench.cpp.o
[ 20%] Building CXX object src/socket/CMakeFiles/socket.dir/coresocket.cpp.o
In file included from /root/openlitespeed-1.8.1/../third-party/include/modsecurity/rules_set_properties.h:35,
                 from /root/openlitespeed-1.8.1/../third-party/include/modsecurity/rules_set.h:31,
                 from /root/openlitespeed-1.8.1/src/modules/modsecurity-ls/mod_security.cpp:25:
/root/openlitespeed-1.8.1/../third-party/include/modsecurity/actions/action.h: In member function 'void modsecurity::actions::Action::set_name_and_payload(const std::string&)':
/root/openlitespeed-1.8.1/../third-party/include/modsecurity/actions/action.h:115:34: error: 'size' is not a member of 'std'
  115 |         if (data.compare(0, std::size(t) - 1, t) == 0) {
      |

iv. This causes the entire build process to conclude like this:

make: *** [Makefile:136: all] Error 2
cp: cannot stat 'build/src/openlitespeed': No such file or directory
cp: cannot stat 'build/src/modules/CMakeFiles/*.so': No such file or directory
cp: cannot stat 'build/src/modules/Makefile/*.so': Not a directory
cp: cannot stat 'build/src/modules/cache/*.so': No such file or directory
cp: cannot stat 'build/src/modules/cmake_install.cmake/*.so': Not a directory
cp: cannot stat 'build/src/modules/modgzip/*.so': No such file or directory
cp: cannot stat 'build/src/modules/modsecurity-ls/*.so': No such file or directory
cp: cannot stat 'build/src/modules/pagespeed/*.so': No such file or directory
Start to pack files.
-e Building finished, please run ./install.sh for installation.
-e You may want to update the ols.conf to change the settings before installation.
-e Enjoy.

Even though the build failed, the script still advises the user to run install.sh, which will of course not install OLS due to the fact that no binary was produced.

Running build.sh with -o OFF -p OFF -l OFF does not fix the issue due to the libbcrypt problem.

[Code-Egg]

Thank you for pointing that out, the command has been updated in the documentation.

Regarding the build from source issue, it’s likely caused by missing dependencies. I just launched a Debian 12 instance on DigitalOcean, and the build/install.sh worked, although it did show some errors due to missing headers.

service lsws status
● lsws.service - LSB: lshttpd
     Loaded: loaded (/etc/init.d/lsws; generated)
     Active: active (running) since Tue 2024-12-31 06:00:03 UTC; 6min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 91098 ExecStart=/etc/init.d/lsws start (code=exited, status=0/SUCCESS)
      Tasks: 3 (limit: 2316)
     Memory: 5.9M
        CPU: 545ms
     CGroup: /system.slice/lsws.service
             ├─91120 "openlitespeed (lshttpd - main)"
             ├─91125 "openlitespeed (lscgid)"
             └─91138 "openlitespeed (lshttpd - #01)"

[kenballus]

Correct me if I'm wrong, but doesn't build.sh install all the necessary dependencies?

What packages do you have installed on the DigitalOcean system?