diff --git a/devlog/2024-02-09-Intro-to-optimizing-packages-on-solus.md b/devlog/2024-02-09-Intro-to-optimizing-packages-on-solus.md index 8181c65b5..68de9a3a4 100644 --- a/devlog/2024-02-09-Intro-to-optimizing-packages-on-solus.md +++ b/devlog/2024-02-09-Intro-to-optimizing-packages-on-solus.md @@ -15,6 +15,7 @@ We'll explore how to build packages with advanced compiler techniques in order t + # Cual es la causa Linux distributions have a lot of control over how a source-based package gets compiled and shipped to users as part of a binary repository. Aggressive and advanced compiler optimization techniques, as well as other methods can be used to provide greater out of the box performance for end users. This can greatly benefit users running on older hardware to provide a snappier end-user experience; reducing time waiting on a heavy workload to finish; or even improved battery life; amongst other improvements. @@ -123,6 +124,7 @@ Awesome, these binaries do exactly what we need to benchmark `libwebp`, but, we One extra step we have to do is ensure these binaries are actually linking against their own library, as upstream developers can have a habit of making sure their binaries don't link against their own libraries and end up being self-contained. Run `ldd` to verify. + ``` $ ldd /usr/bin/dwebp linux-vdso.so.1 (0x00007ffed8733000) @@ -135,6 +137,7 @@ $ ldd /usr/bin/dwebp libz.so.1 => /usr/lib/libz.so.1.3.0 (0x00007f7473200000) /usr/lib64/ld-linux-x86-64.so.2 (0x00007f7473bea000) ``` + Awesome in this case both `dwebp` and `cwebp` link against `libwebp.so` so we can be confident that any performance improvements will be applicable to all packages in the repository linking against `libwebp`. @@ -224,6 +227,7 @@ Benchmark 1: cwebp ~/PNG_Test.png -o /dev/null Well... That's interesting. We actually regress in performance for decode performance whilst gaining another small bump in encoding performance. Worse still, we get a bunch of `profile count data file not found [-Wmissing-profile]` warning messages during the optimized build indicating to us our profiling workload isn't comprehensive enough and doesn't cover enough code paths. The lack of a handy `make check` that could be used as a profiling workload is really hurting us here. For now, let's put a pin in exploring PGO, it isn't a dead end but more work needs to be done curating a more comprehensive workload to chuck at it in this particular case, whilst other, easier, optimization techniques are still available to us. + ## 256 Vector Units go brrrrrr... The next obvious step is to explore `glibc` hardware capabilities. For those unaware both `clang` and `gnu` compilers provide `x86_64-v2`, `x86_64-v3` and `x86_64-v4` micro-architecture build options on top of the baseline of `x86_64`. These enable the use of targeting additional CPU instruction sets during compilation for better performance. For example, `-sse4.2` for `x86_64-v2`, `-avx2` for `x86_64-v3`, and `-avx512` for `x86_64-v4`. @@ -256,6 +260,7 @@ We now see these additional files in the `pspec_x86_64.xml` file Let's rerun `lld` on `dwebp` after installing the new package and... + ``` $ ldd /usr/bin/dwebp linux-vdso.so.1 (0x00007ffeab5b1000) @@ -268,6 +273,7 @@ $ ldd /usr/bin/dwebp libz.so.1 => /usr/lib/glibc-hwcaps/x86-64-v3/libz.so.1.3 (0x00007f9a34dbb000) /usr/lib64/ld-linux-x86-64.so.2 (0x00007f9a3520b000) ``` + We can crucially see that `dwebp` is now loading the `x86-64-v3` built `libwebp.so` lib from `/usr/lib/glibc-hwcaps/x86-64-v3/libwebp.so.7.1.8`, success! Let's what our performance looks like. @@ -396,6 +402,7 @@ However, there is some light in this tunnel as various forks of zlib having been Let's just go for it, replacing Solus' `zlib` package with zlib-ng built in compatible mode. It's a bit scary due to how integral zlib is in a typical Linux install, but, how hard could it be? + ## I Zee a Purty lil' Package Well that was simple. Here's what our zlib-ng `package.yml` recipe looks like. @@ -429,6 +436,7 @@ install : | check : | %ninja_check ``` + After building it, all the files seem to be in the right place and the test suite is passing. Let's just install it overwriting our canonical `zlib` package and hope our system doesn't die... I think the word is YOLO. diff --git a/docs/packaging/advanced-config/local-repository.md b/docs/packaging/advanced-config/local-repository.md index b16dcb76c..0e1974030 100644 --- a/docs/packaging/advanced-config/local-repository.md +++ b/docs/packaging/advanced-config/local-repository.md @@ -97,6 +97,7 @@ sudo eopkg ar Solus https://cdn.getsol.us/repo/unstable/eopkg-index.xml.xz This should yield output similar to: + ``` $ sudo eopkg ar Local /var/lib/solbuild/local/eopkg-index.xml.xz Repo Local added to system. @@ -110,6 +111,7 @@ eopkg-index.xml.xz.sha1sum (40.0 B)100% 765.61 KB/s [00:00:00] [complete eopkg-index.xml.xz (2.1 MB)100% 914.38 KB/s [00:00:01] [complete] Package database updated. ``` + 3. Check that the dependency resolution order is correct so that packages from the local `solbuild` repository are preferred over the upstream Solus repository. @@ -131,6 +133,7 @@ To reset the system to use only packages from the official Solus repository, dis The output should look similar to: + ``` $ sudo eopkg disable-repo Local $ sudo eopkg lr @@ -139,11 +142,13 @@ Local [inactive] Solus [active] https://cdn.getsol.us/repo/unstable/eopkg-index.xml.xz ``` + The "Local" `eopkg` repository can be re-enabled with `sudo eopkg enable-repo Local`. + ``` $ sudo eopkg enable-repo Local $ sudo eopkg lr @@ -152,6 +157,7 @@ Local [active] Solus [active] https://cdn.getsol.us/repo/unstable/eopkg-index.xml.xz ``` + ## Closing thoughts diff --git a/docs/packaging/prepare-for-packaging.md b/docs/packaging/prepare-for-packaging.md index 357495e93..19faf440d 100644 --- a/docs/packaging/prepare-for-packaging.md +++ b/docs/packaging/prepare-for-packaging.md @@ -96,9 +96,11 @@ Create your own fork of [getsolus/packages](https://github.com/getsolus/packages Create a local clone of the package repository you just forked. Here we are using the name `solus-packages` and cloning it into our home directoy. The rest of the documentation will presume this structure. You can choose a different name and path but will have to make sure to replace it in every command that refers to the `solus-packages` directory. + ```bash gh repo clone packages ~/solus-packages ``` + ## Initialize git hooks diff --git a/docs/packaging/submitting-a-pull-request.mdx b/docs/packaging/submitting-a-pull-request.mdx index 30b8154ba..0466ba2c4 100644 --- a/docs/packaging/submitting-a-pull-request.mdx +++ b/docs/packaging/submitting-a-pull-request.mdx @@ -86,6 +86,7 @@ There are multiple ways to create a Pull Request with GitHub, either from the we The text editor used by `github-cli` may not the same one that `git` uses. To change this, consult the [`gh config set` command](https://cli.github.com/manual/gh_config_set). + diff --git a/docs/packaging/translation-instructions.md b/docs/packaging/translation-instructions.md index c4575aa31..8b5b9fd1b 100644 --- a/docs/packaging/translation-instructions.md +++ b/docs/packaging/translation-instructions.md @@ -17,12 +17,12 @@ Please join our [Matrix rooms](/docs/user/contributing/getting-involved.md#matri - Strings which include curly braces `{ }` should be translated leaving the curly braces, _and anything inside the braces_, untouched. - Examples: - + - `{DE} Successfully installed, please reboot to continue.` translates to: - `{DE} Instalado exitosamente, reinicie para continuar.` - `Successfully Installed {}` translates to: - `Instalado exitosamente {}` - + - Do not translate the following strings; leave them untouched: - `Packagekit` - `XDG_SESSION_DESKTOP`