Skip to content

Commit

Permalink
samba: Add autobuild.yml files for dep resolution
Browse files Browse the repository at this point in the history
**Summary**
This is the initial attempt at showing how to use the power of autobuild
for getting the Solus monorepo under control re. dependency cycles.

The purely additive nature of the autobuild.yml configuration files added
with this commit, should make it trivial to modify and extend the ignored
set of dependencies for a package as our cycle-discovery journey
advances.

To gain an appreciation of just how useful this tool might become for us,
consider the following:

```
ermo@solbox:~/repos/getsolus/soluspkgs
$ autobuild -q query src:packages/ samba ldb tdb talloc tevent
talloc tdb cifs-utils notmuch tevent ldb samba acccheck ffmpeg \
python-pysmbc gvfs kio-extras kodi mpd rhythmbox vlc \
gnome-control-center nautilus-share nemo-extensions budgie-control-center
```

The above ^ shows how autobuild might resolve a build + rebuild order for
when someone is doing a samba stack upgrade and pushing it to a single
builder; like, say, the local packager's machine or the current Solus
build infrastructure, which is limited to one builder.

Note that autobuild also already supports committing+pushing to the Solus
builder, which implies that it can replace the current `build_pr.sh`
script in short order, if the the appetite to do so is present within
Solus as an organisation.

But that's not all!

In fact, `autobuild` also supports generating a "tiered" builder order
(notice the `-t` argument), which those of you packaging KDE might
already be familiar with:

```
ermo@solbox:~/repos/getsolus/soluspkgs
$ autobuild query -t src:packages/ samba ldb tdb talloc tevent
 (.. some output omitted ..)
 🗸  Successfully parsed state!
 🗸  Found all requested packages in state!
 🗸  Successfully built dependency graph!
 🗸  Build order:
 🗸  Tier 1: talloc tdb
 🗸  Tier 2: cifs-utils notmuch tevent
 🗸  Tier 3: ldb
 🗸  Tier 4: samba
 🗸  Tier 5: acccheck ffmpeg python-pysmbc
 🗸  Tier 6: gvfs kio-extras kodi mpd rhythmbox vlc
 🗸  Tier 7: gnome-control-center nautilus-share nemo-extensions
 🗸  Tier 8: budgie-control-center
```

Tiers (called "Precedence Constraint Groups" in graph theory literature)
effectively show which packages can be built in parallel since they do
not interdepend on each other.

With the -q argument like above, this build order could potentially drive
a set of builders (like summit does in the Serpent infra currently):

```
ermo@solbox:~/repos/getsolus/soluspkgs
$ autobuild -qt query src:packages/ samba ldb tdb talloc tevent
talloc tdb
cifs-utils notmuch tevent
ldb
samba
acccheck ffmpeg python-pysmbc
gvfs kio-extras kodi mpd rhythmbox vlc
budgie-control-center gnome-control-center nautilus-share nemo-extensions
```

I hope this serves to show how useful autobuild can potentially become
to Serpent and Solus in terms of automating the discovery of build and
rebuild orders.

This commit is mostly intended as a way to showcase what an autobuild-
driven packaging process might look like for Solus and Serpent, and as a
starting point for the discussion of how to tag "Auxiliary Dependencies"
in a way that's compatible with both the Solus and Serpent monorepos and
tooling.

See also:

Massive props to @GZGavinZhao for his interest in studying the necessary
graph theory at my suggestion, and also for his being ready, able, and
willing to design and build this tool, and for his willingness to take on
board constructive feedback re. the documentation and UX necessary to
productise this solution for a wider audience of Solus staff and
community packagers.

Signed-off-by: Rune Morling <[email protected]>
  • Loading branch information
ermo committed May 1, 2024
1 parent 4673aff commit 4feb8db
Show file tree
Hide file tree
Showing 12 changed files with 46 additions and 0 deletions.
6 changes: 6 additions & 0 deletions common/autobuild-ignore-doxygen.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# doxygen is a common enough cause for cycles that it deserves its own file
# which can be trivially symlinked to.
solver:
ignore:
- doxygen

5 changes: 5 additions & 0 deletions common/autobuild-ignore-python-pytest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# python-pytest is a common cycle-creator, in that it's typically a checkdep,
# not a true builddep.
solver:
ignore:
- python-pytest
7 changes: 7 additions & 0 deletions packages/l/llvm-bolt/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# A rebuild in git shouldn't really trigger a rebuild of llvm-bolt, and upon
# inspection of llvm-bolt's recipe, we see that the only reason git is in its
# builddeps is probably because of build system requirements, so we can safely
# ignore git here.
solver:
ignore:
- git
1 change: 1 addition & 0 deletions packages/l/llvm/autobuild.yml
5 changes: 5 additions & 0 deletions packages/m/mesalib/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# linux-driver-management is a rundep of mesalib, but a rebuild of LDM
# really shouldn't trigger a rebuild of mesalib, so ignore it.
solver:
ignore:
- linux-driver-management
4 changes: 4 additions & 0 deletions packages/o/openjdk-17/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# jtreg is a test harness for testing the JDK, so can be ignored in builddeps.
solver:
ignore:
- jtreg
7 changes: 7 additions & 0 deletions packages/p/pulseaudio/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# python3-qt5 is a rundep of pulseaudio-equalizer, so we can ignore python3-qt5
# when considering pulseaudio rebuilds.
# If we don't, a python3-qt5 rebuild could trigger a qt6-base rebuild via PA
# which does not seem reasonable.
solver:
ignore:
- python3-qt5
1 change: 1 addition & 0 deletions packages/py/python-chardet/autobuild.yml
7 changes: 7 additions & 0 deletions packages/py/python-jinja/autobuild.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# python-pytest and python-markupsafe shouldn't trigger a rebuild of this,
# as these are marked as checkdeps.
solver:
ignore:
- python-markupsafe
- python-pytest

1 change: 1 addition & 0 deletions packages/py/python-requests/autobuild.yml
1 change: 1 addition & 0 deletions packages/py/python-wheel/autobuild.yml
1 change: 1 addition & 0 deletions packages/t/tpm2-tss/autobuild.yml

0 comments on commit 4feb8db

Please sign in to comment.