Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New build --stop-after #1573

Merged
merged 7 commits into from
Feb 23, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci-toolchain.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ jobs:
run: ./bin/alr -d -n printenv || ./bin/alr -n -v -d printenv

- shell: bash
run: mv ./bin ./bin-old || { sleep 5s && sudo mv ./bin ./bin-old; }
run: mv ./bin ./bin-old || { sleep 5s && mv ./bin ./bin-old; }
# Windows doesn't allow to replace a running exe so the next command
# fails otherwise. Also, this mv fails sometimes so we try twice JIC.

Expand Down
2 changes: 1 addition & 1 deletion alire.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ windows = { ALIRE_OS = "windows" }
[[pins]]
aaa = { url = "https://github.com/mosteo/aaa", commit = "dff61d2615cc6332fa6205267bae19b4d044b9da" }
ada_toml = { url = "https://github.com/mosteo/ada-toml", commit = "da4e59c382ceb0de6733d571ecbab7ea4919b33d" }
clic = { url = "https://github.com/alire-project/clic", commit = "de0330053584bad4dbb3dbd5e1ba939c4e8c6b55" }
clic = { url = "https://github.com/alire-project/clic", commit = "56bbdc008e16996b6f76e443fd0165a240de1b13" }
dirty_booleans = { url = "https://github.com/mosteo/dirty_booleans", branch = "alire" }
diskflags = { url = "https://github.com/mosteo/diskflags", branch = "alire" }
gnatcoll = { url = "https://github.com/alire-project/gnatcoll-core.git", commit = "4e663b87a028252e7e074f054f8f453661397166" }
Expand Down
2 changes: 1 addition & 1 deletion deps/clic
44 changes: 16 additions & 28 deletions doc/user-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,36 +6,24 @@ stay on top of `alr` new features.

## Release `2.0-dev`

### New switch `alr build --stop-after=<build stage>`

### Enable shared dependencies by default

PR [#1449](https://github.com/alire-project/alire/pull/1449)
mosteo marked this conversation as resolved.
Show resolved Hide resolved

Pre-2.0, Alire worked always in "sandboxed" mode; that is, all source
dependencies were found under `<workspace>/alire/cache`. This behavior can be
now enabled with `alr config --set dependencies.shared false`, locally or
globally.

By default, post-2.0, Alire works in "shared" mode, where sources are
downloaded once (to `~/.cache/alire/releases`) and unique builds are created
(under `~/.cache/alire/builds`) for unique configurations. This should minimize
rebuilds across crate configurations and workspaces, and eliminate risks of
inconsistencies.

Disk use is decreased by unique source downloads, but might be increased by
unique build configurations. Cache management and cleanup will be provided down
the road. The build cache can always be deleted to retrieve disk space, at the
cost of triggering rebuilds.

Unique builds are identified by a build hash which takes into account the
following inputs for a given release:

- Build profile
- Environment variables modified in the manifest
- GPR external variables declared or set
- Configuration variables declared or set
- Compiler version
- Vaue of `LIBRARY_TYPE` and `<CRATE>_LIBRARY_TYPE` variables.
- Hash of dependencies
PR [#1573](https://github.com/alire-project/alire/pull/1573)

From `alr help build`:

**Build stages**

Instead of always doing a full build, the process can be stopped early using `--stop-after=<stage>`, where `<stage>` is one of:

* sync: sync pristine sources to build location
* generation: generate configuration-dependent files
* post-fetch: running of post-fetch actions
* pre-build: running of pre-build actions
* build: actual building of sources
* post-build: running of post-build actions

### Automatic index updates

Expand Down
23 changes: 23 additions & 0 deletions src/alire/alire-builds.ads
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,29 @@ package Alire.Builds is
-- many more shared releases in the vault, finding toolchains could take
-- much more time, hence the separate storage.

type Build_Stages is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a nice process for builds, but I see it is separate from how the code will run.
We may add new stages that will not be added to this enum, and vice versa.

Maybe we can have it more explicit with a loop like this:

for Stage in Build_Stages loop
   case Stage is
      when Sync =>
      when Generation =>
      
      -  Etc,
end loop;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I understand what you mean, but not the implementation detail. What is that loop intended to replace?

Maybe the enum can be renamed to Stopping_Points as to make explicit what they are intended to be used for.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe that's an oversimplification of the build process on my part, and the different steps in the build are probably spread across different locations in the code.

Maybe the enum can be renamed to Stopping_Points as to make explicit what they are intended to be used for.

Indeed that would probably be better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. Yes, it's not as clear-cut as that, as there's some recursion involved through the dependency tree. I'll change the name then.

(Sync,
-- Synchronization of pristine sources from the vault to the build dir.
-- This stage does not exist when using sandboxed dependencies.

Generation,
-- Generation of files based on profiles/configuration variables

Post_Fetch,
-- Running of the post-fetch actions, which happens only on the first
-- build after syncing to a new build location.

Pre_Build,
-- Running of the pre-build actions, which happens on every build

Build,
-- The actual building of sources

Post_Build
-- Running of the post-build actions

);

function Sandboxed_Dependencies return Boolean;
-- Queries config to see if dependencies should be sandboxed in workspace

Expand Down
57 changes: 52 additions & 5 deletions src/alire/alire-roots.adb
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
with Ada.Directories;
with Ada.Unchecked_Deallocation;

with Alire.Builds;
with Alire.Conditional;
with Alire.Dependencies.Containers;
with Alire.Environment.Loading;
Expand Down Expand Up @@ -33,13 +32,33 @@ package body Alire.Roots is

use type UString;

----------------
-- Stop_Build --
----------------

function Stop_Build (Wanted, Actual : Builds.Build_Stages) return Boolean
is
use type Builds.Build_Stages;
begin
if Wanted <= Actual then
Trace.Debug ("Stopping build as requested at stage: " & Wanted'Image);
return True;
else
return False;
end if;
end Stop_Build;

-------------------
-- Build_Prepare --
-------------------

procedure Build_Prepare (This : in out Root;
Saved_Profiles : Boolean;
Force_Regen : Boolean) is
Force_Regen : Boolean;
Stop_After : Builds.Build_Stages :=
Builds.Build_Stages'Last)
is
use all type Builds.Build_Stages;
begin
-- Check whether we should override configuration with the last one used
-- and stored on disk. Since the first time the one from disk will be be
Expand Down Expand Up @@ -68,6 +87,10 @@ package body Alire.Roots is
-- Changes in configuration may require new build dirs.
end if;

if Stop_Build (Stop_After, Actual => Sync) then
return;
end if;

-- Ensure configurations are in place and up-to-date

This.Generate_Configuration (Full => Force or else Force_Regen);
Expand All @@ -83,11 +106,15 @@ package body Alire.Roots is
function Build (This : in out Root;
Cmd_Args : AAA.Strings.Vector;
Build_All_Deps : Boolean := False;
Saved_Profiles : Boolean := True)
Saved_Profiles : Boolean := True;
Stop_After : Builds.Build_Stages :=
Builds.Build_Stages'Last)
return Boolean
is
Build_Failed : exception;

use all type Builds.Build_Stages;

--------------------------
-- Build_Single_Release --
--------------------------
Expand Down Expand Up @@ -193,18 +220,29 @@ package body Alire.Roots is
end if;

-- Run post-fetch, it will be skipped if already ran

Properties.Actions.Executor.Execute_Actions
(This,
State,
Properties.Actions.Post_Fetch);

if Stop_Build (Stop_After, Actual => Post_Fetch) then
return;
end if;

-- Pre-build must run always

Properties.Actions.Executor.Execute_Actions
(This,
State,
Properties.Actions.Pre_Build);

if Stop_Build (Stop_After, Actual => Pre_Build) then
return;
end if;

-- Actual build

if Release.Origin.Requires_Build then
Call_Gprbuild (Release);
else
Expand All @@ -213,7 +251,12 @@ package body Alire.Roots is
& ": release has no sources.", Detail);
end if;

if Stop_Build (Stop_After, Actual => Build) then
return;
end if;

-- Post-build must run always

Properties.Actions.Executor.Execute_Actions
(This,
State,
Expand All @@ -224,9 +267,13 @@ package body Alire.Roots is
end Build_Single_Release;

begin

This.Build_Prepare (Saved_Profiles => Saved_Profiles,
Force_Regen => False);
Force_Regen => False,
Stop_After => stop_after);

if Stop_Build (Stop_After, Actual => Generation) then
return True;
end if;

This.Export_Build_Environment;

Expand Down
9 changes: 7 additions & 2 deletions src/alire/alire-roots.ads
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ private with Ada.Finalization;

with AAA.Strings;

with Alire.Builds;
private with Alire.Builds.Hashes;
with Alire.Containers;
with Alire.Crate_Configuration;
Expand Down Expand Up @@ -264,7 +265,9 @@ package Alire.Roots is
function Build (This : in out Root;
Cmd_Args : AAA.Strings.Vector;
Build_All_Deps : Boolean := False;
Saved_Profiles : Boolean := True)
Saved_Profiles : Boolean := True;
Stop_After : Builds.Build_Stages :=
Builds.Build_Stages'Last)
return Boolean;
-- Recursively build all dependencies that declare executables, and finally
-- the root release. Also executes all pre-build/post-build actions for
Expand All @@ -286,7 +289,9 @@ package Alire.Roots is

procedure Build_Prepare (This : in out Root;
Saved_Profiles : Boolean;
Force_Regen : Boolean);
Force_Regen : Boolean;
Stop_After : Builds.Build_Stages :=
Builds.Build_Stages'Last);
-- Perform all preparations but the building step itself. This will require
-- complete configuration, and will leave all files on disk as if an actual
-- build were attempted. May optionally use saved profiles from the command
Expand Down
Loading
Loading