Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci TEST_BENCH: show output #4089

Merged
merged 2 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ci/ci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export RUSTFLAGS="-D warnings"
export CARGO_INCREMENTAL=0
export CARGO_EXTRA_FLAGS="--locked"

# Determine configuration for installed build (used by test-cargo-miri).
# Determine configuration for installed build (used by test-cargo-miri and `./miri bench`).
echo "Installing release version of Miri"
time ./miri install

Expand Down Expand Up @@ -73,7 +73,7 @@ function run_tests {
fi
if [ -n "${TEST_BENCH-}" ]; then
# Check that the benchmarks build and run, but only once.
time HYPERFINE="hyperfine -w0 -r1" ./miri bench $TARGET_FLAG
time HYPERFINE="hyperfine -w0 -r1 --show-output" ./miri bench $TARGET_FLAG --no-install
fi
# Smoke-test `./miri run --dep`.
./miri run $TARGET_FLAG --dep tests/pass-dep/getrandom.rs
Expand Down
11 changes: 7 additions & 4 deletions miri-script/src/commands.rs
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ impl Command {
Command::Doc { flags } => Self::doc(flags),
Command::Fmt { flags } => Self::fmt(flags),
Command::Clippy { flags } => Self::clippy(flags),
Command::Bench { target, benches } => Self::bench(target, benches),
Command::Bench { target, no_install, benches } =>
Self::bench(target, no_install, benches),
Command::Toolchain { flags } => Self::toolchain(flags),
Command::RustcPull { commit } => Self::rustc_pull(commit.clone()),
Command::RustcPush { github_user, branch } => Self::rustc_push(github_user, branch),
Expand Down Expand Up @@ -378,16 +379,18 @@ impl Command {
Ok(())
}

fn bench(target: Option<String>, benches: Vec<String>) -> Result<()> {
fn bench(target: Option<String>, no_install: bool, benches: Vec<String>) -> Result<()> {
// The hyperfine to use
let hyperfine = env::var("HYPERFINE");
let hyperfine = hyperfine.as_deref().unwrap_or("hyperfine -w 1 -m 5 --shell=none");
let hyperfine = shell_words::split(hyperfine)?;
let Some((program_name, args)) = hyperfine.split_first() else {
bail!("expected HYPERFINE environment variable to be non-empty");
};
// Make sure we have an up-to-date Miri installed and selected the right toolchain.
Self::install(vec![])?;
if !no_install {
// Make sure we have an up-to-date Miri installed and selected the right toolchain.
Self::install(vec![])?;
}

let sh = Shell::new()?;
sh.change_dir(miri_dir()?);
Expand Down
11 changes: 9 additions & 2 deletions miri-script/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ pub enum Command {
/// Runs the benchmarks from bench-cargo-miri in hyperfine. hyperfine needs to be installed.
Bench {
target: Option<String>,
/// When `true`, skip the `./miri install` step.
no_install: bool,
/// List of benchmarks to run. By default all benchmarks are run.
benches: Vec<String>,
},
Expand Down Expand Up @@ -121,9 +123,11 @@ install`. Sets up the rpath such that the installed binary should work in any
working directory. Note that the binaries are placed in the `miri` toolchain
sysroot, to prevent conflicts with other toolchains.

./miri bench [--target <target>] <benches>:
./miri bench [--target <target>] [--no-install] <benches>:
Runs the benchmarks from bench-cargo-miri in hyperfine. hyperfine needs to be installed.
<benches> can explicitly list the benchmarks to run; by default, all of them are run.
By default, this runs `./miri install` to ensure the latest local Miri is being benchmarked;
`--no-install` can be used to skip that step.

./miri toolchain <flags>:
Update and activate the rustup toolchain 'miri' to the commit given in the
Expand Down Expand Up @@ -218,16 +222,19 @@ fn main() -> Result<()> {
Some("bench") => {
let mut target = None;
let mut benches = Vec::new();
let mut no_install = false;
loop {
if let Some(val) = args.get_long_opt("target")? {
target = Some(val);
} else if args.get_long_flag("no-install")? {
no_install = true;
} else if let Some(flag) = args.get_other() {
benches.push(flag);
} else {
break;
}
}
Command::Bench { target, benches }
Command::Bench { target, benches, no_install }
}
Some("toolchain") => Command::Toolchain { flags: args.remainder() },
Some("rustc-pull") => {
Expand Down
Loading