Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove mavenix take 2 #3806

Merged
merged 11 commits into from
Nov 13, 2023
Merged

Remove mavenix take 2 #3806

merged 11 commits into from
Nov 13, 2023

Conversation

goodlyrottenapple
Copy link
Contributor

Switch from using mavenix to using buildMavenPackage back-ported from the upcoming nixpkgs 23.11 stable release. This may increase build times somewhat and updating maven dependencies will no longer be automatic, but arguably the automation never works and It's difficult to figure out why it's failing.

I've updated the README troubleshooting section with the two cases of manual intervention, necessary with the new packaging solution, when maven dependencies change.

The hash mismatch should only happen when deps change, but if we want to automate this in ci then we can do something similar to: https://github.com/runtimeverification/hs-backend-booster/blob/175f10b16ad50dd2bb50fcfe68cbcacd79760317/scripts/update-haskell-backend.sh#L21C3-L22

@goodlyrottenapple goodlyrottenapple changed the base branch from master to develop November 10, 2023 10:44
buildPhase = ''
runHook preBuild
'' + lib.optionalString buildOffline ''
mvn org.apache.maven.plugins:maven-dependency-plugin:3.6.1:go-offline -Dmaven.repo.local=$out/.m2 ${mvnDepsParameters}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using latest version of the go-offline plugin which does not build snapshot jars of the local project

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Um, you're not 100% requiring every single dependency to be pulled in by this, are you? go-offline has bugs that means it pulls in most of the dependencies, but not 100%. So if this is going to cause something to break if one or two dependencies isn't included in this, this solution isn't going to work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, there are indeed dependencies that the go-offline plugin does not detect. The missed ones have to be manually included here:

k/flake.nix

Lines 66 to 71 in fc1859b

manualMvnArtifacts = [
"org.scala-lang:scala-compiler:2.12.18"
"ant-contrib:ant-contrib:1.0b3"
"org.apache.ant:ant-nodeps:1.8.1"
"org.apache.maven.wagon:wagon-provider-api:1.0-alpha-6"
];

I've added a section in the README troubleshooting explaining this:

k/README.md

Lines 435 to 436 in fc1859b

- `[ERROR] Failed to execute goal ... org.apache.maven.artifact.resolver.ArtifactNotFoundException: The following artifacts could not be resolved: org.scala-lang:scala-compiler:jar:2.12.18 ...`
Add `"org.scala-lang:scala-compiler:2.12.18"` (without the `jar:`) to `manualMvnArtifacts` in `flake.nix`

Comment on lines +41 to +42
-o -name \*.snapshots.xml \
-o -name \*.snapshots.xml.sha1 \) \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still get ....snapshots.xml files which contain dates of the latest snapshot, which we need to remove to make the build (hopefully) deterministic

@goodlyrottenapple goodlyrottenapple marked this pull request as ready for review November 10, 2023 13:17
@goodlyrottenapple goodlyrottenapple requested a review from a team as a code owner November 10, 2023 13:17
Copy link
Contributor

@Baltoli Baltoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but as discussed let's wait until a local change to a Java file lands in the master branch and gets merged into this branch.

@goodlyrottenapple goodlyrottenapple changed the title Remove mavenix take 2 [DO NOT MERGE] Remove mavenix take 2 Nov 10, 2023
@goodlyrottenapple goodlyrottenapple changed the title [DO NOT MERGE] Remove mavenix take 2 Remove mavenix take 2 Nov 13, 2023
@rv-jenkins rv-jenkins merged commit ce5b128 into develop Nov 13, 2023
16 checks passed
@rv-jenkins rv-jenkins deleted the sam/remove-mavenix branch November 13, 2023 13:42
rv-jenkins pushed a commit that referenced this pull request Nov 16, 2023
The refactor of the K derivation in #3806 broke the booster integration
test shell, because we stopped wrapping llvm-backend bins with the
`NIX_LLVM_KOMPILE_LIBS` env var. This PR restores that functionality and
I have confirmed this fixes the integration tests in booster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants