Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate dummy-ghc from raw ghc source #2183

Merged
merged 89 commits into from
Apr 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
b879fc2
Generate dummy-ghc from raw ghc source
hamishmack Apr 11, 2024
b47022c
Add ghci
hamishmack Apr 11, 2024
5148a83
Add ghci
hamishmack Apr 11, 2024
d3b0b83
Fix reinstallable lib ghc
hamishmack Apr 12, 2024
4bde923
ifdLevel 1
hamishmack Apr 12, 2024
c7e3d0f
ifdLevel 2
hamishmack Apr 12, 2024
eca98b3
Add more packages
hamishmack Apr 12, 2024
a282422
Fix for ghc 8.10
hamishmack Apr 12, 2024
d2ccd7a
Fix utils/iserv-proxy source for older ghc
hamishmack Apr 12, 2024
7ede55e
ifdLevel 3
hamishmack Apr 12, 2024
f96b03d
Include depends in dummy-ghc
hamishmack Apr 16, 2024
cb10298
Merge remote-tracking branch 'origin/master' into hkm/dynamic-dummy-ghc
hamishmack Apr 16, 2024
952e0e5
Fix ghcjs
hamishmack Apr 16, 2024
f9ab8aa
ifdLevel 0
hamishmack Apr 16, 2024
774ccdb
Avoid eval time dependency on ghc.generated
hamishmack Apr 16, 2024
61e3eca
ifdLevel 1
hamishmack Apr 16, 2024
cca7aa0
ifdLevel 2
hamishmack Apr 16, 2024
f01635a
ifdLevel 3
hamishmack Apr 16, 2024
9647a0e
Fix `relude` and `semaphore-compat`
hamishmack Apr 17, 2024
1321986
Fix for ghc 9.10
hamishmack Apr 17, 2024
f58edba
Refactor commment
hamishmack Apr 17, 2024
9eed8ac
ifdLevel 1
hamishmack Apr 17, 2024
7444097
ifdLevel 2
hamishmack Apr 17, 2024
cf9bb26
ifdLevel 3
hamishmack Apr 17, 2024
d58d411
Add cabal2json to static-nix-tools
hamishmack Apr 17, 2024
2997b24
Use ghc928
hamishmack Apr 18, 2024
e78a099
More fixes
hamishmack Apr 18, 2024
c8eb5d0
Fix for darwin
hamishmack Apr 18, 2024
63c4c89
ifdLevel 1
hamishmack Apr 18, 2024
b48c369
Merge remote-tracking branch 'origin/master' into hkm/dynamic-dummy-ghc
hamishmack Apr 18, 2024
2d639b3
update nix-tools-static.nix
Apr 18, 2024
c634d87
ifdLevel 2
hamishmack Apr 18, 2024
5f4bbcb
ifdLevel 3
hamishmack Apr 18, 2024
cdcc6f8
ifdLevel 1
hamishmack Apr 18, 2024
43720c7
ifdLevel 2
hamishmack Apr 18, 2024
7b7b4db
ifdLevel 0
hamishmack Apr 18, 2024
4d02393
Exclude nix-tools for ifdLevel <3
hamishmack Apr 18, 2024
7477217
ifdLevel 1
hamishmack Apr 18, 2024
ab8cf00
Exclude nix-tools for ifdLevel <3
hamishmack Apr 19, 2024
3204a36
ifdLevel 0
hamishmack Apr 19, 2024
6ef4b65
ifdLevel 1
hamishmack Apr 19, 2024
a016440
ifdLevel 2
hamishmack Apr 19, 2024
f9a6acd
Merge branch 'static-nix-tools' into hkm/dynamic-dummy-ghc
hamishmack Apr 19, 2024
8a1e3be
ifdLevel 0
hamishmack Apr 19, 2024
2efa20f
ifdLevel 1
hamishmack Apr 19, 2024
582c7c6
ifdLevel 2
hamishmack Apr 19, 2024
5e6acfb
ifdLevel 3
hamishmack Apr 21, 2024
2ae4192
Merge remote-tracking branch 'origin/master' into hkm/dynamic-dummy-ghc
hamishmack Apr 22, 2024
1a8b993
Keep haskell.nix pinned in nix-tools
hamishmack Apr 22, 2024
882d38b
Add missing evalPackages arg
hamishmack Apr 22, 2024
85188cb
Stick with pinned hadrian plans and haskell.nix
hamishmack Apr 23, 2024
9a017be
Use configured-src for boot packages (to avoid ghc rebuilds for now)
hamishmack Apr 23, 2024
24047ba
Use configured-src for boot packages (to avoid ghc rebuilds for now)
hamishmack Apr 23, 2024
fb862f0
Add terminfo as ghc dep for older ghc
hamishmack Apr 23, 2024
dbef371
ifdLevel 1
hamishmack Apr 23, 2024
fdf50d1
ifdLevel 2
hamishmack Apr 23, 2024
0b4668e
ifdLevel 3
hamishmack Apr 23, 2024
0c82671
Fix for Windows cross compilation
hamishmack Apr 23, 2024
d84ccb2
Avoid old aeson
hamishmack Apr 24, 2024
827f968
Fix `if os(linux)` in cabal files
hamishmack Apr 24, 2024
776ee2d
Avoid old aeson
hamishmack Apr 24, 2024
eba5f07
ifdLevel 2
hamishmack Apr 24, 2024
1304eef
ifdLevel 3
hamishmack Apr 24, 2024
a582c6d
Fix typo
hamishmack Apr 24, 2024
b4dbe1a
Fix typo
hamishmack Apr 24, 2024
baa8178
Avoid old aeson
hamishmack Apr 24, 2024
cc75939
os-string 2 does not work with ghc 8.10
hamishmack Apr 24, 2024
f1d3647
Bump head.hackage
hamishmack Apr 24, 2024
7b72a59
Fix for cabal-install on ghc 8.10
hamishmack Apr 24, 2024
e8b17be
Merge remote-tracking branch 'origin/master' into hkm/dynamic-dummy-ghc
hamishmack Apr 24, 2024
982a3dd
nix flake update ghc910X and ghc911
hamishmack Apr 25, 2024
898a7b1
ifdLevel 1
hamishmack Apr 25, 2024
df54457
ifdLevel 3
hamishmack Apr 25, 2024
a02e250
Skip plugin test for git ghc
hamishmack Apr 25, 2024
6425443
TH dll fix for macOS && ghc<9.4 avoid simdutf
hamishmack Apr 25, 2024
64e079b
TH dll fix for macOS && ghc<9.4 avoid simdutf
hamishmack Apr 26, 2024
bd603ee
Add supported-languages logic
hamishmack Apr 26, 2024
95f4822
Add test for lib/supported-langauges.nix
hamishmack Apr 26, 2024
45770bc
Remove tabs
hamishmack Apr 26, 2024
58ce277
Merge remote-tracking branch 'origin/master' into hkm/dynamic-dummy-ghc
hamishmack Apr 28, 2024
021122f
Exclude packages versions found in hackage from dummy-ghc
hamishmack Apr 28, 2024
aa7b1b5
Exclude packages versions found in hackage from dummy-ghc
hamishmack Apr 29, 2024
5c16f91
Use cabal2json to inspect ghc .cabal files
hamishmack Apr 29, 2024
c09a6e6
Include conditional dependencies (unix and Win32)
hamishmack Apr 29, 2024
17b9703
Fix for ghc 9.11
hamishmack Apr 29, 2024
10664bf
Merge remote-tracking branch 'origin/master' into hkm/dynamic-dummy-ghc
hamishmack Apr 29, 2024
1dba456
Update head.hackage
hamishmack Apr 29, 2024
3833c4e
Revert workarounds for ghc <8.11 (may not be needed now)
hamishmack Apr 29, 2024
be8bc24
Don't use `:override` for head.hackage
hamishmack Apr 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci.nix
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ dimension "Nixpkgs version" nixpkgsVersions (nixpkgsName: pinnedNixpkgsSrc:
// pkgs.lib.optionalAttrs (ifdLevel >= 2 && crossSystemName != "ghcjs")
pkgs.haskell-nix.iserv-proxy-exes.${compiler-nix-name}
// pkgs.lib.optionalAttrs (ifdLevel >= 3) {
hello = (pkgs.haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; inherit compiler-nix-name; }).getComponent "exe:hello";
hello = (pkgs.haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; inherit evalPackages compiler-nix-name; }).getComponent "exe:hello";
})
))
)
Expand Down
11 changes: 11 additions & 0 deletions compiler/ghc/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -633,6 +633,17 @@ stdenv.mkDerivation (rec {
# We could add `configured-src` as an output of the ghc derivation, but
# having it as its own derivation means it can be accessed quickly without
# building GHC.
raw-src = stdenv.mkDerivation {
name = name + "-raw-src";
inherit
version
patches
src;
installPhase = ''
cp -r . $out
'';
phases = [ "unpackPhase" "patchPhase" "installPhase"];
};
configured-src = stdenv.mkDerivation ({
name = name + "-configured-src";
inherit
Expand Down
16 changes: 8 additions & 8 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

159 changes: 148 additions & 11 deletions lib/call-cabal-project-to-nix.nix
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,9 @@ let

fixedProject = replaceSourceRepos rawCabalProject;

inherit (ghc) dummy-ghc-data;
ghcSrc = ghc.raw-src or ghc.buildGHC.raw-src;

fixPlatformString = x: builtins.replaceStrings ["-linux-gnu"] ["-linux"] x;

# Dummy `ghc` that uses the captured output
dummy-ghc = evalPackages.writeTextFile {
Expand All @@ -299,30 +301,54 @@ let
#!${evalPackages.runtimeShell}
case "$*" in
--version*)
cat ${dummy-ghc-data}/ghc/version
echo "The Glorious Glasgow Haskell Compilation System, version ${ghc.version}"
;;
--numeric-version*)
cat ${dummy-ghc-data}/ghc/numeric-version
echo "${ghc.version}"
;;
${pkgs.lib.optionalString (ghc.targetPrefix == "js-unknown-ghcjs-") ''
--numeric-ghc-version*)
cat ${dummy-ghc-data}/ghc/numeric-ghc-version
echo "${ghc.version}"
;;
--numeric-ghcjs-version*)
cat ${dummy-ghc-data}/ghc/numeric-ghcjs-version
echo "${ghc.version}"
;;
''}
--supported-languages*)
cat ${dummy-ghc-data}/ghc/supported-languages
cat ${import ./supported-languages.nix { inherit pkgs evalPackages ghc; }}
;;
--print-global-package-db*)
echo "$out/dumby-db"
;;
--info*)
cat ${dummy-ghc-data}/ghc/info
echo '[("target os", "${
if pkgs.stdenv.targetPlatform.isLinux
then "OSLinux"
else if pkgs.stdenv.targetPlatform.isDarwin
then "OSDarwin"
else if pkgs.stdenv.targetPlatform.isWindows
then "OSMinGW32"
else if pkgs.stdenv.targetPlatform.isGhcjs
then "OSGhcjs"
else throw "Unknown target os ${pkgs.stdenv.targetPlatform.config}"
}")'
echo ',("target arch","${
if pkgs.stdenv.targetPlatform.isx86_64
then "ArchX86_64"
else if pkgs.stdenv.targetPlatform.isAarch64
then "ArchAArch64"
else if pkgs.stdenv.targetPlatform.isJavaScript
then "ArchJavaScript"
else throw "Unknown target arch ${pkgs.stdenv.targetPlatform.config}"
}")'
echo ',("target platform string","${fixPlatformString pkgs.stdenv.targetPlatform.config}")'
echo ',("Build platform","${fixPlatformString pkgs.stdenv.buildPlatform.config}")'
echo ',("Host platform","${fixPlatformString pkgs.stdenv.hostPlatform.config}")'
echo ',("Target platform","${fixPlatformString pkgs.stdenv.targetPlatform.config}")'
echo ']'
;;
--print-libdir*)
echo ${dummy-ghc-data}/ghc/libdir
echo $out/ghc/libdir
;;
*)
echo "Unknown argument '$*'" >&2
Expand All @@ -333,6 +359,117 @@ let
'';
};

ghc-pkgs = [
"Cabal"
"Cabal-syntax"
"array"
"base"
"binary"
"bytestring"
"containers"
"deepseq"
"directory"
"exceptions"
"filepath"
"ghc"
"ghc-bignum"
"ghc-boot"
"ghc-boot-th"
"ghc-compact"
"ghc-experimental"
"ghc-heap"
"ghc-internal"
"ghc-platform"
"ghc-prim"
"ghc-toolchain"
"ghci"
"haskeline"
"hpc"
"integer-gmp"
"mtl"
"os-string"
"parsec"
"pretty"
"process"
"rts"
"semaphore-compat"
"stm"
"template-haskell"
"terminfo"
"text"
"time"
"transformers"
"xhtml"
] ++ (if pkgs.stdenv.targetPlatform.isWindows
then [ "Win32" ]
else [ "unix" ]
);

dummy-ghc-pkg-dump = evalPackages.runCommand "dummy-ghc-pkg-dump" {
nativeBuildInputs = [
evalPackages.haskell-nix.nix-tools-unchecked.exes.cabal2json
evalPackages.jq
];
} (let varname = x: builtins.replaceStrings ["-"] ["_"] x; in ''
PKGS=""
${pkgs.lib.concatStrings
(builtins.map (name: ''
cabal_file=""
if [ -f ${ghcSrc}/libraries/${name}/${name}.cabal ]; then
cabal_file=${ghcSrc}/libraries/${name}/${name}.cabal
elif [ -f ${ghcSrc}/libraries/Cabal/${name}/${name}.cabal ]; then
cabal_file=${ghcSrc}/libraries/Cabal/${name}/${name}.cabal
elif [ -f ${ghcSrc}/libraries/${name}/${name}/${name}.cabal ]; then
cabal_file=${ghcSrc}/libraries/${name}/${name}/${name}.cabal
elif [ -f ${ghcSrc}/compiler/${name}.cabal ]; then
cabal_file=${ghcSrc}/compiler/${name}.cabal
elif [ -f ${ghcSrc}/compiler/${name}.cabal.in ]; then
cabal_file=${ghcSrc}/compiler/${name}.cabal.in
elif [ -f ${ghcSrc}/libraries/${name}/${name}.cabal.in ]; then
cabal_file=${ghcSrc}/libraries/${name}/${name}.cabal.in
fi
if [[ "$cabal_file" != "" ]]; then
fixed_cabal_file=$(mktemp)
cat $cabal_file | sed -e 's/@ProjectVersionMunged@/${ghc.version}/g' -e 's/default: *@[A-Za-z0-9]*@/default: False/g' -e 's/@Suffix@//g' > $fixed_cabal_file
json_cabal_file=$(mktemp)
cabal2json $fixed_cabal_file > $json_cabal_file
EXPOSED_MODULES_${varname name}="$(jq -r '.library."exposed-modules"[]|select(type=="array")[]' $json_cabal_file | tr '\n' ' ')"
DEPS_${varname name}="$(jq -r '.library."build-depends"[]|select(type=="array")[],select(type=="object").then[]' $json_cabal_file | sed 's/^\([A-Za-z0-9-]*\).*$/\1/g' | sort -u | tr '\n' ' ')"
VER_${varname name}="$(jq -r '.version' $json_cabal_file)"
PKGS+=" ${name}"
LAST_PKG="${name}"
fi
'') ghc-pkgs)
}
${ # There is not .cabal file for system-cxx-std-lib
pkgs.lib.optionalString (builtins.compareVersions ghc.version "9.2" >= 0) (
let name="system-cxx-std-lib"; in ''
EXPOSED_MODULES_${varname name}=""
DEPS_${varname name}=""
VER_${varname name}="1.0"
PKGS+=" ${name}"
LAST_PKG="${name}"
'')}
for pkg in $PKGS; do
varname="$(echo $pkg | tr "-" "_")"
ver="VER_$varname"
exposed_mods="EXPOSED_MODULES_$varname"
deps="DEPS_$varname"
echo "name: $pkg" >> $out
echo "version: ''${!ver}" >> $out
echo "exposed-modules: ''${!exposed_mods}" >> $out
echo "depends:" >> $out
for dep in ''${!deps}; do
ver_dep="VER_$(echo $dep | tr "-" "_")"
if [[ "''${!ver_dep}" != "" ]]; then
echo " $dep-''${!ver_dep}" >> $out
fi
done
if [[ "$pkg" != "$LAST_PKG" ]]; then
echo '---' >> $out
fi
done
'');
# Dummy `ghc-pkg` that uses the captured output
dummy-ghc-pkg = evalPackages.writeTextFile {
name = "dummy-pkg-" + ghc.name;
Expand All @@ -342,15 +479,15 @@ let
#!${evalPackages.runtimeShell}
case "$*" in
--version)
cat ${dummy-ghc-data}/ghc-pkg/version
echo "GHC package manager version ${ghc.version}"
;;
${pkgs.lib.optionalString (ghc.targetPrefix == "js-unknown-ghcjs-") ''
--numeric-ghcjs-version)
cat ${dummy-ghc-data}/ghc-pkg/numeric-ghcjs-version
echo "${ghc.version}"
;;
''}
'dump --global -v0')
cat ${dummy-ghc-data}/ghc-pkg/dump-global
cat ${dummy-ghc-pkg-dump}
;;
*)
echo "Unknown argument '$*'. " >&2
Expand Down
5 changes: 0 additions & 5 deletions lib/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -570,11 +570,6 @@ in {
inherit (pkgs.buildPackages.buildPackages) lib runCommand;
};

makeDummyGhcData = import ./make-dummy-ghc-data.nix {
inherit pkgs;
inherit (pkgs.buildPackages.buildPackages) runCommand;
};

# Here we try to figure out which qemu to use based on the host platform.
# This guess can be overridden by passing qemuSuffix
qemuByHostPlatform = hostPlatform:
Expand Down
78 changes: 0 additions & 78 deletions lib/make-dummy-ghc-data.nix

This file was deleted.

Loading
Loading