Skip to content

Commit

Permalink
Add separate cache for getPkgConfigDb
Browse files Browse the repository at this point in the history
Querying pkg-config for the version of every module can be a very
expensive operation on some systems. This change adds a separate,
per-project, cache for pkgConfigDB; reducing the cost from "every plan
change" to "every pkg-config-db change per project".

No input key is required since getPkgConfigDb already specifies the
directories to monitor for changes. These are obtained from pkg-config
itself as `pkg-config --variable pc_path pkg-config` as documented in
pkg-config(1).

A notice is presented to the user when refreshing the packagedb.
  • Loading branch information
andreabedini committed Nov 8, 2023
1 parent 156d8b2 commit 6170fe3
Show file tree
Hide file tree
Showing 140 changed files with 159 additions and 4 deletions.
7 changes: 6 additions & 1 deletion cabal-install/src/Distribution/Client/ProjectPlanning.hs
Original file line number Diff line number Diff line change
Expand Up @@ -638,6 +638,7 @@ rebuildInstallPlan
fileMonitorSourceHashes = newFileMonitorInCacheDir "source-hashes"
fileMonitorElaboratedPlan = newFileMonitorInCacheDir "elaborated-plan"
fileMonitorImprovedPlan = newFileMonitorInCacheDir "improved-plan"
fileMonitorPkgConfigDb = newFileMonitorInCacheDir "pkg-config-db"

newFileMonitorInCacheDir :: Eq a => FilePath -> FileMonitor a b
newFileMonitorInCacheDir = newFileMonitor . distProjectCacheFile
Expand Down Expand Up @@ -725,7 +726,11 @@ rebuildInstallPlan
withRepoCtx
(solverSettingIndexState solverSettings)
(solverSettingActiveRepos solverSettings)
pkgConfigDB <- getPkgConfigDb verbosity progdb

pkgConfigDB <-
rerunIfChanged verbosity fileMonitorPkgConfigDb () $ do
liftIO $ notice verbosity "Querying pkg-config database..."
getPkgConfigDb verbosity progdb

-- TODO: [code cleanup] it'd be better if the Compiler contained the
-- ConfiguredPrograms that it needs, rather than relying on the progdb
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand All @@ -18,8 +19,7 @@ Configuring library 'indef' instantiated with
Data.Map = containers-<VERSION>:Data.Map
for Includes3-0.1.0.0...
Preprocessing library 'indef' for Includes3-0.1.0.0...
Building library 'indef' instantiated with
Data.Map = containers-<VERSION>:Data.Map
Building library 'indef' instantiated with Data.Map = containers-<VERSION>:Data.Map
for Includes3-0.1.0.0...
Configuring executable 'exe' for Includes3-0.1.0.0...
Preprocessing executable 'exe' for Includes3-0.1.0.0...
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/Backpack/Reexport2/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Error:
Problem with module re-exports:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/Backpack/T6385/cabal.out
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/Backpack/bkpcabal01/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/BuildDeps/DepCycle/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Error:
Dependency cycle between the following components:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/BuildToolDepends/setup.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/BuildTools/External/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/BuildTools/Internal/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/CmmSources/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/CmmSourcesDyn/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/CmmSourcesExe/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/ConfigFile/T8487/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O2
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/Configure/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/CopyHie/cabal.out
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/CustomSegfault/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/CustomWithoutCabal/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/Exec/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/ExecModern/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/ExtraPackages/cabal.out
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/ExtraProgPath/setup.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Warning: cannot determine version of <ROOT>/./pkg-config :
""
Warning: cannot determine version of <ROOT>/./pkg-config :
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/FFI/ForeignOptsC/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/FFI/ForeignOptsCxx/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/HaddockArgs/hoogle.out
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/HaddockArgs/quickjump.out
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Downloading the latest package list from test-local-repo
# cabal haddock-project
Warning: haddock-project command is experimental, it might break in the future
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
3 changes: 2 additions & 1 deletion cabal-testsuite/PackageTests/HaddockWarn/cabal.out
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# cabal v2-haddock
Querying pkg-config database...
Resolving dependencies...
No documentation was generated as this package does not contain a library. Perhaps you want to use the --haddock-all flag, or one or more of the --haddock-executables, --haddock-tests, --haddock-benchmarks or --haddock-internal flags.
No documentation was generated as this package does not contain a library. Perhaps you want to use the --haddock-all flag, or one or more of the --haddock-executables, --haddock-tests, --haddock-benchmarks or --haddock-internal flags.
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Error:
Internal libraries only supported with per-component builds.
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/InternalLibraries/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/JS/JsSources/other-arch.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
1 change: 1 addition & 0 deletions cabal-testsuite/PackageTests/ListBin/Script/cabal.out
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal list-bin
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal list-bin
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# cabal v2-update
Downloading the latest package list from test-local-repo
# cabal v2-repl
Querying pkg-config database...
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-repl
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-repl
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-repl
Querying pkg-config database...
Resolving dependencies...
Error: [Cabal-7076]
Cannot open a repl for multiple components at once. The targets 'pkg-a' and 'pkg-b' refer to different components..
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-run
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-build
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-bench
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal v2-bench
Querying pkg-config database...
Resolving dependencies...
Build profile: -w ghc-<GHCVER> -O1
In order, the following will be built:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# cabal bench
Querying pkg-config database...
Resolving dependencies...
Warning: Some RTS options were found standalone, which affect cabal and not the binary. Please note that +RTS inside the --benchmark-options argument suffices if your goal is to affect the benchmarked binary. For example, use "cabal test --benchmark-options='+RTS -N'" to pass the '-N' RTS option to your binary.
Build profile: -w ghc-<GHCVER> -O1
Expand Down
Loading

0 comments on commit 6170fe3

Please sign in to comment.