mapnik: 3.1.0 -> unstable-2022-04-14 #146039

erictapen · 2021-11-14T22:38:57Z

Motivation for this change

Mapnik is currently broken, so I don't see any other choice than to use current master. I also bumped python-mapnik to latest master and fixed the build.

It required some minor patches, that I'm willing to maintain for future releases.

Things done

    mapnik: 3.1.0 -> unstable-2021-11-02
    
    - use CMake. I just didn't got it to work with SCons.
    - patch cmake to find harfbuzz
    - prefetch catch2 for enabling tests
    - add myself as maintainer
    - build mapnik-config with scons, as it isn't build by cmake
    - use python3 exclusively
    - build with harfbuzz-icu
    - change license to lgpl21Plus

    python3Packages.python-mapnik: unstable-2020-02-24 -> unstable-2020-09-08
    
    Also:
    
    - use libxml2 for xml parsing
    - use newest boost
    - patch to find libmapnik{json,wkt}.a
    - fix license
    - add myself as maintainer

dotlambda · 2021-11-14T23:00:58Z

Does

diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index ee2d656c8bc..7026b0da1a8 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -18490,7 +18490,14 @@ with pkgs;

   opencl-clang = callPackage ../development/libraries/opencl-clang { };

-  mapnik = callPackage ../development/libraries/mapnik { };
+  mapnik = callPackage ../development/libraries/mapnik {
+    gdal = gdal.override {
+      libgeotiff = libgeotiff.override { proj = proj_7; };
+      libspatialite = libspatialite.override { proj = proj_7; };
+      proj = proj_7;
+    };
+    proj = proj_7;
+  };

   marisa = callPackage ../development/libraries/marisa {};

not do the trick? It might at least lead to problems when using other packages that depend on PROJ 8.

erictapen · 2021-11-14T23:58:24Z

@dotlambda Mh, sounds like an easy fix. python-mapnik builds with that.

I guess my changes make sense whenever a new mapnik version comes out. Until then I'd try your fix.

mathisloge · 2021-11-20T12:00:49Z

pkgs/development/libraries/mapnik/cmake-harfbuzz.patch

+-    list(APPEND MAPNIK_OPTIONAL_LIBS HarfBuzz::HarfBuzz HarfBuzz::ICU)
+-endif()
+pkg_check_modules(harfbuzz REQUIRED IMPORTED_TARGET harfbuzz)
+list(APPEND MAPNIK_OPTIONAL_LIBS PkgConfig::harfbuzz)


With mapnik/mapnik#4270 merged, you can pass CMAKE_DISABLE_FIND_PACKAGE_harfbuzz=OFF to the configure call to disable find_packge(harfbuzz) and force pkg-config therefore. So the patch can be removed after that.

See https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-search-procedure (at the bottom of the section) for more info.

That's nice!
Any chance we'd get a new mapnik release anytime soon? Using master in Nixpkgs is usually only done for special cases, e.g. when the last stable release is broken/unsecure.

Good question. I don't really know which plans @artemp has regarding mapnik 4.0.0. I've just started contributing first things to mapnik like the CMake build and windows support.

merged. Patch could be removed

dotlambda · 2021-12-28T10:53:30Z

Does it make sense to apply the python-mapnik bump anyway?
~~Also #144544 breaks mapnik, so it might make sense to apply the mapnik bump after all.~~
EDIT: Never mind, the latter was a false positive. See #152448.

erictapen · 2021-12-28T12:53:22Z

@dotlambda Sure, I'll open a separate PR!

erictapen · 2021-12-28T16:15:34Z

@dotlambda
Looked a bit into it and unfortunately python-mapnik needs mapnik master. Their include file layout seems to have changed in many places.
Do you need the bump for anything specific?

dotlambda · 2021-12-28T16:43:36Z

Do you need the bump for anything specific?

No. Let's just wait for the next official mapnik release.

pkgs/development/python-modules/python-mapnik/default.nix

vcunat · 2022-07-07T20:28:32Z

Perhaps reconsider, in light of PR #180578

dotlambda · 2022-07-09T03:07:31Z

Perhaps reconsider, in light of PR #180578

In fact, mapnik's tests fail in #179406, so we need the mapnik upgrade.
I tried applying a60e669 there but mapnik fails to build with

/build/source/benchmark/src/test_png_encoding2.cpp:6:10: error: 'shared_ptr' in namespace 'std' does not name a template type
    6 |     std::shared_ptr<mapnik::image_rgba8> im_;
      |          ^~~~~~~~~~
/build/source/benchmark/src/test_png_encoding2.cpp:3:1: note: 'std::shared_ptr' is defined in header '<memory>'; did you forget to '#include <memory>'?
    2 | #include "compare_images.hpp"
  +++ |+#include <memory>
    3 |
/build/source/benchmark/src/test_png_encoding2.cpp: In constructor 'test::test(const mapnik::parameters&)':
/build/source/benchmark/src/test_png_encoding2.cpp:16:9: error: 'im_' was not declared in this scope
   16 |         im_ = std::make_shared<mapnik::image_rgba8>(reader->width(),reader->height());
      |         ^~~
/build/source/benchmark/src/test_png_encoding2.cpp:16:20: error: 'make_shared' is not a member of 'std'
   16 |         im_ = std::make_shared<mapnik::image_rgba8>(reader->width(),reader->height());
      |                    ^~~~~~~~~~~
/build/source/benchmark/src/test_png_encoding2.cpp:16:20: note: 'std::make_shared' is defined in header '<memory>'; did you forget to '#include <memory>'?
/build/source/benchmark/src/test_png_encoding2.cpp:16:51: error: expected primary-expression before '>' token
   16 |         im_ = std::make_shared<mapnik::image_rgba8>(reader->width(),reader->height());
      |                                                   ^
/build/source/benchmark/src/test_png_encoding2.cpp: In member function 'virtual bool test::validate() const':
/build/source/benchmark/src/test_png_encoding2.cpp:23:31: error: 'im_' was not declared in this scope
   23 |         mapnik::save_to_file(*im_,actual, "png8:m=h:z=1");
      |                               ^~~
/build/source/benchmark/src/test_png_encoding2.cpp: In member function 'virtual bool test::operator()() const':
/build/source/benchmark/src/test_png_encoding2.cpp:31:43: error: 'im_' was not declared in this scope
   31 |             out = mapnik::save_to_string(*im_,"png8:m=h:z=1");
      |                                           ^~~
make[2]: *** [benchmark/CMakeFiles/mapnik-benchmark-test_png_encoding2.dir/build.make:76: benchmark/CMakeFiles/mapnik-benchmark-test_png_encoding2.dir/src/test_png_encoding2.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1509: benchmark/CMakeFiles/mapnik-benchmark-test_png_encoding2.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

vcunat · 2022-07-09T05:50:23Z

Well, that sounds easy to fix, as it hints:

note: 'std::shared_ptr' is defined in header '<memory>'; did you forget to '#include <memory>'?

erictapen · 2022-07-11T17:41:42Z

I just rebased on latest nixpkgs master and bumped to latest mapnik master.

SuperSandro2000 · 2022-07-13T12:32:53Z

I think we should squash the commits together.

erictapen · 2022-07-13T13:55:38Z

@SuperSandro2000 I wanted to wait with that until it is mergeable, so the development of the PR remains observable.

dotlambda · 2022-07-13T15:12:18Z

I fixed the import by using the same version of PROJ for mapnik and python-mapnik.
Python-mapnik's tests are still failing and I lack the time to fix them.

erictapen · 2022-07-13T16:19:30Z

@dotlambda This is great news, thanks a lot! I was almost ready to give up, as I'm having a hard time remembering what I did back then.

How about we just disable the tests for now and do it some other time? I'll squash the commits then so this can be reviewed again.

dotlambda · 2022-07-13T22:41:52Z

How about we just disable the tests for now and do it some other time? I'll squash the commits then so this can be reviewed again.

Okay, sounds good.

pkgs/development/libraries/mapnik/default.nix

dotlambda · 2022-07-16T11:14:02Z

@erictapen Please squash if the changes look good to you. I suggest we move the last two commits to a separate PR targeting staging.

erictapen · 2022-07-18T13:32:29Z

Just returned from my holidays. I squashed everything and removed your last two commits @dotlambda, I also think a separate PR would be good for them, thx.

SuperSandro2000 · 2022-07-18T13:54:37Z

pkgs/development/libraries/mapnik/default.nix

+  # mapnik-config is currently not build with CMake. So we use the SCons for this one.
+  preBuild = ''
+    cd ..
+    ${scons}/bin/scons utils/mapnik-config


Suggested change

${scons}/bin/scons utils/mapnik-config

scons utils/mapnik-config

Please add scons to nativeBuildInputs

Yeah that breaks the build, as stdenv tries to build everything with scons then. I amended the comment to clarify that.

Can we fix this when we send https://github.com/NixOS/nixpkgs/compare/6d76b47387dd183779e39f8433f08443bd4bb3a7..4e0c6386b64b158e29927c5d9472ead45f81b807 to staging?

Looks like the solution, but can't we just merge this and then put these changes on top (via staging)? Seems like a rather unproblematic occurence of non-idiomatic code to me.

We could also use buildPackages.scons and avoid nativeBuildInputs.

Good point about buildPackages, I included that.

dotlambda

The radioboat commit shouldn't be here.

erictapen · 2022-07-18T18:13:35Z

@dotlambda Right, don't know how that slipped in.

- use CMake. I just didn't got it to work with SCons. - patch cmake to find harfbuzz - prefetch catch2 for enabling tests - add erictapen as maintainer - build mapnik-config with scons, as it isn't build by cmake - use python3 exclusively - build with harfbuzz-icu - change license to lgpl21Plus - use PROJ 8 - use latest scons, but use cmake for everything but mapnik-config - don't make postgresql optional Co-Authored-by: Robert Schütz <[email protected]>

…9-08 Also: - use libxml2 for xml parsing - use newest boost - patch to find libmapnik{json,wkt}.a - fix license - add erictapen as maintainer - use same harfbuzz as mapnik - use pytestCheckHook, disable failing tests Co-Authored-by: Robert Schütz <[email protected]>

dotlambda · 2022-10-29T19:11:41Z

included in #179406

erictapen requested review from hrdinka, FRidh, dotlambda and SuperSandro2000 November 14, 2021 22:38

erictapen requested a review from jonringer as a code owner November 14, 2021 22:38

github-actions bot added the 6.topic: python label Nov 14, 2021

ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Nov 14, 2021

erictapen marked this pull request as draft November 14, 2021 23:58

erictapen mentioned this pull request Nov 15, 2021

mapnik: fix build #146050

Merged

12 tasks

erictapen force-pushed the python-mapnik branch from 1bc9288 to 9d438be Compare November 17, 2021 13:04

SuperSandro2000 approved these changes Nov 19, 2021

View reviewed changes

mathisloge reviewed Nov 20, 2021

View reviewed changes

erictapen force-pushed the python-mapnik branch from 9d438be to 468aca4 Compare December 28, 2021 16:16

SuperSandro2000 reviewed Jan 1, 2022

View reviewed changes

pkgs/development/python-modules/python-mapnik/default.nix Show resolved Hide resolved

erictapen force-pushed the python-mapnik branch from 564f837 to 7188b7e Compare July 11, 2022 17:40

erictapen changed the title ~~mapnik: 3.1.0 -> unstable-2021-11-02~~ mapnik: 3.1.0 -> unstable-2022-04-14 Jul 11, 2022

erictapen marked this pull request as ready for review July 11, 2022 17:41

SuperSandro2000 reviewed Jul 15, 2022

View reviewed changes

pkgs/development/libraries/mapnik/default.nix Outdated Show resolved Hide resolved

pkgs/development/libraries/mapnik/default.nix Outdated Show resolved Hide resolved

pkgs/development/libraries/mapnik/default.nix Outdated Show resolved Hide resolved

ofborg bot added 10.rebuild-darwin: 11-100 10.rebuild-linux: 501+ 10.rebuild-linux: 2501-5000 and removed 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jul 16, 2022

dotlambda requested a review from SuperSandro2000 July 16, 2022 17:38

erictapen force-pushed the python-mapnik branch from 6d76b47 to 4e0c638 Compare July 18, 2022 13:31

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 501+ 10.rebuild-linux: 2501-5000 labels Jul 18, 2022

SuperSandro2000 reviewed Jul 18, 2022

View reviewed changes

erictapen force-pushed the python-mapnik branch from 4e0c638 to 87615aa Compare July 18, 2022 13:59

dotlambda requested changes Jul 18, 2022

View reviewed changes

erictapen force-pushed the python-mapnik branch from 87615aa to cfa3d8a Compare July 18, 2022 18:13

erictapen and others added 2 commits July 18, 2022 20:23

erictapen force-pushed the python-mapnik branch from cfa3d8a to dbae7dc Compare July 18, 2022 18:25

dotlambda closed this Oct 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mapnik: 3.1.0 -> unstable-2022-04-14 #146039

mapnik: 3.1.0 -> unstable-2022-04-14 #146039

erictapen commented Nov 14, 2021

dotlambda commented Nov 14, 2021

erictapen commented Nov 14, 2021

mathisloge Nov 20, 2021 •

edited

Loading

erictapen Nov 20, 2021

mathisloge Nov 20, 2021

mathisloge Dec 3, 2021

dotlambda commented Dec 28, 2021 •

edited

Loading

erictapen commented Dec 28, 2021

erictapen commented Dec 28, 2021

dotlambda commented Dec 28, 2021

vcunat commented Jul 7, 2022

dotlambda commented Jul 9, 2022

vcunat commented Jul 9, 2022

erictapen commented Jul 11, 2022

SuperSandro2000 commented Jul 13, 2022

erictapen commented Jul 13, 2022

dotlambda commented Jul 13, 2022

erictapen commented Jul 13, 2022

dotlambda commented Jul 13, 2022

dotlambda commented Jul 16, 2022

erictapen commented Jul 18, 2022

SuperSandro2000 Jul 18, 2022

erictapen Jul 18, 2022

SuperSandro2000 Jul 18, 2022

erictapen Jul 18, 2022

dotlambda Jul 18, 2022

erictapen Jul 18, 2022

dotlambda left a comment

erictapen commented Jul 18, 2022

dotlambda commented Oct 29, 2022

	${scons}/bin/scons utils/mapnik-config
	scons utils/mapnik-config

mapnik: 3.1.0 -> unstable-2022-04-14 #146039

mapnik: 3.1.0 -> unstable-2022-04-14 #146039

Conversation

erictapen commented Nov 14, 2021

Motivation for this change

Things done

dotlambda commented Nov 14, 2021

erictapen commented Nov 14, 2021

mathisloge Nov 20, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda commented Dec 28, 2021 • edited Loading

erictapen commented Dec 28, 2021

erictapen commented Dec 28, 2021

dotlambda commented Dec 28, 2021

vcunat commented Jul 7, 2022

dotlambda commented Jul 9, 2022

vcunat commented Jul 9, 2022

erictapen commented Jul 11, 2022

SuperSandro2000 commented Jul 13, 2022

erictapen commented Jul 13, 2022

dotlambda commented Jul 13, 2022

erictapen commented Jul 13, 2022

dotlambda commented Jul 13, 2022

dotlambda commented Jul 16, 2022

erictapen commented Jul 18, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotlambda left a comment

Choose a reason for hiding this comment

erictapen commented Jul 18, 2022

dotlambda commented Oct 29, 2022

mathisloge Nov 20, 2021 •

edited

Loading

dotlambda commented Dec 28, 2021 •

edited

Loading