Loading PlugIn failed #34

vri-sst · 2021-03-04T09:50:30Z

Hello and thank you for this very useful plugin.

I have some troubles by loading the photolayer_pi plugin in openCPN 5.2.4 in one of my two computers, both running on Ubuntu 18.04.

When the plugin manager load the plugin, i have the following error in the opencpn log :
ERROR dlunix.cpp:2291 ...../lib/opencpn/libphotolayer_pi.so: undefined symbol: proj_get_type

Do you have any idea about what is going on?

Thanks!

VRI

The text was updated successfully, but these errors were encountered:

Rasbats · 2021-03-04T11:15:52Z

The plugin normally uses libgeotiff from the installed system on Linux. It may not be present.
You could try this:
https://www.howtoinstall.me/ubuntu/18-04/libgeotiff-dev/
Which version of the plugin are you using?

vri-sst · 2021-03-04T12:54:55Z

Hi Rasbats,

I have already libgeotiff-dev (1.4.2-2build1) installed on my system.

I am using the v3.4.2 of the photolayer plugin (provided by the openCPN 5.2.4 plugin-manager).

Rasbats · 2021-03-04T14:02:23Z

Thanks. Not running 18.04 here at the moment. I will flash up another HD and install 18.04 and give the plugin a test.

Rasbats · 2021-03-04T15:35:19Z

A lot of work was done after v3.3.1 to make the managed plugin, not change the functionality. This has broken something. If you import v3.3.1 from here you should get a working plugin:
https://cloudsmith.io/~opencpn/repos/photolayer-prod/packages/detail/raw/photolayer-3.3-ubuntu-18.04-tarball/3.3.1+3105.v3.3.1/

If you are using the gtk3 version of 18.0.4 search for that in the list of plugins.

vri-sst · 2021-03-04T16:35:42Z

Thank you very much, it is working properly with the v3.3.1.

Thanks for the link, I did not succeed to compile it by myself and export as tar.gz archive (that could be load by the pluginManager).

Rasbats · 2021-03-04T19:03:44Z

I will keep this open. We need to fix the building method, which is 'borrowed' from ShipDriver.

leamas · 2021-03-05T12:10:49Z

So what happens if you checkout some point back in the history:

$ git log --oneline -12
2c65a59 Revert "align shipdriver"
a2cf0ed Revert "remove proj"
73f5d59 Revert "Update LibGeotiff.cmake"
2e1f71a Revert "use_system_geotiff - ON"
760dd65 use_system_geotiff - ON
c42377f Update LibGeotiff.cmake
b1fbc45 remove proj
0986703 align shipdriver
5d9ba0a v3.4.2
9974833 Update circleci-build-macos.sh
7c8813f Update circleci-build-macos.sh
e30347e Update Proj.cmake

Given this you for example do something like

git push -f origin e30347e:master

This will reset the github master branch to the last commit and build that. Now, breaking master is a bad idea, you can use the build' branch instead:

git push -f origin e30347e:build

This will build the commit 12 revisions back. If that doesn't make it, find an even older state which works. Once you have something which works, lookup how to make a git bisect. This will help to find exactly which commit that broke the build.

Rasbats · 2021-03-05T15:32:42Z

The latest build (master branch) worked on CircleCI but not locally. A linking error:

photolayer.build.log

The error in OpenCPN (log) using v3.4.2 from the master catalog:

07:44:41.150 MESSAGE pluginmanager.cpp:1174 Checking plugin candidate: /home/mike/.local/lib/opencpn/libphotolayer_pi.so
07:44:41.150 MESSAGE pluginmanager.cpp:2124 Plugin is compatible by elf library scan: true
07:44:41.150 MESSAGE pluginmanager.cpp:2232 PlugInManager: Loading PlugIn: /home/mike/.local/lib/opencpn/libphotolayer_pi.so
07:44:41.154 ERROR dlunix.cpp:162 /home/mike/.local/lib/opencpn/libphotolayer_pi.so: undefined symbol: proj_get_type
07:44:41.154 MESSAGE pluginmanager.cpp:2291 PlugInManager: Cannot load library: /home/mike/.local/lib/opencpn/libphotolayer_pi.so

Proj was not finding the sqlite3 library. Installed this package and I got as far as the linking error above when I built locally.

leamas · 2021-03-05T21:57:44Z

The linking error is bad, really bad. It indicates that it tries to use the dynamic proj library instead of the bundled, static one. As usual, I need a link to the complete build log to say anything useful here.

Rasbats · 2021-03-06T08:20:37Z

CircleCI log in a minute. Local build is here:
photolayer.build.log

This one uses proj-6.3.2 but the result is the same with proj-7.2.1

To get this far I installed the sql3 package. Latest proj versions need sql3 to build the proj database.

Rasbats · 2021-03-06T08:22:11Z

And the circleci logs:
https://app.circleci.com/pipelines/github/Rasbats/photolayer_pi?branch=master

leamas · 2021-03-06T08:32:54Z

Do I get this right: The linkage error only occurs on your local build, not on CircleCI?

Rasbats · 2021-03-06T09:55:55Z

That is correct. It appears to build on CircleCI but the plugins do not work on Linux OS.

07:44:41.154 ERROR dlunix.cpp:162 /home/mike/.local/lib/opencpn/libphotolayer_pi.so: undefined symbol: proj_get_type

leamas · 2021-03-06T10:01:24Z

hm... what is the output of ldd? i. e., on linux, something like:

Get the tarball download link from cloudsmith
wget <link to tarball>
tar xf tarball.tar.gz
cd tarball-name
ldd $(find . -name *.so)

Rasbats · 2021-03-06T10:30:39Z

ldd_photolayer_pi.txt

leamas · 2021-03-06T13:15:57Z

hm... This indeed looks problematic. The missing symbol proj_get_type does exist in the 7.1 version of proj which is bundled with the plugin. The ldd output also indicates that the library in not dynamically linked and we can thus assume that the bundled code is statically linked into place.

A shot in the dark: does the cache contain some outdated version of the proj library? What happens if you apply this patch which will invalidate the cache and force a full rebuild? Could you please test and report back (and link to build logs if there is still problem).


diff --git a/.circleci/config.yml b/.circleci/config.yml
index 53eada6..37d9f7e 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -12,10 +12,10 @@ debian-steps: &debian-steps
     - checkout
     - restore_cache:
         keys:
-          - v4-<<parameters.target>>-{{checksum "ci/circleci-build-debian.sh"}}
+          - v5-<<parameters.target>>-{{checksum "ci/circleci-build-debian.sh"}}
     - run: ci/circleci-build-debian.sh
     - save_cache:
-        key: v4-<<parameters.target>>-{{checksum "ci/circleci-build-debian.sh"}}
+        key: v5-<<parameters.target>>-{{checksum "ci/circleci-build-debian.sh"}}
         paths:
           - /home/circleci/.local/lib
           - /home/circleci/.local/include

Rasbats · 2021-03-06T14:52:19Z

No luck. We have the same error message in the OpenCPN log.

From my reading about recent Proj versions. sql3 (spatialite) is used to create the proj.db. It needs a static sql3 to do this when it is built. Which is perhaps why the build works as far as it does locally.

https://app.circleci.com/pipelines/github/Rasbats/photolayer_pi?branch=master

leamas · 2021-03-06T15:47:55Z

This is probably not about sql3 (I assume you mean sqlite3 ?). The dynamic linkage error on linux is the missing symbol proj_get_type, a symbol which exists in the proj static library we link against.

When you test this, which linux version are you using? Ubuntu Focal/20.04? Something else? I need to re-run this build and log in to the circleci builder to investigate more.

It might be about linkage order. Symbols from static libraries are only linked if the code which needs the library already is processed by ld That is, re-ordering the libraries fed to ld might resolve things, or make them worse... Anyway, Saturday dinner is about to be served, wine bottles are opened. I will need to look into this later ;)

leamas · 2021-03-06T15:55:42Z

Hm... When I look at the Focal logs I don't see any error message. What am I missing?

Rasbats · 2021-03-06T16:05:09Z

Yes... I am using Focal to test this. No errors in the build logs just a plugin that installs (jigsaw icon) and does not run.

Enjoy your dinner.

leamas · 2021-03-07T07:35:26Z

Enjoy your Enjoy your dinner.dinner.

Thanks, it was great. Hope you had some time to eat as well

I can reproduce the problem on a clean Focal doing local builds. Will hopefully find some time late today or tomorrow to nail it down.

leamas · 2021-03-07T15:43:11Z

The linkage order is indeed part of the problem. However, fixing this uncovers the fact that libproj.a actually isn't a working static library... an insane mess. Running out of time now, let's see when I can came back to this.

Rasbats · 2021-03-09T17:40:30Z

No luck with the latest PR. The last build that worked for CI was 3.3.1. But the same linking error happens with a local build of 3.3.1. Crazy.

leamas · 2021-03-11T14:09:25Z

I have been successful installing this plugin using a local build on a clean focal. Perhaps something in your environment?

Built the plugin from current master tip at 8768d70

Rasbats · 2021-03-11T14:45:47Z

Does the Cloudsmith plugin work for you as well?

BTW I am up and running with Fedora 64bit on the Pi. Installed the flathub OpenCPN but ... you mentioned needing a special version?

I am getting the incompatible version message with aarch64 flatpak tarballs of ShipDriver.

leamas · 2021-03-11T15:00:38Z

Havn't checked checked the cloudsmith one.

Not surprised about the incompatible version message. This is an insane mess with no real specs. Could you please make a run with -loglevel debug and post the log?

leamas · 2021-03-11T15:02:12Z

BTE: Have you a link to the cloudsmith repo?

leamas · 2021-03-11T15:05:58Z

and... don't forget to clear the cache. Not doing so costed me some cycles.

Rasbats · 2021-03-11T15:10:02Z

Forget ShipDriver - I think I used the wrong tarball.

PhotoLayer build is here:
https://cloudsmith.io/~opencpn/repos/photolayer-alpha/packages/detail/raw/photolayer-3.4-ubuntu-gtk3-20.04-tarball/3.4.2+3832.8768d70/

leamas · 2021-03-11T15:19:21Z

Indeed. the cloudsmith tarball is broken. In fact, it isn't even linked to libgeotiff in any way. Seems like the dependency is missing in ci/control. Can you please try this patch:

diff --git a/ci/control b/ci/control
index ccf0fc0..8e75bbb 100644
--- a/ci/control
+++ b/ci/control
@@ -11,6 +11,7 @@ Build-Depends: debhelper (>= 9),
  libelf-dev,
  libexif-dev,
  libgdk-pixbuf2.0-dev,
+ libgeotiff-dev,
  libgtk2.0-dev,
  libjsoncpp-dev,
  liblz4-dev,

Rasbats · 2021-03-11T15:34:43Z

Taking the ShipDriver conversation to the ShipDriver issues:
Rasbats/shipdriver_pi#153

Rasbats · 2021-03-11T20:45:41Z

New install of Focal. Using the Cloudsmith tarball the OpenCPN log is here:
opencpn.photolayer.focal.log

I will try a local build tomorrow. Interesting to see a new error. LibOpenGL.

leamas · 2021-03-11T21:19:33Z

Also here ldd is the important tool. What does ldd /home/mike/.local/lib/opencpn/libphotolayer_pi.sosay?

leamas · 2021-03-11T21:26:07Z

No, not ldd. This works because it's installed on my focal, it's in the libopengl0 package.

To me. this seems like a very reasonable requriement to have installed. Why isn't it on your focal? If you install that package, is it enough to get it running?

Rasbats · 2021-03-11T21:31:47Z

I will play with that package tomorrow. G'Night.

Rasbats · 2021-03-12T08:04:14Z

Installed libopengl0. Back to the proj_get_type error:
opencpn.log
I will try a local build with my clean focal.
Could you provide a link to your working repo? I will try building that as well.

leamas · 2021-03-12T08:11:00Z

Could you provide a link to your working repo? I will try building that as well.

As I said, it was commit 8768d70. Doing a git checkout 8768d70 gives you an exact copy of the tree I built.

Can you attach the tarball so I can test and look at it?

leamas · 2021-03-12T08:18:32Z

hm... looking at current photolayer master tip: it's very broken. Give me some time.

Rasbats · 2021-03-12T08:21:45Z

photolayer-3.4.2+3832.8768d70_ubuntu-gtk3-20.04-x86_64.tar.gz
photolayer-3.4.3+3837.450ca76_ubuntu-gtk3-20.04-x86_64.tar.gz

450ca76 is the result of your last patch.

leamas · 2021-03-12T08:24:56Z

I don't understand this. The master tip doesn't even build for me. "confused".

leamas · 2021-03-12T08:29:36Z

Same thing. It is not linked to libgeotiff at all, hence the errors. Now I need to understand why it doesn't build for me first

leamas · 2021-03-12T08:32:55Z

OK, it builds on Focal, no problem w that.

The local build is OK, linked to libgeotiff. Have a got a link to your circleci build logs for 450ca76

leamas · 2021-03-12T10:00:30Z

OK. I rebuilt 450ca76 on Circleci

It is available on Cloudsmith: https://dl.cloudsmith.io/public/alec-leamas/opencpn-plugins-unstable/raw/names/photolayer-3.4-ubuntu-gtk3-20.04-tarball/versions/3.4.3+1761.450ca76/photolayer-3.4.3+1761.450ca76_ubuntu-gtk3-20.04-x86_64.tar.gz

And I can install that plugin just fine on focal after removing my experimental wx 3.1 build of OpenCPN

One thing: cache problems.. I had to clear the cache to have it working.

Thoughts?

Rasbats · 2021-03-12T10:55:05Z

Sorry... I may have been chasing phantoms ... again!
Installed the libgeotiff package and 450ca76 works! I had forgotten libgeotiff was required. This mirrors the situation with the legacy plugin.

Neede to reinstall to get it installed properly, without removing the cache.

Installing the libgeotiff package brings in a number of other packages. Not sure if this can be accomodated by the plugin build process. Shall we leave this as it is? The manual will need a note.

The number of PhotoLayer users may be small. If they upgraded from the legacy plugin they will already have libgeotiff on their systems.

Rasbats · 2021-03-12T10:55:34Z

p.s. Now trying a local build to confirm this works.

Rasbats · 2021-03-12T12:25:22Z

Problem with finding wxWidgets. Set the build process up using this:

https://opencpn.org/wiki/dokuwiki/doku.php?id=opencpn:developer_manual:developer_guide:compiling_linux

Is the problem linked with this issue:

seandepagnier/rtlsdr_pi#25

My machine is pretty clean. No self compiled wxWidgets.
Build log here:

photolayer_build.txt

Leaving this to play on the Pi. I will do more tests later, trying Petri fixes.

leamas · 2021-03-12T13:00:34Z

Those docs are crap. Instead, use https://github.com/Rasbats/managed_plugins/wiki/Local-Build ;)

Rasbats · 2021-03-12T14:32:02Z

RTFM... Perfect... the local build now works fine. Something needs adding to the DokuWiki;)

Shall we leave this as it is?

leamas · 2021-03-12T14:57:32Z

Yes, there are more pressing issues for now. If we really should dive in this I'm convinced it's a can of nasty worms...

Rasbats · 2021-03-12T15:18:36Z

Agree... I will get this version in the managed list.

vri-sst closed this as completed Mar 4, 2021

Rasbats reopened this Mar 4, 2021

Rasbats mentioned this issue Mar 5, 2021

Flatpak fixes Rasbats/shipdriver_pi#156

Merged

Loading PlugIn failed #34

Loading PlugIn failed #34

Comments

vri-sst commented Mar 4, 2021

Thanks!

Rasbats commented Mar 4, 2021

vri-sst commented Mar 4, 2021 • edited Loading

Rasbats commented Mar 4, 2021

Rasbats commented Mar 4, 2021 • edited Loading

vri-sst commented Mar 4, 2021

Rasbats commented Mar 4, 2021 • edited Loading

leamas commented Mar 5, 2021

Rasbats commented Mar 5, 2021

leamas commented Mar 5, 2021

Rasbats commented Mar 6, 2021

Rasbats commented Mar 6, 2021

leamas commented Mar 6, 2021

Rasbats commented Mar 6, 2021

leamas commented Mar 6, 2021 • edited Loading

Rasbats commented Mar 6, 2021

leamas commented Mar 6, 2021

Rasbats commented Mar 6, 2021

leamas commented Mar 6, 2021

leamas commented Mar 6, 2021

Rasbats commented Mar 6, 2021

leamas commented Mar 7, 2021

leamas commented Mar 7, 2021

Rasbats commented Mar 9, 2021

leamas commented Mar 11, 2021

Rasbats commented Mar 11, 2021

leamas commented Mar 11, 2021

leamas commented Mar 11, 2021

leamas commented Mar 11, 2021

Rasbats commented Mar 11, 2021

leamas commented Mar 11, 2021

Rasbats commented Mar 11, 2021

Rasbats commented Mar 11, 2021

leamas commented Mar 11, 2021

leamas commented Mar 11, 2021

Rasbats commented Mar 11, 2021

Rasbats commented Mar 12, 2021

leamas commented Mar 12, 2021

leamas commented Mar 12, 2021

Rasbats commented Mar 12, 2021

leamas commented Mar 12, 2021

leamas commented Mar 12, 2021

leamas commented Mar 12, 2021

leamas commented Mar 12, 2021

Rasbats commented Mar 12, 2021

Rasbats commented Mar 12, 2021

Rasbats commented Mar 12, 2021

leamas commented Mar 12, 2021

Rasbats commented Mar 12, 2021

leamas commented Mar 12, 2021

Rasbats commented Mar 12, 2021

vri-sst commented Mar 4, 2021 •

edited

Loading

Rasbats commented Mar 4, 2021 •

edited

Loading

Rasbats commented Mar 4, 2021 •

edited

Loading

leamas commented Mar 6, 2021 •

edited

Loading