TVNC 2.0.91 and gnome-terminal-3.10.2 on SLES 12 SP1

[dev-gmp]

We are performing tests on SLES 12 SP1 and have run into an issue with TVNC. SLES 12 SP1 ships with gnome 3.10 and according to SuSE, they support MESA with llvmpipe on machines which do not have hardware accelerated graphics. Gnome starts fine but we ran into an issue with gnome-terminal, the application crashes as soon as we click on any of the menu items. A stacktrace is below:

#0  0x00002b6625be7139 in g_logv () from /usr/lib64/libglib-2.0.so.0
#1  0x00002b6625be7282 in g_log () from /usr/lib64/libglib-2.0.so.0
#2  0x00002b6624d076f1 in ?? () from /usr/lib64/libgdk-3.so.0
#3  0x00002b6624d0fe41 in ?? () from /usr/lib64/libgdk-3.so.0
#4  0x00002b66260e32cb in _XError () from /usr/lib64/libX11.so.6
#5  0x00002b66260e0367 in ?? () from /usr/lib64/libX11.so.6
#6  0x00002b66260e0415 in ?? () from /usr/lib64/libX11.so.6
#7  0x00002b66260e12f8 in _XReply () from /usr/lib64/libX11.so.6
#8  0x00002b662782fd7b in XIGrabDevice () from /usr/lib64/libXi.so.6
#9  0x00002b6624d014cb in ?? () from /usr/lib64/libgdk-3.so.0
#10 0x00002b6624ce359a in gdk_device_grab () from /usr/lib64/libgdk-3.so.0
#11 0x00002b662476749b in ?? () from /usr/lib64/libgtk-3.so.0
#12 0x00002b66247693fd in gtk_menu_popup_for_device () from /usr/lib64/libgtk-3.so.0
#13 0x00002b6624769932 in gtk_menu_popup () from /usr/lib64/libgtk-3.so.0
#14 0x00002b662476f51b in ?? () from /usr/lib64/libgtk-3.so.0
#15 0x00002b6624770b53 in ?? () from /usr/lib64/libgtk-3.so.0
#16 0x00002b6625954dc8 in g_closure_invoke () from /usr/lib64/libgobject-2.0.so.0
#17 0x00002b6625965f57 in ?? () from /usr/lib64/libgobject-2.0.so.0
#18 0x00002b662596e429 in g_signal_emit_valist () from /usr/lib64/libgobject-2.0.so.0
#19 0x00002b662596e6e2 in g_signal_emit () from /usr/lib64/libgobject-2.0.so.0
#20 0x00002b6624773f46 in ?? () from /usr/lib64/libgtk-3.so.0
#21 0x00002b66247741a0 in ?? () from /usr/lib64/libgtk-3.so.0
#22 0x00002b66247578de in ?? () from /usr/lib64/libgtk-3.so.0
#23 0x00002b6625954ff7 in ?? () from /usr/lib64/libgobject-2.0.so.0
#24 0x00002b662596da88 in g_signal_emit_valist () from /usr/lib64/libgobject-2.0.so.0
#25 0x00002b662596e6e2 in g_signal_emit () from /usr/lib64/libgobject-2.0.so.0
#26 0x00002b6624880264 in ?? () from /usr/lib64/libgtk-3.so.0
#27 0x00002b6624755c7c in ?? () from /usr/lib64/libgtk-3.so.0
#28 0x00002b66247574ba in gtk_main_do_event () from /usr/lib64/libgtk-3.so.0
#29 0x00002b6624d0cfb2 in ?? () from /usr/lib64/libgdk-3.so.0
#30 0x00002b6625be0166 in g_main_context_dispatch () from /usr/lib64/libglib-2.0.so.0
#31 0x00002b6625be04b8 in ?? () from /usr/lib64/libglib-2.0.so.0
#32 0x00002b6625be055c in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0
#33 0x00002b662567c30c in g_application_run () from /usr/lib64/libgio-2.0.so.0
#34 0x00000000004127cf in main (argc=1, argv=0x7ffd2bbdfb98) at server.c:133

Note that, TigerVNC, version 1.4.3, an update provided by SLES 12 SP1 works fine, however tigervnc-1.3.0, shipped with SLES 12 SP1 displays the same behavior as TVNC 2.0.91.

[dcommander]

How are you using llvmpipe? I have never been able to make Mesa work with TurboVNC at all, except by using the following procedure:

http://www.turbovnc.org/Documentation/Mesa

which involves building Mesa with xlib support (necessary because TurboVNC has no GLX extension.)

At any rate, I was able to repro the issue using that method. Here are my observations:

• Mesa 12.0.1 (latest & greatest), added to the head of LD_LIBRARY_PATH in ~/.vnc/xstartup.turbovnc:

  GNOME 3 doesn't start.

  gnome-session-is-accelerated: No hardware 3D support.
  gnome-session-check-accelerated: Helper exited with code 256
  

• Mesa 11.2.2 (latest in the 11.x series) and Mesa 10.0.2 (same version that ships in SLES 12 SP1), added to the head of LD_LIBRARY_PATH in ~/.vnc/xstartup.turbovnc:

  • GNOME 3 starts, but gnome-terminal crashes, per your observations.
  • The mouse pointer disappears and reappears.
  • Performance is very slow (borderline unusable), and interaction is difficult.

This appears to be the same issue as:
https://bugzilla.redhat.com/show_bug.cgi?id=959941

I concur with your assessment that, with the exception of the slow performance, the other issues do not occur with TigerVNC 1.4.3, but it's very unclear why (and TigerVNC unfortunately doesn't maintain any kind of a change log that would facilitate such a discovery, nor did any of their Git log entries reveal where the fixes occurred.)

Another TurboVNC user reported (and I confirmed) that the following works around the mouse pointer issue:

dbus-launch gsettings set org.gnome.settings-daemon.plugins.cursor active false

[dcommander]

Sorry-- posted before I finished writing. I need to do some more investigation to figure out what changed in TigerVNC. Stand by. Meanwhile, I have also confirmed that everything works properly and performs well when using VirtualGL and running TurboVNC with the -3dwm switch, which is the supported method of running 3D window managers in TurboVNC. The other supported method of working around this is to use MATE instead of GNOME, but whereas RHEL provides that via the EPEL repository, SuSE doesn't appear to do so.

[dcommander]

I've managed to build v1.3.0, v1.3.1, and v1.4.3 of the TigerVNC Server from source on SLES 12 (no mean feat, given that v1.3.x doesn't support the version of X.org that SLES 12 uses.) Unfortunately I can't reproduce any errors with any of those builds. My next step is to find the specific TigerVNC 1.3.0 RPM that was shipped by SuSE, then I can examine the spec file for it to see how it was built. It seems that this might have something to do with build details rather than anything in the code. I'll keep you posted.

[dcommander]

Unfortunately, the TigerVNC 1.3.0 RPMs from the original SLES 12 release do not reproduce any of the issues for me. They work fine. I need further information from you regarding how to reproduce this issue using those RPMs. Were you attempting to run TigerVNC 1.3.0 on your SLES 12 SP1 install, or were you running it on an original SLES 12 install without the service pack? What was the configuration of the machine? It just seems like there wasn't anything in the VNC code itself that fixed the issue but rather that it was fixed somehow by upgrading the dependencies, but I need to reproduce it outside of TurboVNC before I can determine what's going on.

[dev-gmp]

I'm able to reproduce this with TigerVNC on a SLES 12(without the Service Pack) VirtualBox VM . The exact version info from zypper is Version: 1.3.0-17.3. No updates applied. The issue does not appear on a SLES 12 box with SP1 and latest updates applied. The version on the working box is Version: 1.4.3-14.1.

[dev-gmp]

I am able to reproduce the issue even when using the -3dwm option with TurboVNC, gnome-terminal crashes when clicking on any of the menu items.

[dcommander]

I'm still fuzzy as to how you are running GNOME 3 without -3dwm. Can you provide more information on your use of llvmpipe with TurboVNC? Are you following the procedure at http://www.turbovnc.org/Documentation/Mesa?

[dev-gmp]

I'm compiling llvmpipe using the official documentation at http://mesa3d.org/llvmpipe.html. Once the build is done, setting LD_LIBRARY_PATH to the generated libraries and then starting gnome worked for me.

glxinfo within TVNC shows that it is using llvmpipe MESA. Note that, I had to use version 10 of the mesa libs, specifically 10.0.5.

[dcommander]

I can't make that work. For starters, there is no "linux-llvm" target as described in the Mesa documentation. I can build llvmpipe by doing:

sudo wget ftp://ftp.freedesktop.org/pub/mesa/older-versions/10.x/10.0.5/MesaLib-10.0.5.tar.gz
sudo tar xvfz MesaLib-10.0.5.tar.gz
cd Mesa-10.0.5
sudo mkdir linux64
cd linux64
sudo ../configure --prefix=$HOME/mesa/10.0.5.llvmpipe
sudo make install

But then...

/opt/TurboVNC/bin/vncserver -noxstartup
DISPLAY=:1 LD_LIBRARY_PATH=$HOME/mesa/10.0.5.llvmpipe/lib glxgears

Error: couldn't get an RGB, Double-buffered visual

Doesn't work. What am I missing?

[dev-gmp]

I used scons to build Mesa

scons build=debug libgl-xlib

Once built, you'll have .so files under ..../gallium/targets/libgl-xlib

[dcommander]

Ah... OK. That makes sense now. That is essentially the same procedure that I'm using, except that I'm using autotools instead of scons. For some reason, the autotools build installs both libGL.so.1.5.0 and libGL.so.1.6.0, and only the 1.5.0 library has llvmpipe, but it symlinks libGL.so.1 to libGL.so.1.6.0. When I manually change the symlink to point to the 1.5.0 library, llvmpipe works.

[dcommander]

I spent most of the week fixing a serious issue in our RANDR implementation that affected GNOME 3, as well as tracking down other unrelated GNOME 3 issues, such as #47 and #46. I wanted to make sure that was all sorted before I started tackling this. Now that I can make the window manager behave properly, I am able to obtain more meaningful data regarding this bug. Unfortunately, this appears to be due to an incompatibility between GNOME 3 and X.org 7.7. I can confirm your observations: the older TigerVNC 1.3.0 RPM from SLES 12 SP0 exhibits the crash, but the newer TigerVNC 1.4.3 RPM from SLES 12 SP1 doesn't. However, this appears to be because the older RPM was built against an older X server code base (xorg-xserver 1.13, which presumably was the X server used in SLES 12 SP0.) The newer RPM was built against xorg-xserver 1.15, the same version currently used in SLES 12 SP1. If I install the cross-compatible TigerVNC 1.4.3 RPM from the TigerVNC Project, which is built against the same version of the X server as TurboVNC (X.org 7.7), it exhibits the crash as well.

Carefully examining the differences in the VNC logs and xdpyinfo output, I identified several areas of interest:

• The failing X servers (TurboVNC and cross-compatible TigerVNC) produced a warning in their logs: "IDLETIME counter not found". This was due to the lack of a device-specific system idle timer in xorg-xserver 1.12. I spent several hours adding the relevant Git patches from xorg-xserver 1.15 to include this functionality, but that did not fix the issue.
• The failing X servers (TurboVNC and cross-compatible TigerVNC) produced a warning in their logs: "Window manager warning: CurrentTime used to choose focus window; focus window may not be correct." All of the information I could find on this seemed to indicate that it was innocuous.
• The failing X servers don't enable backing store by default, but enabling it made no difference.
• The failing X servers don't provide the Present extension, but disabling it in the working X server (TigerVNC 1.4.3/xorg-xserver 1.15) made no difference.

I finally stumbled upon this bug report, which appears to be the same issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1292965

Sure enough, the bug is reproducible with RHEL 7 (gtk3 = 3.14.x) and Fedora 22 (gtk3 = 3.16.x) but not with Fedora 24 (gtk3 = 3.20.x), which is consistent with the above bug report (apparently that bug was fixed in gtk3 3.19.6. I don't think there's anything that can be done about this other than to use MATE (https://en.opensuse.org/Portal:MATE). I recommend that anyhow-- there are numerous issues with GNOME 3 related to the fact that it's slowly but surely eliminating support for X11, so I think the compatibility story with that WM will get worse, not better. The other option would be to contact Novell and ask whether the fix in gtk3 3.19.6 could be backported to the version of GTK they use in their distro.

[dcommander]

The more painful option would be to do another X server overhaul with TurboVNC, to bring it up to xorg-xserver 1.15. I'm willing to do that, but it would be an expensive project. Closing as SEP for now. Please feel free to contact me offline if you want to discuss more details of long-term strategies regarding how to deal with this.

[dcommander]

Verified that the issue also exists when running the WM using VirtualGL, not just with Mesa.

[dcommander]

Adding a "revisit" tag as a reminder to retest under CentOS and SLE if/when gnome-terminal 3.19.6 or later makes it into those distros.

[dcommander]

This issue went away with the migration to Xorg 1.19.x in TurboVNC 2.2. It still exists with TurboVNC 2.1.x when running in SLES 12 SP1, but SLES 12 SP3 now includes gnome-terminal 3.20.x. The newer WM in SLES 12 SP3 experiences a number of problems with the older TurboVNC 2.1.x X server, but this bug isn't among them. Tagging as fixed.