Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove CGo on Windows #911

Open
gonutz opened this issue Apr 28, 2020 · 27 comments
Open

Remove CGo on Windows #911

gonutz opened this issue Apr 28, 2020 · 27 comments
Labels
enhancement New feature or request OS:Windows Tickets affecting only Microsoft Windows

Comments

@gonutz
Copy link

gonutz commented Apr 28, 2020

Installing a C compiler that works with Go on Windows means installing MinGW or Cygwin, both of which are no fun at all. I would like to be able to build a fyne application on Windows without dependencies. Another issue on Windows are missing OpenGL drivers. Windows works with DirectX natively. Yet another issue is cross-compiling for Windows from another OS which means installing the C compiler that produces Windows binaries there.

As long as fyne uses OpenGL on Windows it is complicated to replace CGo. I have not yet seen an OpenGL wrapper for Go that does not use CGo so you would have to create one. A C-less OpenGL still has other problems on Windows. Windows natively only supports GL 1.1 so you depend on a newer graphics driver to be installed on the user's computer. See issue #410 where OpenGL drivers are hard to install on a virtual machine.

I suggest creating a driver for Windows that replaces GLFW and OpenGL with the native WinAPI and Direct3D9. DirectX version 9 has been shipping with Windows XP and is still supported to this day, meaning it supports the widest range of Windows computers currently out there. This solves both the CGo and the driver problem.

I have been creating Windows desktop programs in Go for some years now. I have a WinAPI wrapper and a Direct3D9 wrapper in pure Go. I use these for example in my Windows GUI library 'wui' and in my game framework 'prototype'.

With these libraries it is possible to compile Windows binaries from pure Go and that do not depend on any extra DLLs at runtime. If you build a 32 bit binary, it can be run on any Windows OS >= XP. Cross compiling is supported by the Go toolchain so you just have to set GOOS=windows and can compile.

I would be very happy to contribute. Looking at the code I see that glfw and go-gl are only used in the driver/glfw and painter/gl. I will soon look to see what is necessary to replace these. I wanted to start the issue to see if this is wanted at all and what others have to say first.

@andydotxyz
Copy link
Member

That is a very brave endeavour! You're right that it's a pain to develop on Windows with a C compiler but we didn't have much choice given the time available and the number of platforms we wanted to support.
One of the challenges of adding a new driver is that it will need to continue to be maintained and improved as the toolkit progresses. If you are interested in doing so then it sounds like a great improvement.

You would need to start a new driver (like glfw) that handles window management and other things (file handling etc) - as required by the fyne.Driver interface. Then the painting would be done by another package, like the gl one you saw. It's job would be to render all the primitives declared in the canvas package.
That's really it, once done update app/app_windows.go to use the correct driver and you're away :)

Thanks for your inteterest and best of luck. It looks like you already have this in mind but, outside of a working graphics driver, we have 0 runtime dependencies and wish to keep it that way.

@andydotxyz andydotxyz added enhancement New feature or request OS:Windows Tickets affecting only Microsoft Windows labels Apr 28, 2020
@Bluebugs
Copy link
Contributor

I would like to point out that adding a new painter and maintaining it on top of a new driver, is going to be a lot of code and that the maintenance code for it will be hard as it will only work on one platform. I started my work on porting Fyne to Wasm with just a custom driver and minimal change on the painter, and it was already hard to keep up with the current speed of development. There could be improvement in the painter and driver to abstract them more and reduce the code of a potential new driver/painter couple, but it will be always difficult to maintain. I will really advise you to think about that problematic and see if there is way around.

A potential direction of thinking would be to keep a GL backend and use a wrapper library that map GL->DirectX like the one Google use for Chrome which as a license compatible with Fyne: https://github.com/google/angle . Another possibility to reduce the need for a C compiler would maybe be by providing pre compiled element as Windows as less variety than Linux. Anyway just some element for thinking here.

@windexlight
Copy link

Just to throw these out there... Gio and Ebiten both contain examples of using GL in Go with Syscall instead of CGo (at least on Windows). I know that Gio at some point was using Angle. A quick glance is insufficient to inform me if they still are or not. Ebiten has forked go-gl to remove CGo on Windows, and the author has an open pull request to merge that work into go-gl, but that seems to be stalled.
Gio: https://git.sr.ht/~eliasnaur/gio
Ebiten: https://github.com/hajimehoshi/ebiten

@gonutz
Copy link
Author

gonutz commented May 3, 2020

Getting CGo out of the existing dependencies, that would be a good way to go. Keeping one code path instead of creating a new one sounds more promising. In that case the only real reason left for using DirectX would be that people would not need an up to date graphics driver but I think this is not a problem these days.

Once I get around to it, I will look more into how much CGo is left in what Go wrapper of OpenGL. I could not quickly find it in ebiten or go-gl. The gio page is highly confusing, I never understood why he uses sourcehut for it. If someone knows where to find these C-less libraries, please add a comment here.

@windexlight
Copy link

Ebiten's GL stuff is here, and the GLFW stuff is here. The author seems to be using an interesting technique of building the GLFW DLL with //go:generate, and embedding the binaries as byte slices. The idea is probably to embed the binaries in the library for each platform (or at least Windows), so users don't need a C compiler to build them.

There is a pull request here related to removing CGo dependency from go-gl on Windows. The author links to a design document describing the approach, which also points here for a discussion of why Syscall isn't feasible outside Windows. That, in turn, leads to this, which looks quite experimental, but could be a starting point for eliminating CGo outside Windows as well.

Gio's relevant bits are centralized here. They're again using Syscall (or x/sys/windows, anyway) on Windows for calling graphics APIs, but CGo on other platforms. They look like they've done a lot of work rolling their own window creation, context creation, and input handling, rather than using GLFW/SDL/etc.

@MatejMagat305
Copy link
Contributor

MatejMagat305 commented Aug 17, 2021

Yes it would be nice remove CGO at least on Windows, I need to say that this library is more clear like gio and I like that gomobile is native (in my opinion it is better like gio binding with java and objective c), but install gcc( 64 bit on windows) was main reason why I have writen my bachalor in https://github.com/gonutz/prototype, whose autor open this issues..., but I know, I know it need time and people...
edited: I'm sorry to claim that linux can be without cgo, it can be only on windows

@MatejMagat305
Copy link
Contributor

(maybe next spam from me, but) according web: https://stackoverflow.com/questions/52297027/go-syscall-v-s-c-system-call it would be even faster than now

@andydotxyz
Copy link
Member

It would indeed be good to do this, but to remove completely I think you'd need to replace the go-gl and go-glfw libraries, which essentially means writing a new driver. At this time we don't have the available resources to do so.

@MatejMagat305
Copy link
Contributor

It would indeed be good to do this, but to remove completely I think you'd need to replace the go-gl and go-glfw libraries, which essentially means writing a new driver. At this time we don't have the available resources to do so.

I know, that is only sound far futures....

@MatejMagat305
Copy link
Contributor

MatejMagat305 commented Sep 14, 2021

I am only asking:
I was found this derivation of go-gl(without cgo dependency on window) :
https://github.com/neclepsio/gl
If I try to replace go-gl in vendor, I will need to replace only gl, or I will must replace glfw too (by inspired: https://github.com/hajimehoshi/ebiten/tree/main/internal/glfw) ? And is it theoretically possible or you was change it in this projekt?

@andydotxyz
Copy link
Member

You would need to replace GLFW as well as the GL dependency.
I guess it is theoretically possible, and probably easier than writing a windows specific driver using DirectX :).

@changkun
Copy link
Member

I would love to see this landing. Apparently, a huge amount of work is required here. We should probably do some cleanup to merge all drivers into a single GL(ES) driver first, then remove glfw, and afterward adding more drivers such as DX12, Metal etc. Given the compatibility promise, the procesure probably needs not only years (I assume)

@MatejMagat305
Copy link
Contributor

I would love to see this landing. Apparently, a huge amount of work is required here. We should probably do some cleanup to merge all drivers into a single GL(ES) driver first, then remove glfw, and afterward adding more drivers such as DX12, Metal etc. Given the compatibility promise, the procesure probably needs not only years (I assume)

It look like gl I can replace without paint(all test pass) and it only about ctrl + shift +R(with idle) and glfw it will harder..............., I do not know schould I create pull request of this small step?

@andydotxyz
Copy link
Member

We should probably do some cleanup to merge all drivers into a single GL(ES) driver first,

Hmm, maybe, but beware that we split driver and painter because they serve different, but overlapping purposes.
For example GLFW and mobile drivers are different (for obvious reasons) but they both use the GL painter.
Then consider that the GLFW driver could launch a GL or a Vulkan surface...

As a refresher we have the following drivers:

  • glfw, mobile, test
    And the following painters:
  • OpenGL(Core/ES/mobile), software

The test driver is not an interactive driver like the others, but it does serve a real purpose as a driver implementation.

@andydotxyz
Copy link
Member

I'm sorry @MatejMagat305 I can't understand "without paint(all test pass) and it only about ctrl + shift +R(with idle)" - can you help explain what you meant?

I do not know schould I create pull request of this small step?

Possibly open a draft PR so that people can test and comment? I don't think that it is a complete item but it a PR can be opened and then added to :)

@changkun
Copy link
Member

Ah. I didn't write that sentence clear. So on desktop, we depend on go-gl for gl and gles (e.g. internal/painter/gl/gl_es.go). On mobile, we have a gomobile gles, which is a duplicate of go-gl. What I meant was these drivers can be merged as a single driver.

@MatejMagat305
Copy link
Contributor

MatejMagat305 commented Sep 15, 2021

I'm sorry @MatejMagat305 I can't understand "without paint(all test pass) and it only about ctrl + shift +R(with idle)" - can you help explain what you meant?

I do not know schould I create pull request of this small step?

Possibly open a draft PR so that people can test and comment? I don't think that it is a complete item but it a PR can be opened and then added to :)

I mean that:
I found all acour of "github.com/go-gl/gl..."
and I replace with: "github.com/neclepsio/gl........"
with jetbrains funktion............
and run draw_test.go and gl_test.go

@andydotxyz
Copy link
Member

I see, if there is a way to stop using both of those GLES drivers then that would be good yes.
It is worth noting that the go-mobile GL binding seems to have some efficiencies (mainly batch-processing for CGo) that it may be a performance loss to remove.

@andydotxyz
Copy link
Member

I was found this derivation of go-gl(without cgo dependency on window) :
https://github.com/neclepsio/gl

My only concern is that this repository is not used by many people, and it is not clear whether it is actively maintained or not.
I don't think it is being kept up to date with upstream go-gl where, generally, they fix bugs that are reported.

@changkun
Copy link
Member

It is worth noting that the go-mobile GL binding seems to have some efficiencies (mainly batch-processing for CGo) that it may be a performance loss to remove.

That's fantastic, yes :) The cgo batching in go-mobile is an excellent one. It would also be a lover if this is also landed on desktop, too.

@andydotxyz
Copy link
Member

and run draw_test.go and gl_test.go

If it has worked smoothly you should be able to run the packaged fyne_demo app (in cmd/fyne_demo) and see it work perfectly as well.

@MatejMagat305
Copy link
Contributor

and run draw_test.go and gl_test.go

If it has worked smoothly you should be able to run the packaged fyne_demo app (in cmd/fyne_demo) and see it work perfectly as well.

it works, but I add 2MB to binary....., so I give up to you like profesional..........

@changkun
Copy link
Member

I think it would be ideal to extract the essential GL calls that are used in fyne, then provide them as an internal/driver/gl for other code to use, in this way, we can fully remove the go-gl stuff.

The go-gl is really necessary, it was generated by a generator, and contains too many unused code, and adds too many bytes to the repository. See #2437 as an example of tidy unused GL calls from the mobile driver.

@MatejMagat305
Copy link
Contributor

MatejMagat305 commented May 16, 2022

I know it is litle bit silly notice, but maybe it could be usefull for far future ..., (it is only for interasting) the library of ebiten found way how remove cgo from macOs(they create own dlopen): hajimehoshi/ebiten#2091, ebitengine/purego#1

@andydotxyz
Copy link
Member

That is really very cool - but we do use CGo to access other macOS APIs like notifications and menu interactions.
I did not read all of their discussion but it sounds like we might not be able to solve that in the same way?

@andydotxyz
Copy link
Member

Of course if we can / should then perhaps a new issue to track that would make sense?

@eihigh
Copy link

eihigh commented May 18, 2022

Ebiten tries to remove CGo on windows at this branch. https://github.com/hajimehoshi/ebiten/commits/issue-1764-windows-go-use

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request OS:Windows Tickets affecting only Microsoft Windows
Projects
None yet
Development

No branches or pull requests

7 participants