Use CMake to compile the profiler #655
Comments
|
Thanks for asking first. The current build solution is based on MSVC projects and makefiles that extract source file lists from those projects. This is a solution that works well for me. There have been numerous requests or even PRs to add a CMake build system that have never considered why the current system is in place, or what introducing a second build system would entail. The obvious consequence is the existence of two build systems that require maintenance, and that's not something I want, especially considering that I don't like CMake. Some of the said PRs have even removed my current build solution, implying that it is inferior, which I find insulting. In fact, some of the requests have been downright abusive, as if I owe something to some random guy on the Internet. However, since MSVC and Windows as a platform have been deteriorating for some time now, I am considering migrating the build system to use Meson. This is not so easy, as the tools can be a bit lacking in some areas, which I am currently trying to improve, but it takes time and requires the cooperation of various maintainers, some of whom are unresponsive. I do not think that solving the library availability problem by downloading and compiling the source during the build process is the right solution. This is a problem where the solution was found by Linux distributions a long time ago, and there are consequences, sometimes affecting the security of the system, if you do things in a different way. The real problem here is Debian's insistence on shipping very outdated software packages. (It's always Debian.) |
|
Ahahah I agree, Debian can be a little tedious. Although, i find most projects that rely on system installed dependencies tend to have the same failures. Not all package managers are built equally I have been known to be quite vocal about CMake. I think it comes from use. When you get used to something like CMake or Meson and then run into the platform specific build systems like make or MSVC it seems almost archaic. Not to defend others approaches to the conversation, just understanding where their high horse might have come from 😆 Granted, there are pros and cons to both but I completely understand your point of view when it comes to staying with what works Meson vs CMake is more a matter of market share. Jetbrains cpp ecosystem study found that the majority use CMake. From my experience of throwing third party libraries into projects you tend to see more often CMake or Make. Meson tends to be preferred by project similar to gstreamer or Google projects. Meson has support to import CMake where CMake does not have support to import Meson. This hints that Tracy would have a greater chance of adoption on projects if it tended towards CMake as Meson users can always just import it All that said, it is your project and therefore your choice :) Where does this leave us?
Are you happy for me to submit two PRs. One with a Meson approach and one with a CMake approach. Both will not be complete to start but will provide a starting point to review which system is preferred Also thanks for the prompt reply |
In my experience, you trade one set of annoyances for another between the two approaches.
Currently, Tracy ships with client-side build files for both CMake and Meson, so this should not be a problem. The issue at hand is how the server utilities are built.
There's no need. As I said, the tooling needs to be improved before anything can be done. For example, I need support for selecting a meson.build file that is appropriate for the utility I want to work with. The "one build file to rule them all" approach is a bit naive and not sufficient for what Tracy is doing. For example, the GUI server compiles a source file with a flag to enable calculation of statistics that can be presented to the user, while the CLI capture utility does not need to do these calculations. This results in two separate #ifdefed code paths, and you need to have an appropriate set of defines in compile_commands.json depending on which utility you are working on. Switching between two meson.builds is the only way I can see to do this. Adding support for this is on track in mesonbuild/vscode-meson#179. The same cannot be said for extending the clangd vscode extension to allow selection of the compile_commands.json location, which is stuck in clangd/vscode-clangd#544. I don't really need this to proceed, as I can just override the clangd binary parameters, but that would be an inferior solution, potentially overriding the user's custom configuration. Unfortunately, there is only so much you can do. Then there's the need to provide Windows support. While meson can use cl.exe or even output MSVC project files, I don't think it can discover the dependencies provided by vcpkg. This would also need to be addressed somehow. Vcpkg packages provide pkgconfig files, so adding support for that doesn't sound unreasonable. Once all this is in place, switching to meson should be a breeze. |
|
Ah this seems to be mostly meson issues? I have never had issues with vscode using compile commands, I use the Microsoft c++ extension and CodeLLDB for debugging works smooth on windows, macOS and Linux. Less so on M1 😅 Vcpkg and pkg-config have cmake support so are fairly easy to add in. Vcpkg does this using a toolchain file. Although I'm not the biggest fan of C++ package managers, things tend to break easily. I'm not quite sure what you mean by "one build file to rule them all". Having a single CMakeLists tends to scale horribly. Most of the time I split it into one per target. I've not had issues with building multiple executables with CMakes target system before. I'll make sure I dig around to understand the define issues you discussed. Could you provide a link to these or a file name please? |
|
Grep for |
|
Sorry for the spam today I just wanted to make sure that I approached this conversation properly and not be one of those CMake guys Last question, you have a lot of third party code in the root directory which is copied into the project. Are these third parties heavily edited for Tracy? Is this why they are copied instead of using submodules or pulling with fetch content / meson fetching alternatives? |
|
There have been significant changes in some of these libraries, yes. |
|
Okay, so the Meson solution did not work out in the end. You can see it at https://github.com/wolfpld/tracy/tree/meson, and while it builds fine on Linux, getting it to work on any other system is an exercise in futility, because Meson is broken by design. I made an attempt to set up CMake, and while it looks terrible, it can at least be made to work. This can be seen at https://github.com/wolfpld/tracy/tree/cmake. Currently, there's a CMake config for I can build, run and debug the profiler on Linux, all from VS Code, which is great. I tried to build on Windows and it presented me with a list of toolchains, including MSVC build tools, which is also great. It failed to find the currently hardcoded pkg-config dependencies, which on Windows are provided by vcpkg. This should be supported, but I guess it requires knowledge of how to do things with CMake, which I don't have, and the quality of the documentation available doesn't help here. I would appreciate it if you would take a look at it. There are a few things that need to be done if this is to be merged, namely: support for Windows, Mac and Emscripten, and migrating to autogeneration of Wayland headers + sources (I have it working on the Meson branch). Adding CMake build files for the rest of the utilities can be done later. |
|
Hey @wolfpld, Its great to hear you gave this a shot! I'm open to taking a look at the CMake approach. I had a branch working cross-platform with CMake. The issue with my branch is that I had to reorder some of the folder structure and how the libraries where compiled for the project to work seamlessly - there where issues with headers and linking which are inherent from the current build system. Ill take a look at your PR when I get chance and possibly review my own to see how far off i was from having feature parity |
|
I think i also got it working without vcpkg using CPM |
|
On Windows, binaries are linked statically, so it doesn't really matter how the third-party libraries are built, and vcpkg can be dropped. On Linux, I would expect to link with the system libraries. |
|
They are such small binaries and dependencies aren't mega to compile. On all platforms you could link statically and distribute standalone executables. That would likely be the best option. Linux package managers can have broken dependencies. For instance Ubuntu 20's capstone 4 package is actually capstone 3.0.5 so you can't build Tracy on that version anymore as there are missing header definitions. Bundling the dependencies is better in Tracys case - or at least that was my conclusion after bundling them as it then build on all platforms. Its close to Christmas so my time is more limited but I will try to rebase and update my PR and also take a look at your CMake approach 😄 |
|
Yes, sooner or later you will run into some dumb decision made by someone, be it Debian/Ubuntu providing horrendously outdated packages, or Gnome not supporting server-side decorations in Wayland, or Meson not supporting env variables, or whatever. There's nothing you can do about that, as you can see in the issue trackers for those projects. I'm not interested in making concessions to provide workarounds or support for these cases. In the end, it's people making decisions, and why should I help them if I don't agree with what they're doing? I don't see a problem with downloading and building libs from source as an option, as that may have legitimate uses, but the default should be to link with the libs installed on the system. |
|
Some more potential problems with capstone: #671. |
|
On my branch i just pin it to a version and build from source. Its a very quick build and capstone has pretty flaky support in different distros
|
|
Yes, it's a headache that should probably be avoided by simply compiling from source. It's also the only library that I have to manually compile in order to have it available on emscripten. |
|
Yeah i ran into those emscripten issues too. Are there any other issues with that which you've ran into in the past and would expect to run into again if the project moved to full CMake?? |
|
Nothing comes to mind. |
|
Hey @wolfpld happy new year! I spent a little bit of time productionising my CMake approach today. See #707 This is a WIP. The deleted changes looks scary but I think that is the MSVC project files. In comparison to your above branches this PR:
Pros:
Cons:
I will try do some fresh builds now and look into CI support. Thats fairly easy with CMake targets It would be great to hear your thoughts on this approach! |
|
Vcpkg should only be used on windows, as a way to provide system libraries that you would have installed on unix. |
This falls apart with some linux distros. From my understanding vcpkg falls back to system dependencies if they are present so this approach should support those that have the packages installed and those where the package manager does not support the packages I am pushing closer to the automatic approach as most people that I have tried to persuade to adopt Tracy have failed due to their Linux distro not supporting the correct packages or broken packages which Tracy requires |
|
Will move this discussion to the PR as it is related to the PRs proposed approach |
Can you clarify what you mean by this? Meson does have nontrivial usage on Windows, so I'm guessing the apparent breakage is, at least, workflow-dependent.
Note that meson supports many environment variables, but it serializes them at startup into an internal toolchain description and preserves this during reconfiguration. |
|
|
https://mesonbuild.com/Release-notes-for-1-3-0.html#global_source_root-and-dirname-in-machine-files Current versions of meson do allow the toolchain file to refer to the current directory (or to the project source root), which means you can stick the toolchain file into the same directory as vcpkg -- or have vcpkg generate one automatically, which could be worth a vcpkg feature request. |
|
In the case of vcpkg, it's a bit more complicated, as you can have multiple triplets, e.g. This solution still requires the user to manually specify a path to the machine file from the command line, which unnecessarily complicates setup for users who expect to just open the project checkout in an IDE and have everything work as long as they have the proper paths set in environment variables. In some cases (e.g. corporate environments with strict security policies), the user may not be able to manually inject a machine file into the SDK, which is a requirement for the current directory relative paths to work. Even if this were not a problem, requiring users to manually inject the machine file every time they update some SDKs (for example, the Vulkan SDK has versioned paths, like |
|
Meson does have native support for I agree that that isn't a general solution. ;) I'm not very familiar with vcpkg but I thought it used the same basic approach as the meson toolchain suggestion... it produces a cmake toolchain file and you specify the path to that file via |
It's a bit sad that these "ugly hacks" are in use when the reality hits, but the general solution is still not available to users.
I'm not able to comment on the interaction between CMake and vcpkg. For the IDE use case, I would like to be able to specify in the meson build file that Ideally, Meson would be able to pick up vcpkg as a dependency provider on its own, but you need to solve the vcpkg checkout path problem and the triplet selection problem. As things stand, you have to pass a machine file specifying where the vcpkg pkg-config executable is located as a parameter to the The lack of support for env variables, even in machine files, requires an additional step of running some sort of shell script to sed the correct path into the file. But in doing so, we are creating a generator for the build system generator, which is a strange direction to go in. |
|
Done. |
Hi @wolfpld,
I've been using Tracy for a while on personal projects. Lately, I've been attempting to add Tracy as a project profiler for Vulkan Samples here's the PR
As there are no redistributables for a standalone Linux version of the profiler I have added a script
./scripts/tracy.pywhich attempts to build Tracy (or download it if on Windows). This is a little cumbersome on Linux due to Linux things. For instance, on Ubuntu 20.04 libcapstone-dev installs 4.0.1+realy+3.0.5 which is very annoying as Tracy relies on 4+. You have highlighted some of the build nuances in your documentation which is greatTo cut a long story short. I would like to be able to use CMake to build the profiler to make it easier for devs to debug in a cross-platform manner without the need to dig through platform-specific build configs. Alternatively, it might be useful to create a deb or rpm distributable using CPACK. We can use things such as
find_libraryto make sure system dependencies are installed or even add them via CPM so that they are downloaded and built-in for the userI am happy to work on a PR here and prove the concept out. It would be great to get your opinion beforehand. Maybe you have already attempted this in the past? Or you have a specific reason for not using CMake for the profiler?
The text was updated successfully, but these errors were encountered: