Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3.0 Roadmap #5527

Open
3 of 10 tasks
waruqi opened this issue Aug 28, 2024 · 89 comments
Open
3 of 10 tasks

v3.0 Roadmap #5527

waruqi opened this issue Aug 28, 2024 · 89 comments

Comments

@waruqi
Copy link
Member

waruqi commented Aug 28, 2024

If there are proposed changes that you would like to see introduced in xmake v3.0, you can submit them here.

It will be adopted if the vote result >= 80%.

we can use set_policy("compatibility.version", "3.0") or xmake f --policies=compatibility.version:3.0 to attempt v3.0

Roadmap (Adopted)

Roadmap (Draft)

@waruqi waruqi added this to the v3.0.0 milestone Aug 28, 2024
@waruqi waruqi pinned this issue Aug 28, 2024
@24bit-xjkp
Copy link
Contributor

Support dynamical script load, so we can distribute some xmake scripts by packages. This would be useful when sharing some scripts between projects.

@waruqi
Copy link
Member Author

waruqi commented Aug 28, 2024

Support dynamical script load, so we can distribute some xmake scripts by packages. This would be useful when sharing some scripts between projects.

This key point is difficult to be implemented because the package is not yet installed when it is first compiled. We can currently only distribute packages rules.

@SirLynix
Copy link
Member

A better way for xmake packages to pull off info from their build xmake.lua, to avoid having to duplicate defines and such

@heheda123123
Copy link
Contributor

I think it is very important to improve the documentation, now the documentation of a few more serious problems
1 Even if I select Chinese in xmake.io, the search still comes up with English results
2 Some methods are not found in the documentation

@KkemChen
Copy link
Contributor

支持对apt,pkg-config等系统库指定使用动态库或静态库

@jingkaimori
Copy link
Contributor

jingkaimori commented Aug 28, 2024

elements of target

Re-consider relationship of rules, toolchains, config variables, values and policies.

Table below shows which entity can hold which element:

entity type rule toolchain config variable value policies
option v
target v v v v v
(global space) v v v v
package
xpack

Option

should option be a type of task, or just a object like value? should option checks itself?

@waruqi
Copy link
Member Author

waruqi commented Aug 28, 2024

I think it is very important to improve the documentation, now the documentation of a few more serious problems 1 Even if I select Chinese in xmake.io, the search still comes up with English results 2 Some methods are not found in the documentation

new document, https://docs.xmake.io/en-us/index.html

But I don't have a lot of time alone to update and perfect it.

@24bit-xjkp
Copy link
Contributor

Maybe we can use document generators like LDoc and embed the document into annotations.

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Supports specifying the use of dynamic libraries or static libraries for system libraries such as apt and pkg-config.

@waruqi
Copy link
Member Author

waruqi commented Aug 28, 2024

Remove support for built-in variables in script scope, support it only in the description scope, Poll: #5529

@qudix
Copy link
Contributor

qudix commented Aug 28, 2024

A way to reuse rules in a project that uses xmake, but in a package definition without duplicating work.
https://github.com/shad0wshayd3-FO4/CommonLibF4/blob/master/CommonLibF4/xmake.lua#L133

If such a thing requires some modifications on the lib side, that's fine.

@WubiCookie
Copy link
Contributor

Rename buildir to builddir

@waruqi
Copy link
Member Author

waruqi commented Aug 29, 2024

Rename buildir to builddir

Poll: #5533

@A2va
Copy link
Contributor

A2va commented Aug 29, 2024

For my project xmake-rs, I wanted xmake to send the resolved list of a target's links to the rust part. This way the user wouldn't need to link anything in the build.rs.
To do this, I had to copy a large part of the xmake source code, my idea would be to add a target:orderlinks() function which would return a resolved list of links on a binary target.
I thought about using target:linkcmd(), but the flags are different for different compilers and you'd have to parse them.

@star-hengxing
Copy link
Contributor

Use ninja generator for cmake package by default.

@waruqi
Copy link
Member Author

waruqi commented Aug 30, 2024

Use ninja generator for cmake package by default.

Poll: #5545

@jube
Copy link

jube commented Aug 30, 2024

Add a way to really clean everything. Sometimes, I need to delete .xmake directories (and/or build directory) in order to fix a build that should work. A nice command like xmake mrproper would be nice to delete everything, as if it was a fresh installation of xmake (like on a CI).

@waruqi
Copy link
Member Author

waruqi commented Aug 30, 2024

Add a way to really clean everything. Sometimes, I need to delete .xmake directories (and/or build directory) in order to fix a build that should work. A nice command like xmake mrproper would be nice to delete everything, as if it was a fresh installation of xmake (like on a CI).

xmake f -c will ignore all configuration cache.

@SirLynix
Copy link
Member

A way to change config without having to respecify all config, maybe separate some configs such as mode from other configs. It's quite painful to work with a lot of config because of that.

@KkemChen
Copy link
Contributor

希望option域里面 能够支持 add_rules 接口

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


It is hoped that the add_rules interface can be supported in the option domain.

@xq114
Copy link
Contributor

xq114 commented Aug 31, 2024

希望定义option时可以直接指定default值,而不用这样再包一层 https://github.com/xmake-io/xmake-repo/blob/dev/packages/a/angle/port/6288/xmake.lua

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I hope that when defining options, I can directly specify the default value without having to wrap it up again https://github.com/xmake-io/xmake-repo/blob/dev/packages/a/angle/port/6288/xmake.lua

@waruqi
Copy link
Member Author

waruqi commented Sep 1, 2024

希望定义option时可以直接指定default值,而不用这样再包一层 https://github.com/xmake-io/xmake-repo/blob/dev/packages/a/angle/port/6288/xmake.lua

原本就可以直接指定,这里的问题是你用了 is_plat ,想要在 option 里用,只能在脚本域。这个是本身机制决定的,没法在描述域实现

@heheda123123
Copy link
Contributor

heheda123123 commented Sep 2, 2024

跑测试的时候,是否有必要把所有的config也测试到呢?
比如下面的pr,其实是测试不到新添加的config的
xmake-io/xmake-repo#5115

或者说写包描述的时候可以给config添加一个字段指定是否对这个config进行自动测试

@waruqi
Copy link
Member Author

waruqi commented Sep 2, 2024

跑测试的时候,是否有必要把所有的config也测试到呢? 比如下面的pr,其实是测试不到新添加的config的 xmake-io/xmake-repo#5115

或者说写包描述的时候可以给config添加一个字段指定是否对这个config进行自动测试

这个不太现实,新增一个config ,排列组合下,ci 的测试 job 是数量级暴增的,一个 config 如果不是 boolean 值,是 string values 的,组合就更多, 还可能引入各种 deps 编译,没这么多资源,不仅耗时,耗ci资源,跑出问题,也没这么多人力修。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


When running tests, is it necessary to test all configs? For example, in the PR below, the newly added config cannot be tested xmake-io/xmake-repo#5115

Or when writing a package description, you can add a field to the config to specify whether to automatically test this config.

This is not realistic. If you add a new config, the number of ci test jobs will increase by orders of magnitude under the permutation and combination. Without so many resources, it will not only be time-consuming, but also consume ci resources. If problems occur, there will not be so many manpower to repair them.

@SnowinterCat
Copy link

我需要一个接口来遍历我的项目依赖的三方库的列表,我需要将三方库的二进制打包到我的安装包中。
我现在的做法是在脚本域中使用以下代码来获取三方库的二进制的:

        import("core.project.project")
        cprint("${yellow}Export Shared libs")
        for pkg_name, info in pairs(project:required_packages()) do
            local libfiles = info:get("libfiles")
        end

但是这只能获取到三方库的库文件,对于一些拥有插件或者工具的三方库,例如gstreamer、ffmpeg、qt,我需要用到他们的工具和插件,并将其打包到安装包中。目前我没有办法做到直接从xmake的脚本域中找到它们
能否提供一个函数例如project:all_packages(),target:all_packages()来获取到他们的manifest信息?

又或者为package的包管理中添加一个on_export_runtime(),可以用于一键导出三方库的运行时?

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I need an interface to traverse the list of third-party libraries that my project depends on, and I need to package the binaries of the third-party libraries into my installation package.
My current approach is to use the following code in the script domain to obtain the binary of the third-party library:

        import("core.project.project")
        cprint("${yellow}Export Shared libs")
        for pkg_name, info in pairs(project:required_packages()) do
            local libfiles = info:get("libfiles")
        end

But this can only obtain the library files of third-party libraries. For some third-party libraries with plug-ins or tools, such as gstreamer, ffmpeg, and qt, I need to use their tools and plug-ins and package them into the installation package. Currently I have no way to find them directly from xmake's script domain
Can you provide a function such as project:all_packages(), target:all_packages() to obtain their manifest information?

Or add an on_export_runtime() to the package management of the package, which can be used to export the runtime of third-party libraries with one click?

@waruqi
Copy link
Member Author

waruqi commented Sep 25, 2024

我需要一个接口来遍历我的项目依赖的三方库的列表,我需要将三方库的二进制打包到我的安装包中。 我现在的做法是在脚本域中使用以下代码来获取三方库的二进制的:

        import("core.project.project")
        cprint("${yellow}Export Shared libs")
        for pkg_name, info in pairs(project:required_packages()) do
            local libfiles = info:get("libfiles")
        end

但是这只能获取到三方库的库文件,对于一些拥有插件或者工具的三方库,例如gstreamer、ffmpeg、qt,我需要用到他们的工具和插件,并将其打包到安装包中。目前我没有办法做到直接从xmake的脚本域中找到它们 能否提供一个函数例如project:all_packages(),target:all_packages()来获取到他们的manifest信息?

又或者为package的包管理中添加一个on_export_runtime(),可以用于一键导出三方库的运行时?

这种问题,没必要在这里问,可以到 discussion 里提,target:pkgs() ,pkg:installdir() 原本就能取到。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I need an interface to traverse the list of third-party libraries that my project depends on, and I need to package the binaries of the third-party libraries into my installation package. My current approach is to use the following code in the script domain to obtain the binary of the third-party library:

import("core.project.project")
cprint("${yellow}Export Shared libs")
for pkg_name, info in pairs(project:required_packages()) do
local libfiles = info:get("libfiles")
end

But this can only obtain the library files of third-party libraries. For some third-party libraries with plug-ins or tools, such as gstreamer, ffmpeg, and qt, I need to use their tools and plug-ins and package them into the installation package. Currently, I have no way to find them directly from the script domain of xmake. Can you provide a function such as project:all_packages(), target:all_packages() to get their manifest information?

Or add an on_export_runtime() to the package management of the package, which can be used to export the runtime of third-party libraries with one click?

There is no need to ask this kind of question here. You can mention it in the discussion. Target:pkgs() and pkg:installdir() can be obtained originally.

@Wzshun
Copy link

Wzshun commented Oct 10, 2024

A way to change config without having to respecify all config, maybe separate some configs such as mode from other configs. It's quite painful to work with a lot of config because of that.

I also want it, something like a file xmake.lua.user, and xmake build will check it if change, auto config with it.

@jingkaimori
Copy link
Contributor

c51sdcc工具链的默认plat不要设为当前系统,建议为裸机(没有系统的单片机,自立实现)添加名为bare或者standalone的目标平台plat属性值。c51工具链的默认架构arch建议也弄成mcs51,不然下面的日志太难绷:

> xmake config --clean
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33811
> xmake build --rebuild
[ 50%]: cache compiling.release src\main.c
[ 75%]: linking.release hello
[ 75%]: generating.release build\windows\x64\release\hello.hex
[100%]: build ok, spent 0.125s

这个无解,每个 target 可以绑定多个不同 toolchain 的,而全局平台 toolchain 只有一个,跟他们独立。

那xmake-repo里的包,如何适配c51或者sdcc工具链,同时不影响默认工具链的编译?

@waruqi
Copy link
Member Author

waruqi commented Oct 16, 2024

c51sdcc工具链的默认plat不要设为当前系统,建议为裸机(没有系统的单片机,自立实现)添加名为bare或者standalone的目标平台plat属性值。c51工具链的默认架构arch建议也弄成mcs51,不然下面的日志太难绷:

> xmake config --clean
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33811
> xmake build --rebuild
[ 50%]: cache compiling.release src\main.c
[ 75%]: linking.release hello
[ 75%]: generating.release build\windows\x64\release\hello.hex
[100%]: build ok, spent 0.125s

这个无解,每个 target 可以绑定多个不同 toolchain 的,而全局平台 toolchain 只有一个,跟他们独立。

那xmake-repo里的包,如何适配c51或者sdcc工具链,同时不影响默认工具链的编译?

add_requires("xxx", {configs = {toolchains = ""}}) 也是可以设置 toolchain 的

@SirLynix
Copy link
Member

SirLynix commented Oct 21, 2024

How would you feel about having

add_requires("xx", { configs = { foo = "bar" } })

automatically add

add_requireconfs("**.xx", { configs = { foo = "bar" }})

?

Also

add_requires("libsdl 2.30")

would add

add_requireconfs("**.libsdl", { version = "2.30" }})

This way, package config and versions would automatically replicate to their dependencies as well, because currently:

add_requires("libsdl 2.28", "libsdl_ttf")

will make xmake install this:

$ xmake f --check
checking for Microsoft Visual Studio (x64) version ... 2022
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> libsdl 2.28.5 [runtimes:"MD"]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

same goes for

add_requires("libsdl", { configs = {sdlmain = false}})
add_requires("libsdl_ttf")

=>

in xmake-repo:
  -> libsdl 2.30.8 [runtimes:"MD", sdlmain:n]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

it's been a long time I've been thinking about making a feature request for a policy for this, and I think it would be a great default to have that in xmake 3.0.

what do you think?

(also thinking about it, package.librarydeps.strict_compatibility would be great to enable by default in that case, to avoid subtles bugs when you change your mind about a package config also used as a dependency)

@waruqi
Copy link
Member Author

waruqi commented Oct 21, 2024

How would you feel about having add_requires("xx", { configs = { foo = "bar" } }) automatically add add_requireconfs("**.xx", { configs = { foo = "bar" }})? Also add_requires("libsdl 2.30") => add_requireconfs("**.libsdl", { version = "2.30" }}).

This way, package config and versions would automatically replicate to their dependencies as well, because currently:

add_requires("libsdl 2.28", "libsdl_ttf")

will make xmake install this:

$ xmake f --check
checking for Microsoft Visual Studio (x64) version ... 2022
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> libsdl 2.28.5 [runtimes:"MD"]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

same goes for

add_requires("libsdl", { configs = {sdlmain = false}})
add_requires("libsdl_ttf")

=>

in xmake-repo:
  -> libsdl 2.30.8 [runtimes:"MD", sdlmain:n]
  -> libsdl#1 2.30.8 [runtimes:"MD", from:libsdl_ttf]
  -> libsdl_ttf 2.22.0 [runtimes:"MD"]
please input: y (y/n/m)

it's been a long time I've been thinking about making a feature request for a policy for this, and I think it would be a great default to have that in xmake 3.0.

what do you think?

(also thinking about it, package.librarydeps.strict_compatibility would be great to enable by default in that case, to avoid subtles bugs when you change your mind about a package config also used as a dependency)

The add_requires at the user's top level are completely independent. They should not affect the dependency configurations of other packages, as this can lead to many hidden problems.

@SirLynix
Copy link
Member

SirLynix commented Oct 21, 2024

The add_requires at the user's top level are completely independent. They should not affect the dependency configurations of other packages, as this can lead to many hidden problems.

But in the examples I'm showing this is already causing many issues, if you decide to use libsdl as a shared library, libsdl_ttf will use it as a static library causing bugs (libsdl uses global variables which would not be the same between libsdl and libsdl_ttf in that case).

Another example would be openssl, if you use package locking for example and you're locking your openssl package version, it won't lock it as a dependency (and openssl is used by many packages).
Your app cannot have two differents versions of openssl and it will error out on init.

Having that behavior as an opt-out seems safer to me

@waruqi
Copy link
Member Author

waruqi commented Oct 21, 2024

The add_requires at the user's top level are completely independent. They should not affect the dependency configurations of other packages, as this can lead to many hidden problems.

But in the examples I'm showing this is already causing many issues, if you decide to use libsdl as a shared library, libsdl_ttf will use it as a static library causing bugs (libsdl uses global variables which would not be the same between libsdl and libsdl_ttf in that case).

so we should use add_requireconfs

Another example would be openssl, if you use package locking for example and you're locking your openssl package version, it won't lock it as a dependency (and openssl is used by many packages).

This is just another issue about requires-lock. We can improve requires-lock instead of add_requires.

Your app cannot have two differents versions of openssl and it will error out on init.

However, two completely independent targets can be bound to different versions of OpenSSL. If their dependency behaviors are globally modified through add_requires, these unrelated targets will all be affected.

Having that behavior as an opt-out seems safer to me

In the future, I will try to improve and solve this problem, but not by modifying the default behavior of add_requires. Modifying it would lead to even more problems than now.

@SirLynix
Copy link
Member

I understand even though I don't agree on the current behavior being safer than what I suggest (especially because it causes subtle bugs one can fix only with advanced xmake usage and knowing how libs depends on each other, my students fall in this trap every year).

(About package lock, the issue arise simply when you set a package version through add_requires, at best it duplicates symbols and at worst it causes weird bugs in your app)

How about having this behavior as an opt-in policy? Because even though we can already do this using add_requireconfs, it means duplicating every add_requires with a add_requireconfs which is redundant and more error-prone.

@waruqi
Copy link
Member Author

waruqi commented Oct 21, 2024

I understand even though I don't agree on the current behavior being safer than what I suggest (especially because it causes subtle bugs one can fix only with advanced xmake usage and knowing how libs depends on each other, my students fall in this trap every year).

(About package lock, the issue arise simply when you set a package version through add_requires, at best it duplicates symbols and at worst it causes weird bugs in your app)

How about having this behavior as an opt-in policy? Because even though we can already do this using add_requireconfs, it means duplicating every add_requires with a add_requireconfs which is redundant and more error-prone.

I haven't figured out how to improve and solve it yet. You can open an issue first.

@Bluelari
Copy link

A way to change config without having to respecify all config, maybe separate some configs such as mode from other configs. It's quite painful to work with a lot of config because of that.

这个+1,甚至希望xmake config的默认行为就是只修改而不要清除之前的config,类似于git config。并且如果真的需要覆盖已有config的话也可以先跑xmake config --clean再修改

@for2years
Copy link

for2years commented Nov 7, 2024

Assuming target A depends on target B, the build.fence policy will cause the object files of target A to wait until target B is fully linked before compiling, resulting in no-full-usage of the CPU.

On the other hand, build.target_across_parallel leads to even lower CPU utilization. Will xmake 3.0.0 consider optimizing CPU utilization for codegen/proto targets? We have a log of codegen / proto tasks in our repo, so we're quite interested in this feature.

@Dozingfiretruck
Copy link

#4878

Hopefully 3.0 will support menu like menuconfig

@A2va
Copy link
Contributor

A2va commented Nov 13, 2024

Hopefully 3.0 will support menu like menuconfig

It's already supported with xmake config --menu

@Dozingfiretruck
Copy link

Hopefully 3.0 will support menu like menuconfig

It's already supported with xmake config --menu

no , see this #4878

@waruqi
Copy link
Member Author

waruqi commented Nov 13, 2024

Hopefully 3.0 will support menu like menuconfig

It's already supported with xmake config --menu

no , see this #4878

You didn't explain it clearly. It's a dependency issue of option, not menuconfig.

@Arthapz
Copy link
Member

Arthapz commented Nov 16, 2024

as version 3 come with breaking changes, isn't it an opportunity to remove experimental support for modules (-fmodules-ts for gcc and clang and /experimental:modules for cl) and only support compilers with official support ? (-fmodules for gcc, -std=c++20 for clang and /std:c++latest or /std:c++20 for cl)
old compilers experimental support is very broken, nobody will use them seriously in a project

or at least disable them by default when compatibility policy for 3.x is set ?

@waruqi
Copy link
Member Author

waruqi commented Nov 17, 2024

as version 3 come with breaking changes, isn't it an opportunity to remove experimental support for modules (-fmodules-ts for gcc and clang and /experimental:modules for cl) and only support compilers with official support ? (-fmodules for gcc, -std=c++20 for clang and /std:c++latest or /std:c++20 for cl) old compilers experimental support is very broken, nobody will use them seriously in a project

or at least disable them by default when compatibility policy for 3.x is set ?

you can use compatibility.version >= 3.0 policy to remove it.

@ifarbod
Copy link
Contributor

ifarbod commented Nov 29, 2024

Opinions on this?

#5904

@SirLynix
Copy link
Member

Talking about renaming things, how about renaming iphoneos platform to ios and macosx to macos?

@waruqi
Copy link
Member Author

waruqi commented Nov 29, 2024

Opinions on this?

#5904

Poll: #5906

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests