From e590c723a9f684bc9f2099992c248536c6a369cb Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 9 Mar 2022 13:24:19 -0800 Subject: [PATCH 01/29] Copy over RFC text from https://github.com/Manishearth/namespacing-rfc/ --- text/0000-packages-as-optional-namespaces.md | 160 +++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 text/0000-packages-as-optional-namespaces.md diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md new file mode 100644 index 00000000000..11bacacec46 --- /dev/null +++ b/text/0000-packages-as-optional-namespaces.md @@ -0,0 +1,160 @@ +# Pre-RFC: Packages as Optional Namespaces + +## Summary + +Grant exclusive access to publishing crates `parent/foo` for owners of crate `parent`. + +Namespaced crates can be named in Rust code using underscores (e.g. `parent_foo`) + +## Motivation + +While Rust crates are practically unlimited in size, it is a common pattern for organizations to split their projects into many crates, especially if they expect users to only need a fraction of their crates. + +For example, [unic](https://crates.io/search?page=1&per_page=10&q=unic-), [tokio](https://crates.io/search?page=1&per_page=10&q=tokio-), [async-std](https://crates.io/search?page=1&per_page=10&q=async-), [rusoto](https://crates.io/search?q=rusoto) all do something like this, with lots of `projectname-foo` crates. At the moment, it is not necessarily true that a crate named `projectname-foo` is maintained by `projectname`, and in some cases that is even desired! E.g. `serde` has many third party "plugin" crates like [serde-xml-rs](https://github.com/RReverser/serde-xml-rs). Similarly, [async-tls](https://crates.io/crates/async-tls) is a general crate not specific to the async-std ecosystem. + +Regardless, it is nice to have a way to signify "these are all crates belonging to a single organization, and you may trust them the same". Recently, when starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). + +This is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. + +## Guide-level explanation + +If you own a crate `foo`, you may create a crate namespaced under it as `foo/bar`. Only people who are owners of `foo` may _create_ a crate `foo/bar` (and all owners of `foo` are implicitly owners of `foo/bar`). After such a crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. + +The crate can be imported in Cargo.toml using its name as normal: + +```toml +[dependencies] +"foo/bar" = "1.0" +``` + + +In Rust code, the slash gets converted to an underscore, the same way we do this for dashes. + +```rs +use foo_bar::Baz; +``` + +## Reference-level explanation + +`/` is now considered a valid identifier inside a crate name Crates.io. For now, we will restrict crate names to having a single `/` in them, not at the beginning or end of the name, but this can be changed in the future. + +When publishing a crate `foo/bar`, if the crate does not exist, the following must be true: + + - `foo` must exist + - The user publishing the crate must be an owner of `foo` + +For the crate `foo/bar`, all owners of `foo` are always considered owners of `foo/bar`, however additional owners may be added. People removed from ownership of `foo` will also lose access to `foo/bar` unless they were explicitly added as owners to `foo/bar`. + +Crates.io displays `foo/bar` crates with the name `foo/bar`, though it may stylistically make the `foo` part link to the `foo` crate. + +The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo/bar` in `foo@/bar`, placed next to where `foo` is in the trie (i.e. the full path will be `fo/foo@/bar`). + +No changes are made to `rustc`. When compiling a crate `foo/bar`, Cargo will automatically pass in `--crate-name foo_bar`, and when referring to it as a dependency Cargo will use `--extern foo_bar=....`. This is the same thing we currently do for `foo-bar`. + +If you end up in a situation where you have both `foo/bar` and `foo-bar` as active dependencies of your crate, your code will not compile and you must [rename](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml) one of them. + +## Drawbacks + +### Slashes +So far slashes as a "separator" have not existed in Rust. There may be dissonance with having another identifier character allowed on crates.io but not in Rust code. Dashes are already confusing for new users. Some of this can be remediated with appropriate diagnostics on when `/` is encountered at the head of a path. + + +Furthermore, slashes are ambiguous in feature specifiers: + +```toml +[dependencies] +"foo" = "1" +"foo/std" = { version = "1", optional = true } + +[features] +# Does this enable crate "foo/std", or feature "std" of crate "foo"? +default = ["foo/std"] +``` + +### Namespace root taken +Not all existing projects can transition to using namespaces here. For example, the `unicode` crate is reserved, so `unicode-rs` cannot use it as a namespace despite owning most of the `unicode-foo` crates. In other cases, the "namespace root" `foo` may be owned by a different set of people than the `foo-bar` crates, and folks may need to negotiate (`async-std` has this problem, it manages `async-foo` crates but the root `async` crate is taken by someone else). Nobody is forced to switch to namespaces, of course, so the damage here is limited, but it would be _nice_ for everyone to be able to transition. + + +### Dash typosquatting + +This proposal does not prevent anyone from taking `foo-bar` after you publish `foo/bar`. Given that the Rust crate import syntax for `foo/bar` is `foo_bar`, same as `foo-bar`, it's totally possible for a user to accidentally type `foo-bar` in `Cargo.toml` instead of `foo/bar`, and pull in the wrong, squatted, crate. + +We currently prevent `foo-bar` and `foo_bar` from existing at the same time. We _could_ do this here as well, but it would only go in one direction: if `foo/bar` exists, neither `foo-bar` nor `foo_bar` will be allowed to be published. However, if `foo-bar` or `foo_bar` exist, we would choose to allow `foo/bar` to be published, because we don't want to limit the use of names within a crate namespace due to crates outside the namespace existing. This limits the "damage" to cases where someone pre-squats `foo-bar` before you publish `foo/bar`, and the damage can be mitigated by checking to see if such a clashing crate exists when publishing, if you actually care about this attack vector. There are some tradeoffs there that we would have to explore. + +One thing that could mitigate `foo/bar` mapping to the potentially ambiguous `foo_bar` is using something like `foo::crate::bar` or `~foo::bar` or `foo::/bar` in the import syntax. + + +### Slow migration + +Existing projects wishing to use this may need to manually migrate. For example, `unic-langid` may become `unic/langid`, with the `unic` project maintaining `unic-langid` as a reexport crate with the same version number. Getting people to migrate might be a bit of work, and furthermore maintaining a reexport crate during the (potentially long) transition period will also be some work. Of course, there is no obligation to maintain a transition crate, but users will stop getting updates if you don't. + +A possible path forward is to enable people to register aliases, i.e. `unic/langid` is an alias for `unic-langid`. + +## Rationale and alternatives + +This change solves the ownership problem in a way that can be slowly transitioned to for most projects. + +### `foo::bar` on crates.io and in Rust + +While I cover a bunch of different separator choices below, I want to call out `foo::bar` in particular. If we went with `foo::bar`, we could have the same crate name in the Rust source and Cargo manifest. This would be _amazing_. + +Except, of course, crate `foo::bar` is ambiguous with module `bar` in crate `foo` (which might actually be a reexport of `foo::bar` in some cases). + +This can still be made to work, e.g. we could use `foo::crate::bar` to disambiguate, and encourage namespace-using crates to ensure that `mod bar` in crate `foo` either doesn't exist or is a reexport of crate `foo::bar`. I definitely want to see this discussed a bit more. + +### Separator choice + +A different separator might make more sense. + +We could perhaps have `foo-*` get autoreserved if you publish `foo`, as outlined in https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 . I find that this can lead to unfortunate situations where a namespace traditionally used by one project (e.g. `async-*`) is suddenly given over to a different project (the `async` crate). Furthermore, users cannot trust `foo-bar` to be owned by `foo` because the vast number of grandfathered crates we will have. + +Another separator idea would be to use `::`, e.g. `foo::bar`. This looks _great_ in Rust code, provided that the parent crate is empty and does not also have a `bar` module. See the section above for more info. + +Triple colons could work. People might find it confusing, but `foo:::bar` evokes Rust paths without being ambiguous. + +We could use `~` which enables Rust code to directly name namespaced packages (as `~` is no longer used in any valid Rust syntax). It looks extremely weird, however. + +We could use dots (`foo.bar`). This does evoke some similarity with Rust syntax, however there are ambiguities: `foo.bar` in Rust code could either mean "the field `bar` of local/static `foo`" or it may mean "the crate `foo.bar`". + +Note that unquoted dots have semantic meaning in TOML, and allowing for unquoted dots would freeze the list of dependency subfields allowed (to `version`, `git`, `branch`, `features`, etc). + +### Separator mapping + +The proposal suggests mapping `foo/bar` to `foo_bar`, but as mentioned in the typosquatting section, this has problems. There may be other mappings that work out better: + + - `foo::bar` (see section above) + - `foo::crate::bar` + - `foo::/bar` + - `~foo::bar` + +and the like. + + +### User / org namespaces + +Another way to handle namespacing is to rely on usernames and GitHub orgs as namespace roots. This ties `crates.io` strongly to Github -- currently while GitHub is the only login method, there is nothing preventing others from being added. + +Furthermore, usernames are not immutable, and that can lead to a whole host of issues. + +### Registry trie format + +Instead of placing `foo/bar` in `foo@/bar`, it can be placed in `foo@bar` or something else. + +## Prior art + +This proposal is basically the same as https://internals.rust-lang.org/t/pre-rfc-packages-as-namespaces/8628 and https://internals.rust-lang.org/t/pre-rfc-idea-cratespaces-crates-as-namespace-take-2-or-3/11320 . + +Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing-on-crates-io/8571 , https://internals.rust-lang.org/t/pre-rfc-domains-as-namespaces/8688, https://internals.rust-lang.org/t/pre-rfc-user-namespaces-on-crates-io/12851 , https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 , https://internals.rust-lang.org/t/blog-post-no-namespaces-in-rust-is-a-feature/13040/4 , https://internals.rust-lang.org/t/crates-io-package-policies/1041/37, https://internals.rust-lang.org/t/crates-io-squatting/8031, and many others. + + +## Unresolved questions + + - Is `/` really the separator we wish to use? + - How do we avoid ambiguity in feature syntax + - Is there a way to avoid `foo/bar` turning in to the potentially ambiguous `foo_bar`? + - Can we mitigate some of typosquatting? + - How can we represent namespaced crates in the registry trie? + +## Future possibilities + +We can allow multiple layers of nesting if people want it. \ No newline at end of file From 5e6ad7c5d60ae603dc1eb548a920bdadde549a5d Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 9 Mar 2022 13:26:11 -0800 Subject: [PATCH 02/29] Fix template --- text/0000-packages-as-optional-namespaces.md | 43 +++++++++++--------- 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 11bacacec46..59014b17875 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -1,12 +1,15 @@ -# Pre-RFC: Packages as Optional Namespaces +- Feature Name: `packages_as_namespaces` +- Start Date: (fill me in with today's date, 2022-03-09) +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) -## Summary +# Summary Grant exclusive access to publishing crates `parent/foo` for owners of crate `parent`. Namespaced crates can be named in Rust code using underscores (e.g. `parent_foo`) -## Motivation +# Motivation While Rust crates are practically unlimited in size, it is a common pattern for organizations to split their projects into many crates, especially if they expect users to only need a fraction of their crates. @@ -16,7 +19,7 @@ Regardless, it is nice to have a way to signify "these are all crates belonging This is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. -## Guide-level explanation +# Guide-level explanation If you own a crate `foo`, you may create a crate namespaced under it as `foo/bar`. Only people who are owners of `foo` may _create_ a crate `foo/bar` (and all owners of `foo` are implicitly owners of `foo/bar`). After such a crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. @@ -34,7 +37,7 @@ In Rust code, the slash gets converted to an underscore, the same way we do this use foo_bar::Baz; ``` -## Reference-level explanation +# Reference-level explanation `/` is now considered a valid identifier inside a crate name Crates.io. For now, we will restrict crate names to having a single `/` in them, not at the beginning or end of the name, but this can be changed in the future. @@ -53,9 +56,9 @@ No changes are made to `rustc`. When compiling a crate `foo/bar`, Cargo will aut If you end up in a situation where you have both `foo/bar` and `foo-bar` as active dependencies of your crate, your code will not compile and you must [rename](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml) one of them. -## Drawbacks +# Drawbacks -### Slashes +## Slashes So far slashes as a "separator" have not existed in Rust. There may be dissonance with having another identifier character allowed on crates.io but not in Rust code. Dashes are already confusing for new users. Some of this can be remediated with appropriate diagnostics on when `/` is encountered at the head of a path. @@ -71,11 +74,11 @@ Furthermore, slashes are ambiguous in feature specifiers: default = ["foo/std"] ``` -### Namespace root taken +## Namespace root taken Not all existing projects can transition to using namespaces here. For example, the `unicode` crate is reserved, so `unicode-rs` cannot use it as a namespace despite owning most of the `unicode-foo` crates. In other cases, the "namespace root" `foo` may be owned by a different set of people than the `foo-bar` crates, and folks may need to negotiate (`async-std` has this problem, it manages `async-foo` crates but the root `async` crate is taken by someone else). Nobody is forced to switch to namespaces, of course, so the damage here is limited, but it would be _nice_ for everyone to be able to transition. -### Dash typosquatting +## Dash typosquatting This proposal does not prevent anyone from taking `foo-bar` after you publish `foo/bar`. Given that the Rust crate import syntax for `foo/bar` is `foo_bar`, same as `foo-bar`, it's totally possible for a user to accidentally type `foo-bar` in `Cargo.toml` instead of `foo/bar`, and pull in the wrong, squatted, crate. @@ -84,17 +87,17 @@ We currently prevent `foo-bar` and `foo_bar` from existing at the same time. We One thing that could mitigate `foo/bar` mapping to the potentially ambiguous `foo_bar` is using something like `foo::crate::bar` or `~foo::bar` or `foo::/bar` in the import syntax. -### Slow migration +## Slow migration Existing projects wishing to use this may need to manually migrate. For example, `unic-langid` may become `unic/langid`, with the `unic` project maintaining `unic-langid` as a reexport crate with the same version number. Getting people to migrate might be a bit of work, and furthermore maintaining a reexport crate during the (potentially long) transition period will also be some work. Of course, there is no obligation to maintain a transition crate, but users will stop getting updates if you don't. A possible path forward is to enable people to register aliases, i.e. `unic/langid` is an alias for `unic-langid`. -## Rationale and alternatives +# Rationale and alternatives This change solves the ownership problem in a way that can be slowly transitioned to for most projects. -### `foo::bar` on crates.io and in Rust +## `foo::bar` on crates.io and in Rust While I cover a bunch of different separator choices below, I want to call out `foo::bar` in particular. If we went with `foo::bar`, we could have the same crate name in the Rust source and Cargo manifest. This would be _amazing_. @@ -102,7 +105,7 @@ Except, of course, crate `foo::bar` is ambiguous with module `bar` in crate `foo This can still be made to work, e.g. we could use `foo::crate::bar` to disambiguate, and encourage namespace-using crates to ensure that `mod bar` in crate `foo` either doesn't exist or is a reexport of crate `foo::bar`. I definitely want to see this discussed a bit more. -### Separator choice +## Separator choice A different separator might make more sense. @@ -118,7 +121,7 @@ We could use dots (`foo.bar`). This does evoke some similarity with Rust syntax, Note that unquoted dots have semantic meaning in TOML, and allowing for unquoted dots would freeze the list of dependency subfields allowed (to `version`, `git`, `branch`, `features`, etc). -### Separator mapping +## Separator mapping The proposal suggests mapping `foo/bar` to `foo_bar`, but as mentioned in the typosquatting section, this has problems. There may be other mappings that work out better: @@ -130,24 +133,24 @@ The proposal suggests mapping `foo/bar` to `foo_bar`, but as mentioned in the ty and the like. -### User / org namespaces +## User / org namespaces Another way to handle namespacing is to rely on usernames and GitHub orgs as namespace roots. This ties `crates.io` strongly to Github -- currently while GitHub is the only login method, there is nothing preventing others from being added. Furthermore, usernames are not immutable, and that can lead to a whole host of issues. -### Registry trie format +## Registry trie format Instead of placing `foo/bar` in `foo@/bar`, it can be placed in `foo@bar` or something else. -## Prior art +# Prior art This proposal is basically the same as https://internals.rust-lang.org/t/pre-rfc-packages-as-namespaces/8628 and https://internals.rust-lang.org/t/pre-rfc-idea-cratespaces-crates-as-namespace-take-2-or-3/11320 . Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing-on-crates-io/8571 , https://internals.rust-lang.org/t/pre-rfc-domains-as-namespaces/8688, https://internals.rust-lang.org/t/pre-rfc-user-namespaces-on-crates-io/12851 , https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 , https://internals.rust-lang.org/t/blog-post-no-namespaces-in-rust-is-a-feature/13040/4 , https://internals.rust-lang.org/t/crates-io-package-policies/1041/37, https://internals.rust-lang.org/t/crates-io-squatting/8031, and many others. -## Unresolved questions +# Unresolved questions - Is `/` really the separator we wish to use? - How do we avoid ambiguity in feature syntax @@ -155,6 +158,6 @@ Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing- - Can we mitigate some of typosquatting? - How can we represent namespaced crates in the registry trie? -## Future possibilities +# Future possibilities -We can allow multiple layers of nesting if people want it. \ No newline at end of file +We can allow multiple layers of nesting if people want it. From d9d15564b5ac411eba23ce37f179bf138b4c666f Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 9 Mar 2022 13:51:54 -0800 Subject: [PATCH 03/29] Add basic FAQ --- text/0000-packages-as-optional-namespaces.md | 26 ++++++++++++++------ 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 59014b17875..36f96e3935d 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -7,7 +7,7 @@ Grant exclusive access to publishing crates `parent/foo` for owners of crate `parent`. -Namespaced crates can be named in Rust code using underscores (e.g. `parent_foo`) +Namespaced crates can be named in Rust code using underscores (e.g. `parent_foo`). # Motivation @@ -15,7 +15,7 @@ While Rust crates are practically unlimited in size, it is a common pattern for For example, [unic](https://crates.io/search?page=1&per_page=10&q=unic-), [tokio](https://crates.io/search?page=1&per_page=10&q=tokio-), [async-std](https://crates.io/search?page=1&per_page=10&q=async-), [rusoto](https://crates.io/search?q=rusoto) all do something like this, with lots of `projectname-foo` crates. At the moment, it is not necessarily true that a crate named `projectname-foo` is maintained by `projectname`, and in some cases that is even desired! E.g. `serde` has many third party "plugin" crates like [serde-xml-rs](https://github.com/RReverser/serde-xml-rs). Similarly, [async-tls](https://crates.io/crates/async-tls) is a general crate not specific to the async-std ecosystem. -Regardless, it is nice to have a way to signify "these are all crates belonging to a single organization, and you may trust them the same". Recently, when starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). +Regardless, it is nice to have a way to signify "these are all crates belonging to a single organization, and you may trust them the same". When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). This is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. @@ -56,13 +56,15 @@ No changes are made to `rustc`. When compiling a crate `foo/bar`, Cargo will aut If you end up in a situation where you have both `foo/bar` and `foo-bar` as active dependencies of your crate, your code will not compile and you must [rename](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml) one of them. +The `features = ` key in Cargo.toml continues parsing `foo/bar` as "the feature `bar` on dependency `foo`", however it now will unambiguously parse strings ending with a slash (`foo/` and `foo/bar/`) as referring to a dependency, as opposed to feature on a dependency. Cargo may potentially automatically handle the ambiguity or error about it. + # Drawbacks ## Slashes -So far slashes as a "separator" have not existed in Rust. There may be dissonance with having another identifier character allowed on crates.io but not in Rust code. Dashes are already confusing for new users. Some of this can be remediated with appropriate diagnostics on when `/` is encountered at the head of a path. +So far slashes as a "separator" have not existed in Rust. There may be dissonance with having another non-identifier character allowed on crates.io but not in Rust code. Dashes are already confusing for new users. Some of this can be remediated with appropriate diagnostics on when `/` is encountered at the head of a path. -Furthermore, slashes are ambiguous in feature specifiers: +Furthermore, slashes are ambiguous in feature specifiers (though a solution has been proposed above for this): ```toml [dependencies] @@ -74,8 +76,9 @@ Furthermore, slashes are ambiguous in feature specifiers: default = ["foo/std"] ``` + ## Namespace root taken -Not all existing projects can transition to using namespaces here. For example, the `unicode` crate is reserved, so `unicode-rs` cannot use it as a namespace despite owning most of the `unicode-foo` crates. In other cases, the "namespace root" `foo` may be owned by a different set of people than the `foo-bar` crates, and folks may need to negotiate (`async-std` has this problem, it manages `async-foo` crates but the root `async` crate is taken by someone else). Nobody is forced to switch to namespaces, of course, so the damage here is limited, but it would be _nice_ for everyone to be able to transition. +Not all existing projects can transition to using namespaces here. For example, the `unicode` crate is reserved, so `unicode-rs` cannot use it as a namespace despite owning most of the `unicode-foo` crates. In other cases, the "namespace root" `foo` may be owned by a different set of people than the `foo-bar` crates, and folks may need to negotiate (`async-std` has this problem, it manages `async-foo` crates but the root `async` crate is taken by someone else). Nobody is forced to switch to using namespaces, of course, so the damage here is limited, but it would be _nice_ for everyone to be able to transition. ## Dash typosquatting @@ -91,7 +94,7 @@ One thing that could mitigate `foo/bar` mapping to the potentially ambiguous `fo Existing projects wishing to use this may need to manually migrate. For example, `unic-langid` may become `unic/langid`, with the `unic` project maintaining `unic-langid` as a reexport crate with the same version number. Getting people to migrate might be a bit of work, and furthermore maintaining a reexport crate during the (potentially long) transition period will also be some work. Of course, there is no obligation to maintain a transition crate, but users will stop getting updates if you don't. -A possible path forward is to enable people to register aliases, i.e. `unic/langid` is an alias for `unic-langid`. +A possible path forward is to enable people to register aliases, i.e. `unic-langid` is an alias for `unic/langid`. # Rationale and alternatives @@ -148,7 +151,6 @@ Instead of placing `foo/bar` in `foo@/bar`, it can be placed in `foo@bar` or som This proposal is basically the same as https://internals.rust-lang.org/t/pre-rfc-packages-as-namespaces/8628 and https://internals.rust-lang.org/t/pre-rfc-idea-cratespaces-crates-as-namespace-take-2-or-3/11320 . Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing-on-crates-io/8571 , https://internals.rust-lang.org/t/pre-rfc-domains-as-namespaces/8688, https://internals.rust-lang.org/t/pre-rfc-user-namespaces-on-crates-io/12851 , https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 , https://internals.rust-lang.org/t/blog-post-no-namespaces-in-rust-is-a-feature/13040/4 , https://internals.rust-lang.org/t/crates-io-package-policies/1041/37, https://internals.rust-lang.org/t/crates-io-squatting/8031, and many others. - # Unresolved questions @@ -161,3 +163,13 @@ Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing- # Future possibilities We can allow multiple layers of nesting if people want it. + +# FAQ + +## What if I don't want to publish my crate under a namespace? + +You don't have to, namespaces are completely optional when publishing. + +## Does this stop people from squatting on `coolcratename`? + +No, this proposal does not intend to address the general problem of squatting (See [crates.io's policy](https://crates.io/policies#squatting), a lot of this has been discussed many times before). Instead, it allows people who own an existing crate to publish sub-crates under the same namespace. In other words, if you own `coolcratename`, it stops people from squatting `coolcratename/derive`. From e29aa55a9071c2a1614de5937aebe855db004460 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 9 Mar 2022 14:50:22 -0800 Subject: [PATCH 04/29] Add more separator options --- text/0000-packages-as-optional-namespaces.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 36f96e3935d..b279f55e5d7 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -112,6 +112,8 @@ This can still be made to work, e.g. we could use `foo::crate::bar` to disambigu A different separator might make more sense. +We could continue to use `/` but also use `@`, i.e. have crates named `@foo/bar`. This is roughly what npm does and it seems to work. The `@` would not show up in source code, but would adequately disambiguate crates and features in Cargo.toml and in URLs. + We could perhaps have `foo-*` get autoreserved if you publish `foo`, as outlined in https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 . I find that this can lead to unfortunate situations where a namespace traditionally used by one project (e.g. `async-*`) is suddenly given over to a different project (the `async` crate). Furthermore, users cannot trust `foo-bar` to be owned by `foo` because the vast number of grandfathered crates we will have. Another separator idea would be to use `::`, e.g. `foo::bar`. This looks _great_ in Rust code, provided that the parent crate is empty and does not also have a `bar` module. See the section above for more info. @@ -124,6 +126,10 @@ We could use dots (`foo.bar`). This does evoke some similarity with Rust syntax, Note that unquoted dots have semantic meaning in TOML, and allowing for unquoted dots would freeze the list of dependency subfields allowed (to `version`, `git`, `branch`, `features`, etc). + +We could reverse the order and use `@`, i.e. `foo/bar` becomes `bar@foo`. This might be a tad confusing, and it's unclear how best to surface this in the source. + + ## Separator mapping The proposal suggests mapping `foo/bar` to `foo_bar`, but as mentioned in the typosquatting section, this has problems. There may be other mappings that work out better: @@ -159,6 +165,7 @@ Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing- - Is there a way to avoid `foo/bar` turning in to the potentially ambiguous `foo_bar`? - Can we mitigate some of typosquatting? - How can we represent namespaced crates in the registry trie? + - How do we represent namespaced crates in the URLs of crates.io and docs.rs? # Future possibilities From 3a0b0996f5491dbda9b28b193fd87b8b582bdbcc Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 9 Mar 2022 17:25:44 -0800 Subject: [PATCH 05/29] leaf crate names --- text/0000-packages-as-optional-namespaces.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index b279f55e5d7..4935f7a9a3a 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -17,6 +17,8 @@ For example, [unic](https://crates.io/search?page=1&per_page=10&q=unic-), [tokio Regardless, it is nice to have a way to signify "these are all crates belonging to a single organization, and you may trust them the same". When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). +It's worth clarifying, the use of "organization" here can refer to "projects" as well, where a project wishes multiple sub-crates of a particular project to be under the same umbrella. For example, `serde-derive` refers to "the `derive` component of the `serde` project", and `icu-provider` refers to "the provider component of the `icu` project". + This is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. # Guide-level explanation @@ -100,6 +102,13 @@ A possible path forward is to enable people to register aliases, i.e. `unic-lang This change solves the ownership problem in a way that can be slowly transitioned to for most projects. + +## Using identical syntax in Cargo.toml and Rust source + +This RFC in its current form does not propose changes to the Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#Separator choice) for more options); this RFC is attempting to be minimal in its effects on rustc. + +However, the divergence between Cargo.toml and rustc syntax does indeed have a complexity cost, and may be confusing to some users. Furthermore, it increases the chances of [Dash typosquatting](#Dash typosquatting) being effective. + ## `foo::bar` on crates.io and in Rust While I cover a bunch of different separator choices below, I want to call out `foo::bar` in particular. If we went with `foo::bar`, we could have the same crate name in the Rust source and Cargo manifest. This would be _amazing_. @@ -108,6 +117,13 @@ Except, of course, crate `foo::bar` is ambiguous with module `bar` in crate `foo This can still be made to work, e.g. we could use `foo::crate::bar` to disambiguate, and encourage namespace-using crates to ensure that `mod bar` in crate `foo` either doesn't exist or is a reexport of crate `foo::bar`. I definitely want to see this discussed a bit more. + +## Whole crate name vs leaf crate name in Rust source + +It may be potentially better to use just the leaf crate name in Rust source. For example, when using crate `foo/bar` from Cargo.toml, the Rust code would simply use `bar::`. Cargo already supports [renaming dependencies](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml) which can be used to deal with any potential ambiguities here. This also has the added benefit of not having to worry about the separator not parsing as valid Rust. + +A major drawback to this approach is that while it addresses the "the namespace is an organization" use case quite well (e.g. `unicode/segmentation` vs `unicode/line-break` and `rust-lang/libc` vs `rust-lang/lazy-static`, etc), this is rather less amenable to the "the namespace is a _project_" case (e.g. `serde` vs `serde/derive`, `icu/datetime` vs `icu/provider`, etc), where the crates are related not just by provenance. In such cases, users may wish to rename the crates to avoid confusion in the code. This may be an acceptable cost. + ## Separator choice A different separator might make more sense. From 4422adf5136f2c2ead3bdd71dc058f367ae834a9 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 9 Mar 2022 23:45:03 -0800 Subject: [PATCH 06/29] sep choice --- text/0000-packages-as-optional-namespaces.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 4935f7a9a3a..c7d01d9c7df 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -102,7 +102,6 @@ A possible path forward is to enable people to register aliases, i.e. `unic-lang This change solves the ownership problem in a way that can be slowly transitioned to for most projects. - ## Using identical syntax in Cargo.toml and Rust source This RFC in its current form does not propose changes to the Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#Separator choice) for more options); this RFC is attempting to be minimal in its effects on rustc. @@ -111,6 +110,9 @@ However, the divergence between Cargo.toml and rustc syntax does indeed have a c ## `foo::bar` on crates.io and in Rust + +**For discussions about separator choice, please discuss them in [this issue](https://github.com/Manishearth/namespacing-rfc/issues/1) to avoid overwhelming the main RFC thread.** + While I cover a bunch of different separator choices below, I want to call out `foo::bar` in particular. If we went with `foo::bar`, we could have the same crate name in the Rust source and Cargo manifest. This would be _amazing_. Except, of course, crate `foo::bar` is ambiguous with module `bar` in crate `foo` (which might actually be a reexport of `foo::bar` in some cases). @@ -120,12 +122,18 @@ This can still be made to work, e.g. we could use `foo::crate::bar` to disambigu ## Whole crate name vs leaf crate name in Rust source + +**For discussions about separator choice, please discuss them in [this issue](https://github.com/Manishearth/namespacing-rfc/issues/1) to avoid overwhelming the main RFC thread.** + It may be potentially better to use just the leaf crate name in Rust source. For example, when using crate `foo/bar` from Cargo.toml, the Rust code would simply use `bar::`. Cargo already supports [renaming dependencies](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml) which can be used to deal with any potential ambiguities here. This also has the added benefit of not having to worry about the separator not parsing as valid Rust. A major drawback to this approach is that while it addresses the "the namespace is an organization" use case quite well (e.g. `unicode/segmentation` vs `unicode/line-break` and `rust-lang/libc` vs `rust-lang/lazy-static`, etc), this is rather less amenable to the "the namespace is a _project_" case (e.g. `serde` vs `serde/derive`, `icu/datetime` vs `icu/provider`, etc), where the crates are related not just by provenance. In such cases, users may wish to rename the crates to avoid confusion in the code. This may be an acceptable cost. ## Separator choice + +**For discussions about separator choice, please discuss them in [this issue](https://github.com/Manishearth/namespacing-rfc/issues/1) to avoid overwhelming the main RFC thread.** + A different separator might make more sense. We could continue to use `/` but also use `@`, i.e. have crates named `@foo/bar`. This is roughly what npm does and it seems to work. The `@` would not show up in source code, but would adequately disambiguate crates and features in Cargo.toml and in URLs. From 24db22741f518db082a76d350267ab55a0f06fc4 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Thu, 10 Mar 2022 10:25:43 -0800 Subject: [PATCH 07/29] Clarify motivation on projects vs organizations --- text/0000-packages-as-optional-namespaces.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index c7d01d9c7df..ceb3f89f04f 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -15,11 +15,14 @@ While Rust crates are practically unlimited in size, it is a common pattern for For example, [unic](https://crates.io/search?page=1&per_page=10&q=unic-), [tokio](https://crates.io/search?page=1&per_page=10&q=tokio-), [async-std](https://crates.io/search?page=1&per_page=10&q=async-), [rusoto](https://crates.io/search?q=rusoto) all do something like this, with lots of `projectname-foo` crates. At the moment, it is not necessarily true that a crate named `projectname-foo` is maintained by `projectname`, and in some cases that is even desired! E.g. `serde` has many third party "plugin" crates like [serde-xml-rs](https://github.com/RReverser/serde-xml-rs). Similarly, [async-tls](https://crates.io/crates/async-tls) is a general crate not specific to the async-std ecosystem. -Regardless, it is nice to have a way to signify "these are all crates belonging to a single organization, and you may trust them the same". When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). +Regardless, it is nice to have a way to signify "these are all crates belonging to a single project, and you may trust them the same". When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). -It's worth clarifying, the use of "organization" here can refer to "projects" as well, where a project wishes multiple sub-crates of a particular project to be under the same umbrella. For example, `serde-derive` refers to "the `derive` component of the `serde` project", and `icu-provider` refers to "the provider component of the `icu` project". +It's worth spending a bit of time talking about "projects" and "organizations", as nebulous as those terms are. This feature is *primarily* motivated by the needs of "projects". By this I mean a _single_ piece of software developed as multiple crates, for example `serde` and `serde/derive`, or `icu` and `icu/provider`, or `servo/script` and `servo/layout`. One would expect "projects" like this to live under a single Git repository according to the norms of project organization; they are logically a single project even if they are multiple crates. -This is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. +The feature suggested here _might_ be used by "organizations" too, by which I mean a group of people who are coming together to build likely related crates, under the same "brand". One would expect "organizations" like this to use multiple repos under a GitHub organization, and the use case on crates.io would be to be able to have a similar "namespace" under which they can name their crates whatever they want, and prevent squatting. Personally I find this use case less compelling; and it is not the primary motivation behind the RFC. In particular, this use case is more prone to wanting to move crates between organizations and dependency management as a space is not in general super amenable to renames (though we can come up with ways to make this more pleasant). + + +The motivation here is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. # Guide-level explanation From 1986c3dac3bfbe791aa6c0efdb4525a28ec08e6e Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 25 Mar 2022 10:42:44 -0700 Subject: [PATCH 08/29] Switch to colons Fixes https://github.com/Manishearth/namespacing-rfc/issues/1 Fixes https://github.com/Manishearth/namespacing-rfc/issues/2 --- text/0000-packages-as-optional-namespaces.md | 118 +++++++++---------- 1 file changed, 57 insertions(+), 61 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index ceb3f89f04f..796348a04b3 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -17,55 +17,86 @@ For example, [unic](https://crates.io/search?page=1&per_page=10&q=unic-), [tokio Regardless, it is nice to have a way to signify "these are all crates belonging to a single project, and you may trust them the same". When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). -It's worth spending a bit of time talking about "projects" and "organizations", as nebulous as those terms are. This feature is *primarily* motivated by the needs of "projects". By this I mean a _single_ piece of software developed as multiple crates, for example `serde` and `serde/derive`, or `icu` and `icu/provider`, or `servo/script` and `servo/layout`. One would expect "projects" like this to live under a single Git repository according to the norms of project organization; they are logically a single project even if they are multiple crates. +It's worth spending a bit of time talking about "projects" and "organizations", as nebulous as those terms are. This feature is *primarily* motivated by the needs of "projects". By this I mean a _single_ piece of software developed as multiple crates, for example `serde` and `serde::derive`, or `icu` and `icu::provider`, or `servo::script` and `servo::layout`. One would expect "projects" like this to live under a single Git repository according to the norms of project organization; they are logically a single project even if they are multiple crates. -The feature suggested here _might_ be used by "organizations" too, by which I mean a group of people who are coming together to build likely related crates, under the same "brand". One would expect "organizations" like this to use multiple repos under a GitHub organization, and the use case on crates.io would be to be able to have a similar "namespace" under which they can name their crates whatever they want, and prevent squatting. Personally I find this use case less compelling; and it is not the primary motivation behind the RFC. In particular, this use case is more prone to wanting to move crates between organizations and dependency management as a space is not in general super amenable to renames (though we can come up with ways to make this more pleasant). +The feature suggested here _might_ be used by "organizations" too, by which I mean a group of people who are coming together to build likely related crates, under the same "brand". One would expect "organizations" like this to use multiple repos under a GitHub organization, and the use case on crates.io would be to be able to have a similar "namespace" under which they can name their crates whatever they want, and prevent squatting. Personally I find this use case less compelling; and it is not the primary motivation behind the RFC. In particular, this use case is more prone to wanting to move crates between organizations and dependency management as a space is not in general super amenable to renames (though we can come up with ways to make this more pleasant, considered out of scope for this RFC). The motivation here is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. # Guide-level explanation -If you own a crate `foo`, you may create a crate namespaced under it as `foo/bar`. Only people who are owners of `foo` may _create_ a crate `foo/bar` (and all owners of `foo` are implicitly owners of `foo/bar`). After such a crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. +If you own a crate `foo`, you may create a crate namespaced under it as `foo::bar`. Only people who are owners of `foo` may _create_ a crate `foo::bar` (and all owners of `foo` are implicitly owners of `foo/bar`). After such a crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. The crate can be imported in Cargo.toml using its name as normal: ```toml [dependencies] -"foo/bar" = "1.0" +"foo::bar" = "1.0" ``` -In Rust code, the slash gets converted to an underscore, the same way we do this for dashes. +In Rust code, the colons still work: ```rs -use foo_bar::Baz; +use foo::bar::Baz; ``` +In case there is also an in-scope crate `foo` with an exported `bar` item, this will cause an ambiguity error unless both the item `foo::bar` and the crate `foo::bar` are actually resolving to the same item. This is similar to what rustc already does when it encounters in-use clashes in glob imports. + # Reference-level explanation -`/` is now considered a valid identifier inside a crate name Crates.io. For now, we will restrict crate names to having a single `/` in them, not at the beginning or end of the name, but this can be changed in the future. +`::` is now considered valid inside crate names on Crates.io. For now, we will restrict crate names to having a single `::` in them, not at the beginning or end of the name, but this can be changed in the future. -When publishing a crate `foo/bar`, if the crate does not exist, the following must be true: +When publishing a crate `foo::bar`, if the crate does not exist, the following must be true: - `foo` must exist - The user publishing the crate must be an owner of `foo` -For the crate `foo/bar`, all owners of `foo` are always considered owners of `foo/bar`, however additional owners may be added. People removed from ownership of `foo` will also lose access to `foo/bar` unless they were explicitly added as owners to `foo/bar`. +For the crate `foo::bar`, all owners of `foo` are always considered owners of `foo::bar`, however additional owners may be added. People removed from ownership of `foo` will also lose access to `foo::bar` unless they were explicitly added as owners to `foo::bar`. -Crates.io displays `foo/bar` crates with the name `foo/bar`, though it may stylistically make the `foo` part link to the `foo` crate. +Crates.io displays `foo::bar` crates with the name `foo::bar`, though it may stylistically make the `foo` part link to the `foo` crate. -The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo/bar` in `foo@/bar`, placed next to where `foo` is in the trie (i.e. the full path will be `fo/foo@/bar`). +The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo::bar` as just `foo::bar`. No changes are made to `rustc`. When compiling a crate `foo/bar`, Cargo will automatically pass in `--crate-name foo_bar`, and when referring to it as a dependency Cargo will use `--extern foo_bar=....`. This is the same thing we currently do for `foo-bar`. -If you end up in a situation where you have both `foo/bar` and `foo-bar` as active dependencies of your crate, your code will not compile and you must [rename](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml) one of them. +`rustc` will need some changes. When `--extern foo::bar=crate.rlib` is passed in, `rustc` will include this crate during resolution as if it were a module `bar` living under crate `foo`. If crate `foo` is _also_ in scope, this will not automatically trigger any errors unless `foo::bar` is referenced, `foo` has a module `bar`, and that module is not just a reexport of crate `foo::bar`. + +The autogenerated `lib.name` key for such a crate will just be `bar`, the leaf crate name, and the expectation is that to use such crates one _must_ use `--extern foo::bar=bar.rlib` syntax. There may be some better things possible here, perhaps `foo_bar` can be used here. -The `features = ` key in Cargo.toml continues parsing `foo/bar` as "the feature `bar` on dependency `foo`", however it now will unambiguously parse strings ending with a slash (`foo/` and `foo/bar/`) as referring to a dependency, as opposed to feature on a dependency. Cargo may potentially automatically handle the ambiguity or error about it. # Drawbacks -## Slashes + +## Namespace root taken +Not all existing projects can transition to using namespaces here. For example, the `unicode` crate is reserved, so `unicode-rs` cannot use it as a namespace despite owning most of the `unicode-foo` crates. In other cases, the "namespace root" `foo` may be owned by a different set of people than the `foo-bar` crates, and folks may need to negotiate (`async-std` has this problem, it manages `async-foo` crates but the root `async` crate is taken by someone else). Nobody is forced to switch to using namespaces, of course, so the damage here is limited, but it would be _nice_ for everyone to be able to transition. + + +## Slow migration + +Existing projects wishing to use this may need to manually migrate. For example, `unic-langid` may become `unic::langid`, with the `unic` project maintaining `unic-langid` as a reexport crate with the same version number. Getting people to migrate might be a bit of work, and furthermore maintaining a reexport crate during the (potentially long) transition period will also be some work. Of course, there is no obligation to maintain a transition crate, but users will stop getting updates if you don't. + +A possible path forward is to enable people to register aliases, i.e. `unic-langid` is an alias for `unic::langid`. + + +## Requires rustc changes + +There are alternate solutions below that don't require the _language_ getting more complex and can be done purely at the Cargo level. Unfortunately they have other drawbacks. + + +# Rationale and alternatives + +This change solves the ownership problem in a way that can be slowly transitioned to for most projects. + +## Slash as a separator + +**For discussions about separator choice, please discuss them in [this issue](https://github.com/Manishearth/namespacing-rfc/issues/1) to avoid overwhelming the main RFC thread.** + +A previous version of the RFC had `/` as a separator. It would translate it to `foo_bar` in source, and disambiguated in feature syntax with `foo/bar/` vs `foo/bar`. It had the following drawbacks: + + +### Slashes So far slashes as a "separator" have not existed in Rust. There may be dissonance with having another non-identifier character allowed on crates.io but not in Rust code. Dashes are already confusing for new users. Some of this can be remediated with appropriate diagnostics on when `/` is encountered at the head of a path. @@ -81,12 +112,7 @@ Furthermore, slashes are ambiguous in feature specifiers (though a solution has default = ["foo/std"] ``` - -## Namespace root taken -Not all existing projects can transition to using namespaces here. For example, the `unicode` crate is reserved, so `unicode-rs` cannot use it as a namespace despite owning most of the `unicode-foo` crates. In other cases, the "namespace root" `foo` may be owned by a different set of people than the `foo-bar` crates, and folks may need to negotiate (`async-std` has this problem, it manages `async-foo` crates but the root `async` crate is taken by someone else). Nobody is forced to switch to using namespaces, of course, so the damage here is limited, but it would be _nice_ for everyone to be able to transition. - - -## Dash typosquatting +### Dash typosquatting This proposal does not prevent anyone from taking `foo-bar` after you publish `foo/bar`. Given that the Rust crate import syntax for `foo/bar` is `foo_bar`, same as `foo-bar`, it's totally possible for a user to accidentally type `foo-bar` in `Cargo.toml` instead of `foo/bar`, and pull in the wrong, squatted, crate. @@ -95,33 +121,21 @@ We currently prevent `foo-bar` and `foo_bar` from existing at the same time. We One thing that could mitigate `foo/bar` mapping to the potentially ambiguous `foo_bar` is using something like `foo::crate::bar` or `~foo::bar` or `foo::/bar` in the import syntax. -## Slow migration - -Existing projects wishing to use this may need to manually migrate. For example, `unic-langid` may become `unic/langid`, with the `unic` project maintaining `unic-langid` as a reexport crate with the same version number. Getting people to migrate might be a bit of work, and furthermore maintaining a reexport crate during the (potentially long) transition period will also be some work. Of course, there is no obligation to maintain a transition crate, but users will stop getting updates if you don't. - -A possible path forward is to enable people to register aliases, i.e. `unic-langid` is an alias for `unic/langid`. - -# Rationale and alternatives - -This change solves the ownership problem in a way that can be slowly transitioned to for most projects. -## Using identical syntax in Cargo.toml and Rust source +### Using identical syntax in Cargo.toml and Rust source -This RFC in its current form does not propose changes to the Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#Separator choice) for more options); this RFC is attempting to be minimal in its effects on rustc. +The `/` proposal does not require changes to Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#Separator choice) for more options); this RFC is attempting to be minimal in its effects on rustc. However, the divergence between Cargo.toml and rustc syntax does indeed have a complexity cost, and may be confusing to some users. Furthermore, it increases the chances of [Dash typosquatting](#Dash typosquatting) being effective. -## `foo::bar` on crates.io and in Rust - - -**For discussions about separator choice, please discuss them in [this issue](https://github.com/Manishearth/namespacing-rfc/issues/1) to avoid overwhelming the main RFC thread.** +Some potential mappings for `foo/bar` could be: -While I cover a bunch of different separator choices below, I want to call out `foo::bar` in particular. If we went with `foo::bar`, we could have the same crate name in the Rust source and Cargo manifest. This would be _amazing_. - -Except, of course, crate `foo::bar` is ambiguous with module `bar` in crate `foo` (which might actually be a reexport of `foo::bar` in some cases). - -This can still be made to work, e.g. we could use `foo::crate::bar` to disambiguate, and encourage namespace-using crates to ensure that `mod bar` in crate `foo` either doesn't exist or is a reexport of crate `foo::bar`. I definitely want to see this discussed a bit more. + - `foo::bar` + - `foo::crate::bar` + - `foo::/bar` + - `~foo::bar` +and the like. ## Whole crate name vs leaf crate name in Rust source @@ -137,7 +151,7 @@ A major drawback to this approach is that while it addresses the "the namespace **For discussions about separator choice, please discuss them in [this issue](https://github.com/Manishearth/namespacing-rfc/issues/1) to avoid overwhelming the main RFC thread.** -A different separator might make more sense. +A different separator might make more sense. See the [previous section](#slash-as-a-separator) for more on the original proposal of `/` as a separator. We could continue to use `/` but also use `@`, i.e. have crates named `@foo/bar`. This is roughly what npm does and it seems to work. The `@` would not show up in source code, but would adequately disambiguate crates and features in Cargo.toml and in URLs. @@ -157,27 +171,13 @@ Note that unquoted dots have semantic meaning in TOML, and allowing for unquoted We could reverse the order and use `@`, i.e. `foo/bar` becomes `bar@foo`. This might be a tad confusing, and it's unclear how best to surface this in the source. -## Separator mapping - -The proposal suggests mapping `foo/bar` to `foo_bar`, but as mentioned in the typosquatting section, this has problems. There may be other mappings that work out better: - - - `foo::bar` (see section above) - - `foo::crate::bar` - - `foo::/bar` - - `~foo::bar` - -and the like. - - ## User / org namespaces Another way to handle namespacing is to rely on usernames and GitHub orgs as namespace roots. This ties `crates.io` strongly to Github -- currently while GitHub is the only login method, there is nothing preventing others from being added. Furthermore, usernames are not immutable, and that can lead to a whole host of issues. -## Registry trie format - -Instead of placing `foo/bar` in `foo@/bar`, it can be placed in `foo@bar` or something else. +The primary goal of this RFC is for _project_ ownership, not _org_ ownership, so it doesn't map cleanly anyway. # Prior art @@ -187,12 +187,8 @@ Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing- # Unresolved questions - - Is `/` really the separator we wish to use? - - How do we avoid ambiguity in feature syntax - - Is there a way to avoid `foo/bar` turning in to the potentially ambiguous `foo_bar`? - - Can we mitigate some of typosquatting? - How can we represent namespaced crates in the registry trie? - - How do we represent namespaced crates in the URLs of crates.io and docs.rs? + - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? # Future possibilities From 42046f8c22b9197440dc158d7c135048495cc49e Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Fri, 25 Mar 2022 18:42:38 -0700 Subject: [PATCH 09/29] fixes --- text/0000-packages-as-optional-namespaces.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 796348a04b3..719f0def972 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -5,9 +5,9 @@ # Summary -Grant exclusive access to publishing crates `parent/foo` for owners of crate `parent`. +Grant exclusive access to publishing crates `parent::foo` for owners of crate `parent`. -Namespaced crates can be named in Rust code using underscores (e.g. `parent_foo`). +Namespaced crates can be named in Rust code with their full name (`parent::foo`). # Motivation @@ -59,8 +59,6 @@ Crates.io displays `foo::bar` crates with the name `foo::bar`, though it may sty The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo::bar` as just `foo::bar`. -No changes are made to `rustc`. When compiling a crate `foo/bar`, Cargo will automatically pass in `--crate-name foo_bar`, and when referring to it as a dependency Cargo will use `--extern foo_bar=....`. This is the same thing we currently do for `foo-bar`. - `rustc` will need some changes. When `--extern foo::bar=crate.rlib` is passed in, `rustc` will include this crate during resolution as if it were a module `bar` living under crate `foo`. If crate `foo` is _also_ in scope, this will not automatically trigger any errors unless `foo::bar` is referenced, `foo` has a module `bar`, and that module is not just a reexport of crate `foo::bar`. The autogenerated `lib.name` key for such a crate will just be `bar`, the leaf crate name, and the expectation is that to use such crates one _must_ use `--extern foo::bar=bar.rlib` syntax. There may be some better things possible here, perhaps `foo_bar` can be used here. @@ -157,8 +155,6 @@ We could continue to use `/` but also use `@`, i.e. have crates named `@foo/bar` We could perhaps have `foo-*` get autoreserved if you publish `foo`, as outlined in https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 . I find that this can lead to unfortunate situations where a namespace traditionally used by one project (e.g. `async-*`) is suddenly given over to a different project (the `async` crate). Furthermore, users cannot trust `foo-bar` to be owned by `foo` because the vast number of grandfathered crates we will have. -Another separator idea would be to use `::`, e.g. `foo::bar`. This looks _great_ in Rust code, provided that the parent crate is empty and does not also have a `bar` module. See the section above for more info. - Triple colons could work. People might find it confusing, but `foo:::bar` evokes Rust paths without being ambiguous. We could use `~` which enables Rust code to directly name namespaced packages (as `~` is no longer used in any valid Rust syntax). It looks extremely weird, however. @@ -187,7 +183,6 @@ Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing- # Unresolved questions - - How can we represent namespaced crates in the registry trie? - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? # Future possibilities From 2fff101fb5fd2babf660edff6d937df2e80431e3 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Sat, 26 Mar 2022 12:49:44 -0500 Subject: [PATCH 10/29] Document Python prior art Based on this comment: https://github.com/rust-lang/rfcs/pull/3243#issuecomment-1065194578 THere might be some techncial nuance missing but this should generally get the point that users of a language with close-enough semantics have been wanting this feature that it was added in 2003 and then improved in 2012. I decided to forego the examples as that will be better in the guide-level explanation. --- text/0000-packages-as-optional-namespaces.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 719f0def972..71115667388 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -181,6 +181,10 @@ This proposal is basically the same as https://internals.rust-lang.org/t/pre-rfc Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing-on-crates-io/8571 , https://internals.rust-lang.org/t/pre-rfc-domains-as-namespaces/8688, https://internals.rust-lang.org/t/pre-rfc-user-namespaces-on-crates-io/12851 , https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 , https://internals.rust-lang.org/t/blog-post-no-namespaces-in-rust-is-a-feature/13040/4 , https://internals.rust-lang.org/t/crates-io-package-policies/1041/37, https://internals.rust-lang.org/t/crates-io-squatting/8031, and many others. +Python has a similar coupling of top-level namespaces and modules with the filesystem. Users coming from other packaging systems, like Perl, wanted to be able to split up a package under a common namespace. A hook to support this was added in Python 2.3 (see [PEP 402](https://peps.python.org/pep-0402/#the-problem). In [PEP 420](https://peps.python.org/pep-0420/) they formalized a convention for packages to opt-in to sharing a namespace. Differences: +- Python does not have a coupling between package names and top-level namespaces so there is no need for extending the package name format or ability to extend their registry for permissions support. +- In Python, nothing can be in the namespace package while this RFC allows the namespace package to also provide an API. + # Unresolved questions - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? From 8e667ce5dad56ed84e09db23f91ccc05369eb624 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Sat, 26 Mar 2022 15:42:40 -0500 Subject: [PATCH 11/29] Expand on the guide-level explanation This is pulled from https://github.com/rust-lang/rfcs/pull/3243#issuecomment-1064664638 This shifted the guide to starting from the consumer of a namespace, as that is the majority case, and then moving into the role of a crate owner. This leaves out details on feature flags to not duplicate a future "why not feature flags" section. --- text/0000-packages-as-optional-namespaces.md | 28 +++++++++++++++----- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 719f0def972..0c928019e04 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -26,23 +26,37 @@ The motivation here is distinct from the general problem of squatting -- with ge # Guide-level explanation -If you own a crate `foo`, you may create a crate namespaced under it as `foo::bar`. Only people who are owners of `foo` may _create_ a crate `foo::bar` (and all owners of `foo` are implicitly owners of `foo/bar`). After such a crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. +The owners of the `foo` crate may provide other crates under the `foo` namespace, like `foo::bar`. For users, this makes its official status clearer and makes it easier to discover. -The crate can be imported in Cargo.toml using its name as normal: +Users import these crates in Cargo.toml as normal: ```toml [dependencies] -"foo::bar" = "1.0" +"foo" = "1.0.42" +"foo::bar" = "3.1" ``` - -In Rust code, the colons still work: +They will then access this through a facade made of `foo` and all `foo::*` crates, for example: ```rs -use foo::bar::Baz; +let baz = foo::bar::Baz::new(); +foo::render(baz); ``` -In case there is also an in-scope crate `foo` with an exported `bar` item, this will cause an ambiguity error unless both the item `foo::bar` and the crate `foo::bar` are actually resolving to the same item. This is similar to what rustc already does when it encounters in-use clashes in glob imports. +Some reasons for `foo`s owner to consider using namespaces: +- Avoid name conflicts with third-party authors (since they are reserved) +- Improve discoverability of official crates +- As an alternative to feature flags for optional subsystems +- When different parts of your API might have different compatibility guarantees + +When considering this, keep in mind: +- Does it makes sense for this new crate to be presented in the `foo` facade? +- How likely is a crate to move into or out of the namespace? + - Moving the crate in or out of a namespace is a breaking change though it can be worked around by having the old crate re-export the new crate but that does add extra friction to the process. + - There is not currently a mechanism to raise awareness with users that a crate has migrated into or out of a namespace and you might end up leaving users behind. +- If users import both `foo` and `foo::bar` but `foo` also has a `bar` item in its API that isn't just `foo::bar` re-exported, then rustc will error. + +Only the owners of `foo` may _create_ the `foo::bar` crate (and all owners of `foo` are implicitly owners of `foo/bar`). After the `foo::bar` crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. # Reference-level explanation From 01d37b2440047f22e38a746e9ffbed15f32ab8f9 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Sat, 26 Mar 2022 15:54:50 -0500 Subject: [PATCH 12/29] Document feature flags drawbacks This tries to summarize https://github.com/rust-lang/rfcs/pull/3243#issuecomment-1079725129. --- text/0000-packages-as-optional-namespaces.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 719f0def972..61a28dd45c6 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -175,6 +175,17 @@ Furthermore, usernames are not immutable, and that can lead to a whole host of i The primary goal of this RFC is for _project_ ownership, not _org_ ownership, so it doesn't map cleanly anyway. +## Feature Flags + +This proposal allows for optional subsystems. This can be created today with feature flags by adding a dependency as optional and re-exporting it. + +Draw backs to feature flags +- Solutions for documenting feature flags are limited +- Feature flags can be cumbersome to work with for users +- A semver breakage in the optional-subsystem crate is a semver breakage in the namespace crate +- The optional-subsystem crate cannot depend on the namespace crate +- There is limited tooling for crate authors to test feature combinations especially in workspaces with feature unification and its slow (re-running all tests even if they aren't relevant) + # Prior art This proposal is basically the same as https://internals.rust-lang.org/t/pre-rfc-packages-as-namespaces/8628 and https://internals.rust-lang.org/t/pre-rfc-idea-cratespaces-crates-as-namespace-take-2-or-3/11320 . From f7be349cd612a7fa12e33eb20cd5679dc637716d Mon Sep 17 00:00:00 2001 From: Ed Page Date: Mon, 28 Mar 2022 11:31:41 -0500 Subject: [PATCH 13/29] Fix a couple of typos - Lack of closing paren - A copy/paste lingering `/` separator --- text/0000-packages-as-optional-namespaces.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 9e677a44bce..46a1af756a8 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -56,7 +56,7 @@ When considering this, keep in mind: - There is not currently a mechanism to raise awareness with users that a crate has migrated into or out of a namespace and you might end up leaving users behind. - If users import both `foo` and `foo::bar` but `foo` also has a `bar` item in its API that isn't just `foo::bar` re-exported, then rustc will error. -Only the owners of `foo` may _create_ the `foo::bar` crate (and all owners of `foo` are implicitly owners of `foo/bar`). After the `foo::bar` crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. +Only the owners of `foo` may _create_ the `foo::bar` crate (and all owners of `foo` are implicitly owners of `foo::bar`). After the `foo::bar` crate is created, additional per-crate publishers may be added who will be able to publish subsequent versions as usual. # Reference-level explanation @@ -206,7 +206,7 @@ This proposal is basically the same as https://internals.rust-lang.org/t/pre-rfc Namespacing has been discussed in https://internals.rust-lang.org/t/namespacing-on-crates-io/8571 , https://internals.rust-lang.org/t/pre-rfc-domains-as-namespaces/8688, https://internals.rust-lang.org/t/pre-rfc-user-namespaces-on-crates-io/12851 , https://internals.rust-lang.org/t/pre-rfc-hyper-minimalist-namespaces-on-crates-io/13041 , https://internals.rust-lang.org/t/blog-post-no-namespaces-in-rust-is-a-feature/13040/4 , https://internals.rust-lang.org/t/crates-io-package-policies/1041/37, https://internals.rust-lang.org/t/crates-io-squatting/8031, and many others. -Python has a similar coupling of top-level namespaces and modules with the filesystem. Users coming from other packaging systems, like Perl, wanted to be able to split up a package under a common namespace. A hook to support this was added in Python 2.3 (see [PEP 402](https://peps.python.org/pep-0402/#the-problem). In [PEP 420](https://peps.python.org/pep-0420/) they formalized a convention for packages to opt-in to sharing a namespace. Differences: +Python has a similar coupling of top-level namespaces and modules with the filesystem. Users coming from other packaging systems, like Perl, wanted to be able to split up a package under a common namespace. A hook to support this was added in Python 2.3 (see [PEP 402](https://peps.python.org/pep-0402/#the-problem)). In [PEP 420](https://peps.python.org/pep-0420/) they formalized a convention for packages to opt-in to sharing a namespace. Differences: - Python does not have a coupling between package names and top-level namespaces so there is no need for extending the package name format or ability to extend their registry for permissions support. - In Python, nothing can be in the namespace package while this RFC allows the namespace package to also provide an API. From 9a144fb79e80ad7b5cbd8483981fa8173fe95756 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Mon, 28 Mar 2022 11:44:34 -0500 Subject: [PATCH 14/29] Updaet motivation for `::` semantics This was meant to pull in the guide-level explanation's reasons for this feature but they were mostly covered. Instead, I ended up re-wording parts because of the semantics of `::` changes slightly the problem this solves. This is no longer a general project solution but specific to when everything makes sense in the same API. These updates are intended to convey that. --- text/0000-packages-as-optional-namespaces.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 9e677a44bce..4b4ca4e8e27 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -11,15 +11,15 @@ Namespaced crates can be named in Rust code with their full name (`parent::foo`) # Motivation -While Rust crates are practically unlimited in size, it is a common pattern for organizations to split their projects into many crates, especially if they expect users to only need a fraction of their crates. +While Rust crates are practically unlimited in size, it is a common pattern for organizations to split their projects into many crates, especially if they expect users to only need a fraction of their crates or they have different backwards compatibility guarantees. For example, [unic](https://crates.io/search?page=1&per_page=10&q=unic-), [tokio](https://crates.io/search?page=1&per_page=10&q=tokio-), [async-std](https://crates.io/search?page=1&per_page=10&q=async-), [rusoto](https://crates.io/search?q=rusoto) all do something like this, with lots of `projectname-foo` crates. At the moment, it is not necessarily true that a crate named `projectname-foo` is maintained by `projectname`, and in some cases that is even desired! E.g. `serde` has many third party "plugin" crates like [serde-xml-rs](https://github.com/RReverser/serde-xml-rs). Similarly, [async-tls](https://crates.io/crates/async-tls) is a general crate not specific to the async-std ecosystem. -Regardless, it is nice to have a way to signify "these are all crates belonging to a single project, and you may trust them the same". When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). +Regardless, it is nice to have a way to signify "these are all crates belonging to a single project, and you may trust them the same" and discover these related crates. When starting up [ICU4X](https://github.com/unicode-org/icu4x/), we came up against this problem: We wanted to be able to publish ICU4X as an extremely modular system of `icu-foo` or `icu4x-foo` crates, but it would be confusing to users if third-party crates could also exist there (or take names we wanted to use). -It's worth spending a bit of time talking about "projects" and "organizations", as nebulous as those terms are. This feature is *primarily* motivated by the needs of "projects". By this I mean a _single_ piece of software developed as multiple crates, for example `serde` and `serde::derive`, or `icu` and `icu::provider`, or `servo::script` and `servo::layout`. One would expect "projects" like this to live under a single Git repository according to the norms of project organization; they are logically a single project even if they are multiple crates. +It's worth spending a bit of time talking about "projects" and "organizations", as nebulous as those terms are. This feature is *primarily* motivated by the needs of "projects". By this I mean a _single_ Rust API developed as multiple crates, for example `serde` and `serde::derive`, or `icu` and `icu::provider`, or `servo::script` and `servo::layout`. One would expect "projects" like this to live under a single Git repository according to the norms of project organization; they are logically a single project and API even if they are multiple crates. -The feature suggested here _might_ be used by "organizations" too, by which I mean a group of people who are coming together to build likely related crates, under the same "brand". One would expect "organizations" like this to use multiple repos under a GitHub organization, and the use case on crates.io would be to be able to have a similar "namespace" under which they can name their crates whatever they want, and prevent squatting. Personally I find this use case less compelling; and it is not the primary motivation behind the RFC. In particular, this use case is more prone to wanting to move crates between organizations and dependency management as a space is not in general super amenable to renames (though we can come up with ways to make this more pleasant, considered out of scope for this RFC). +The feature suggested here is unlikely to be used by "organizations" as this would put independent concerns in the same Rust API. By "organizations", I mean a group of people who are coming together to build likely related crates, under the same "brand", likely developed in multiple repos under a GitHub organization. The motivation here is distinct from the general problem of squatting -- with general squatting, someone else might come up with a cool crate name before you do. However, with `projectname-foo` crates, it's more of a case of third parties "muscling in" on a name you have already chosen and are using. From 11c63543dfb8f520cfde673659fff2642d316969 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Mon, 28 Mar 2022 19:44:39 -0500 Subject: [PATCH 15/29] Update summary to focus on `::` semantics I can see dropping the first paragraph or moving it to another section if we want to keep the summary more trim / focused. --- text/0000-packages-as-optional-namespaces.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 27cf52c31f3..8537627eac2 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -5,9 +5,9 @@ # Summary -Grant exclusive access to publishing crates `parent::foo` for owners of crate `parent`. +Languages like C++ have open namespaces where anyone can write code in any namespace. In C++'s case, this includes the `std` namespace and is only limited by convention. In contrast, Rust has closed namespaces which can only include code from the original namespace definition (the crate). -Namespaced crates can be named in Rust code with their full name (`parent::foo`). +This proposal extends Rust to have partially open namespaces by allowing crate owners to create crates like `parent:foo` that will be available as part of the crate `parent`'s namespace. To protect the use of open namepsaces, the owners of `parent` has exclusive access to publishing crates in that namespace. # Motivation From 65f708335fb11b7654cb2770d47bc2f68b41058d Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Tue, 29 Mar 2022 09:27:37 -0700 Subject: [PATCH 16/29] add renames as unresolved q --- text/0000-packages-as-optional-namespaces.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 8537627eac2..cb2df8edf5b 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -213,6 +213,7 @@ Python has a similar coupling of top-level namespaces and modules with the files # Unresolved questions - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? + - Should we allow renames like `"foo::bar" = { package = "foo_bar", version = "1.0" }` in Cargo.toml? # Future possibilities From 5c497c5b2f22264c2e6a2cdb43f07a56a1c4eb43 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 24 May 2023 09:03:52 -0700 Subject: [PATCH 17/29] fix syntax --- text/0000-packages-as-optional-namespaces.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index cb2df8edf5b..63f5f4e2c6e 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -7,7 +7,7 @@ Languages like C++ have open namespaces where anyone can write code in any namespace. In C++'s case, this includes the `std` namespace and is only limited by convention. In contrast, Rust has closed namespaces which can only include code from the original namespace definition (the crate). -This proposal extends Rust to have partially open namespaces by allowing crate owners to create crates like `parent:foo` that will be available as part of the crate `parent`'s namespace. To protect the use of open namepsaces, the owners of `parent` has exclusive access to publishing crates in that namespace. +This proposal extends Rust to have partially open namespaces by allowing crate owners to create crates like `parent::foo` that will be available as part of the crate `parent`'s namespace. To protect the use of open namepsaces, the owners of `parent` has exclusive access to publishing crates in that namespace. # Motivation @@ -227,4 +227,4 @@ You don't have to, namespaces are completely optional when publishing. ## Does this stop people from squatting on `coolcratename`? -No, this proposal does not intend to address the general problem of squatting (See [crates.io's policy](https://crates.io/policies#squatting), a lot of this has been discussed many times before). Instead, it allows people who own an existing crate to publish sub-crates under the same namespace. In other words, if you own `coolcratename`, it stops people from squatting `coolcratename/derive`. +No, this proposal does not intend to address the general problem of squatting (See [crates.io's policy](https://crates.io/policies#squatting), a lot of this has been discussed many times before). Instead, it allows people who own an existing crate to publish sub-crates under the same namespace. In other words, if you own `coolcratename`, it stops people from squatting `coolcratename::derive`. From 67ef91389814c395dee765630bdc4f9c975c41a0 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 24 May 2023 14:59:03 -0700 Subject: [PATCH 18/29] fixes --- text/0000-packages-as-optional-namespaces.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 63f5f4e2c6e..27f43731bde 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -60,7 +60,7 @@ Only the owners of `foo` may _create_ the `foo::bar` crate (and all owners of `f # Reference-level explanation -`::` is now considered valid inside crate names on Crates.io. For now, we will restrict crate names to having a single `::` in them, not at the beginning or end of the name, but this can be changed in the future. +`::` is now considered valid inside crate names on Crates.io and Cargo. For now, we will restrict crate names to having a single `::` in them, not at the beginning or end of the name, but this can be changed in the future. When publishing a crate `foo::bar`, if the crate does not exist, the following must be true: @@ -71,7 +71,7 @@ For the crate `foo::bar`, all owners of `foo` are always considered owners of `f Crates.io displays `foo::bar` crates with the name `foo::bar`, though it may stylistically make the `foo` part link to the `foo` crate. -The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo::bar` as just `foo::bar`. +The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo::bar` as just `foo/bar` or using some special character.. `rustc` will need some changes. When `--extern foo::bar=crate.rlib` is passed in, `rustc` will include this crate during resolution as if it were a module `bar` living under crate `foo`. If crate `foo` is _also_ in scope, this will not automatically trigger any errors unless `foo::bar` is referenced, `foo` has a module `bar`, and that module is not just a reexport of crate `foo::bar`. From 56fae2e42f67e90c855b5b573339c23c7bba41cb Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Wed, 24 May 2023 15:03:05 -0700 Subject: [PATCH 19/29] trie-unresolved --- text/0000-packages-as-optional-namespaces.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 27f43731bde..2f93d8be9b7 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -214,6 +214,7 @@ Python has a similar coupling of top-level namespaces and modules with the files - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? - Should we allow renames like `"foo::bar" = { package = "foo_bar", version = "1.0" }` in Cargo.toml? + - How precisely should this be represented in the index trie? # Future possibilities From 482fae89e524edfe3ac883c0d867169b887473e7 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Sun, 23 Jul 2023 22:53:22 +0000 Subject: [PATCH 20/29] Update 0000-packages-as-optional-namespaces.md --- text/0000-packages-as-optional-namespaces.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 2f93d8be9b7..4280058694d 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -60,18 +60,20 @@ Only the owners of `foo` may _create_ the `foo::bar` crate (and all owners of `f # Reference-level explanation -`::` is now considered valid inside crate names on Crates.io and Cargo. For now, we will restrict crate names to having a single `::` in them, not at the beginning or end of the name, but this can be changed in the future. +_This section will maintain a distinction between "package" (a crates.io package) and "crate" (the actual rust library). The rest of the RFC does not attempt to make this distinction_ -When publishing a crate `foo::bar`, if the crate does not exist, the following must be true: +`::` is now considered valid inside package names on Crates.io. For now, we will restrict package names to having a single `::` in them, not at the beginning or end of the name, but this can be changed in the future. + +When publishing a package `foo::bar`, if the package does not exist, the following must be true: - `foo` must exist - - The user publishing the crate must be an owner of `foo` + - The user publishing the package must be an owner of `foo` -For the crate `foo::bar`, all owners of `foo` are always considered owners of `foo::bar`, however additional owners may be added. People removed from ownership of `foo` will also lose access to `foo::bar` unless they were explicitly added as owners to `foo::bar`. +For the package `foo::bar`, all owners of `foo` are always considered owners of `foo::bar`, however additional owners may be added. People removed from ownership of `foo` will also lose access to `foo::bar` unless they were explicitly added as owners to `foo::bar`. -Crates.io displays `foo::bar` crates with the name `foo::bar`, though it may stylistically make the `foo` part link to the `foo` crate. +Crates.io displays `foo::bar` packages with the name `foo::bar`, though it may stylistically make the `foo` part link to the `foo` package. -The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo::bar` as just `foo/bar` or using some special character.. +The [registry index trie](https://doc.rust-lang.org/nightly/cargo/reference/registries.html#index-format) may represent subpackages by placing `foo::bar` as just `foo::bar`. `rustc` will need some changes. When `--extern foo::bar=crate.rlib` is passed in, `rustc` will include this crate during resolution as if it were a module `bar` living under crate `foo`. If crate `foo` is _also_ in scope, this will not automatically trigger any errors unless `foo::bar` is referenced, `foo` has a module `bar`, and that module is not just a reexport of crate `foo::bar`. From 4c14f9c4cfe43d43286fbf617ddd28f1cf9417bd Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Sat, 4 Nov 2023 12:43:08 -0700 Subject: [PATCH 21/29] Update text/0000-packages-as-optional-namespaces.md Co-authored-by: Eric Huss --- text/0000-packages-as-optional-namespaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 4280058694d..a3a2e646d94 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -138,7 +138,7 @@ One thing that could mitigate `foo/bar` mapping to the potentially ambiguous `fo ### Using identical syntax in Cargo.toml and Rust source -The `/` proposal does not require changes to Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#Separator choice) for more options); this RFC is attempting to be minimal in its effects on rustc. +The `/` proposal does not require changes to Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#separator-choice) for more options); this RFC is attempting to be minimal in its effects on rustc. However, the divergence between Cargo.toml and rustc syntax does indeed have a complexity cost, and may be confusing to some users. Furthermore, it increases the chances of [Dash typosquatting](#Dash typosquatting) being effective. From d9a3c90e1c1d219c27086165da130902cd07c7ca Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Sat, 4 Nov 2023 12:43:24 -0700 Subject: [PATCH 22/29] Update text/0000-packages-as-optional-namespaces.md Co-authored-by: Eric Huss --- text/0000-packages-as-optional-namespaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index a3a2e646d94..4418e593873 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -140,7 +140,7 @@ One thing that could mitigate `foo/bar` mapping to the potentially ambiguous `fo The `/` proposal does not require changes to Rust compiler to allow slash syntax (or whatever) to parse as a Rust path. Such changes could be made (though not with slash syntax due to parsing ambiguity, see [below](#separator-choice) for more options); this RFC is attempting to be minimal in its effects on rustc. -However, the divergence between Cargo.toml and rustc syntax does indeed have a complexity cost, and may be confusing to some users. Furthermore, it increases the chances of [Dash typosquatting](#Dash typosquatting) being effective. +However, the divergence between Cargo.toml and rustc syntax does indeed have a complexity cost, and may be confusing to some users. Furthermore, it increases the chances of [Dash typosquatting](#dash-typosquatting) being effective. Some potential mappings for `foo/bar` could be: From 8bd7fbc1202c757a91cd1ccd45faca6738f77f4d Mon Sep 17 00:00:00 2001 From: Ed Page Date: Mon, 13 Nov 2023 10:35:46 -0600 Subject: [PATCH 23/29] Add .crate file as unresolved --- text/0000-packages-as-optional-namespaces.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 4418e593873..4da07c33638 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -217,6 +217,7 @@ Python has a similar coupling of top-level namespaces and modules with the files - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? - Should we allow renames like `"foo::bar" = { package = "foo_bar", version = "1.0" }` in Cargo.toml? - How precisely should this be represented in the index trie? + - How we should name the `.crate` file / download URL # Future possibilities From abde93f0d2df463690fc1b74d0e7f182cb394458 Mon Sep 17 00:00:00 2001 From: Ed Page Date: Mon, 13 Nov 2023 10:36:49 -0600 Subject: [PATCH 24/29] Be explicit that open questions are deferred out --- text/0000-packages-as-optional-namespaces.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 4da07c33638..84265e7b42f 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -214,10 +214,11 @@ Python has a similar coupling of top-level namespaces and modules with the files # Unresolved questions - - How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? - - Should we allow renames like `"foo::bar" = { package = "foo_bar", version = "1.0" }` in Cargo.toml? - - How precisely should this be represented in the index trie? - - How we should name the `.crate` file / download URL +Deferred to tracking issue to be resolved pre-stabilization: +- How exactly should the Cargo.toml `lib.name` key work in this world, and how does that integrate with `--extern` and `-L` and sysroots? +- Should we allow renames like `"foo::bar" = { package = "foo_bar", version = "1.0" }` in Cargo.toml? +- How precisely should this be represented in the index trie? +- How we should name the `.crate` file / download URL # Future possibilities From 7d77485d3e24aa18cf63047627599c6b544d02ef Mon Sep 17 00:00:00 2001 From: Ed Page Date: Mon, 13 Nov 2023 10:41:11 -0600 Subject: [PATCH 25/29] Call out distributions As this isn't an unresolved question for us, I wasn't quite sure where to put it but felt like I should acknowledge the dilemna they are facing somewhere. --- text/0000-packages-as-optional-namespaces.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 84265e7b42f..9a671507a7b 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -220,6 +220,12 @@ Deferred to tracking issue to be resolved pre-stabilization: - How precisely should this be represented in the index trie? - How we should name the `.crate` file / download URL +Third-parties, like Linux distributions, will need to decide how to encode +cargo package names in their distribution package names according to their +individual rules. +Compared to existing ecosystems with namespaces that they package, the only new +wrinkle is that there can be 0-1 namespace levels. + # Future possibilities We can allow multiple layers of nesting if people want it. From 0db096f16998d93ed062c0c6656e589591ce9c70 Mon Sep 17 00:00:00 2001 From: Manish Goregaokar Date: Mon, 11 Mar 2024 13:21:08 -0700 Subject: [PATCH 26/29] Update text/0000-packages-as-optional-namespaces.md Co-authored-by: Oli Scherer --- text/0000-packages-as-optional-namespaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/0000-packages-as-optional-namespaces.md index 9a671507a7b..6913eed8dc1 100644 --- a/text/0000-packages-as-optional-namespaces.md +++ b/text/0000-packages-as-optional-namespaces.md @@ -7,7 +7,7 @@ Languages like C++ have open namespaces where anyone can write code in any namespace. In C++'s case, this includes the `std` namespace and is only limited by convention. In contrast, Rust has closed namespaces which can only include code from the original namespace definition (the crate). -This proposal extends Rust to have partially open namespaces by allowing crate owners to create crates like `parent::foo` that will be available as part of the crate `parent`'s namespace. To protect the use of open namepsaces, the owners of `parent` has exclusive access to publishing crates in that namespace. +This proposal extends Rust to have partially open namespaces by allowing crate owners to create crates like `parent::foo` that will be available as part of the crate `parent`'s namespace. To protect the use of open namespaces, the owners of `parent` has exclusive access to publishing crates in that namespace. # Motivation From 6cc888605a97109b882433a69996e51bdabc873d Mon Sep 17 00:00:00 2001 From: Oli Scherer Date: Mon, 11 Mar 2024 22:40:11 +0100 Subject: [PATCH 27/29] Rename 0000-packages-as-optional-namespaces.md to 3243-packages-as-optional-namespaces.md --- ...onal-namespaces.md => 3243-packages-as-optional-namespaces.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0000-packages-as-optional-namespaces.md => 3243-packages-as-optional-namespaces.md} (100%) diff --git a/text/0000-packages-as-optional-namespaces.md b/text/3243-packages-as-optional-namespaces.md similarity index 100% rename from text/0000-packages-as-optional-namespaces.md rename to text/3243-packages-as-optional-namespaces.md From a93b1e4fb81cf6157b1fd5c4ff7a2f1fe50fef7c Mon Sep 17 00:00:00 2001 From: Oli Scherer Date: Mon, 11 Mar 2024 22:41:10 +0100 Subject: [PATCH 28/29] Add RFC PR link --- text/3243-packages-as-optional-namespaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3243-packages-as-optional-namespaces.md b/text/3243-packages-as-optional-namespaces.md index 6913eed8dc1..29dab294f28 100644 --- a/text/3243-packages-as-optional-namespaces.md +++ b/text/3243-packages-as-optional-namespaces.md @@ -1,6 +1,6 @@ - Feature Name: `packages_as_namespaces` - Start Date: (fill me in with today's date, 2022-03-09) -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- RFC PR: [rust-lang/rfcs#3243](https://github.com/rust-lang/rfcs/pull/3243) - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) # Summary From 656203f28c2abab062a098bb60a2e5148be76b77 Mon Sep 17 00:00:00 2001 From: Oli Scherer Date: Mon, 11 Mar 2024 23:30:45 +0100 Subject: [PATCH 29/29] Link to rust tracking issue --- text/3243-packages-as-optional-namespaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3243-packages-as-optional-namespaces.md b/text/3243-packages-as-optional-namespaces.md index 29dab294f28..4d8a2e404ad 100644 --- a/text/3243-packages-as-optional-namespaces.md +++ b/text/3243-packages-as-optional-namespaces.md @@ -1,7 +1,7 @@ - Feature Name: `packages_as_namespaces` - Start Date: (fill me in with today's date, 2022-03-09) - RFC PR: [rust-lang/rfcs#3243](https://github.com/rust-lang/rfcs/pull/3243) -- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) +- Rust Issue: [rust-lang/rust#122349](https://github.com/rust-lang/rust/issues/122349) # Summary