-
Notifications
You must be signed in to change notification settings - Fork 593
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Support for cpp:identifier
Metadata, and the Groundwork for xxx:identifier
Metadata in General
#3292
Add Support for cpp:identifier
Metadata, and the Groundwork for xxx:identifier
Metadata in General
#3292
Conversation
We should not do that, just like we don't allow cs::identifier on modules in "new Slice": |
I find this sub-optimal. All our compilers generate code for only one language, so it would be nicer for the compiler to tell the Parser from the get-go "my language is xxx, please lookup xxx:identifier when resolving mapped identifiers". |
It's worth noting that modules are pretty different in IceRPC vs Ice Slice. Theoretics aside, how do you propose a C++ user fixes this then?
Or even worse, this:
|
Alright, I'll see what I can do to improve it. But I'll wait until we figure out the module situation. |
It would be logical to add a new Now, back to new Slice: why do we have It's because the semantics are different and more specialized: We should do the same for Ice Slice. The question is which metadata do we use? Say we use ["cpp:ns:Reservation::Bundles"]
module Res::Vacation
{
...
} maps to C++ namespace ["cpp:ns:Foo::Reservation"]
module Res
{
["cpp:ns:Bundles"]
module Vacation { ... }
} maps to: namespace Foo::Reservation
{
namespace Bundles
{
}
} Keep in mind the situation is simpler with new Slice since we don't support the |
Yes, but you're just describing the exact semantics of
And it will be mapped to:
Since we're targeting C++17, which supports nested namespace definitions. However... There might be a bug here with putting Using a name like And it's easier to explain Back to new Slice though, modules are quite different between Ice and IceRPC. And while the design of modules in IceRPC is simpler, I don't think that the way we handle and treat modules in Ice is wrong, just different. IceRPC uses the Java approach ( |
Also, as an aside, if we allow If you have:
It is completely backwards compatible to update this to:
|
The argument for cs::namespace in new Slice is not an identifier but a scoped name. You think it's ok to use lang:identifier in Ice Slice on modules where the argument of this "identifier" metadata is not an identifier? With your example: // module M is mapped to namespace MyWidget.M
["cs:identifier:MyWidget.M"] // MyWidget.M - the argument of cs:identifier - is not an identifier.
module M {} |
Interestingly, cppreference names ns-name "identifier": probably because it was an identifier before C++17. |
Your point isn't lost on me. But... yes, I do think it's okay. Although I agree it's not great. But think mechanistically. With your idea, we'd have two metadata: And they would have the following effect:
when compiled with
These effects are indistinguishable. In both cases, instead of using the Slice-provided identifier, a different name was generated based on some metadata. So, these sure feel like the same metadata to me. Just that it was artificially split into 2. |
My example is a little contrived though, because we're only ever dealing with identifiers. But okay. Nested module syntax looks like this:
An actual question: What would you call the Internally, in the Slice compiler, we call it a Line 382 in 50fbf7e
identifier , but it is still a kind of identifier.
And as you point out, the cppreference calls it an A similar argument applies to how you called it a "scoped name". Yes, Also, interestingly, cppreference has been using |
Regardless of which metadata we eventually select for modules, I recommend you rework this PR to focus exclusively on cpp:identifier for constructs other than modules. Metadata for modules / namespaces / scopes deserves its own PR. For now, just rename the Slice "keyword" modules in the C++ tests to something else. |
0018278
to
2c3b573
Compare
@@ -83,9 +83,9 @@ Slice::validateMetadata(const UnitPtr& p, string_view prefix, map<string, Metada | |||
MetadataInfo deprecatedInfo = { | |||
.validOn = | |||
{typeid(InterfaceDecl), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just swapped the order here to be consistent with other lists.
@@ -4555,19 +4551,25 @@ Slice::DataMember::DataMember( | |||
// ---------------------------------------------------------------------- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All the code below this is to allow the compilers to inject their languagePrefix
into libSlice
.
{ | ||
_scoped = cont->scoped(); | ||
} | ||
_scoped += "::" + _name; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There isn't much point to _scoped
now.
Most of the calls to scoped()
and scope()
are for mapped identifiers, and now we have to compute their values on the fly. So, there's even less point to caching the _scoped
name up-front. So, I removed it.
[[nodiscard]] std::string name() const; | ||
[[nodiscard]] std::string scoped() const; | ||
[[nodiscard]] std::string scope() const; | ||
[[nodiscard]] std::string flattenedScope() const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
flattenedScope
was only used by slice2cpp
, so I moved the logic into that compiler.
cpp/src/Slice/Parser.h
Outdated
@@ -453,7 +453,6 @@ namespace Slice | |||
EnumeratorList enumerators() const; | |||
EnumeratorList enumerators(const std::string& identifier) const; | |||
ContainedList contents() const; | |||
std::string thisScope() const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is only used internally.
Instead of keeping it public and adding the mappedName
flag, I just made it protected, so only things in libSlice
(which don't care about mapped names) can access it.
{ | ||
string s = typeToString(type, optional, scope, metadata, typeCtx); | ||
out << nl << s << ' ' << fixedName << ';'; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is only 2 lines... but takes 7 arguments. I inlined it in the few places where it was used.
// "cpp:identifier" | ||
MetadataInfo identifierInfo = { | ||
.validOn = | ||
{typeid(InterfaceDecl), | ||
typeid(Operation), | ||
typeid(ClassDecl), | ||
typeid(Slice::Exception), | ||
typeid(Struct), | ||
typeid(Sequence), | ||
typeid(Dictionary), | ||
typeid(Enum), | ||
typeid(Enumerator), | ||
typeid(Const), | ||
typeid(Parameter), | ||
typeid(DataMember)}, | ||
.acceptedArgumentKind = MetadataArgumentKind::SingleArgument, | ||
}; | ||
knownMetadata.emplace("cpp:identifier", std::move(identifierInfo)); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the entry which allows the new cpp:identifier
through the metadata validator.
Note that as-of-now, we do not support cpp:identifier
on Modules.
But it is supported on every other relevant Slice construct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the new mapped identifiers, I would use camel case and no leading underscore, for example:
Slice param throw
=> C++ param cppThrow
.
[[nodiscard]] std::string name() const; | ||
[[nodiscard]] std::string scoped() const; | ||
[[nodiscard]] std::string scope() const; | ||
[[nodiscard]] std::string flattenedScope() const; | ||
[[nodiscard]] std::string mappedName() const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be helpful to add doc-comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added some short doc-comments to each of these 6 functions.
@@ -446,14 +449,13 @@ namespace Slice | |||
TypeList lookupTypeNoBuiltin(const std::string& identifier, bool emitErrors, bool ignoreUndefined = false); | |||
ContainedList lookupContained(const std::string& identifier, bool emitErrors); | |||
ExceptionPtr lookupException(const std::string& identifier, bool emitErrors); | |||
[[nodiscard]] UnitPtr unit() const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any reason for the removal of these [[nodiscard]]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This wasn't intentional.
I must of missed this when I was resolving the merge conflicts with main.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added these [[nodiscard]]
s back. And added it to the functions above which should also be marked with it.
These functions aren't marked const
because they can lead to errors being reported, which causes errors++;
.
So I'm not surprised the automatic fix didn't mark them [[nodiscard]]
. But, we should never be called createStruct
and throwing away the value it returns. I'm sure there's other places that need this fix as well, but I didn't want to get carried away here.
cpp/src/Slice/Parser.h
Outdated
@@ -483,6 +485,8 @@ namespace Slice | |||
} | |||
|
|||
protected: | |||
std::string thisScope() const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing [nodiscard]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
cpp/src/Slice/Parser.h
Outdated
@@ -1039,11 +1046,12 @@ namespace Slice | |||
[[nodiscard]] std::set<std::string> getTopLevelModules(const std::string& file) const; | |||
|
|||
private: | |||
Unit(bool all, MetadataList defaultFileMetadata); | |||
Unit(std::string_view languagePrefix, bool all, MetadataList defaultFileMetadata); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find this use of string_view over the top. Using strings is preferable, especially for something you adopt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fine, I switched it to use string
instead.
cpp/src/Slice/Parser.cpp
Outdated
while ((pos = s.find("::", pos)) != string::npos) | ||
// First check if any 'xxx:identifier' has been applied to this element. | ||
// If so, we return that instead of the element's Slice identifier. | ||
static const string metadata = string(_unit->languagePrefix()) + ":identifier"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find this static const
dubious. Yes, all the units in your program will most likely have the same language prefix, but at the same time, you made language prefix a per-unit field, not a global field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find it very unlikely that this would actually cause any problems.
But, I removed the static
anyways.
So we'll re-compute this string every time the function is called.
Thankfully the compilers are already pretty fast, so it's fine I guess.
cpp/src/ice2slice/Main.cpp
Outdated
@@ -176,7 +176,7 @@ compile(const vector<string>& argv) | |||
} | |||
else | |||
{ | |||
UnitPtr p = Unit::createUnit(false); | |||
UnitPtr p = Unit::createUnit("icerpc", false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How does this work since icerpc is not a mapped language in Parser.cpp?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't. Good catch that I forgot to add it to the list of known languages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is icerpc
meaningful here? What does this "prefix" mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think adding 'icerpc' to the list of known language mappings is a good fix. You could make this first parameter std::optional or treat "" as a special value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed icerpc
. Now ice2slice
just passes the empty string.
And I added an assert to mappedName
which enforces that the language string is non-empty.
Since ice2slice
should never be using mapped names anyways.
cpp/src/Slice/Parser.cpp
Outdated
{ | ||
assert(binary_search(&languages[0], &languages[sizeof(languages) / sizeof(*languages)], _languagePrefix)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How does this work for 'icerpc'?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It didn't, but I fixed it. And opened an issue to run ice2slice
in CI.
Right now, we have 0 testing of this tool, and it's way too easy to break it without realizing...
} | ||
|
||
/// Returns a doxygen formatted link to the provided Slice identifier. | ||
/// TODO we need to add a way for the doc-comment generation to use 'cpp' identifier! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't make sense for C++ doc-comment links to link to Slice identifiers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, but this is going to need to be it's own PR.
To fix this, the doc-comment parsing code need to ability to look up Slice definitions by name, and check metadata on them. Right now it has no way to access any of this. And adding this functionality will affect all the languages since this comment-parsing code is shared.
This is a good thing to do anyways, so we can report broken links.
But it's going to a sub-project in-of-itself.
@@ -28,8 +28,8 @@ exception TestImpossibleException | |||
|
|||
void unknownExceptionWithServantException(); | |||
|
|||
string impossibleException(bool throw) throws TestImpossibleException; | |||
string intfUserException(bool throw) throws TestIntfUserException, TestImpossibleException; | |||
string impossibleException(["cpp:identifier:_cpp_throw"] bool throw) throws TestImpossibleException; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not use leading underscores for the new C++ identifiers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed all the identifiers outside of the escape
test.
If you really want, I can also do the escape
test, but this will render the file utterly unreviewable.
And I don't think the names for that test matters, since that test is already using insane names like break
and return
.
Fine with me. I was following our 'old' escaping convention, which I agree isn't very nice. At some point, we should decide what escaping convention we want to use in other languages before I start implementing them. |
cpp/src/Slice/Parser.h
Outdated
@@ -1039,11 +1056,12 @@ namespace Slice | |||
[[nodiscard]] std::set<std::string> getTopLevelModules(const std::string& file) const; | |||
|
|||
private: | |||
Unit(bool all, MetadataList defaultFileMetadata); | |||
Unit(std::string languagePrefix, bool all, MetadataList defaultFileMetadata); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is languagePrefix? Does it mean for metadata only? What else is it a prefix of?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renamed to the more general "languageName"
cpp/src/ice2slice/Main.cpp
Outdated
@@ -176,7 +176,7 @@ compile(const vector<string>& argv) | |||
} | |||
else | |||
{ | |||
UnitPtr p = Unit::createUnit(false); | |||
UnitPtr p = Unit::createUnit("icerpc", false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is icerpc
meaningful here? What does this "prefix" mean?
@@ -563,7 +563,7 @@ Slice::Python::compile(const vector<string>& argv) | |||
return EXIT_FAILURE; | |||
} | |||
|
|||
UnitPtr u = Unit::createUnit(false); | |||
UnitPtr u = Unit::createUnit("python", false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems wrong
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, but that's just how it is. slicepy
internally uses python
for it's metadata.
It's unfortunate, but we can't realistically change it now.
@@ -614,7 +614,7 @@ Slice::Python::compile(const vector<string>& argv) | |||
} | |||
else | |||
{ | |||
UnitPtr u = Unit::createUnit(all); | |||
UnitPtr u = Unit::createUnit("python", all); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
@@ -172,7 +172,7 @@ Slice::Ruby::compile(const vector<string>& argv) | |||
return EXIT_FAILURE; | |||
} | |||
|
|||
UnitPtr u = Unit::createUnit(false); | |||
UnitPtr u = Unit::createUnit("ruby", false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
Re: associating a language mapping with Unit This PR introduces this association, so that Unit can provide various helper functions like: /// @return The mapped identifier that this element will use in the target language.
[[nodiscard]] std::string mappedName() const; (BTW, this is not a correct doc-comment. A doc-comment should not be @return - only) This works fine for Slice compilers like slice2cpp, slice2cs etc. It's not usable by ice2slice - especially once we support multiple language mappings - but this doesn't matter as we wouldn't want to use this helper to figure out the mapped identifiers in ice2slice. If we could use such a helper and called it from ice2slice, we would generate xxx::identifier attributes for pretty much all converted code, which is obviously undesirable. |
47a0803
to
3dc8134
Compare
My thinking was that it would be usable by ice2slice. Say you have this in your Slice definition:
It would generate an uncompilable TL;DR: I removed this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
This description has been updated and is up-to-date
This PR implements #3264 and is a proof of concept for #2864 by adding support for this metadata to C++.
It does this by mimicking the 3 fundamental functions in the
libSlice
Parser:name
: returns the identifier of a Slice definitionscope
: returns the enclosing scope of a Slice definition (with a trailing::
)scoped
returns the fully-scoped identifier of a Slice definition (scope + name
)Currently on main, all 3 of these return the Slice identifier, and it's up the language mappings to massage them.
Now, we have a separate set of 3 functions:
mappedName
,mappedScope
, and the less-than-ideal name ofmappedScoped
.These functions work exactly the same as the original trio, except that they first check for any
xxx:identifier
metadata.If it's present, we use it's argument instead of the Slice-defined name.
In order to make the API smooth, we need a way to inject which language we're working with into
libSlice
.So I added a new
string_view langaugePrefix
field toUnit
, which each compiler passes in.When checking for mapped names, the parser substitutes this string in place of the
xxx
inxxx:identifier
.To see exactly how this change has affected the generated code, see
InsertCreativityHere/compiler-comparison@6d0fafb