Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop leaking identifiers in parser generator and use more DSLv2 types #977

Merged
merged 8 commits into from
May 24, 2024

Conversation

Xanewok
Copy link
Contributor

@Xanewok Xanewok commented May 23, 2024

Part of #638, ticks the "stop leaking identifiers" box

  • We leverage the existing codegen_language_definition::model::Identifier in PG rather than rely on &'static strs from DSL v1, so we don't leak anymore and we start using the DSLv2 type in PG
  • Similarly, we move some of the old DSLv1 utilities in favor of directly using the DSLv2 counterparts
  • Version-dependent codegen is now extracted into a dedicated module rather than split ad-hoc in trie.rs and parser_definition.rs
  • We separate the accumulator-specific state from the model itself to better convey what is required and necessary during the accumulation and what data is finally being derived from the accumulation and stored in the resulting ParserModel used in the MVC fashion for PG codegen

The remaining work is to stop using the DSLv1 Grammar(Visitor) but rather directly collect items from and use the DSLv2 model to construct the final ParserModel for codegen.

I wouldn't look too much into details like duplicated doc comments for the state accumulator/model because it's going away in the next PR, similarly naming for stuff like Versioned(Quote) that's also renamed temporarily and will be changed/removed since we will move off the DSLv1 model.

@Xanewok Xanewok requested a review from a team as a code owner May 23, 2024 08:53
Copy link

changeset-bot bot commented May 23, 2024

⚠️ No Changeset found

Latest commit: b408825

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@Xanewok Xanewok changed the title Stop leak identifiers in parser generator and use more DSLv2 types Stop leaking identifiers in parser generator and use more DSLv2 types May 23, 2024
};
use crate::parser::versioned::{Versioned as _, VersionedQuote as _};

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest removing the as _ from import statements, to enable unused warnings when it is no longer needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not true: after removing the usages I still get correct unused import: `Versioned as _` rustc diagnostic; like I mentioned, this is somewhat special-cased by rustc.

Copy link
Contributor

@OmarTawfik OmarTawfik May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is somewhat special-cased by rustc

hmmm, this is surprising. it goes against how underscore behaves in other places! thanks for pointing out 🤔
I wonder what is the benefit of that syntax then, rather than just use Versioned;? in this case, the underscore is meaningless to rustc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It exists when you want to import a trait but also use a type with the same name - in this case, using the trait in scope allows you to use the trait methods but you can still use the other data type with a given name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@Xanewok Xanewok added this pull request to the merge queue May 24, 2024
Merged via the queue into NomicFoundation:main with commit 099f2d5 May 24, 2024
1 check passed
@Xanewok Xanewok deleted the dont-leak-in-pg-and-cleanup branch May 24, 2024 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants