Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Code Search, II] A tree-sitter parser alternative #1160

Draft
wants to merge 22 commits into
base: new-parser-staging
Choose a base branch
from
Draft

[Code Search, II] A tree-sitter parser alternative #1160

wants to merge 22 commits into from

Conversation

Ellpeck
Copy link
Member

@Ellpeck Ellpeck commented Nov 19, 2024

No description provided.

@Ellpeck Ellpeck linked an issue Nov 19, 2024 that may be closed by this pull request
[Code Search, II] A tree-sitter parser alternative #1126
Open
Ellpeck and others added 18 commits November 19, 2024 16:50
@Ellpeck
wip: stuff we talked about
87404ff
@Ellpeck
refactor: move tree sitter wasm to git lfs
77e3259
@Ellpeck
wip: lots of work, lots of issues
9d5821e
@Ellpeck
refactor: use the recommended name for tree sitter vars
4e40b40
@Ellpeck
wip: more work
5349747
@Ellpeck
feat: a lot more work on supporting tree-sitter types
3f14f25
@Ellpeck
feat: support more tree-sitter types
4e32d5e
@Ellpeck
feat: finished basic tree sitter normalization
fc38b67
@Ellpeck
test: automatically test for equality between the two ast variations
10e2b38
@Ellpeck
test: check out lfs when testing to fix wasm loading
6839c2e
@Ellpeck
test-fix: no that was awful
63d6e67
@Ellpeck
feat: full parity between RShell and TS outputs
58db836
@Ellpeck
refactor: remove dangling true
d4499de
@EagleoutIce
Merge branch 'main' into 1126-code-search-ii-a-tree-sitter-parser-alt…
f557c39 
…ernative
@EagleoutIce
feat: separate tree sitter and r shell normalize contexts
cab5c71
@EagleoutIce
refactor: switch from deprecated external marker
3b59ceb
@EagleoutIce
refactor(tests): avoid ast reparse
7438219
@EagleoutIce
refactor: synchronize step names
f04947a
@EagleoutIce
Copy link
Member

EagleoutIce commented Nov 22, 2024 •
edited by Ellpeck
Loading

Still open (to be done within this PR or to be outsourced):

  • switch to npm package once available, to avoid wasm and (hopefully init)
    • Still allow the wasm backend (e.g., if we want to develop/explore part of it) with
      a user-supplied .wasm file.
  • For this we maybe want to create a startup configuration (like a feature list that determines what things are done/initialized on startup) (Julian) (Reconsider Startup Configuration #942)
    • Allow for configuration to switch between the tree sitter version and the RShell
  • Verify that docker etc. work (we should incorporate this into the system tests as done by @gigalasr in Basic System Tests for flowR #1164) -> needs RShell etc. to work
  • pass the TreeSitterExecutor object similar to the RShell (Julian)
  • do not use an RShellExecutor when using TreeSitter for Sourcing (Julian)
  • Verify the source tests with tree-sitter too
  • Reflect the backend in the HelloMessage and the other server Requests
  • Document the new backend and the way we can configure it (Flo, later)
  • Also execute benchmarks with tree-sitter and r shell separately

Please add if there are more @Ellpeck

@EagleoutIce EagleoutIce changed the base branch from main to new-parser-staging November 22, 2024 07:04
@Ellpeck Ellpeck mentioned this pull request Nov 23, 2024
Open
@DavisVaughan DavisVaughan mentioned this pull request Nov 25, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Code Search, II] A tree-sitter parser alternative
2 participants
@Ellpeck @EagleoutIce