Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include Bulk Requests support + RPC trace related tables #3

Merged
merged 5 commits into from
May 23, 2023

Conversation

cortze
Copy link

@cortze cortze commented May 17, 2023

Description

With the coming merge/review of lotus#10405, the level of traces emitted and the way of pushing them will change. The PR includes trace batching through Elastic Search Bulk Requests and aggregating the Send/Recv RPC traces. Thus, this PR aims to upgrade the trace-catcher to support those changes.

Tasks

  • Add a _bulk endpoint to the HTTP server to read batched traces
  • Add the DB functionalities to support the RPC events (Broadcast Messages, Iwants, and Ihaves )
    - rpc_event (sent / recv)
    - broadcast_message (linked to the RPC event)
    - ihave_message (linked to the RPC event)
    - ìwant_message (linked to the RPC event)
  • Update the MessageID encoding format to be persistable in the DB
  • Fix some minor hanging typos
  • Format the code to be ready for review :)

Testing

I'm currently testing this same branch on my testing server, and everything seems to be working fine with lotus#10405

@cortze
Copy link
Author

cortze commented May 17, 2023

This PR also uses #2 as a base, and I'm unsure if we want to merge this and drop the prev one.

Copy link
Contributor

@iand iand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Most of my comments are about logging :D

server.go Show resolved Hide resolved
server.go Outdated Show resolved Hide resolved
server.go Outdated Show resolved Hide resolved
server.go Outdated Show resolved Hide resolved
db.go Outdated Show resolved Hide resolved
db.go Outdated Show resolved Hide resolved
db.go Outdated Show resolved Hide resolved
db.go Outdated Show resolved Hide resolved
db.go Outdated Show resolved Hide resolved
db.go Show resolved Hide resolved
cortze pushed a commit to cortze/tracecatcher that referenced this pull request May 22, 2023
cortze pushed a commit to cortze/tracecatcher that referenced this pull request May 22, 2023
@cortze
Copy link
Author

cortze commented May 22, 2023

Hey @iand ! I just pushed a commit with the comments addressed - thank you so much for the feedback!
I also rebased the PR to the latest change on the missing peer_score branch, including the v2 tables. Let me know if anything is missing :)

Copy link
Contributor

@iand iand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I will get this built and deployed so we can use it as a testbed. Thanks again for your contribution.

@iand iand merged commit cf9990a into probe-lab:main May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants