[E2E Test] Utilities for State Sync Test

[Olshansk]

Description

Summary generated by Reviewpad on 03 Aug 23 20:11 UTC

This pull request introduces several changes across multiple files.

Here is a summary of the changes:

1. go.mod:

   • The golang.org/x/text dependency is now listed explicitly instead of being an indirect dependency.

2. scenario_test.go:

   • Added a new feature called "State Sync Namespace" that includes various commands and waits for specific amounts of time.
   • Code improvements and TODO comments have been added.

3. FAQ.md:

   • Updated an issue related to starting LocalNet with SELinux on an operating system. Replaced the command make compose_and_watch with make lightweight_localnet to avoid permission denied errors.

4. .gitignore:

   • Removed the entry "main" from the list of ignored files.
   • Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.
   • Added the entry "**/gomock_reflect_*/" to ignore mock temporary files.

5. e2e/README.md:

   • Added a new section on Keywords.
   • Modified scenario descriptions and code examples to replace instances of "Validator" with "Node".
   • Included a flowchart depicting the E2E scenarios with updated terminology.

6. Consensus module files:

   • Added a new logging statement in the HandleDebugMessage function.
   • Simplified the handling of the DEBUG_CONSENSUS_RESET_TO_GENESIS action.

7. .tiltignore:

   • Removed the entry "main" from the list of ignored files.
   • Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.

8. CHANGELOG.md:

   • Updated build commands and added a new section on Keywords.

9. persistence/docs/CHANGELOG.md:

   • Several changes related to deprecation, addition, and fixing of functions and issues.

10. validator.feature:

    • Renamed file from "valdator.feature" to "validator.feature".
    • Updated scenario titles and step descriptions to use more descriptive terminology.
    • Replaced references to "validator" with "node".

11. tilt_helpers.go:

    • Added a new file containing functions related to syncing network configuration and checking package installation.

12. debug.go:

    • Added new debug commands and subcommands.
    • Updated existing functions and added new functions for debug actions.

13. account.feature:

    • Added a new file containing scenarios for testing node account functionalities.

14. README.md files:

    • Updated sections, titles, dependencies, and instructions in various README.md files.

15. build/config/README.md:

    • Updated usage instructions, changing the command make compose_and_watch to make lightweight_localnet.

16. iteration_3_end_to_end_tx.md:

    • Updated commands to start LocalNet and consensus debugger.

17. Deleted files:

• validator.go
• watch_build.sh

18. New files added:

• debug.feature
• tilt_helpers.go
• account.feature

Please let me know if you need more information about any specific change.

Issue

Fixes par of #579

Type of change

Please mark the relevant option(s):

• New feature, functionality or library
• Bug fix
• Code health or cleanup
• Major breaking change
• Documentation
• Other

List of changes

• s/compose_and_watch/lightweight_localnet and all related helpers
• s/validator/node in e2e tests for clarity
• Add fire-and-forget Debug CLI w/ several useful initial subcommands
• Add keywords to the e2e document
• Add an e2e debug test to trigger views and track the blockchain increasing
• Avoid rebuilding the actors if the CLI changes
• Small miscellaneous improvements & code cleanup

Testing

• make develop_test; if any code changes were made
• make test_e2e on k8s LocalNet; if any code changes were made
• e2e-devnet-test passes tests on DevNet; if any code was changed
• Docker Compose LocalNet; if any major functionality was changed or introduced
• k8s LocalNet; if any infrastructure or configuration changes were made

Required Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added, or updated, godoc format comments on touched members (see: tip.golang.org/doc/comment)
• I have tested my changes using the available tooling
• I have updated the corresponding CHANGELOG

If Applicable Checklist

• I have updated the corresponding README(s); local and/or global
• I have added tests that prove my fix is effective or that my feature works
• I have added, or updated, mermaid.js diagrams in the corresponding README(s)
• I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

[gitguardian]

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.

---

<sup><sub>🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!</sub></sup>

[Olshansk]

@dylanlott @red-0ne @adshmh Please note that this test is still a bit flaky for now (will solidify it next week), but I think this change is close enough to start getting reviews on.

[0xBigBoss]

I didn't review everything in detail but this approach for handling e2e looks great. I was taking a bit of a different approach instead of waiting for a specified time after triggering the next view, I added more state to the test suite and track pending transaction hashes waiting for them to be all committed. Using the tilt executable too is a nice touch for introducing more participants.

potentially instead of a simple wait, perhaps we can add a step that gets the current height, triggers next view, and then polls for the height to change.

All in all in this is a great direction for the stability of the e2e tests.

See https://github.com/pokt-network/pocket/pull/948/files#diff-01dec4121ae8acb7a1f4bb72a6c2104827d2c2d2197eb5f45fa5c032ffba32cdR93

[Olshansk]

@red-0ne Could you please prioritize taking a look at this?

[Olshansk]

@0xBigBoss I've created #959 and added TODO comments so we can either poll on heights on healthchecks in a followup commit. Also, to avoid PRs that are too large, would appreciate if you could split out things that are possible.

For example, this used to be part of the state sync PR, but I split it out to make it easier for the reviewer instead of having one multi-thousand line PR.

[red-0ne]

Have minor comments.
It's a good base to test state sync against. LGTM 👍

Actually, I'm having trouble running making the e2e test succeed. Will comeback after resolving it.

[0xBigBoss]

all in all a great base addition to the e2e tests. i think it's good to merge and work on removing the timers in separate PRs.