Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for libtest JSON output #1152

Open
1 of 9 tasks
sunshowers opened this issue Dec 10, 2023 · 0 comments
Open
1 of 9 tasks

Tracking issue for libtest JSON output #1152

sunshowers opened this issue Dec 10, 2023 · 0 comments

Comments

@sunshowers
Copy link
Member

sunshowers commented Dec 10, 2023

Still to do before stabilizing the format:

  • Include stderr and stdout for failing tests (Add support for combined test output #1088, landed as Add support for combined test output (to land) #1212)
  • Produce warning if --message-format-version isn't specified, similar to cargo metadata without --format-version
  • Figure out situation around splitting vs combining stdout and stderr (may need to provide a little command-line configuration for this, e.g. --message-format-version 0.1,test-output=split)
  • Currently, --message-format libtest-json causes both the human and the machine-readable formats to be produced. We need something like Cargo's --message-format json and --message-format json-render-diagnostics.
  • Provide deserializers for the format in nextest-metadata
  • Switch integration tests over to this format (the ones in integration-tests/ scrape the current, unstable format, which sucks
  • Include messages for start-of-test-run and end-of-test-run (maybe only under the libtest-json-plus variant)
  • Write documentation
  • Get real-world use and feedback

cc @Jake-Shadle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant