Improve testing functionality

[WorldSEnder]

Description

This PR contains a nyew crate, yew-test-runner, that exposes component testing functionality. It re-implements layout_tests with only public API and has a procedural API. The focus should be on the latter, as I think it's more principled and leads to more readable test code, and can be more easily expanded to further functionality.

The reason to have a separate crate is to decouple releases and added functionality, on top of dog feeding our own API. For example, I've had to add a __unstable_start_now function to the scheduler to get it to work. Unstable until further consensus on its meaning. A further argument to have a separate crate is that a module exposed only with #[cfg(test)] is weird and doesn't show up in cargo docs. Adding a feature flag enabling testing code would somewhat complicate internal tests.

Open questions and issues:

• __unstable_start_now could be stable, but in this case, should it panic when the scheduler is already active (and it's not actually doing something?). Should it guarantee that all current events were consumed or is it a "best effort to make some progress"?
• Docs. Should they even mention the layout_tests module, since the API is so crude? I've only ported that for compatibility. I think the conversion is not too bad, maybe we can deprecate it from the start.

Subsumes the testing API in #2679, internal tests are not converted (yet). It's much smaller and easier to review.

Checklist

• I have reviewed my own code
• I have added tests

[github-actions]

Visit the preview URL for this PR (updated for commit d02e0bd):

https://yew-rs-api--pr2858-testing-v2-cvc0aw2k.web.app

_{(expires Mon, 19 Sep 2022 01:54:34 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

[github-actions]

Benchmark - SSR

Yew Master

Benchmark	Round	Min (ms)	Max (ms)	Mean (ms)	Standard Deviation
Baseline	10	307.423	323.716	309.695	4.959
Hello World	10	607.018	611.333	608.121	1.586
Function Router	10	2261.245	2276.906	2266.495	4.760
Concurrent Task	10	1007.766	1009.367	1008.449	0.533

Pull Request

Benchmark	Round	Min (ms)	Max (ms)	Mean (ms)	Standard Deviation
Baseline	10	353.920	356.465	355.015	0.849
Hello World	10	617.497	620.506	618.956	0.958
Function Router	10	2255.166	2267.458	2260.876	3.486
Concurrent Task	10	1007.559	1009.179	1008.552	0.481

[github-actions]

Size Comparison

examples	master (KB)	pull request (KB)	diff (KB)	diff (%)
boids	170.233	170.239	+0.006	+0.003%
communication_child_to_parent	90.466	90.467	+0.001	+0.001%
communication_grandchild_with_grandparent	105.418	105.421	+0.003	+0.003%
communication_grandparent_to_grandchild	101.320	101.319	-0.001	-0.001%
communication_parent_to_child	87.601	87.599	-0.002	-0.002%
contexts	107.849	107.847	-0.002	-0.002%
counter	85.501	85.503	+0.002	+0.002%
counter_functional	86.031	86.030	-0.001	-0.001%
dyn_create_destroy_apps	88.372	88.370	-0.002	-0.002%
file_upload	100.081	100.084	+0.003	+0.003%
function_memory_game	163.779	163.783	+0.004	+0.002%
function_router	348.458	348.471	+0.013	+0.004%
function_todomvc	158.626	158.628	+0.002	+0.001%
futures	221.975	221.988	+0.014	+0.006%
game_of_life	105.739	105.740	+0.001	+0.001%
immutable	181.605	181.604	-0.002	-0.001%
inner_html	82.368	82.369	+0.001	+0.001%
js_callback	111.385	111.383	-0.002	-0.002%
keyed_list	195.571	195.568	-0.003	-0.001%
mount_point	85.122	85.121	-0.001	-0.001%
nested_list	112.596	112.597	+0.001	+0.001%
node_refs	92.851	92.851	0	0.000%
password_strength	1548.171	1548.170	-0.001	-0.000%
portals	96.287	96.284	-0.003	-0.003%
router	318.126	318.120	-0.006	-0.002%
simple_ssr	152.170	152.179	+0.009	+0.006%
ssr_router	394.003	394.010	+0.007	+0.002%
suspense	109.106	109.111	+0.005	+0.004%
timer	88.317	88.317	0	0.000%
todomvc	139.742	139.743	+0.001	+0.001%
two_apps	86.116	86.115	-0.001	-0.001%
web_worker_fib	152.222	152.229	+0.008	+0.005%
webgl	84.834	84.834	0	0.000%

✅ None of the examples has changed their size significantly.

[ranile]

Can you try adding the new crate as a dependency in Yew? I don't think it can be used for Yew tests because of circular dependency

[WorldSEnder]

Can you try adding the new crate as a dependency in Yew? I don't think it can be used for Yew tests because of circular dependency

Adding it as a dev-dependency seems to be allowed. But I found rust-lang/cargo#4242 which is still open. Tl;dr of that is that publishing could become more complicated. The same problem exists in cargo, and a workaround is to remove the 'cyclic' dev-dependency during publishing.

Also relevant is rust-lang/cargo#6765 (comment), which explains how this is allowed.

Note that external tests, such as in the tests/ directory should work just fine with this sort of semi-cyclic dependency setup, it's only internal tests (cfg(test) in the crate) that can be confusing. I'd argue that these sort of tests shouldn't be super-interested in the test-runner the PR introduces.

[ranile]

Can you add documentation here for what diff_layouts does, steps it goes through and an example on how to use it?

Module level documentation for those who are unfamiliar with layout/snapshot testing would be good to have as well. We don't need to thoroughly explain what it is, a link to somewhere that explains it well alongside a short summary would be enough

Also, changing the signature to accept IntoIterator would be better.

Suggested change

	pub fn diff_layouts(layouts: Vec<TestLayout<'_>>) {
	pub fn diff_layouts<'a>(layouts: impl IntoIterator<Item = TestLayout<'a>>) {

This will allow the following to compile:

diff_layouts([layout]);
diff_layouts(vec![layout]);

[ranile]

Suggested change

	/// Automatically gets an implementation of [`TestCase`].
	/// Automatically gets an implementation of [`TestRunner`].

I think you meant TestRunner, not TestCase

[ranile]

Suggested change

	/// Common methods to [`TestRunner`] and [`TestStep`]. Automatically implemented for any type
	/// implementing [`TestContext`].
	/// Common methods to [`TestRunner`] and [`TestStep`].
	///
	/// Automatically implemented for any type implementing [`TestContext`].

Just formatting. This helps rustdoc only display the important line on the home page

[ranile]

See documentation part of #2858 (comment)

[ranile]

it's only internal tests (cfg(test) in the crate) that can be confusing. I'd argue that these sort of tests shouldn't be super-interested in the test-runner the PR introduces.

There's many of them that use layout test. Wouldn't those break?

[futursolo]

I think there are 3 areas that this new testing API needs to improve if we want to endorse it for component testing of user authored crates:

Tests are synchronous

Asynchronous events and network requests are prevalent in a web application and tests usually need to accommodate them.
(This may be less prevalent when testing components, but still not uncommon. e.g.: Users may implement a throttle function with setTimeout for a text field.)

These require tests to run asynchronously. However, both procedural and layout tests are running tests synchronously.

Tests fail as soon as an assertion fails once

Historically, large number of Selenium tests are difficult to maintain and intermittently fails because they would fail as soon as a test command fails an assertion once.

Modern testing frameworks (Cypress, Playwright, react-testing-library, etc.) usually operates asynchronously by retrying testing operation in a short interval until either the test command times out or it becomes successful.
With this approach, it also frees us from exposing (scheduler::start_now()) and we can update the scheduler running logic without breaking tests. (e.g.: We may move use_effect to run after browser painting like what React is doing.)

Tests are encouraged to assert the entire rendered content of a component

This behaviour might be more desirable if we are verifying the behaviour of the Yew renderer.
For tests for Yew itself, it's better to assert all rendered content as sometimes the renderer might try to modify nodes in a seemingly irrelevant location if it is broken.
In most cases of Yew's internal tests, the rendered content of these tests are also not very complex.

However, asserting all contents might not be as desirable when used to verify user defined components.

There are 2 reasons:

1. For complex components, modifying a single element (<h5> -> <h6>) would cause all test cases to fail. This would make tests difficult to maintain if a component is complex and requires a lot of test cases. (e.g.: a DataGrid)
2. Rendered content contains irrelevant information that is not stable.
   If a component is styled with stylist, stylist would create a random css class name to each Style. Most of time, the style class would not be part of the assertions of a test if partial assertions based on css selectors or a11y is used. This testing structure would require stylist class names to be considered in the test as well.