You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sometimes it can be really helpful to see what is going on in the eFPS test suite. This PR adds an option to record videos for runs. It is off by default, cause I don't think we necessarily need the videos for every run - but we may consider changing it if we see it being really useful.
efps — editor "frames per second". The number of updates assumed to be possible within a second.
Derived from input latency. efps = 1000 / input_latency
Detailed information
🏠 Reference result
The performance result of sanity@latest
Benchmark
latency
p75
p90
p99
blocking time
test duration
article (title)
56ms
62ms
84ms
202ms
1114ms
13.2s
article (body)
16ms
18ms
21ms
87ms
128ms
5.0s
article (string inside object)
53ms
57ms
62ms
163ms
947ms
8.8s
article (string inside array)
68ms
71ms
81ms
258ms
1857ms
9.9s
recipe (name)
36ms
39ms
42ms
92ms
87ms
9.5s
recipe (description)
31ms
32ms
36ms
101ms
59ms
6.4s
recipe (instructions)
7ms
9ms
10ms
21ms
0ms
3.5s
synthetic (title)
65ms
68ms
75ms
312ms
2041ms
15.4s
synthetic (string inside object)
65ms
69ms
91ms
683ms
2668ms
10.7s
🧪 Experiment result
The performance result of this branch
Benchmark
latency
p75
p90
p99
blocking time
test duration
article (title)
55ms
57ms
65ms
193ms
1182ms
13.7s
article (body)
17ms
19ms
24ms
215ms
313ms
6.0s
article (string inside object)
51ms
53ms
62ms
340ms
1030ms
8.8s
article (string inside array)
68ms
73ms
80ms
182ms
1733ms
9.8s
recipe (name)
36ms
38ms
40ms
105ms
96ms
9.9s
recipe (description)
31ms
34ms
38ms
66ms
61ms
6.5s
recipe (instructions)
7ms
8ms
10ms
21ms
0ms
3.5s
synthetic (title)
67ms
69ms
81ms
329ms
2206ms
15.6s
synthetic (string inside object)
65ms
70ms
91ms
470ms
2237ms
11.0s
📚 Glossary
column definitions
benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
test duration — how long the test run took to complete.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Sometimes it can be really helpful to see what is going on in the eFPS test suite. This PR adds an option to record videos for runs. It is off by default, cause I don't think we necessarily need the videos for every run - but we may consider changing it if we see it being really useful.
Notes for release
None