Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(efps): allow recording video in efps suite #7606

Merged
merged 1 commit into from
Oct 9, 2024
Merged

Conversation

rexxars
Copy link
Member

@rexxars rexxars commented Oct 9, 2024

Description

Sometimes it can be really helpful to see what is going on in the eFPS test suite. This PR adds an option to record videos for runs. It is off by default, cause I don't think we necessarily need the videos for every run - but we may consider changing it if we see it being really useful.

Notes for release

None

@rexxars rexxars requested a review from a team as a code owner October 9, 2024 09:52
@rexxars rexxars requested review from juice49 and removed request for a team October 9, 2024 09:52
Copy link

vercel bot commented Oct 9, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 9, 2024 9:53am
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 9, 2024 9:53am
test-compiled-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 9, 2024 9:53am
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 9, 2024 9:53am
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 9, 2024 9:53am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Oct 9, 2024 9:53am

@rexxars rexxars requested review from ricokahler and removed request for juice49 October 9, 2024 09:52
Copy link
Contributor

github-actions bot commented Oct 9, 2024

No changes to documentation

@rexxars rexxars added this pull request to the merge queue Oct 9, 2024
Merged via the queue into next with commit 39e331a Oct 9, 2024
52 checks passed
@rexxars rexxars deleted the ci/record-efps-run-flag branch October 9, 2024 09:53
Copy link
Contributor

github-actions bot commented Oct 9, 2024

Component Testing Report Updated Oct 9, 2024 9:58 AM (UTC)

✅ All Tests Passed -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 46s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 8s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 32s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 39s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 17s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 11s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 2m 33s 1 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 45s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 39s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 16s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 9s 3 9 0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx ✅ Passed (Inspect) 26s 9 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 18s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 35s 12 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Oct 9, 2024

⚡️ Editor Performance Report

Updated Wed, 09 Oct 2024 10:11:39 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 17.9 efps (56ms) 18.2 efps (55ms) -1ms (-1.8%)
article (body) 63.5 efps (16ms) 59.7 efps (17ms) +1ms (+6.3%)
article (string inside object) 18.9 efps (53ms) 19.6 efps (51ms) -2ms (-3.8%)
article (string inside array) 14.8 efps (68ms) 14.8 efps (68ms) +0ms (-/-%)
recipe (name) 27.8 efps (36ms) 28.2 efps (36ms) -1ms (-1.4%)
recipe (description) 32.8 efps (31ms) 32.3 efps (31ms) +1ms (+1.6%)
recipe (instructions) 99.9+ efps (7ms) 99.9+ efps (7ms) +0ms (-/-%)
synthetic (title) 15.4 efps (65ms) 14.9 efps (67ms) +2ms (+3.1%)
synthetic (string inside object) 15.4 efps (65ms) 15.4 efps (65ms) +0ms (-/-%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 56ms 62ms 84ms 202ms 1114ms 13.2s
article (body) 16ms 18ms 21ms 87ms 128ms 5.0s
article (string inside object) 53ms 57ms 62ms 163ms 947ms 8.8s
article (string inside array) 68ms 71ms 81ms 258ms 1857ms 9.9s
recipe (name) 36ms 39ms 42ms 92ms 87ms 9.5s
recipe (description) 31ms 32ms 36ms 101ms 59ms 6.4s
recipe (instructions) 7ms 9ms 10ms 21ms 0ms 3.5s
synthetic (title) 65ms 68ms 75ms 312ms 2041ms 15.4s
synthetic (string inside object) 65ms 69ms 91ms 683ms 2668ms 10.7s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 55ms 57ms 65ms 193ms 1182ms 13.7s
article (body) 17ms 19ms 24ms 215ms 313ms 6.0s
article (string inside object) 51ms 53ms 62ms 340ms 1030ms 8.8s
article (string inside array) 68ms 73ms 80ms 182ms 1733ms 9.8s
recipe (name) 36ms 38ms 40ms 105ms 96ms 9.9s
recipe (description) 31ms 34ms 38ms 66ms 61ms 6.5s
recipe (instructions) 7ms 8ms 10ms 21ms 0ms 3.5s
synthetic (title) 67ms 69ms 81ms 329ms 2206ms 15.6s
synthetic (string inside object) 65ms 70ms 91ms 470ms 2237ms 11.0s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants