"WebDriverError: session deleted because of page crash from tab crashed" for scripted journeys on sitespeed.io v35.3.0

[CharlsChacko]

Have you read the documentation?

• Yes, I've read the how to make a reproducable bug guide
• Yes, I've read the how to debug my script guide

URL

Error is only on scripted journeys and not when run using URL(s)

What are you trying to accomplish

• Since the upgrade of sitespeed.io from 34.7.0 to 35.3.0, we are seeing the below error for scripted journeys.

ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms WebDriverError: unknown error: session deleted because of page crash
from tab crashed
(Session info: chrome=130.0.6723.31)
from tab crashed
(Session info: chrome=130.0.6723.31)
at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)

• Scripting is done using javascript and all it does is hit the first page of the site and run measure commands. And then move on to the next page URL and measure metrics again.
• The script works without any errors on the version 34.7.0 but fails with above error on the latest version 35.3.0
• Attached is the compressed file which contains the js file, config.json and a reference to test execution commands:
  github-issue-files.zip
• Please note: The issue is not seen with URLs of certain clients of ours and not with all URLs. And the same URLs used to work on the earlier version we were on (34.7.0).

What browser did you use?

Chrome

How to reproduce

1. Please refer to the attached files.

2. Run sitespeed.io test using the script and config.json shared on version '35.3.0' using below command to replicate the issue:
> docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:35.3.0 sitespeedio-journey.js --config config.json --multi

3. Now update the `cpu` flag on config to `false` and you will see the test completes successfully without the error and of course with limited data on the CPU tab of the result.

4. Also the error is not seen with `cpu` flag set as `true` when the test is executed on version '34.7.0'
> docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:34.7.0 sitespeedio-journey.js --config config.json --multi

Log output

Chromium 130.0.6723.31 
Mozilla Firefox 131.0.3
[2024-11-11 18:17:26] INFO: Versions OS: linux 6.6.22-linuxkit nodejs: v20.17.0 sitespeed.io: 35.3.0 browsertime: 23.2.0 coach: 8.0.2
[2024-11-11 18:17:26] INFO: Running tests using Chrome - 1 iteration(s)
[2024-11-11 18:17:27] INFO: Use Chrome trace categories: -*,disabled-by-default-lighthouse,v8,v8.execute,blink.user_timing,devtools.timeline,disabled-by-default-devtools.timeline,disabled-by-default-devtools.timeline.stack,disabled-by-default-devtools.timeline.frame,disabled-by-default-devtools.timeline.invalidationTracking,loading,latencyInfo
[2024-11-11 18:17:27] INFO: ================== In perfTest stage ==================
[2024-11-11 18:17:27] INFO: ================== Start measuring Homepage ==================
[2024-11-11 18:17:27] INFO: Start to measure home-page
[2024-11-11 18:17:29] INFO: Navigating to url https://uk.tommy.com iteration 1
[2024-11-11 18:17:36] INFO: Take after page complete check screenshot
[2024-11-11 18:17:37] INFO: Take cumulative layout shift screenshot
[2024-11-11 18:17:37] INFO: Take largest contentful paint screenshot
[2024-11-11 18:17:38] INFO: ================== Start measuring next page ==================
[2024-11-11 18:17:38] INFO: Start to measure next-page
[2024-11-11 18:17:39] INFO: Navigating to url https://uk.tommy.com/women iteration 1
[2024-11-11 18:17:40] INFO: URL failed to load, trying 5 more time(s): Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
[2024-11-11 18:17:40] ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms WebDriverError: unknown error: session deleted because of page crash
from tab crashed
  (Session info: chrome=130.0.6723.31)
from tab crashed
  (Session info: chrome=130.0.6723.31)
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
[2024-11-11 18:17:41] INFO: URL failed to load, trying 4 more time(s): Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
[2024-11-11 18:17:41] ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms NoSuchSessionError: invalid session id
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
[2024-11-11 18:17:42] INFO: URL failed to load, trying 3 more time(s): Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
[2024-11-11 18:17:42] ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms NoSuchSessionError: invalid session id
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
[2024-11-11 18:17:43] INFO: URL failed to load, trying 2 more time(s): Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
[2024-11-11 18:17:43] ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms NoSuchSessionError: invalid session id
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
[2024-11-11 18:17:44] ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms NoSuchSessionError: invalid session id
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
[2024-11-11 18:17:44] INFO: URL failed to load, trying 1 more time(s): Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
[2024-11-11 18:17:45] ERROR: Failed waiting on page https://uk.tommy.com/women to finished loading, timed out after 120000 ms NoSuchSessionError: invalid session id
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
[2024-11-11 18:17:45] INFO: URL failed to load, trying 0 more time(s): Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
[2024-11-11 18:17:45] ERROR: Could not load URL https://uk.tommy.com/women UrlLoadError: Failed waiting on page https://uk.tommy.com/women  to finished loading, timed out after 120000 ms 
    at SeleniumRunner._waitOnPageCompleteCheck (file:///usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:210:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async SeleniumRunner.loadAndWait (file:///usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:298:11)
    at async runMeasurementCommands (/sitespeed.io/sitespeedio-journey.js:15:5)
    at async perfTest (/sitespeed.io/sitespeedio-journey.js:26:5)
    at async file:///usr/src/app/node_modules/browsertime/lib/core/engine/run.js:4:7
    at async Iteration.run (file:///usr/src/app/node_modules/browsertime/lib/core/engine/iteration.js:162:9)
    at async Engine.runByScript (file:///usr/src/app/node_modules/browsertime/lib/core/engine/index.js:308:20)
    at async analyzeUrl (file:///usr/src/app/lib/plugins/browsertime/analyzer.js:214:12)
    at async BrowsertimePlugin.processMessage (file:///usr/src/app/lib/plugins/browsertime/index.js:173:26)
[2024-11-11 18:17:45] ERROR: Couldn't execute script named PAGE_URL error:NoSuchSessionError: invalid session id
[2024-11-11 18:17:45] INFO: Mark run as failure with message: invalid session id
[2024-11-11 18:17:45] ERROR: NoSuchSessionError: invalid session id
    at Object.throwDecodedError (/usr/src/app/node_modules/selenium-webdriver/lib/error.js:521:15)
    at parseHttpResponse (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:514:13)
    at Executor.execute (/usr/src/app/node_modules/selenium-webdriver/lib/http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (/usr/src/app/node_modules/selenium-webdriver/lib/webdriver.js:744:17)
    at async timeout (file:///usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:52:20)
    at async SeleniumRunner.runScript (file:///usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:412:22)
    at async Measure.stop (file:///usr/src/app/node_modules/browsertime/lib/core/engine/command/measure.js:402:8)
    at async perfTest (/sitespeed.io/sitespeedio-journey.js:30:5)
    at async file:///usr/src/app/node_modules/browsertime/lib/core/engine/run.js:4:7
    at async Iteration.run (file:///usr/src/app/node_modules/browsertime/lib/core/engine/iteration.js:162:9)
    at async Engine.runByScript (file:///usr/src/app/node_modules/browsertime/lib/core/engine/index.js:308:20)
    at async analyzeUrl (file:///usr/src/app/lib/plugins/browsertime/analyzer.js:214:12)
    at async BrowsertimePlugin.processMessage (file:///usr/src/app/lib/plugins/browsertime/index.js:173:26)
[2024-11-11 18:17:46] ERROR: Caught error from Browsertime TypeError: Cannot read properties of null (reading 'split')
    at pathToFolder (file:///usr/src/app/node_modules/browsertime/lib/support/pathToFolder.js:21:51)
    at Collector.perIteration (file:///usr/src/app/node_modules/browsertime/lib/core/engine/collector.js:494:14)
    at async Engine.runByScript (file:///usr/src/app/node_modules/browsertime/lib/core/engine/index.js:312:9)
    at async analyzeUrl (file:///usr/src/app/lib/plugins/browsertime/analyzer.js:214:12)
    at async BrowsertimePlugin.processMessage (file:///usr/src/app/lib/plugins/browsertime/index.js:173:26)
[2024-11-11 18:17:46] ERROR: TypeError: Cannot read properties of null (reading 'split')
    at pathToFolder (file:///usr/src/app/lib/core/resultsStorage/pathToFolder.js:20:49)
    at Object.relativeSummaryPageUrl (file:///usr/src/app/lib/core/resultsStorage/resultUrls.js:29:14)
    at DataCollector._addUrl (file:///usr/src/app/lib/plugins/html/dataCollector.js:20:31)
    at DataCollector.addErrorForUrl (file:///usr/src/app/lib/plugins/html/dataCollector.js:92:10)
    at HTMLPlugin.processMessage (file:///usr/src/app/lib/plugins/html/index.js:59:27)
    at file:///usr/src/app/lib/core/queueHandler.js:228:32
    at drainItem (/usr/src/app/node_modules/concurrent-queue/index.js:96:21)
    at Immediate.drain [as _onImmediate] (/usr/src/app/node_modules/concurrent-queue/index.js:77:84)
    at process.processImmediate (node:internal/timers:483:21)
[2024-11-11 18:17:46] INFO: HTML stored in /sitespeed.io/sitespeed-result/sitespeedio-journey_js/2024-11-11-18-17-25

[soulgalore]

Hi @CharlsChacko thank you for creating the issue. I wonder if something changed in Chrome between the release (since it worked in previous versions). Let me check how you can get more info out from Chrome, see if we can hold off why the page crashes the tab/Chrome. Is it possible for you to run the same script in a non Dockerized environment?

[soulgalore]

Also if Chrome crashes I would check the memory on the instance where you run the test, see if you can increase it (if it could be an issue).

[CharlsChacko]

Hello @soulgalore
Thank you for your quick response. Please find below responses to the details you had asked for above:

1. Tried running the script on a non-dockerized sitespeed.io instance of version 35.3.0 and observed the same error. Please see the log from the same here: log-output-non-docker-sitespeedio.txt

2. Also had a look at the memory usage from livecharts extension on docker desktop for the sitespeedio tests on each of the versions. Monitoring the memory usage, looks like the memory usage pattern is quite similar - there is a jump in memory when accessing the next page on the script but is same on both the tests except that the tests on version 35.3.0 did crash and the memory usage drops after the test fails. Please have a check from your side and let me know your thoughts.
   Please see below the comparison of memory usage:
   

[soulgalore]

I see, are you able to downgrade or run an upcoming version of Chrome on your machine and see if it's been fixed in coming versions? You can add --browsertime.chrome.binaryPath add the path to the executable. For example running the Canary build on Mac the path is "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary" . I think Chrome 131 is released any day now ... well it's actually today: https://chromiumdash.appspot.com/schedule I will make sure I do a new release of the container when it's available for Ubuntu.

[soulgalore]

There's a new version of sitespeed.io (35.5.0) rolling out now with Chrome 131 in the Docker container, but if you could reproduce it locally, you can just update the browser and test. Please let me know how it works out.

[CharlsChacko]

Hello @soulgalore

Tried running tests with the latest version sitespeed.io 35.5.0 - unfortunately, seeing the same error on this version on Chromium 131 as well. Here is the log from the run: sitespeedio-35.5.0-log-output.txt

Wanted to check with you if you were able to replicate the issue on your end with the details shared.

[soulgalore]

Sorry I haven't tested, let me do that tonight. I think if you get both in Docker and desktop I will probably be able to reproduce it. If I can reproduce, I'll create the Chrome bug.

[soulgalore]

Hi @CharlsChacko yes I could reproduce, thank you for sharing exactly how I should run. If I omit --cpu it works for me. CPU adds trace categories to Chrome and it seems some of them makes it crash. I will try to reproduce with a simple Selenium script and add an issue to Chromedriver or Chrome. I also tried with Chrome canary (133) and I get the same there.

[soulgalore]

I could track in down to the trace disabled-by-default-devtools.timeline.invalidationTracking if I delete that it works for me (I reproduced using standalone Selenium). So for now until there's a fix I would try something like this:
--browsertime.chrome.traceCategories "-*,disabled-by-default-lighthouse,v8,v8.execute,blink.user_timing,devtools.timeline,disabled-by-default-devtools.timeline,disabled-by-default-devtools.timeline.stack,disabled-by-default-devtools.timeline.frame,loading,latencyInfo" hopefully that will work. That will miss some CLS information in the trace but I guess that is better than just getting errors.

[CharlsChacko]

Thank you @soulgalore for looking at this issue in such quick time.
For now, the cpu browsertime config is set to false to avoid failure of tests. We will wait for a fix before we toggle the config back to true.

[soulgalore]

I created https://issues.chromium.org/issues/379170483 please follow that issue :)