QualityTest never resolves (10-15% of the tests)

[KokoChris]

We have been using this package "opentok-network-test-js" in production for about half a year now.

A few months ago we got some reports that the network tests were never finishing.

Our initial assumption (based on the code we wrote around error handling) was that the .testConnectivity() part was working as expected

The prime suspect was the testQuality and that the promise was never resolving.

We deployed the following fix in production to monitor the situation

 const forceStopNetwork = () =>
            new Promise(
              (resolve, reject) =>
                (forceStopTimeout.current = setTimeout(
                  () => reject({name: KNOWN_QUALITY_ERRORS.TestTookTooLong}),
                  timeoutWithBufferForDelays
                ))
            );
          
Promise.race([otNetworkTest.current.testQuality(), forceStopNetwork()])

What this piece of code does is the following.

It races the quality test with another promise we have which has significantly higher expected resolution time than the whole timeout configured for the NetworkTest

If our promise fails before the quality test resolves then we throw an error

We expected to see a few incidents here and there but after monitoring the issue for about a week we noticed the following results

UnsupportedBrowser | 2020-07-01 | 2 |  
TestTookTooLong | 2020-07-01 | 128 | <--- these are the cases our new error handling caught (the name of the error is ours)
InitPublisherError | 2020-07-01 | 85 |  
acceptable | 2020-07-01 | 106 |  
poor | 2020-07-01 | 165 |  
excellent | 2020-07-01 | 334

Even though we are currently working around the issue the number of occurrences is alarming.

Do you have any similar reports?

Any suggestions on how we should proceed with this?