Skip to content
This repository has been archived by the owner on Apr 17, 2021. It is now read-only.

QA

Jump to bottom
liuche edited this page Jul 4, 2019 · 10 revisions

QA Test Plan: https://wiki.mozilla.org/QA/Mobile/FirefoxTVTestPlan

Fretboard Experiments

We run A/B tests in FFTV, but in testing we want the ability to launch the FFTV app with a specific experiment branch, so we can test/verify specific changes (rather than hoping to be put into a specific bucket).

FFTV can be started from commandline with experiment branches specified (with some sample experiments included):

adb shell am start -n org.mozilla.tv.firefox/org.mozilla.tv.firefox.MainActivity -a android.intent.action.MAIN --esa qaActiveExperiments TvGuideChannels-2195:B,HintBar-2011:A

Experiment input is of the format (and the list must be comma-separated, no spaces): experimentName:branch

Terms

  • experimentName: experiment names can be found in this config file
  • branch: this determines which branch of the experiment will be launched (this is inconsistent: for TV Guide channels, branch B will show the channels; for HintBar, branch A will show the hint bar)

Caveats

  • If you set an override with this command, the app will remain with that set of specified experiments, until either another override with a different set of experiments is run, or the app data is cleared.
  • App must be restarted to guarantee the changes show.

This functionality was added in #2320.

Pocket background fetch

To make testing the Pocket background fetch easier, we have exposed a way to set the delay to a specified number of seconds. You can start the app with a custom intent through the command line with this command:

adb shell am start -n org.mozilla.tv.firefox/org.mozilla.tv.firefox.MainActivity -a android.intent.action.MAIN --el qaFetchDelaySeconds 10

The important flag here is qaFetchDelaySeconds, which is the number of seconds from now that you want the fetch to happen.

When you run this command, the specified fetch will only happen once. This ensures that we do not accidentally leave this build installed and end up spamming the server. The next time you open the app, it will return to the regular production fetch schedule. This means that everytime you want a customized delay, you need to open the app with this command. Another thing to note is that Pocket only updates their videos every 6 hours currently, so the content will not change every time you fetch. If you uninstall the app, you will see the default videos, and then can test fetching the new content.

This functionality was added in #2325.

Clone this wiki locally