Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Safe Creation] A/B test the new design #968

Closed
katspaugh opened this issue Oct 25, 2022 · 1 comment · Fixed by #1013 or #1017
Closed

[Safe Creation] A/B test the new design #968

katspaugh opened this issue Oct 25, 2022 · 1 comment · Fixed by #1013 or #1017
Assignees

Comments

@katspaugh
Copy link
Member

katspaugh commented Oct 25, 2022

What is the feature about

We need to analyze how the new design performs against the old one.

The most correct way to do it is to run both versions in parallel and measure the drop-off rate in both.

The list of requirements

  • The new Safe creation should be on a new URL (e.g. /new-safe/create) and its components in a corresponding new folder
  • The old Safe creation should stay on the old URL
  • Add a new A/B hook that would flip a coin and save the variant to the localStorage
  • If a user with the B coin goes to the Safe creation, redirect them to /new-safe/create
  • Add the variant from the localStorage to all GTM events

The A/B hook

The hook should be generic and work for other A/B tets in the future.

So it should be something like this:

const useABTest = (name: string): boolean => {
  const [isB, setB] = useLocalStorage<boolean>(`AB__${name}`, false)
  
  useEffect(() => {
    // Flip the coin if the variant hasn't been determined yet
    setB((prev) => prev ?? Math.random() > 0.5)
  }, [setB])
  
  return isB
}

Links

Epic: #864

@katspaugh katspaugh moved this to Todo in Web Squad Oct 26, 2022
@iamacook iamacook self-assigned this Oct 28, 2022
@iamacook iamacook moved this from Todo to In Progress in Web Squad Oct 28, 2022
@iamacook iamacook moved this from In Progress to Peer Review in Web Squad Oct 28, 2022
This was referenced Oct 31, 2022
@iamacook iamacook linked a pull request Oct 31, 2022 that will close this issue
@iamacook iamacook moved this from Peer Review to In Progress in Web Squad Oct 31, 2022
@iamacook iamacook moved this from In Progress to Peer Review in Web Squad Nov 1, 2022
@iamacook iamacook moved this from Peer Review to In Progress in Web Squad Nov 1, 2022
@iamacook iamacook linked a pull request Nov 1, 2022 that will close this issue
@iamacook iamacook moved this from In Progress to Peer Review in Web Squad Nov 1, 2022
@iamacook iamacook moved this from Peer Review to Ready for QA in Web Squad Nov 1, 2022
@francovenica francovenica moved this from Ready for QA to QA in progress in Web Squad Nov 1, 2022
@francovenica francovenica moved this from QA in progress to QA done in Web Squad Nov 2, 2022
@francovenica
Copy link
Contributor

Is 50/50 alright.
LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
3 participants