Sprint 3 Score Display User Testing

Format

The following details three short user testing sessions that took place over several days between 24th and 29th of September, 2021.

Each testing session had the Participant take part in two short activities, and then answer several questions relating to those activities and general questions about the proposed design and function of the Score Screen 'Runtime'.

Activity 1

This activity was conducted to both familiarise users with the format of numbers being presented and discussed in these interviews, as well as assess clarity and readability of a variety of different numbers in that format.

The first activity had the users presented with a series of small pieces of paper, each with a different number on it, and asked to sort them highest to lowest as quickly as possible. The numbers used in Activity 1 can be seen below;

00037, 00137, 06786, 17858, 04168, 14293, 00007, 07998, 79998, 02010, 24700, 32200

These numbers were randomly generated by an online number generator, and then the numbers that were not five digits were extended to be five digits long. The reason why a series of five digit numbers were presented was in reference to Arcade Machine score boards, which would most typically display in either five or six digits, varying from game to game. Scores lower than five digits would be presented the way the first number in Activity 1, "00037" would appear.

Activity 2

This activity was conducted to assess user feelings, primarily those of reward and accomplishment depending on score presented.

The second activity had users complete an task of spinning around once, and then underarm throwing a ball at the target. This part of the activity was almost completely pointless. Before they did this, each user was told they would be scored on technique, accuracy, and style, with the metrics for what earns them points not revealed to them. In truth there was no metric, and the activity was simply to make them feel invested that the score they received was a reflection of their effort.

Users all received the same scores across their four attempts at throwing the ball, each attempt having a minor alteration, such as throwing arm or direction of spin, with users also invited to "fancy up" their techniques in the second to fourth attempts if they felt they would get them a better score. Again, it would not affect the score received as the scores were predetermined. The scores received, in order they were told to the user are;

90, 50, 25000, 37000

As the results for the activity were predetermined, and it was the users sense of satisfaction that was being assessed, there are no direct findings from this specific activity, but the experience was discussed in the several following questions. The participants were not informed that their 'scores' were fabricated until after the testing session had concluded.

Activity 3

This activity was aimed to collect user thoughts and present a mock-up (seen below) of the score screen to gather opinion and see how it correlates with their experience in the prior activities.

The third activity had each user answer six short questions, regarding the first two activities and a stimuli of a potential score screen mock-up to gauge their opinion.


Question 1

Starting with Activity 2, how did you feel receiving a 90 for your first attempt? Were you satisfied, and how impressive a score did you think it was?

Question 2

And then when the result for your second attempt was revealed, how did that change your opinion of your first score?

Question 3

Did your opinion change further when the third and fourth scores were revealed? How did you feel about your initial performance then?

Question 4

Focusing back on Activity 1 now, did you find it challenging? What numbers stood out to you with the greatest clarity?

Question 5

Looking now at this mock-up, would you be able to identify which five digit score was the best when they were unordered, at a glance?
Question 6

Do you think you would personally feel more elated with a score of between 10,000 and 99,999, versus a score between 1 and 99?

Results

First Tester

Activity 1

16.11 Seconds

Activity 3

Question 1

Well I thought the score would be out of 100 so I thought I did really well. I was very satisfied.

Question 2

I didn't know why my score was worse because you didn't tell me what changed the score. I didn't know what I had done wrong. But like I guess it made the first score seem better? I think I was more disappointed by a lower score for what I thought was doing the same thing then happier that I got 90 on my first try.

Question 3

Really confused. I have no idea how you were calculating score, it wasn't fair obviously, because I did almost exactly the same thing but just dabbed afterwards for the third one. And I'm pretty sure dabbing didn't affect it at all because I didn't dab on the fourth throw and got a better score. It made me think my first two scores were really bad, because I thought the cap was 100 but suddenly it wasn't.

Question 4

Oh, uh, it wasn't too hard? Hardest part was probably moving the paper around honestly, but I guess like the 24700 and 32200 were the easiest to read because I only really needed to look at two digits, if that makes sense. Like the 00037 had only two significant digits but I had to look at all five to know that, you know?

Question 5

I guess that depends how they are formatted? If they are like the 24700 and 32200, or even less number like just three digits, probably. Having it be five digits is like how like Arcade Machines were, right, but its not super readable if I have to look at all five digits. On this mock-up that would be like twenty digits.

Question 6

Bigger number, more dopamine. Unless it's like bills or something.

Second Tester

Activity 1

19.16 Seconds

Activity 3

Question 1

90 was a really bad score right? Maybe at the time I thought it was fine but I'm not happy with it anymore.

Question 2

90 was bad, 50 was worse. You should have asked this before revealing my next two scores because I'm not gonna pretend I was happy with them after getting 20000 or whatever.

Question 3

I guess having a bunch of points was good. It just wasn't established why I got the points.

Question 4

It was super easy. Did I get a better time than [Participant 1]? None of it was challenging and it was super easy and quick to sort.

Question 5

I can read score boards so yes.

Question 6 It doesn't really change anything. Still like the same percent of the max. But maybe getting like 50,000 would feel better than 50 if I knew what I had done to get the 50,000.

Third Tester

Activity 1

18.83 Seconds

Activity 3

Question 1

I thought I got 90 out of 100, so I pogged a bit. Like it's pretty good for a first attempt, I thought. At first.

Question 2

Made me think my first throw must have been really special. I thought it must have been because I was accurate or had better form when I used my right arm in the first throw. But at the same time I like still hit the target and you don't have like score regions marked on it or anything so I don't know how you would have determined I did worse.

Question 3

Made me think my first two scores were shit. But suddenly getting a big point boost was nice. I wish I knew how I went from 50 to 25,000 though. Or 25,000 to 37,000. Like what did I even do that was special?

Question 4

It wasn't hard but I feel like I took way longer than I should have honestly. After I got used to reading the five digits I found them all pretty clear, I suppose. I think maybe the ones with more zeroes were easier?

Question 5

If it's all the same format then yeah, probably could get it a glance. Would they have been ordered? (Participant was answered "No, they wouldn't be.") Oh, ok, well, still pretty quick. Not having to sort them would make it quicker.

Question 6

Dude 99,999 for sure. Got way more range for like actually assessing performance in a game context too.