User Testing for Features Enhancements in Sprint 4

Overview

This testing round was conducted to answer several hypotheses that were arose after the team managed to address the problems that surfaced in the previous user testing.

Hypotheses

This testing round tried to answer these hypotheses:

1. Movement and pathfinding are well-integrated to the friendly soldier units
2. Visual feedback for selections and attacking functionalities make the game interactive and improve its experience overall
3. The players understand how the spell economics work

Testing Plan

Data collection that used for this testing round was observation and brief interviews. Observation was conducted to see how the users interact with the gameplay, where they struggle, and gather some insights that somehow might have been missed. The users were asked to say what they think during playing the game and doing several tasks (Think Out Loud). Brief interview was conducted to answer several hypotheses regarding the features that have been developed so far. The tasks that the users needed to do were:

1. Start the game
2. Watch cutscenes
3. Watch tutorial a. Skip tutorial
4. Highlight units
5. Move units
6. Use units to chase/attack enemy
7. Cast spell
8. Quit the game

The questions asked during the interview session were:

1. What do you think about the way you move a unit?
2. What do you think about the way you select soldiers to attack an enemy?
3. What do you think about the spell mechanism?
4. From a scale of 1-5, how easy it is to move soldiers to attack an enemy?
5. From a scale of 1-5, how easy it is to use the spell mechanism?

Findings

There were several insights and bugs found during this testing round. The number of participants involved in this testing round was three. The table below shows excerpts collected during observations for each tasks.

Tasks	Insights
Task 2	"Yeah sure, cutscenes to introduce the game is interesting."
Task 3	"But how do I interact with the units though? I feel like the tutorial is clear for the most part but lacking in introducing some features." The game quit unexpectedly "Wait what.. Is it supposed to be like this?"
Task 3.a	Were asked about their opinion regarding skipping the cutscenes "I agree that we should be able to skip the cutscenes once in a while."
Task 4	"So, how do I do it?" "Is it me or some units are not supposed to be selected?"
Task 5	"Now, how to move this unit?" "Why don't you make it so we only use mouse left click to highlight and move the units? Because that way I think it's more natural."
Task 6	"Have I selected the enemy that I want to attack? Because there's no signs (visual feedback) at all." "It seems like some of the soldiers don't have attacking animation yet?" "Are these units attacking the selected enemy at all? I mean, they don't touch each other, they are still far from each other." "It looks like when my selected soldier attack an enemy, they don't look interacting with each other. But suddenly, either of them is killed."
Task 7	Player clicked the spawn unit button "Wait, where is my freshly spawned soldiers?" "Is it me or it doesn't work? The resources are not decreasing as well."
Task 8	"I like how the UI tell me how big the radius of the spell is." "I bet there's some kind of a cooldown system for this? But I can't see the count down or anything." "I love the spell animation as well."

Below are the summary of the answers from the interview with the participants. Some of the answers were not included here since they are basically saying the same thing.

Questions	Answers
1	"Yeah as I said, I think it's unnatural. Though, it's not an issue when you find out how it works." "It's nice. I don't find any problem with it. The only thing missing is the visual feedback for some units when they're selected." "Well, it works. But, as I said, it seems like some of them don't have walking animation and attacking animation."
2	"Again, the visual feedback when an enemy is selected to be attacked by the soldier is missing. I couldn't be sure whether my soldiers were chasing the selected enemy or not." "Still about the animation. Some attack animation are missing. Also, I saw some soldiers with no hp left floating around the map for some reason. Is it supposed to be like that?"
3	"I think the animation is awesome. It's easy to use as well. The crosshair helps a lot." "Yeah I love the animation. The only thing that needs improvement is the visual feedback to show the cooldown time. I mean, I wasn't aware of it."
4	"3" "4" "3"
5	"5" "5" "5"

There were several bugs found during the observation sessions. The bugs were:

1. The game quit unexpectedly when a player finish the tutorial
2. The barrack still unable to spawn soldiers and reduce the resources inventory

There were several suggestions as well. The suggestions were:

1. The control for selecting and moving units feel unnatural. But it doesn't cause any functional issue.
2. Lack of visual feedback regarding a soldier unit attacking an enemy. The selected enemy is not highlighted.
3. The distance between the soldier and the enemy when both of them killing each other is too far.
4. The spell mechanism is great. However, the cooldown visual feedback could be better since the participants couldn't see the count down
5. Some units still don't have complete animations

Evaluation

This testing round successfully answered the hypotheses asked. For the first hypotheses, it's true that moving and path-finding mechanism work as expected. However, some participants found that the UX was lacking. Although it worked fine, some participants didn't understand how to do it. For the second hypotheses, it's true that it improves UX. However, some units still don't have visual feedback. Lastly, for the third hypotheses, it's true that all participants found that the spell economics work well and easy to use. It's proven by how all participants rated the spell mechanism high (5).