Testing and Validation
To justify the feature of the guidebook and ensure it is needed for the game, user-tested was completed on the final result from sprint 2. The testing that occurred was observational testing, which meant that I let the users play the game and recorded what they did and where they got stuck. I tested a total of five people to help establish what is intuitive in the game and what is not.
I started the game for the users, and let them play through the game while I watched. I took notes of what they did, where they got stuck and what they said while playing the game to help establish emotion.
After they finished working through the game and said that they don't think there are any more aspects to explore, I explain how to actually use the game.
From testing five users, only two figured out how to make the main character move, and attack. No users figured out how to or even that you could place buildings. Many of the users just kept on clicking around the screen expecting something to happen. One user went into the shop five times trying to find something useful to help the player move.
From this many of the users did not figure out what the point of the game was, thus, the game is not intuitive. This means that the game does need instructions and a guide to know how to play and win the game.
To best present the information in a meaningful and helpful manner, we conducted user testing on our low fidelity guidebook screen wireframes to understand which format was the most legible, captivating and suitable for the target audience. The following testing plan outlines the tests undertaken and how we measured success to produce the best outcome for users. See this link to view low-fidelity prototypes and more about wireframes. From left to right --> Options 1, 2 and 3 respectively.
The testing undertaken was customer preference testing. Whereby users are given three options to choose from and must rank in order of which they like the most, with 1 being their first preference and 3 being their last preference. After scoring the options, the user must respond with an explanation for their ranking. Preference testing is fundamental to grasping users' dislikes and likes early on.
-
Preference: 2,3 & 1.
Reasoning: "I really like the idea of creating a book as the layout adds character to the aesthetic and ties in the idea of the oceanic adventures out on the sea with those old leather sailing books. So that’s why I chose 2 and 3 as my top preferences and made 1 last as it doesn’t have as much of a special effect." -
Preference: 2,1 & 3. Reasoning: "Just by looking at option 3, I have never seen anything done like that, and I’d say it would be because one it’s hard to produce and two people would prefer to read a flat book than a sentence or a paragraph that ascends upwards. Although it’s creative, I think for game instructions I just want it plainly laid out for me, like given directly. So that’s why I went with 2 as you still get that cool book design but, it’s easier to quickly read the instructions."
-
Preference: 2,1 & 3. Reasoning: "Option two appeals to me the most as it is similar to how I read on my iPad. Which is for those who don’t know a flat top-down view of the text. I guess it works the best as there is no curvature or effects that make it difficult to read, and that is probably why I put 3 last. I don’t mind option 1, but it is a bit bland and I personally appreciate games that make the effort to intertwine their game theme throughout all interfaces."
-
Preference: 1,2 & 3. Reasoning: "I strongly dislike the angled book layout (option 3) as it is hard to read text angled like that. If I were playing a game, I’d find it a pain and almost nauseating to have to tilt my head to read text. For that reason, I prefer the 2D format of both 1 and 2."
-
Preference: 2,1 & 3 Reasoning: "I like the middle option as the layout is the most intuitive. What I mean by this is that I can easily find the exit button, see how to flick through pages and the content on the right-hand side is presented neatly. I ranked the first option second as I also like the simple layout of that option. But I just didn’t enjoy the buttons “next and prev”, I prefer the arrows so I went with the middle one."
After gathering the scores, we found that there was a strong favour towards option 2. Option 2 has a flat view and is made to appear like an old-fashioned novel. The users leaned towards this option as they found it was the perfect mix of integration of the game and legibility. There was a common preference to be able to quickly read the information and return to the game, and not make the process a hassle. Additionally, the option still captured the theme nicely with a lot of the users after the testing saying it would also look great when produced with the pixel design. Therefore, customer preference testing was very beneficial in grasping how users would like the guidebook to be organised and depicted.
A big task for the guidebook was arranging the topics put forward by our studio members in a logical order that a user would expect to view the information. There is no better way than hearing what users themselves have to say by doing some testing. Therefore, we asked users to arrange the topics in how they'd like to view the content which is relative to their first experiences playing the game. Since the topics are subject to the studio's opinions, we gave the user the ability to transform the topics by adding or removing topics too.
Please list in order (1 being the first option, 2 being the second option, and so on…) the order you would like to view the game information from the guidebook. Some motives for ranking could be the importance you associate with the topic or logical timeline or sequence of the game. We also have the option to ‘not include’ a topic; do so by placing this topic name in the red cell. If excluded from the guidebook, please give a valid reason why. And if you identify any elements you wish to add to the topics, please add them in the green cell and provide a reason why and where you’d place them.
See the image below which outlines the feedback from user testing.
The user testing brought to light the irrelevance of certain topics and a common understanding of what users like to know about in a particular order. The majority of users didn't think cut scenes, camera-related adjustments, and NPCs were important enough to be outlined in the guidebook. More specifically, users were perplexed as to why cut scene controls exist and said just to give users the option to skip. And for camera-related adjustments, they should just be found in settings and don't really require a full-on explanation. In terms of the generally agreed-upon ordering of the table of contents for the guidebook, it seems as though users would like to know about the crystal and main character, then who they have to protect themselves from (enemies) when to expect the enemies (night cycle), what to do in the day cycle (buildings, shop etc). A final version of the list will be presented below. However, we have also manipulated the table of contents topics to include suggestions from users, which were generic how-to-play instructions, currency and the final boss level. These were added based on frustrations and pain points experienced by the users when playing the game and not having enough information.
Table of Contents - Revised:
- How to Play
- Crystal
- Main Player (their movements, health, lives, and objectives)
- Day and Night cycles
- Enemies (Generic Enemies and hint to boss level)
- Types of Buildings
- Resource Buildings
- Currency
- Shop resources
- Inventory