Crystal Design Testing
User Test and Results Analysis (Programming) Test Details: The purpose of this test is to test functionality of implemented crystal functions and investigate user behaviour with the crystal.
Purpose: To ensure decreasing of the health status on the crystal is functional.
Method: To test the functionality of the above details, partakers of user testing here are required to follow the below gameplay instruction: Launch Atlantis Sink game and prompt for ‘Start Game’ Once gameplay begins, begin to familiarise yourself with gameplay, character controls and navigation. Locate the crystal and the enemies and take particular notice of the lifelines above these elements. As enemies begin to spawn, navigate your way through the map to avoid them until caught (on trigger for Sprint 1). Notice the effect of this on the crystal health status itself.
Results/Feedback Summary : Participant 1 Results: Success. Participant 1 paid close attention to the health status of both the character and the crystal itself; tended to take particular notice of the crystal health after noticing declining health status.
(Design) Test Details: https://github.com/UQdeco2800/2022-studio-1/wiki/Crystal-Design-Detail#internal--external-testing
Preliminary testing involves a delineation of two separate audiences - that is, those internal to the Atlantis Sinks project and those external to it. The purpose of doing so is to not only hear the insights of those with an understanding of the crystal’s role in the game, but also those who come with no pre-conceived ideas of the ‘who’, ‘what’ or ‘why’ of the crystal.
Preliminary testing hopes to elicit key insight into the expectations, of the crystal. Ideally, it should have the ability to answer the following guide question:
- Should the crystal design upgrade as the crystal itself upgrade? How should this be demonstrated in terms of design?
- Should the crystal design conform to the design styles of all other game elements or are there justified reasonings for it to alter in style?
- Will we be required to broaden our current colour-scheme?
- What are our participant’s general behaviours and attitudes toward the crystal idea?
To achieve the best responses to these questions, our approach to testing here is not quite verbal, but more so hands-on. Required by our participants is to draw their own iterations of the crystal.
As mentioned, our participants will be taking a more hands-on approach to testing here, requiring them to sketch their own iterations of the crystal. For our internal partakers, they use the basic, previously sketched crystal from Sprint 1 and iterate their thoughts for upgrade from there. However, for our external partakers, this test would look like a brief explanation/backstory of the function and role of the crystal followed by a blank canvas requiring them to ideate and sketch their own meaning of the crystal and perform upgrades of their own on their crystals from there. Inclusively, we as testers will have the critical role of observing the behaviours of our participants in extracting responses to guide question (4) above.
Overall, this preliminary user test can be applauded for its highly effective structure which offered not only an insight into the user's preferences, but additionality it's specific 'role' in Atlantis Sinks. With the focus of upgrading the crystal in Sprint 2 here, our designs aught to reflect this. What our users iterations suggested to us is to consider what comes with an upgrade - that is perhaps adverbs such as 'more', 'enhancing', 'levelling up'. For example, participant 1 sketched this understanding below.
Participant one suggested that 'more' means adding more dimension, more depth and hence more dominance over the gameboard. This helps us answer Guide Question 1 and 4 specifically. Participant 1 demonstrated an approach to upgrading the crystal by adding more dimension and their behaviour (that is, adding more depth and detail to their sketch) suggests to us a level of 'beauty' that should also be entailed. However, more tests had to be undertaken to resolve Guide Question 2 and 3 equally.
In all later tests conducted, an interesting finding was that no users felt the need to nor addressed any need in their sketches or comments for a change in the colour-palette adopted. This hints to us as designers more than the fact that the colour scheme may be appropriate, but rather it suggests detailing is what is critical. However, this leads us to question - why is detailing important? Analysing a comment made my a participant (Participant 3), she suggested the 'crystal has to be beautiful'. 'Beauty' in her sketches was portrayed through detail in shading.
This leads directly into answering Guide Question 2, suggesting that with such importance as a game element, the crystal will be required to essentially 'outshine' other game elements to demonstrate this dominance. This differs from our findings in Sprint 1, where it was intended to, as best as possible, conform to the common design approach as adopted by the rest of the game elements. Hence with this, from Sprint 1 to Sprint 2, the crystal has seen drastic improvement in dimension, shading, beauty, detail, interactivity and hence its importance as a key gameplay element. It's design successfully adopts a design portraying of its importance through added shading and 'shine' and dominance in Atlantis Sinks (please find sketches under Crystal Design Detail. Future tests (the Eye-Catcher below) hopes to validate our iterated designs of the overall Crystal and Crystal upgrade design.
The Eye-Catcher test is a rather rapid yet can be deemed highly efficient approach to testing the implemented crystal design. From ‘User Test Plan 1’, we found that participants saw great importance as well as dominance for the crystal which they expressed through their sketch iterations and behaviour. Using this understanding, we’ve altered the design from Sprint 1 to this.
As for the intentions of the Eye-Catcher test here, it is expected to be found key insights into the success of the crystal design. We hypothesise that the user will be immediately drawn to the crystal due to its significance in gameplay. This would occur if and only if the design of the crystal is able to properly capture the importance of the crystal in its design. Should it not be the item that catches the users eye at first glance, this will suggest potential flaws in the crystal’s interface. Method:
- Launch the game.
- Comment on the following:
- What is the very first thing you notice?
- What did you expect it to be?
- Did you expect there to be a change in the upgrade at all?
As conductors, it’s again not only required by us to take note of their verbal results and perspectives of the game, but its further required that we observe and comment on their behaviours during and after the testing session. Here this is most critical as the comments users make about the gameboard and the crystal within it will deliver insight into the emotions sent through the crystal.
With the purpose of more-so validating the design iterations inspired from Sprint 1's User Test Plan and Sprint 2's User Test Plan 1, results here gave insight into the success or pitfalls in the final adopted design. Our user's from User Test Plan 1 suggested to us the importance and dominance the crystal has as an individual game element. Our iterations hence reflect this, it was simply a matter of having the Eye Catcher test validate our design. Should user's find it doesn't conform to their expectations, flaws may pertain.
Of all tests conducted, the most concerning finding was that of Participant 3's. Participant 3 accredited the crystal for its 'intrinsic' design approach, however questioned the contrast between that of the crystal and blue water/background. According to this participant, his "attention is being fought for by both the crystal and the water at a 50/50 ratio". This hints to us as users that perhaps a further element is to be added to enhance the crystal's dominance over the gameboard.
Regardless of this, the remaining 4 test participants reported the crystal's design to be 'beautiful' and 'outshining'. Many complemented its design approach for the dimensions added and the consistent crystal shape adopted per fractal added. Suggestions were however noted for enhancing the size of the crystal to better its dominance over the gameboard.
The Think Aloud testing protocol is one many have adopted as a first tool for UX enhancement. It requires users to "think out loud" when participating in the user testing method, or in this case - exhibit. In the case of testing the crystal here however, the test will be accompanied with pre and post-exhibit questions to stimulate user ideation on the crystal and the crystal in terms of Atlantis Sinks specifically.
The purpose of the Think Aloud user test here is to retrieve key insight into the live user’s experience itself. With mid-exhibit questions centred around the crystal, the user will be provoked into the sharing their at-current thoughts on the crystal design within gameplay as opposed to stand-alone as in previous sprints, ceasing the flow of any pre-conceived prejudices. Ultimately, it is hoped to gain real-time emotions and thoughts of the crystal design within gameplay. For the sake of finding newer perspectives on the crystal, all test users must be external to the project and not tested in any of the previous sprints.
Stage 1 - Think Aloud (Exhibit) Pre-Exhibit Questionnaire:
- List words that you would associate the ‘crystal’ with in the game of Atlantis Sinks.
- List words that you would associate a ‘damaged crystal’ with in the game of Atlantis Sinks.
After prompting for ‘Start Game’, the user will be required to loop around the course of the map once. Following this, the user will be asked to upgrade crystal (instructions are able to be shared if necessary) and comment on their thoughts of crystal upgrade level 2. Again, the user must comment their views on the crystal upgrade level 3.
Post-Exhibit Questionnaire
- Edit your list of words (add more/remove/change) regarding the ‘crystal’ with consideration of Atlantis Sinks specifically.
- Again, list words that you would associate a ‘damaged crystal’ with in the game of Atlantis Sinks.
Stage 2 - Iterate
Once the user has had their go of the exhibit, the second stage of the user test is for the user to iterate their ideas for they would manipulate the existing crystals with the words they have comprised in their wordlists as well as the theme of Atlantis Sinks. Note that their iterations are to be sketched alongside the existing crystal.
The Think Aloud user testing protocol has seen great successes throughout many UX testing scenarios. It’s robust capability in prompting for qualitative feedback throughout any user test format has seen many projects evolve drastically for the better. In the case of testing the design of the crystal, this success was exampled to a degree.
Whilst most users engaged well with the game and commented the crystal for its contrasting colour, the words they described prior to the exhibit did not conform well to what the crystal truly was. For example, both Participant 2 and 3 described a crystal to be ‘transparent’ – concerningly for us as designers, the crystal demonstrated little to no transparency or attempt to transparency. Users did however comment on the crystal interface during upgrade- particularly in terms of its shading.
What concerned us however was the limited responses we received during actual gameplay. As mentioned under ‘Purpose’, the intention of Think Aloud is to have users comment during the exhibit. Regardless, the most rewarding aspect of the user test a whole was the intrinsic questions posed to users during completion of the exhibit. As an observer, it was clear to see users had been focused on gameplay and the crystal to a higher degree. We however noted that when it came to re-answering the question on listing words associated with a crystal in terms of Atlantis Sinks, users tended to retract several words associated with transparency and replaced these with textured.
Observing their sketches after this, particular for the damaged crystal – we as observers noted darker outlines alongside a darker pigment a whole. For the crystal a whole, shading was noted across almost all participant sketches. Using the ideas from this, we developed the crystal into the following:
Given the success of our Eye Catcher user test from the previous sprint, we have decided to use it here again as a tool for validation and hope for any further iterations. Within this, we are not only testing the design of the crystal, upgrade crystal and destroyed crystal but also the ‘juiciness’ of implementations with the popup, the newly devised pedestal and the overall integration style and how well we meet the design criteria specifications under ‘Crystal Design Details’.
As mentioned, the purpose is to validate the iterations performed on the crystal from User Test 1 as well as the installations of overall ‘juiciness’, to the popup and partial glistening effects. With our new enhancements, we hypothesise that the user will be immediately drawn to the crystal due to its significance in gameplay. This would occur if and only if the design of the crystal can properly capture the importance of the crystal in its design. Should it not be the item that catches the user’s eye at first glance, this will suggest potential flaws in the crystal’s interface.
As per the previous Eye Catcher Test:
- Launch the game - ask : • What is the very first thing you notice? • What did you expect it to be? • Did you expect there to be a change in the upgrade at all?
As conductors, it’s again not only required by us to take note of their verbal results and perspectives of the game, but its further required that we observe and comment on their behaviours during and after the testing session.
With greater success, the crystal can be proven the most critical element toward the game. Not only had our users demonstrated an understanding of this from the introductory Guidebook and Storyline, but in gameplay, observers were able to see much attention drawn to the crystal itself. Most rewardingly, we noted a greater eye for the crystal after it’s placement on the newly devised pedestal. A particularly intriguing insight was that of Participant 2’s – stating “the crystal could look as though its floating on the podium.” Unfortunately, this was not implementable due to timing constraints as well as overall issues with sizing when implementing in the code. As a team, we agree that future design directions would however hope to work on this on making the crystal 3-dimensional to conform better and truer to its role in Atlantis Sinks. Overall, the crystal design can be validated