1
Table of content Introduction (Katrine) 3 Problem definition (Katrine).3 Test Purpose (Katrine)..3 Test method (Katrine) 4 Use of Scientific Methods (Charlotte).5 Test Persons (Charlotte)..6 Test plan (Charlotte)..6 Test material (Charlotte).7 Environment (Charlotte)..7 Test setup (Charlotte) 7 Conducting the test (Katrine)..8 Data collection and data analysis (Katrine) 9 Reporting (Charlotte)..11 Conclusion (Katrine).12 Biography (Katrine)..13 Appendix 14 Appendix 1: Primary Persona (Charlotte) 14 Appendix 2: Profile of test persons (Charlotte).14 Appendix 3: Test plan (Katrine & Charlotte) 18 Appendix 4: Test documentation (Katrine &Charlotte)..20 Appendix 5: Minute and Click (Katrine).30 Appendix 6: Test report for the client (Katrine)...31 2
Introduction When addressing usability problems, we will need to know how to approach the issues. To be usable a product or a service need to be useful, efficient, effective, satisfying learnable and accessible. 1 This we need to remember when analyzing websites. We will need to look at the balance between human, context and activity on the website. There need to be balance between the three points, and it need to be integrated on the website. 2 When we know which points to measure a website on, we will need to find a website. We have chosen aarstiderne.com. It is an internet based shop, selling organic food, mostly fruit and vegetable. Aarstiderne.com is based on selling boxes with fruit and vegetable with weakly delivery. The website is selling boxes for 1-4 persons. The website offers also recipes, lunch boxes and activists. Now we know a bit about the website we will conduct test on, then we need to set some goal for the test. Problem definition When setting goal for the testing we are looking at which part of the website is the most important and which parts is secondary. Buying food is the most important and finding information, activities/courses, and the recipes are secondary. We will need to test the usability on both parts, but keep the focus on the most important. How easy is it for a new customer to buy products on aarstiderne.com? Were they able to find the information they needed to buy the products? Did they get frustrated and defeatist when navigating though the website? Did the website services satisfy the users? Test Purpose By looking and trying out aarstiderne.com we learned that it were difficult to find information on the different subjects. We thought that there was not enough focus on information on the vegetable, fruit, meat and so on. We learned that text, design and navigation were not in harmony. We wanted to make a usability test to test the usability of the website and find ways to make the product more attractive. First we needed to find the target group of aarstiderne.com and find out how we wanted to conduct the testing. 1 Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) p.3-6 2 Bailey's Human Preformance Model Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) p.7 3
Test method 3 To conduct the testing we chose to do individual interviews, a questionnaire and a think aloud test. There are a lot of different test methods 4, but we needed to find some test that could both analyze, design and test. Individual interviews 5 : To find test persons within the target group we first needed to set some criteria for the target group. We thought that the main target group is families with two or more members. They are living in Denmark and have a job. We chose to do individual interviews on people that fitted these criteria. Furthermore we added that people shouldn t have bought food on the website before (shouldn t be familiar with the website). We talked to people on their terms and made sure it were relaxed and then had a conversation were we talked about aarstiderne.com and their relation to it. We also talked about what usability testing is and their role in it. Questionnaire: We made a questionnaire in connection with the think aloud test. We made both a pre questionnaire that should give an image of the test person and a post questionnaire that should evaluate aarstiderne.com. The questionnaires were in Danish and each had around 10 questions. We made sure it didn t tooktoo long, to answer them and that they were easy to understand. Think aloud test: We made a think aloud test on aarstiderne.com to see if and were there are usability problems on the website. The test gives opportunity to observe the user in their own environment with minimal disturbance from the observer. We chose these methods because we believe that this will give us the best result on the circumstances. We are testing the website without making a buy and without cooperate with aarstiderne.com. The website is already running and functions therefore should a usability test, not change the concept but help the website to be more usable. If we had the time, we would conduct the think aloud test on 8 persons. We would also have made an open and closed cardsorting test on 6 persons. The cardsorting should be able to give us an idea of the hierarchy on aarstiderne.com. By using both people who know and don t know the website, we will see how difficult it is to navigate through the site, and how it could be easier. 3 Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) ch. 2 and http://usability.gov/methods/index.html 4 http://usability.gov/methods/index.html 5 http://usability.gov/methods/analyze_current/learn/individual.html 4
Use of Scientific Methods 6 : We want to introduce the theories and methods we are going to apply in our usability. We will reflect on quantitative and qualitative data compared to our test. We used scientific methods in our usability process to expand the analyticalways of thinking: Inscience, we questioned thethings.there must beanexplanation ofthe issues. Compared to the test we used both Hermeneutic and Positivism method in our work with Aarstidernes website. The technicalis about finding the solutions; you go from why to explain how. Positivism is a method used in nature science. Observations stand first to be scientific cognition when empirical data has been achieved through systematic collection. For positivism is science rational and objective. Theories in positivism are proved through observations, therefor is the objective verification. Hermeneutic a method used in human sciences1. The focus is on understanding text and interpretation what has been read. In hermeneutics the assumption is that you have some knowledge, which you will use. There is no neutral spot, you always be prejudice. The hermeneutics works with the idea that everything evolves, also your knowledge, therefore you will start with one assumption and as you get new knowledge you will get another assumption. The Hermeneutic works with seven principles, the most important is: hermeneutic is continuously moving back and forth, cause the idea is that every understanding is better understanding. The interpretation is done when a good gestalt is reached and that testing the parts though the work with the subject. We used Hermeneutic method because the test persons give their personal answer to what they think about Aarstidernes website. They use their knowledge in the Interpretative theory to navigate on aarstiderne.com, and try to find the subjects they were told to find. They express their own action to Aarstidenes website. Positivism is used in our Think aloud test. We have collected data from our pre-and posttest and have some specific data and proved fact in the test result. The theories we have on the test persons are based on our observations and we achieve empiricism knowledge. 6 Philosophy of Science: A Very Short Introduction 5
Test Persons: Target group From what we can see out of Aarstiderne.com their target group is private persons and companies. For private its mostly families their live a busy weekday. Aarstiderne.com make it easy for them, and deliver the food straight to the door once a week and can chose which box they want. For both private persons and companies is it people there be absorbed in Danish and ecologic food. They think about what they eat. Persona The benefit by using the persona is that it helps us focus on the users goals and needs compared to aarstiderne.com. We use the Persona in our process because it gives a picture of the design area and the empathy. We describe a person out of the user perspective when they are user at aarstiderne.com. We chose to use a persona in our project so we can identify us with the user. When we make the persona we take the most important elements for Aarstiderne.com and we make them into a Primary persona 7. Test persons The usability test of the website aarstiderne.com we chose people in the age range 29-52 years old both men and women. The test person is not a user of aarstiderne.com. We want to focus on the user friendliness and the new impression the website should give to new user. We used our resources to find relevant test persons with a job, and living together with someone. So we know that they have afford to bye food on the website. Aarstiderne.com deliver food all over Denmark, and on the basis of that we chose to test people on Fyn and in Jylland. Collected data from the pretest 8 Test plan 9 Before the test we made a test plan to structure our tests. Then we know that the test with high certainty being completely and we get the necessary data. We formulate the question to the test, and we used a combination of both open and closed questions. Then we give the test person opportunity to clarify what he or she means. We find out how we will prepare the tests that we use Think Aloud and use pre- and posttest to collect data. We make a valuation that the posttest will give us a great result in statistical terms. We disused 7 Appendix 1: Primary Persona 8 Appendix 2: Data from pretest. 9 Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) ch. 5 6
which places we would test in, and agreed on testing at the test person home. Test material We used a mac laptop to conducting the tests. We used both Silverback and Quick time player because Silverback was discontinued from the one laptop that we used because we installed it when we write the Milestone project. We use Quick time player instead with the screen and audio recording. Silverback 10 Quick time player Screen recording 11 Audio recording 12 Environment 13 Before we began the test we discuss the pros and cons by testing at peoples home. We think it is important that it is a calm environment with least disruption. We also know that when people normally use a website like aarstiderne.com they will use it at home. In this way the test persons can feel calm and it was their own safety environment. There is room for us, because we make an appointment with the test person, and we bring our laptops and some paper with us, and prepare it all for the test. The think aloud test is about the test person talks loud when they navigate on the website, and the observer make notes, and that help us to focus on the result. Test setup Before the test person started the Think Aloud test we asked the test person to fill a short questionnaire about their background 14. Then we have prepared a role and a posttest. The role is the test person who get instruction to find elements and navigate on the website. The posttest is were they explaining what their general impression is. During the test we were one moderator and one observer. The moderator was reading the task scenarios together with the test person and the observers were making notes of the usersbehavior during the tasks. We have the same procedure with all test sessions. We used our laptop with both Sivlerback and Quick time player for testing. The moderator was concentrated in the test and the user, 10 http://silverbackapp.com/ 11 http://www.macstories.net/reviews/lions-quicktime-player-screen-recording-improvements-and-new-sharing-features/ 12 http://mactips.info/2011/02/make-a-screencast-or-audio-recording-with-quicktime-player-10 13 Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) p. 93 14 Appendix: Profile test person 7
while the observer writes notes about the users behavior. When we have conducted the tests we have all the answers we needed to make our data collection. And we know what the test person thinks about the website, and what they think could be different. Conducting the test 15 After conducting individual interviews with the participants, we choose our test persons. We send an email to each person asking them on which days fitted them best and where it was easiest for them to meet. We agreed on time and place and send them a closing email with confirmation on date and time and general information on the test. When we arrived to the participants home, only the moderator spoke. For us it was important to respect them and their home. We explained what they should do and set up the test environment. We have asked the participants to have three chairs and a table ready for the test. This meant that they were able to decide where the test should be conducted. Our demand was that the room was big enough to fit participant, moderator and observer.when the moderator explained the participants about the test 16, the observer set up the computer and the chairs. We made sure that the participant knew how to use a mac computer. Some of them didn t and they got a quick tour on how they worked, and the moderator said that if they doing the test had problems they should just ask. Before starting the test, the moderator asked if they had any question. Then we were ready to start the test. The moderator started by asking the participants to fill out the prestudy. When he/she had completed the questionnaire, the think aloud test started recording. The moderator was mostly quiet doing the test, but many times it was necessary to help the participant finding the right way. The moderator helped by adding positive comments that motivated the participant to try a new way of completing the task. We made sure always to smile, not cross our arms and show we were committed. Even though the participants were frustrated the moderator was optimistic. When the participant had answered the question in the think aloud test, the moderator stopped the recording and asked him/her to answer the postquestionnaire. When the test session was over she asked the participant if they had any follow up question and then said thanks for their help, and went on to the next. In the time after the testing we compared our immediately findings. And made sure we had saved the material. 15 Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) Ch. 9 16 Appendix 3: Test plan 8
Data collection and data analysis 17 Hot spot analyze (worst problems): If we look at the most obvious, it was that the participants had difficulties finding the data. Many times they were very confused. They had an idea of where to look, but finding the right information often required many steps. Compile data 18 We collected all our data and but it into a folder. We watch the videos and made notes to it. When looking though the videos we observed different elements. We took each question were one wrote notes to how many steps it took and one wrote notes to the impression that the participant expressed. Summarize data When looking at the data we collected we noticed that the participants often got confused then navigating though the website. They couldn t find what they were looking for. Most of them said that they would just call aarstiderne.com and had given up long time ago if they were looking by themselves. The participants found different tasks difficult. One had difficulties with getting back to the frontpage. One gave up finding out if the food all were organic and Danish. And one weren t able to but the boxes in the basket. Even think placed on the frontpage, like deliveryday, took many steps to complete. Finding a box that fitted the participant needs was difficult. The names on the boxes didn t describe their content and how many persons it was for, were difficult to see. Some of the participants weren t able to open an account, and found this very frustrating. This is one of the most important think you should be able to do. Analyze data By looking at the raw material we have collected, we find that the participants all found the website very confusing. If it weren t for the test they would never have proceeded to buying on aarstiderne.com. If they wanted to buy the boxes they had contacted aarstiderne.com by phone. They all need general information to be clearer and easy to access. Right now is it difficult to find general information on the company and their politic, this need to be more accessible. 17 Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests (2008) Ch. 11 18 Appendix 4: Test documentation 9
The test has been completed by 5 persons. Here is see-through of the result. Q1. Is all food organic and Danish 2 people completed it with ease 1 person completed it with difficulty 2 people failed to complete the task Q2. What are you benefits as a user 0 people completed it with ease 1 person completed it with difficulty 4 people failed to complete the task Q3. How long shall you be a user if you create anaccount on aarstiderne.com 1 person completed it with ease 1 person completed it with difficulty 3 people failed to complete the task Q4. What shall you do with the wooden box 2 people completed it with ease 0 people completed it with difficulty 2 people failed to complete the task Q5. When will you get your food delivered 1 person completed it with ease 0 people completed it with difficulty 4 people failed to complete the task Q6. Add a SingleKasse and a DogmeKasse to the basket 3 people completed it with ease 0 people completed it with difficulty 2 people failed to complete the task Q7. Add 6. fl. Moulin de Breuil Chardonnay to the basket 3 people completed it with ease 0 people completed it with difficulty 2 people failed to complete the task Q8. Choose the fruit box that fit you and your family best 3 people completed it with ease 0 people completed it with difficulty 2 people failed to complete the task Q9. Where do you create an account 0 people completed it with ease 2 people completed it with difficulty 3 people failed to complete the task Q10. Create an account (do not approve) 0 people completed it with ease 2 people completed it with difficulty 3 people failed to complete the task Q11. In your basket this week there are Enoki mushrooms, but you don t know how to use them in your cooking. What do you do 4 people completed it with ease 1 person completed it with difficulty 0 people failed to complete the task Q12. This week you got oranges instead of grapefruits what is the reason 1 person completed it with ease 0 people completed it with difficulty 4 people failed to complete the task Q13. You got a melon and would like to know hold it stays fresh 1 person completed it with ease 0 people completed it with difficulty 4 people failed to complete the task In average it took 2 minutes to complete the different tasks 19. Around question 3, people started being annoyed. There had no patience and gave up on the task, often the just clicked in frustration. 19 Appendix 5: time and click 10
Reporting Result from Think Aloud test 20 The usability test of Aarstiderne.com is true experiments with five test persons. The test purpose is that we see individual response of the test. The website shall have a better hierarchy, the menus link have to be more visible and conspicuous for the user. Relevant info is something you have to search after, but because there is important information about the delivery and the food, it has to be more obvious at the front pages. The boxes with fruit shall have some other names, there is for example a box called the boring box and who thinks that is interesting to buy. It has to be clear for the user to see how many people the box is for, and what is in the boxes. We could have used closed card sorting to specify that. 1 pers. 2 pers. 3-4 pers. 4-6 pers. - Box 1 - Box 2 - Box 1 - Box 2 - Box 1 - Box 2 - Box 1 - Box 2 A very important thing to change is when the users add a box and they have to push the subscription button. It takes too many clicks, because the users think they can simply add the box with only one click. So generally shall the website be more simple and the categories need to be better placed and below under the appropriatemenus 20 HUT - Usability testing 11
Conclusion By testing aarstiderne.com we learned that the website have many usability problems. The website is already being used by many, but have potential for many more. The test participants we talked to were positive about the concept. They think that the idea is good. But when the visit the website the think it is messy and confusing. None of them have ever tried to buy on aarstiderne.com, and after the test none of them would do it. If they want to buy food they would call. They all found it difficult to find general information on the concept and on fruit and vegetables. The participant got annoyed when finding the right boxes. They didn't feel like the name described what was in the box. Furthermore adding the box to the basket were difficult for most of the participants, the just pressed on læg i kurv. When the test was completed felt defeated and had given up on the website. To make the website more usable there will need to be a clearer information guide. Except for buying the fruit information should be one of the most visible elements on the website. The front-page have many elements right now, making it simpler will bring focus on the basket and will be less confusing. Name the menu points so everybody knows what to find under them the same applies for the box names. By adding hierarchy on the aarstiderne.com will it highlight the most important and give people possibility to explore the website instead of getting lost in it. 12
Biography Online Aarstiderne.com (31.5.2012 kl. 15.50) http://www.aarstiderne.com/ Usability.gov (31.5.2012 kl. 15.50) http://usability.gov/methods/index.html Usability.gov, Individuel interview (31.5.2012 kl. 15.50) http://usability.gov/methods/analyze_current/learn/individual.html Silverbackapp (31.5.2012 kl. 15.53) http://silverbackapp.com/ Mactories (31.5.2012 kl. 15.53)http://www.macstories.net/reviews/lions-quicktime-player-screenrecording-improvements-and-new-sharing-features/ Mactips (31.5.2012 kl. 15.53)http://mactips.info/2011/02/make-a-screencast-or-audio-recordingwith-quicktime-player-10 Offline Philosophy of Science: A Very Short Introduction. Okasha, Samir. 2002. Oxford university press Handbook of Usability Testing, Second Edition: How to Plan, Design, and Conduct Effective Tests. Rubin, Jeff et.al. (2008) 2. Ed. Wiley Published, Inc. 13
Appendix Appendix 1: Primary Persona Name: Merete Holm Bangsø Age: 32 Lives in: Brøns, Skærbæk Merete loves to exercise and she absorbed in what she eats every day. That s why she uses Aarstiderne.com in her daily life. It is a big help because they deliver a box with food straight to the door once a week. She spends money on buying healthy food and she focus on organic food because then she knows that it is clean ingredients. Merete live together with her boyfriend Lars and they don t have any children. Lars likes the ingredients to because he uses them in his cooking. Merete travel a lot because of her work as purchasing manager, and she think it is a great option that you can skip a week delivery or only deliver food for Lars, and then you don t get unnecessary goods. Appendix 2: Profileof test persons 14
15
16
KØN ALDER 4 3 3 2 1 0 Dame Mand 2 1 0 20-29 30-39 40-49 50-59 Køn Dame Mand Medlemmer i Husstanden Hørt om Aarstiderne.com 4 2 0 1 2 3 4 4 3 2 1 0 JA NEJ Dame Mand Dame Mand Brugt Aarstiderne.com Ansvarlig for indkøb 4 3 2 1 0 JA Dame Mand NEJ 2,5 2 1,5 1 0,5 0 Er ansvarlig Delvis ansvarlig Ikke ansvarlig Dame Mand Internetbrug 4 3 2 1 0 Hver dag Næsten hver dag Hver anden dag Sjældnere Dame mand 17
Appendix 3: Test plan Hej Vi er Charlotte og Katrine. Tak, fordi i vil hjælpe os med at teste. Som i ved regner vi med at testen tager ca. 45. minutter. Katrine vil gå i gang med at gøre klar til testen hvis i vil vise hende hvor det passer bedst. Vi vil starte med at bede dig udfylde dette spørgeskema. Derefter vil vi forsætte med test delen på computeren og tilslut er der er afsluttende spørgeskema. Som skrevet i mailen, vil filmen blive brugt i forbindelse med vores eksamens projekt. Har i nogle spørgsmål ellers tror jeg vi er klar til at gå i gang. (Pretest) Navn: Køn: Mand/Kvinde Alder: Postnr.: Hvor ofte burger du internettet: Hver dag/næsten hver dag/hver anden dag/sjældnere Har du en Iphone/Smartphone/bærbar/Mac/Computer/Tablet/Ipad/mobil - Uden internet adgang/mobil - med internet adgang. Antal I husstanden: 1/2/3/4/5/6+ Hjemmeboende børn Ja/nej Hvem er ansvarlig for dagligvarer indkøb: Jeg er ansvarlig/ jeg er delvis ansvarlig/ jeg er ikke ansvarlig Hvor køber du normalt dine dagligvarer: Supermarked/købmand/kiosk/special butik/grønthandler/slagter/bager/internettet/ Har du hørt om aarstiderne.com? Ja/nej Har du brugt aarstiderne.com? Ja/nej Hvis du kender aarstiderne.com, hvad er dit indtryk af konceptet? ----------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------- 18
Usability test: Think aloud test: Role: Du besøger hjemmesiden for første gang, med det ønske at købe frugt og grønt til husstanden. I lever sundt og vil derfor se hvad aarstiderne.com tilbyder. Gå ind på hjemmesiden www.aarstiderne.com Start med at beskrive forsiden ud fra de første følgende spørgsmål: 1. før du trykker videre: Fortæl hvad er dit første indtryk af aarstiderne.com? 2. før du trykker videre: Fortæl hvad siden handler om/ hvad den indeholder 3. før du trykker videre: Fortæl hvad du kan købe på siden 4. Er alle varer økologiske og danske? 5. Hvad er fordelene når man er burger 6. Hvor lang tid binder jeg mig for når du opretter dig som bruger? 7. Hvad gør du med trækassen når den er tømt for varer? 8. Find ud af hvornår du kan få maden leveret 9. Tilføj en SingleKasse og en DogmeKasse til kurven. 10. læg 6. fl Moulin de Breuil Chardonnay i kurven 11. Vælg den frugt kasse der passer dig og din familie bedst a. - hvorfor valgte du netop den? Nu kender du lidt til hjemmesiden, og vil nu gerne tilmelde dig. 1. Hvor tilmelder du dig henne? 2. Opret dig som bruger (Du skal ikke godkende) 3. Du har i denne uge fået Enoki Svampe med, men ved ikke helt hvordan du skal bruge dem. Hvad gør du? (brug hjemmesiden) 4. Du har nu modtaget en frugtkasse men har fået appelsiner i stedet for grapefrugter hvad kan årsagen være til det? 5. Du har modtaget en melon og vil gerne vide hvordan den holder sig bedst Posttest: Hvad var dit første indtryk af hjemmesiden? 10(godt)----------------------------------------------1(dårligt) Kommentar------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------- Var det nemt at finde hvad du ledte efter? 10(meget nemt)----------------------------1(meget svært) Kommentar ------------------------------------------------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- Er hjemmesiden brugervenlig? (Ved du hvordan du skal navigere rundt på siden, og tror du det er nemt for alle) 10(God)----------------------------------------------1(dårlig) Kommentar ------------------------------------------------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- 19
Hvad syntes du om designet på hjemmesiden? 10(godt)----------------------------------------------1(dårligt) Kommentar ------------------------------------------------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- Kunne du tænke dig at købe mad via aarstiderne.com 10(Meget)----------------------------------------------1(Slet ikke) Kommentar ------------------------------------------------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- Opfylder hjemmesiden dine behov og forventninger? 10(Meget)----------------------------------------------1(Slet ikke) Kommentar ------------------------------------------------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- Hvor let var det at oprette dig som bruger? 10(meget nemt)----------------------------1(meget svært) Kommentar ------------------------------------------------------------------------------------------------------------------ ----------------------------------------------------------------------------------------------------------------------------------- Hvad syntes du mangler på hjemmesiden? Kommentar------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------------------- Hvad vil du gerne ændre? Kommentar------------------------------------------------------------------------------------------------------------------- Appendix 4: Test documentation 20