MSN.com

Quantitative Research
Goal
Improve the site navigation
Methods
Card sort, tree jack test
Contributions
  • Organized and conducted card sort
  • Conducted tree jack test
  • Analyzed data to simplify site navigation
Overview
MSN.com organized a lot of information from a variety of categories. Because of this the navigation can be confusing.
Research offers:
  • Improved navigation
  • Better categorization of difficult items
  • Statistical evidence
The Process
Four
We conducted a card sort followed by a tree jack test to identify areas of confusion and offer guidance for a better categorization.
  • Card sort
  • Data analysis
  • Tree jack test
  • Data analysis
1.0 Card Sort
Sumitaive
We started with a closed card sort to identify areas of confusion in the navigation.
  • The original navigation had 15 categories and 144 sub-categories.
  • The study had 10 participants.
  • Participants were asked to match labels to the headings that fit best.
  • Pre-survey questions were designed to gathered information about where participants typically sourced their news, what type of news they were interested in, their frequency of use for msn.com, and their impression of the site.
2.0 Data Analysis
Overview
Analysis included agreement score, dendrograms, and multidimensional scaling.
Agreement Score
The agreement score indicates how much agreement there was among participants in sorting the categories. The higher the agreement score, the less confusion there was.
Dendrograms
The Ward and Centroid Dendrograms gave us the same 10 groupings, which correlated with the 10 different categories.
Multidimensional Scaling
A Multidimensional Scaling chart confirmed the grouping. The colored groupings in the chart represent each category from the dendrograms.
3.0 Tree Jack Test
Overview
We wanted to better understand how the number of categories would impact the user experience. We fourmulated three hypothesis.
  • Hypothesis 1: A menu with fewer category listings would allow users to find a specific article faster.
  • Hypothesis 2: Users would find the navigation less confusing when a site has fewer category listings.
  • Hypothesis 3: More users will find "careers" under "lifestyle" than "money."
  • The experiment consisted of between-subject tests created in optimal workshop. Each test had 10 participants.
4.0 Data Analysis
Overview
Analysis included time on task, self-reported confusion, lostness, and success rate.
  • The results of time were not significant, but the task with fewer categories generally took less time.
  • More participants rated the test with more categories as somewhat confusing or extremely confusing.
  • In both perceived confusion and lostness the number of categories did not make a statistically significant difference. More users found the target on the test with fewer categories.
Task 2 Pie Trees
We see more branching in the tree test with more categories, indicating more confusion.
Task 1 Pie Trees
The pie trees show that the majority of the people found the correct path the first time.
Task 3 Pie Trees
The branch with lifestyle or health and lifestyle is slightly preferred on each tree.
Limits & Challenges
Limitations
  • Low sample size- The sample size was not large enough to show a statistically significant difference.
  • Optimal Workshop Limitations- We were limited in the number of categories and sub-categories that we could test due to the software used.
Challenges
  • It was challenging to take the many points of data we gathered from questions to statistical data and discover the underlying patterns.
Results & 
Reflections
Results
  • Although the tests did not show statistical differences, we are able to show that data points towards a better experience with fewer categories.
  • It was interesting to dive into statistical analysis for this study. In the future I would like to have more participants for this study.
Copyright Marissa Thompson 2021
linkedinInstagram