user research - prototyping
Role & Team
One of two designers.
Worked with design lead, head of product, developers, researchers, tutors, and program administrators.
Key Skills
A/B Testing & Think Aloud Interviews
Designing for competitive stakeholder interests & conflicting user needs.
My Impact
I was an integral part of every step of our process; I ran interviews to uncover key insights, analyzed data from hundreds of responses, facilitated effective critique sessions, drove decision making, and flexibly adapted our designs at each challenge we faced.
PLUS: a math tutoring software & program.
500+ tutors
3000+ students
100 sessions per week
The Problem
The existing reflection form was usable, but not useful.
The previous reflection form had gone out of usage because while it had good usability, it didn't actually collect useful data.
The Solution
A personalized form collecting actionable data to improve the program & useful reflection for tutors.
We designed a new form that collected the data our stakeholders need while still making it easy & helpful for tutors to fill out.
Defining the Problem Space (1/5)
Identifying & balancing stakeholder needs.
We came into this project with the ambiguous request to redesign the existing reflection form. The old form had been easy to fill out, but didn't collect the data needed to accomplish our stakeholders' needs. To define those needs, we interviewed our 3 stakeholders: tutors, admins, and researchers.
Stakeholder Need #1
Tutors needed to communicate issues through the form, as well as for the questions to help them reflect and improve.
Stakeholder Need #2
Admins needed to identify specific areas tutors were struggling in and subsequently how to adapt the program and training that’s given to tutors to address those.
Stakeholder Need #3
Researchers needed to measure how effective different lessons and the form itself are in improving tutoring skills.
The needs of the admins & researchers overlapped, but competed with the tutors' needs.
Admins & researchers needed to collect quantitative data that could be analyzed to detect patterns. Tutors benefited from qualitative questions that helped them reflect on how to improve.
To address this, we applied the strategy of prioritization & accommodation.
We chose to prioritize the admin & researcher interests, as our highest priority was to identify areas of improvement, while still accommodating tutors and making the form as easy and interesting to fill out as possible.
A/B Testing & Data Analysis (2/5)
Usability testing 2 prototypes to gauge conflicting user needs.
To quickly get feedback, we used Google Forms (no dev effort needed!) to obtain a large sample of feedback, which we used to better inform our designs.
Our Head of Product and us had conflicting design opinions, so we tested 2 versions of our prototype to better inform our decisions.
Our Head of Product was concerned that our version would take too long to fill out and strongly advocated for a different question format to make the form faster.
Over the next few weeks, we ended up with 570+ form responses.
We analyzed them to surface usability problems & make an informed decision between the 2 versions.
Response patterns varied between the 2 versions, indicating that form structure influenced the tutors.
Our Head of Product was concerned that our version would take too long to fill out and strongly advocated for a different question format to make the form faster.
Analyzing quality of response by question helped us determine which to keep vs. cut.
Certain questions provoked far more lengthy & detailed responses than others, indicating them as more useful for tutor reflection. User interviews showed that tutors preferred spending time on questions they deemed useful.
Think Aloud Interviews (3/5)
Conducting user interviews to identify conflicting user needs.
Our data analysis showed that not only was there a wide spectrum of users, the types of question created interaction effects with different kinds of tutors. To better understand how users were thinking, we structured the interview protocol to compare the 2 versions.
Our interviews supported that we had a wide range of users whose needs conflicted.
The “get it done” end needed to fill out the form as quickly & easily as possible, often during the session, while the “get it right” end needed the affordance of reflecting in detail after the session.
To design for the entire spectrum, we again applied prioritization & accommodation.
We chose to prioritize the “get it right” users, because that’s the kind of behavior we wanted to encourage. But, we still wanted to accommodate the “get it done” users.
Key Insights & Design Decisions (4/5)
Key insights informed changes to the form's navigation, structure, and more.
Key Insight #1
Some tutors need to fill out the form during tutoring sessions, while others needed to fill it out after (conflicting need).
Design Decision #1
Navigation structure enabling multiple different sequences.
Using a sidebar promoted filling out the student section first, helping tutors write things down while their memory was fresh, while still allowing tutors to fill out the form in any order.
Design Decision #2
Responsive design to allow side-by-side filling out.
Using a sidebar promoted filling out the student section first, helping tutors write things down while their memory was fresh, while still allowing tutors to fill out the form in any order.
Key Insight #2
Some tutors were more motivated to fill out the form, even when it was longer, if they deemed the questions more useful.
Design Decision #1
Personalization to improve form experience while minimizing effort needed to fill it out.
Each question a tutor is shown depends on their answer from the previous question, ensuring that tutors only see relevant questions, allowing them to better reflect on that specific experience while also reducing the effort needed to fill out the form.
Design Decision #2
Progressive disclosure to prevent information overload while reducing clicks.
Some tutors preferred having the content split up between many sections, allowing them to better focus, but others didn’t like to have to click so many times. To address these conflicting needs, we applied progressive disclosure - thus preventing information overload while also reducing clicks.
Key Insight #3
Nearly all tutors had difficulty remembering student names & interactions after the session.
Design Decision #1
Recognition over recall for selecting student names.
The date picker is autofilled to the current date to save time, but depending on the date selected, the dropdown below is populated by sessions that happened that day. When a session is selected, student names are pre-populated from those registered for that session.
Learnings (5/5)
Learning & applying how to be a more effective team.
Throughout this project, my team and I reflected many times on how to collaborate & communicate more effectively with one another, allowing us to keep growing throughout the entire process.
Learning #1
Organized documentation from the start.
This project hammered in the importance of establishing an organized way to document EVERYTHING from the very start of any project - I hadn’t realized just how important that was until this project because it was both remote and quite on-and-off, interspersed with schoolwork, so this was an issue we faced midway through.
Learning #2
Align on scope, purpose, and deadline before even starting.
This was a lesson in just how critical it is for not just myself, but my entire team to have a clear & aligned understanding of the scope, purpose, and deadline of a project before starting and to use that as a north star to guide decisions.
Learning #3
Always dig into feedback to uncover 'why'.
The design reviews and meetings in this project were great exercises in digging deeper into provided context or feedback, and always asking “why” to figure out the motivations behind what people say.
