Informing Design with Data
—
A Story by Ryan LaBouve (@ryanlabouve)
I wanted to do this talk on the sheer power of quantifiable data driving design
Pump through a few hundred thousand views.
Show half Button A and half Button B.
Track conversion, whichever one wins is better
But...
What if both of the options are just plain wrong?
You've just decided which of the wrong options is better
Then all I've shown you is how to make the best wrong decision
Instead...
This talk will guide you through the stages of making objective design decisions.
Using data.
Ryan LaBouve
Today's Agenda
- Problem Definition
- User Research
- Design Sprint
- Technical Prototyping
- Measure, Audit, and Plan with Numbers
Problem Definition
Problem Definition
Defining a problem w/ Business, Design, and Engineering
How it Informs Design
Unifies the design product vision with information from business, design and engineering
A problem well-stated is half-solved.
—John Dewey
A quick tale about this...
In an agile shoppe close to you...
Grab next open task from JIRA
Design the Dashboard (3 points)
What's Wrong with this?
How did this happen?
We need a dashboard
—Business
I have no idea what this means... Let's just find some cool dashboard screenshots in dribbble, and brand them for our company?
—Design
We literally have none of this data... Let's just cut out what doesn't make sense.
—Engineering
This misses like... every critical business goal. There is 0 ROI.
—Business
This should have been obvious, here's a lot more detail about what we actually want.
—Business
Re-designs...—Design
Re-engineers...—Engineering
This wastes a lot of cycles and is a common story in many companies
Lets go back to the start of this conversation
We need a dashboard
—Business
Problems are not Solutions
We need a dashboard
—Business
And this is a problem
Which largely lacks definition.
And we have to work together to do this.
Defining a problem Step 1:
Get the right people involved
Defining a problem Step 1:
Get the right people involved
Create a document answering...
- What are you making?
- What problems are you solving by doing this?
- How do you design success?
It's important that the whole group comes to terms here.
This is the time where it's cheap and easy to ask questions, disagree, etc.
Refer to this document throughout the lifecycle of a project.
It can help maintain focus, and prevent scope creep
Project Definition
Defining a problem w/ Business, Design, and Engineering
How it Informs Design
Unifies the design product vision with information from business, design and engineering
User Research
Defining who your users are and communicating that with the team.
How it Informs Design
Users are the lens through which we look to evaluates whether a design makes sense or not.
Users are the lens through which we look to evaluates whether a design makes sense or not.
#Longread
'There is this company that I worked with a couple of years ago that were doing coupons. Their site looked a little crazy. We said, “We can make this better.” We redesigned all the visual design... We put it back in front of customers, and they looked at this new, fancy visual design that looked really good, and they said, “Wait, who’s behind this thing? Are people making money off of me?” We realized that we screwed it up. We tried to make it beautiful for us, but in doing so, we broke the product.'
https://blog.intercom.io/podcast-braden-kowitz-talks-design-and-startups/It's important to know our users
Users.What comes to your mind?
Google image search: Users
Tons of blank faces...
Knowing our users helps us frame our problems to our solutions.
Know our users help us know we are solving the right problem, the right way.
“We wanted to find out who really uses MailChimp. It was a question posed to us by data analyst Allison last year. We could broadly generalize about our users (savvy, self-reliant, techie, motivated), but we realized that we couldn't rattle off the four or five archetypal MailChimp users.”Jason from Mailchimp
- Step 1: Interview Steakholders to see who we assume our customers are.
- Step 2: Rank our pool of active users by industry
- Step 3: Identify subjects from popular industries and interview, interview, interview
- Step 4: Analyze what was seen and heard
- Step 5: Share findings with team
Step 1
Interview Steakholders to see who we assume our customers are.
Step 2
Rank our pool of active users by industry
Step 3
Identify subjects from popular industries and interview, interview, interview.
How we interview users.
Step 4
Discuss what was seen and heard
Who is involved here?
Step 5: Deliver finding to company.
Examples of Hi-fi personas
https://blog.mailchimp.com/new-mailchimp-user-persona-research/
Sketched Personas
- http://www.uxdesignedge.com/2011/06/personas-dead-yet/
- http://www.uxapprentice.com/discovery/
- http://www.slideshare.net/toddwarfel/data-driven-design-research-personas
- http://www.fastcodesign.com/1671571/4-secrets-for-doing-gonzo-user-research
- http://www.uxbooth.com/articles/complete-beginners-guide-to-design-research/
- http://uxmag.com/articles/personas-the-foundation-of-a-great-user-experience
Design Sprints
Utilize your whole team: The quick way to solve hard design problems, using your whole team.
How it Informs Design
Decide the right solution to build, and then build it. (Instead of deciding on a solution to build, and then figuring out you were wrong.)
Best application: Large new features/products
http://www.gv.com/sprint/Sometimes the best solution will come from the least likely spot
If you are depriving teams or employees from participating in the design process, you are missing your best ideas.
Give everyone on your team a way to take part in design process
The 5 Day, 6 Stage Sprint (GV)
- Understand
- Define
- Diverge
- Decide
- Prototype
- Validate
Product Definition
We already did this!
Brainstorm Alone
Empower everyone to do this, not just designers.
Your most surprising and innovative ideas will come from the least likely of places
A few ways to empower individuals to brainstorm.
Especially those who don't feel "that creative"
Bad Sketching
Group Convergence
Sticky Notes & Dot Voting
How a design shop might do this:
- The facilitator presents the design on a big display, or prints it out and put it on the wall.
- We go around the table, over and over again. During their turn, each person can share one (and only one) tension she or he has with the presented design, elaborating on the tension to the extent they deem necessary.
- Tensions don’t have to be exclusive — you can repeat or add to another person’s prior tension.
How a design shop might do this (con't):
- Or you can skip (pass) your round if you can’t think of a tension at this moment.
- Cross-talk (other people speaking out of their turn) is discouraged, unless it is used for clarification (“Wait, what did you mean by…?”).
- The whole thing is capped at 20 minutes, or whenever we run out of things to say (everybody in the room skipped their round in sequence). The facilitator takes care of the order and keeps the meeting running smoothly.
Some of our personal rules (at Raisemore):
- Leave your ego at the door
- Strong oppinions held losely
- Be kind, or be quite
Prototype
"We’re going for “goldilocks level fidelity; just good enough to suspend disbelief."
This also give design a good place to start working.
Validate
Talk to those users we defined earlier!
(Prototype, User-Satisfaction) => { Future Direction }
Design Sprint
Use the whole team to explore a design problem.
How it Informs Design
Produces prototypes, validated by users, agreed upon by your team that help you avoid costly mistakes.
Effective Prototyping
Building highly expressive prototypes to help
How it Informs Design
Determine whether an idea is worth investing in to build using minimal resources.
We talked a little about prototyping in the last slide.
"We’re going for “goldilocks level fidelity; just good enough to suspend disbelief."
Low Fidelity = Primitive Form
Old Version = Primitive Form
Prototype = Smoke and Mirrors
Potentially, All 3!
But today, we'll focus on smoke and mirror...
In a previous job, I worked for OU...
Boss: We are re-designing the OU homepage. We need a final direction on design by tomorrow, and would love your oppinion.
Me:...
Original Site
TL;DR
This is what I delivered the next day
MGMT's Reaction
This truth is, all of this took 1 hour...
Smoke and Mirrors via Dev Tools
Modern, with Traditional Roots
...Harvard
Sturdy and Approachable
...Bates
Techie and Scientific
...Michigan
Classic
...Cornell
Rock Magazine
...Whatever this site
Honest and Trendy
Australian Catholic University
Effective Prototyping
Trying other solutions on for size
How it Informs Design
Quick, cheap, and surprising helpful in making decisions.
Measure, Audit, and Plan with Numbers
- We defined the problems
- We know our users.
- We are using our whole team to design future features
- We have prototyped and built features/products
Everything, to this point, was a precursor for...
Using Numbers!
Numbers are good at telling you what's not working, or what to work on next.
Not how to fix something.
- Track Stuff (lots of it)
- Measure Changes
- Make it accessible to folks
- Use it to help you make decisions
Your Data Is Your Lifeblood — Set up the Analytics It Deserves
Ben Porterfield http://firstround.com/review/your-data-is-your-lifeblood-set-up-the-analytics-it-deserves/"Seriously, Do Not Wait. As soon as your company has users, you need to set up a solid analytics framework. It's not a waste of time or money."
Ben Porterfield http://firstround.com/review/your-data-is-your-lifeblood-set-up-the-analytics-it-deserves/DON'T FLY BLIND!
Basic Usage
How much traffic did we get last week from facebook?
What was the response like from our email campaign this week?
What was the response like from our email campaign this week?
Segmentation, Funnels, Retention, A/B Testing, Etc.
Tracking Funnel, allows you to measure changes and improve funnel
Awesome Resources
Lean Analytics
Intercom on Product Management
Expanding this to Guide Roadmaps
https://blog.intercom.io/before-you-plan-your-product-roadmap/
https://blog.intercom.io/wp-content/uploads/2012/07/Graph-Starred.jpg
For features, it's easy to think:
https://www.intercom.io/books/product-management
What you will likely find:
https://www.intercom.io/books/product-management
Possible Disruption:
Measure, Audit, and Plan with Numbers
Track everything.
How it Informs Design
Numbers can tell you what's working, what's not, and what might be worth it.
Not everything that matters can be measured, and not everything that is measured matters. — Elliot Eisner