The deadline for YCombinator applications was this week and Paul Graham, the founder of YC tweeted that if he had to distill the most important advice he has for startups it would be the title of this post: “Explain what you have learned from your users” It may sound trivial, but many products/startups that no longer exist have ignored this advice at their peril.
In his own words:
“Even if you only thought of the idea yesterday, you can still talk to users. In fact, that’s the first thing I’d do after working out the first version of most ideas: run it by some potential users to see how they reacted. Incidentally, it won’t carry much weight if you merely say “Potential users love the idea!” You have to say what you *learned* from users. I.e. in what ways your initial model of them was mistaken. If you can’t answer that, it’s probably not because your model was perfect.”
Paul Graham, Ycombinator
As startup founders ask themselves this question, some of the answers might be specific to their product: i.e. we should improve our onboarding, change our freemium model, etc. But many of the answers are true universally, and have to do with how startups collect, digest and act on user feedback.
So I took a short poll of some of our portfolio CEOs at Remagine Ventures and got some insightful quotes I’d like to share with you:
Everything we build is a hypothesis until the users are using the feature (hands-on) and we have a tangible way to measure its impact. Throughout our journey, we had a few ‘solid’ theories and concepts that we were confident would work and we invested many resources into developing them. Now, we design every feature as lean and fast as possible to get early validation and only after the initial validation we build more robust capabilities.
Or Briga, Co-founder and CEO of Novos.gg
We co-develop our product with our community. Roblox is all about the community, and they are highly involved in everything we do. We never tell them what they need, we listen to what they want and implement it in our development cycles and our LiveOps on a weekly basis. If we have a new feature or game mechanic we’ll usually invite them to play test with us, get their feedback and make the changes they asked for.
Anat Shperling, founder and CEO of Toya, a Roblox gaming studio
Listen, not speak-is probably the most important thing you could do prior to PMF, and as you move forward developing your business. Many startups founders look to solve a problem that may affect them personally, but the founders are often not the main persona that they are building for. So early conversations (prior to building) becomes even more essential in your product strategy. Features should not be the topic of any discussion with users, but rather, truly listening to the issues/problems that brought them to your product in the first place, and their expectations against their experience-that should be the essence of these conversations
Yoav Oren and Matan Guttman, Co-founders of Zoog (a B2C mobile app)
Your product’s “Time to Hello World” should be minimal! Users should be able to set up an account or complete the installation as fast as possible and see the product in action! This will allow they to experience the product’s value right away.
Alon Grinshpoon, Co-founder and CEO of Echo3D
You can have all the advanced features you want, if you don’t reach feature parity [with competitors] on some features, it’s an uphill adoption fight. Once you have achieved that balance, there is no one like you.
David Stiff, Co-founder and CEO of Vault AI
5 tips to nurture user feedback to build better products
1. The Mom Test
It turns out that people have been lying to you. When you ask your mom “what do you think about my startup idea?” she might say it’s great, because she loves you and doesn’t want to hurt your feelings. The Mom Test is a book by the wonderful Rob Fitzpatrick focusing on how to talk to customers, ask open ended questions and get honest user feedback.
For example, below are good questions to dig into feature requests:
- “Why do you want that?”
- “What would that let you do?”
- “How are you coping without it?”
- “Do you think we should push back the launch add that feature, or is it
- something we could add later?”
- How would that fit into your day?
2. Create a Sandbox
A sandbox is an isolated environment where you can test and experiment before making changes to production.
The process of building products is one big experiment, broken down into many small experiments. As Or Briga pointed out in his quote, every feature is an hypothesis that was tested and validated. But if we constantly change the product for all users, it can backfire and break the trust: imagine you went to Google and everyday it looked different? Even though Google also runs hundreds of experiments in a given month, it’s done subtly using A/B testing and the concept of Sandbox.
Get in the habit of testing, and always keep a control group. The best companies learn do to this at scale and it’s true for both B2B and B2C businesses. When assumptions have been tested and properly validated, they can be rolled out to all users gradually. There are now many SaaS tools to manage A/B testing (and even some that will create product optimisation ideas using AI) – give them a try!
3. Eat your own dog food
Put yourself in the shoes of your user. Would you rave about your product? how can you make it more accessible or easier to use?
Google was very disciplined at testing products internally before they went out to real users. It doesn’t meant that they didn’t make mistakes (there are plenty to choose from), but I’m sure this strategy help them catch more bugs and get critical user feedback before taking it to the public.
Developers, designers, product managers in your team should use your product every day.
4. Pay close attention to your user feedback
I love this quote by Brian Chesky, co-founder of AirBnb:
“It’s really hard to get even 10 people to love anything, but it’s not hard if you spend a ton of time with them. If I want to make something amazing, I just spend time with you…. Early on, Joe Gebbia and I literally commuted to New York from Mountain View [to visit our Airbnb hosts in person]. We literally would knock on the doors of all of our hosts. We had their addresses and we say, “Knock knock. Hello. Hey, this is Brian, Joe, we’re founders and we just want to meet you… And then you literally start designing touchpoint by touchpoint. The creation of the peer review system, customer support, all these things came from — we didn’t just meet our users, we lived with them. And I used to joke that when you bought an iPhone, Steve Jobs didn’t come sleep on your couch. But I did”
Brian Chesky, AirBnb co-founder and CEO
Not only user feedback will help you design a better product, but you can also start leveraging the feedback as validation for other users. We often have a herd mentality when it comes to purchases. When faced with the challenges of choice, we look for external validation: reviews, testimonials, referrals… can all make the difference between a conversion or a churn.
5. Get serious about UX testing
To start with a simple definition: UI (User Interface) is how things look. At the end of the day, it’s a deliverable, a set of mocks/design assets. UX (User experience) is a process, that defines how things work/flow and it should be iterated regularly.
In startups, we often conflate these terms as resources are limited, but it would be a mistake to skip usability testing and focus only on aesthetics.
Did you know that Google offers a free UX design certificate course? The program follows the design process from beginning to end: empathising with users, defining their pain points, coming up with ideas for design solutions, creating wireframes and prototypes, and testing your designs with users to get feedback. By the end of the program, you’ll have a professional UX portfolio with three design projects.
Never stop experimenting. User feedback should be a guiding voice in the product decisions you make and testing time should be baked in to your product roadmap. In addition to running experiments, it’s good practice to build a community of users using Discord, Slack or any other group that enables you have a dialogue and shorten the feedback loop.
* Image credit: Stable Diffusion
I'm a former general partner at google ventures, head of Google for Entrepreneurs in Europe and founding head of Campus London, Google's first physical hub for startups.
I'm also the founder of Techbikers, a non-profit bringing together the startup ecosystem on cycling challenges in support of Room to Read. Since inception in 2012 we've built 11 schools and 50 libraries in the developing world.
