Hypothesis Validation 101: The Clubhouse Guide for testing frameworks
We sat down with John Milinovich, Head of Discovery Product at Clubhouse, to dive into some best practices for startups building from zero.
Previously, he co-founded and served as CEO at Aesthetic and URX (acquired by Pinterest). In both cases, John and his early team went through Y Combinator. In addition to his 2x founder roles, he spent time at Pinterest as a Senior PM and at Google as a Program Manager.
In the second installment of our two-part conversation with John, we dive into:
- Why knowing your current scope and limits is crucial
- How to structure effective hypothesis testing
- Mistakes to avoid in early team hires
"Hypothesis testing is valuable in all cases. If something proves your hypothesis right, you learn. If something disproves it, you learn even more."
To Validate Hypotheses, Talk to Everyone You Can
There are two overarching systems a startup can have:
- Closed system — These startups converge toward low entropy AKA brand death. It's easy for new founders to think everything they need is in their heads. But there are substantial unknown factors they need to discover.
- Open system — These startups maintain high entropy, meaning they constantly receive new information which drives learning and growth.
John advises leaning toward the second for an open-book mentality as you build.
For instance, the best way to test and validate brand hypotheses is by talking to as many people as possible. It’s an obvious insight, but the method is underutilized.
The second time John went through Y Combinator, he accomplished two major benchmarks simply by talking to his peers in the program:
- Accelerated growth — He moved more swiftly in his early stages because he had so many people to bounce ideas off of and gain inspiration from.
- “Users” to test — He validated both his brand's customer segment and pain points by unofficially testing them (casually running them by many potential users).
He notes that a great way to determine if you've found a good founder-market fit is if you have a lot of viable people in your life who actively want to interact with your product.
"You'll know your brand has a viable customer base because you won't be sending cold outbounds early on. You’ll reach out to your invested audience."
How to Structure Hypothesis-Testing Frameworks
If you’re looking to structure conversations for hypothesis testing without asking leading questions, turn to The Mom Test by fellow YC alum Rob Fitzpatrick.
He explains that, if a founder asks their mom if she’d use their newest app for searching recipes, she’ll definitely say “yes.” Even if she doesn't benefit from the app at all, she likely won’t admit the truth that it’s useless to her child.
However, the founder could switch up their delivery.
If they asked their mom how she finds recipes and whether she’d generally enjoy a digital method, she’d more easily say, “I don’t need that.”
That second question and answer are far more informative about the app’s viability. Essentially, the Mom Test slices up a problem, starts at the highest level, and works its way down.
The Aesthetic Playbook: Real-Time Interview Questions
John's second startup, Aesthetic, focused on design services — every company clearly needs them. But they had more important questions to address like:
- How much design capacity do our potential clients already have?
- How design-aware is the founding team?
To acquire info, they'd approach potential users with questions like:
- Are you invested in design? If so, where exactly in your company?
- Why do you think design can benefit or help you?
- How do you generally approach design?
- Is your model B2B or B2C?
These lines of questioning validate whether or not your company hypothesis actually aligns with pain points for brands. Based on the answer, you can engage in solution mode.
If it turns out that brands aren’t struggling with the problem you imagined, you’ll know to stop pursuing that angle. Or, if your hypothesis did ring true, you could dig deeper.
- Once you found a designer, did you know how to work with them?
- Was it hard to find your in-house or outsourced design team?
- Did you consider the designer’s product high-quality?
- How did you pay the designer?
In John’s experience, the key to discovery conversations is remembering the difference between pitching your solution to companies and determining if your brand is pursuing a real problem.
- In a sales conversation, you're always going to lead with your ask
- In a discovery conversation, the person may never find out what you're selling
The best case scenario is when, at the end of a discovery conversation, you can let the user know that you're working on something that may solve their problem.
"The frequent issue with hypothesis testing is that people try to combine discovery and research with sales. Doing so yields bad results."
Why Early-Stage Hiring Should Resemble Mitosis
At the earliest stage, every hire is significant for the company. After all, if your brand has two employees, your third employee represents 50% team growth.
The best way to go about early-stage hiring is to view it like mitosis. Take one function or task from an existing employee and reassign it entirely to a new hire.
By doing so, you can strategically offload one employee’s weakest work onto someone who’ll be best in class in tackling that area.
In addition, with the mitosis rule, onboarding is a clear, straightforward process because the new hire is shadowing a founder.
Avoid Hiring Outside Your Existing Scope
At the other end of the equation, an all-too-common error to avoid is hiring for an entirely new function that no existing employee already handles.
Hiring outside your existing scope doesn't create operating leverage because it brings in a new function the founders don't understand themselves.
For instance, at a two-person company, if one member works on product design and sales, they should make their next hire a product designer or a salesperson.
Once the new hire is onboarded and fully ramped, the founder can learn a new skill like marketing with their newfound free time — and only then consider hiring a marketer.
Overall, hiring outside your current scope is a risk that startups can't afford to take.
It also suggests that there's a realm of work within their startup that the founders are avoiding dealing with themselves.
John has made this mistake, and, while he recovered from it, it's not an ideal place to be.
If a startup must hire for an entirely new task, they should look for someone who's an excellent individual contributor rather than a great manager or executive.
Trained managers often join startups with the intent to hire and scale rapidly, but that's not always what the company needs.
That new hire should ideally go with the flow, see the startup's vision, and align with its goals.
"New hires for startups are unambiguously crucial because they hold a third of the company’s output and hours."
Ready to learn more about listing on the LTSE Exchange?
