Writing helps you think
I’m a big proponent of writing things down. I believe that it brings clarity to your thinking and is a great way for others to help you find flaws in your logic. When I worked at Microsoft, we used to put together (very) long Product Specs that covered everything from the businesses case to the intricate details of how a feature should be implemented. While I believe that it’s often unnecessary to go to the level of granularity of interactions for every click (unless you work in a large or non-co-located team), the notion of writing down why you’re pitching the thing you’re pitching is highly valuable.
The likes of Jeff Bezos and Stephen Sinofsky are also huge supporters of getting their teams to spend the time to write.
1/ “Writing is thinking” is my favorite saying in “how to work” in a company. It is very interesting to dive into this a bit because I often get so much pushback, especially from startups and/or those focused on agility.
— Steven Sinofsky ॐ (@stevesi) 19 April 2018
While many startups believe that “documentation is not Agile,” I believe in the opposite. At bare minimum, I believe that everything that a Product Manager pitches should be paired with an Experiment Document. Here’s a basic outline.
1. Background & State of the World
Provide some context for why this work has bubbled up – how does it tie into the company’s current quarterly goals and OKRs?
2. Customer problem you’re solving for
Bring the customer into the story. Do you have quotes, customer usage patterns, or recordings that help solidify the customer problem?
3. Hypothesis
Write a clear hypothesis in the form of
- We believe that by doing X
- Y will happen
- We know that we’re successful if we achieve metrics A, B, and C in T time
Speak to the potential impact to the company’s most important metric here. For example, if increasing usage by 30 minutes per day can be correlated with $X,000 of revenue, model that out.
4. Assumptions
This part is really important – what are the assumptions you’re making that led you to believe that this hypothesis is true? If this experiment fails, you need to jump straight to this section to identify which assumption was wrong and fold it into your learnings.
5. Past Data
Do you have historical data or past experiment results to justify why this work is important?
6. Experiment Design
Are you going to A/B test this design? To which cohort? How long is this test going to run for?
7. Next Steps
What are the next few experiments or decisions you’ll make if this experiment proves to be successful?
Do you have any other experiment doc formats you’d like to share? Do reach out to me at [email protected]. Would love to learn more about what you’re doing.
