Lesson: Product Growth with Gustaf Alströmer

Step #5 Tools: If you do not keep score around your product, it is hard to win games

I think the tools are primarily about metrics in trying to understand — this is another reason probably for someone else, but how do you keep score and how do you win games. You don’t keep score on your product, it’s hard to win games. You basically just want to know which metric that’s really important and track that metric really carefully.

You can do that either with third-party tools or internally. I think if you’re a small company you want to start something third-party, so you have a mobile application and you want to make it start using [inaudible 00:39] to do some basic web tracking externally. At some point, which can be at a very late stage, you need to do more analysis in-house and it’s important for you to keep all the metrics in-house and the you probably have some data team, data design people that can help you with those things.

If you’re really small, if you’re a new startup, I would probably make sure I know SQL and [inaudible 01:07] database. But I also like to rely on a lot of external software, I think, external client and metrics, solutions.

Some external is great for just client event tracking. Optimize.ly is great for A/B testing. I think Google Analytics is probably basic in that for if you’re running something on web. I think there are a lot of highly vertical specialized tools as well. Everything else beyond that is probably very special in some way.

I think the way if you ask the questions, “Why am I using this tool?” you want to build funnels, you want to understand how does my user flow through the product that I built and you want to know why things are happening. Just tracking the number of downloads you’re getting every day is not that interesting. It’s more interesting why do I have more downloads today than yesterday. You want to focus on metrics that help you understand the picture versus metrics that just gives you a vanity number. Growth rate is more important than the actual growth.

For someone that works in growth there are many benefits of working in an API ecosystem. APIs open up channels. There are APIs that increase relevance. There are APIs that do great things for you if you are an app developer. I think mobile web is somewhat limited in many of these channels, so I probably wouldn't spend my time thinking about how to build an app.

Android is kind of the most interesting area right now because there are so many APIs that either people don't know about or it’s too exciting to take advantage of them and build really great experiences. I think iOS doesn't have the same kind of level of deepness in the API right now.

So for A/B testing it's probably true that you want to do some web view or mobile web, or you do it on the web, and then take your hypothesis to mobile. If you want to think about channels, you want to think about platform. You would generally want to think about repeat usage and engagement and all of those things. I think you'd definitely have to be on an app. But if you just think purely about A/B testing, there's no one standard or one way of doing A/B testing on mobile. There are many different people right now trying to do it but it's very fragmented, I would say.