00:00we are very grateful to have sueldo she
00:03from Mixpanel who co-founded Mixpanel
00:10almost 10 years ago now and is going to
00:14talk about how to measure your product
00:17which as you heard from Gustav is really
00:20the other side of the coin of growth and
00:24everything that helps you build
00:26something that people truly want and
00:28will use so with that Suhail all right
00:42awesome I thought I would start off by
00:45just kind of like I sometimes try to
00:47gate try to change my presentation a
00:50little bit on the fly depending on the
00:52audience and so I thought I would just
00:53ask like a really quick question just a
00:56quick show of hands um I'm gonna define
00:59like a user's like just someone that
01:00uses your product whether it's a whether
01:02it be to be company paying free doesn't
01:04really matter like they did something in
01:06your product how many people have like
01:09zero users right now okay
01:12cool awesome and then how many people
01:14have a hundred or less well okay you
01:18have greater than zero between 0 1 and
01:21okay awesome great and then how many
01:23people have like more than 10,000 okay
01:27awesome all right so we've got it's a
01:29majority of people have zero users right
01:32cool awesome great so I'm gonna start
01:35off with just a quick about me
01:38for those who don't haven't heard of
01:40Mixpanel um I started this company
01:41called Mixpanel I think the first line
01:44of code was written in like October 2008
01:47technically I was like 20 years old I
01:50was in my parents house in my bedroom
01:53and and you know 10 years later the
01:57company has about 300 employees we've
01:59raised a bunch of money and we have
02:01about 7,000 paying customers and we're
02:03starting to close in on around 100
02:05million dollars in annual recurring
02:06revenues just to give you a sense of
02:09yeah 2008 to now cool
02:13awesome um if you don't know what mix
02:15phenol is we make pretty graphs like
02:18this we help you measure what people are
02:20doing inside the product so this is like
02:23you know someone inviting a colleague to
02:25their app we might help you like measure
02:27a funnel and see the conversion rate of
02:30people opening your app to like viewing
02:31an article will help you measure
02:33retention I know Gustav went over that a
02:35little bit today and I'm just showing
02:38you this just because like everything
02:39that I talked about today is is
02:42basically is something that you could
02:44probably do in Mixpanel or even any
02:46other tool there are a bunch of tools
02:49that are great to measure these things
02:50but I want you to know that if you
02:52walked out the door and you wanted to
02:53apply any of it you totally could
02:55there's there's like a freemium plan and
02:58we cater very very much to startups
03:00those are all of our early customers so
03:02feel free to do that awesome great so
03:06today what I wanted to dive into was I
03:09thought I would take a top-down approach
03:11to thinking about analytics and metrics
03:14instead of kind of coming at you with
03:16charts and visualizations and numbers
03:18and things like that I thought it would
03:21be better to kind of think about the
03:23problem that you're trying to actually
03:24solve and it's sort of ignore for a
03:27moment that you have to like do complex
03:29analysis and calculations um you will
03:32figure that out it's not hard or you'll
03:34just use tools that will make that a lot
03:35easier for you so that's one thing that
03:38I thought I would do and so I picked
03:41these three things um I tried to think
03:43pretty hard about like what the heck did
03:46I care about when I started a company
03:48like what mattered to me
03:50what was on my mind and I thought well I
03:52think the first thing is just like do
03:54people even understand what I made and
03:56mixed well I had a really hard time with
03:58that in its first 18 months we had to
04:00change a lot of things and most people
04:03do the second thing is okay great
04:05they kind of understand what it is but
04:08do they like find it relatively easy to
04:11use the product on day one and that
04:14that's kind of this never-ending mission
04:17that you'll end up taking when you start
04:20your company and when you ship your
04:22product and then they'll and the last
04:23thing is you know I kind of tried to
04:26pare it down a little bit but I thought
04:27most important thing would be then next
04:29are people coming back and using my
04:31product and I think that you'll find
04:33that like I know Gustaf talked about
04:35this and I'm going to be extremely
04:37repetitive because it's one of the most
04:38important things that early founders
04:40make a mistake with yeah so from there I
04:46thought that I would try to also I
04:48thought that it'd also be helpful for
04:50you to see it kind of broken down in a
04:53formula and so this is the formula and I
04:55think I like to think about things from
04:57first principles right like what are the
04:59things that would cause you to grow and
05:02they're pretty simple right there's just
05:04like people that visit some page or the
05:07app write basic you experience this
05:09every day and then there's like people
05:11that sign up and then of those people
05:14like who did something how many of those
05:16people did something valuable like
05:17watching a video or making a recipe or
05:21you know taking a scooter a scooter ride
05:24and then of the people like how many of
05:27those people end up coming back some
05:29period of time later and then how many
05:32people will like spread that product
05:34like tell a friend share it maybe you
05:37even have a sales team who knows right
05:39and these these five things like
05:41basically equal growth and these are
05:44your lovers and the thing that I wanted
05:46to kind of point out and I want to
05:47stress is that today I'm gonna cover the
05:49basics but I want to kind of implore on
05:53you that the basics are really important
05:57even thousand-person companies get the
06:00basics wrong in fact you'll find that if
06:04you don't get the even these thousand
06:06person companies that get it wrong they
06:07tend to overcomplicate the number of
06:11things that they should actually measure
06:12and track and it's really easy to make
06:15twenty five metrics conceive of them
06:19align them you know divide them ball by
06:21you know give each team five and then
06:23really over complicate something that is
06:27actually quite simple to do when you
06:29over complicate it what happens is is
06:30that companies even large companies have
06:33immediate paralysis even midsize large
06:37companies have this like decision
06:40making problem used to have this funny
06:43phrase at Mixpanel where I used to say
06:44that all data is like contestable you
06:47know you could like see this number go
06:48off into the right and go oh well I
06:50don't know it was like sunny outside and
06:52like and that's why people did it and
06:55people will like have these arguments no
06:56matter what because causation is really
06:59hard to discover and so what I find is
07:02that just picking like three to five
07:05things on Northstar and really
07:09simplifying it will do more good for you
07:11and I know because not only have I
07:13watched thousands of companies make this
07:15mistake at all sizes I have made this
07:18mistake I had a piece of paper that had
07:20like 25 sub metrics and it was nearly
07:22impossible to actually keep track of it
07:24and what and people end up doing is in
07:27reality what people care about is there
07:29are there are metrics that guide the
07:31team and there are metrics that help you
07:33monitor and assess if anything is going
07:35wrong and today we're gonna talk about
07:37things that guide the team okay cool
07:39great so with this formula you should be
07:45able to assess anything going wrong in
07:48the company right if you're if you have
07:51a really confusing landing page it you
07:55should be able to figure that out if
07:56your signup process is too difficult you
07:58should be able to figure that out if
08:00you've a product that people don't
08:01really value you will figure that out
08:03with just that formula it's just that
08:06simple and you can get more complicated
08:09after that right you could say okay
08:11landing page views are down all of a
08:13sudden why who is it by Clemmy look at
08:17that segmented by country maybe
08:18something happened maybe I changed
08:20something in a different country so you
08:22should be able to assess any reason why
08:24you're not growing just starting out
08:25with that and that's why I say starting
08:27simple will last a very long time so
08:30this is what it looks like right linking
08:32those two ideas together you have this
08:35funnel of people going from visits to
08:37you know eventually retaining and
08:38spreading the product and like on each
08:41one of these things you're trying to
08:43basically solve one of these three
08:45problems is it easy to understand is it
08:47easy to get started are people coming
08:48back because they find it valuable and
08:50and then it loops and the funny thing
08:53about this is even if
08:54you're like Airbnb and you're thousands
08:56and thousands of people this like this
08:58process of like optimizing each one of
09:01these steps is never-ending no matter
09:04how you're no matter what scale you're
09:06at there will always be changing
09:07something and you probably have
09:09competition so you'll have no choice so
09:12awesome and then once you are able to
09:16measure those five things and assess
09:20you have your levers and you should be
09:22able to take action and fix any of those
09:24things put a team on it maybe you're
09:26fixing it it was really that simple
09:30awesome so we're gonna start with the
09:35first thing which is is my product easy
09:38to understand in order in order to
09:40illustrate this idea I decided that it
09:43would be somewhat sympathetic because I
09:45could show you like really awesome
09:46landing pages of like great companies or
09:50I could just show you like what it's
09:51like to start out and and I also wanted
09:54to mention that having a really bad
09:56landing page um there's this really
09:59awesome quote that PG used to have in
10:00our batch that I thought I would impart
10:02on you which is that like people
10:04basically just have their mouths like
10:05hovering on the back button and they're
10:07just waiting to say I don't know what
10:11this is confusing back and that's
10:13immediately how most companies are just
10:15like losing most of their um users from
10:17moving on and so just remember that your
10:21enemy is the back button so this is what
10:25mixed panels first front page look like
10:28in 2009 that's not what our logo looks
10:34like anymore this is a design by me
10:38original design by me and and and in all
10:44of its glory if you like actually goalie
10:46read the words there's like tons of
10:47grammatical mistakes because I was
10:48horrible at writing and for some reason
10:52in my mind I thought I know what the
10:54tagline for the company should be
10:55metrics that'll make you drool
11:02I don't I and I guess I think the reason
11:04why was because I thought these graphs
11:06were like so pretty looking compared to
11:08what was out there that I thought like
11:10that would be like the value proposition
11:13of the company and of course is like
11:16totally wrong like you're all laugh I
11:17wish I could have like made this landing
11:18page presented it to you then you don't
11:20laugh at me okay I should change that
11:21thing but I didn't have that I had zero
11:24users at the time you know and I tried
11:28to target you know small startup big
11:30company I didn't know who I was
11:31targeting application developer I like
11:33trying to target everybody like this is
11:35like every mistake that one could make
11:38on a landing page is like right here
11:40right like the signup button at least I
11:41kind of got that right thank you Endora
11:43for teaching me how to optimize buttons
11:44that slide like made it like yellow but
11:48that's about that's about maybe all that
11:50I got right right just maybe they are
11:52roughly the signup button and then you
11:54know after a lot of work a lot of hard
11:57work a lot of iteration a lot of talking
11:58to customers a lot of measurement you
12:01know eventually a couple years later it
12:02kind of turned into this right I like
12:05managed to get like the logo right a
12:08sign-up buttons like ginormous and
12:11yellow and and and what we figured out
12:15though after after a lot of time and a
12:18lot of energy was that we transform for
12:20metrics that'll make you drool to
12:22actions speak louder than pages and and
12:25you'll notice it's like extremely
12:26prominent the reason why was because the
12:28number one question that people that we
12:30would get as a company was that's cool
12:32but like how are you different than
12:34Google Analytics I don't get it right
12:36and so it turned out like this
12:38tagline for us at the front page lasted
12:41like the next five years of the company
12:43and and because it like very much
12:46differentiated us as something where we
12:48would measure engagement we did
12:50something different like Google was
12:51about pageviews we were about like what
12:53people did in your app and that
12:55resonated and then and then from there
12:57again we tried to explain like what
12:59various we try to urge people to go down
13:01the website and like look at different
13:04features and see if those things were
13:05valuable we tried to be benefits
13:06oriented and the copy tried to show
13:08customers and case studies who maybe
13:12we've probably made like four changes
13:14after this but I thought I'd present
13:16this one just because it was the one
13:18that maybe lasted like the the copy and
13:21the general essence of what we did
13:22instead we did what we said we did kind
13:26largely but lots of design changes so um
13:30the real question is you know great so
13:34like how do I measure how did you
13:37measure whether people really understood
13:39whether they understood what you were
13:41doing and so I think there's like one
13:43tried and trued number and it's like
13:45very simple it's just did people even
13:47bother to sign up bothering to sign up
13:49doesn't mean that they're gonna use your
13:50products that's a whole different matter
13:52bothering to sign up is like I don't
13:55know I'm kind of interested I'll kick
13:57the tires this seems kind of interesting
14:00and that'll tell you a lot about whether
14:02what you're doing is even something
14:03valuable for people the second thing
14:06that you can try these are just ideas
14:09trying to find the ratio of people that
14:12are going from just like hitting your
14:13page right and then there's and then
14:16doing any kind of thing after that so
14:18instead of aka instead of hitting the
14:21back button right like did they click on
14:23segmentation or funnels or attention in
14:26that case right did they bother to do
14:29that I know there are a lot of b2b
14:31companies um these days actually one
14:32quick question how many people are
14:33making like a b2b company yeah great so
14:36that's about half the room so I thought
14:39I'd give a b2b slant one thing that I
14:41discover was that people a lot of your
14:44users will click on a pricing page link
14:46at the top it's like second or third
14:48like most clicked-on link if your b2b
14:50company I think generally speaking I
14:52Jenna I think you know this is totally
14:55contestable but I find that like people
14:57clicking on your pricing page is like
14:59actually a pretty decent indicator of
15:01like okay cool you have something kind
15:02of valuable okay how much and so even
15:05something like that I think can tell you
15:07a lot and then if you're in consumer the
15:10benefit of being in consumer is that you
15:12generally have usually will have like
15:15hundreds or thousands of users such that
15:17you could actually go do an a/b test
15:19whereas in b2b it's a lot harder to
15:21achieve that and so I really I think
15:24Gustav is absolutely right that you
15:26to conduct as many experimentation
15:28experiments as you can and really a B
15:30test your copywriting and find out like
15:32what resonates with people but these are
15:34all things that you can do to basically
15:37is this easy cool so the second the
15:43second one is well is it easy to get
15:45started with my product and the thing
15:49about this is like a lot of people the
15:52thing about about these first two things
15:55is that if you don't make the first step
15:58easy if it's easy to understand a lot of
16:01people put so much energy when we're
16:03trying to build a company we try to
16:04think about like all the cool features
16:05and things that we could make that'll be
16:07really valuable for customers the
16:08problem with that is like if you don't
16:10even get step one right you don't even
16:11have a chance at any of the subsequent
16:13steps so when we think about like is it
16:17easy to get started with my product you
16:19know forgot about password is kind of a
16:21useless feature if you have like a very
16:23small percentage of people that are
16:25basically like not even bothering to get
16:27started in fact I remember there was a
16:29YC company called Zen tur which turned
16:31into like Google slides and I remember a
16:33Robby Walker during one of our one of
16:35one of our Tuesday dinners said that
16:38they had they didn't even implement the
16:39save button for like their online
16:41PowerPoint because they knew that none
16:43of their users would ever asked like hey
16:46they just even bother to do it because
16:49they wanted to see what anyone even use
16:51the PowerPoint presentation product it
16:53was like that was in that one day and
16:55part of that on me I was kind of like
16:56wow that's crazy and for a like I don't
16:59know maybe like 18 months of Mixpanel we
17:02didn't even have forgot about password
17:03because we were just so ruthlessly
17:05trying to prioritize like things that
17:07really would really matter to customers
17:08and it turned out we didn't get very
17:10many requests anyway so this is really
17:14important and here's an example of like
17:16easy to use something you use every
17:18that is literally one-step you just go
17:22to google you type it in and you
17:24immediately get to use the product
17:26there's no signup there's no CAPTCHAs
17:28there's none of that right a more
17:32complex example life is like Airbnb who
17:35might have like you know ten steps or
17:37something like that we worked with air
17:40me for a while helping them optimize
17:42initial user experience and you know
17:45even though Airbnb probably has like 11
17:48pages of you know like you have to go to
17:50Airbnb and then you have to fill out
17:51this form and then you get taken to the
17:53results and then you click on those
17:55results and then you you know you click
17:57on many different results because you're
17:59trying to price compare and you're going
18:00through this very complex checkout
18:01process so they're actually many steps
18:03to booking something online but Airbnb
18:07had to kind of figure out like how do we
18:09a describe what we're doing what we
18:13offer what's useful for the world and be
18:15how do we help people get started as
18:18quickly as easily as possible and like
18:20this is Airbnb calm right now and it's
18:24you know they have this very simple like
18:26book unique homes and experiences you
18:28fill out these very basic things and you
18:30hit search and that's it that's their
18:31getting started experience it's not
18:34their entire flow but it's the first way
18:36to get started and they even try to like
18:39kind of impute something like what you
18:41might what you might be able to find
18:42something imaginative on Airbnb but the
18:46thing is is that this seems simple right
18:48this seems like something that you could
18:50just like easily make it doesn't take
18:52how hard is that right we can do that
18:54with Twitter bootstrap in no time the
18:56problem the thing that's really hard is
18:58like figuring out that that's the right
19:00thing that's really hard that takes like
19:02they probably change that you know 50
19:04times before they figure it out like
19:06this is the right current optimal thing
19:08at this stage for the company and so it
19:11takes a lot of work in and you have to
19:13grind very hard to get to this at this
19:14point so question is like okay great so
19:17how do you how would you measure
19:19something like that well I think the
19:22really simple answer is a funnel
19:25how many people go from a landing page
19:26to signing up to doing that valuable
19:29thing like watching a video doing an
19:32Airbnb space who had been just doing a
19:34search just one search at least once you
19:38get past that measuring your funnel for
19:41the entire initial user experience
19:43there's like weird things we're like
19:45there's actually situations in games
19:47games can have like 20 steps and I've
19:49noted and I've seen hundreds of gaming
19:53and even though they might have like 20
19:55or 30 steps sometimes you can they
19:57actually convert very highly like they
19:59might get like 8090 percent conversion
20:00rate through all those 20 steps just the
20:02first two are like actually the ones
20:05that have the greatest drop-off and then
20:08the last thing is just like actually
20:10speed this is one that I think a lot of
20:12people don't actually do they're just
20:14like how fast how fast can someone just
20:17get started and that will tell you a lot
20:20about how complicated your experience is
20:22sometimes slow is okay it depends on the
20:24business but speed is really important
20:26does it take them like five minutes to
20:27get something figured out or does it
20:28take a minute in mixed panels case we
20:32cared a lot about like we had a
20:33complicated flow because you not only
20:35did you have to sign up but you have to
20:36be a little to write code just really
20:39complicated and that was the one
20:40downside we weren't really sure where
20:42people with Google Analytics you just
20:43copy and paste the JavaScript with mixed
20:45family had to like actually go and track
20:46a line of code and so we had this
20:49question our mind was that easy enough
20:50and so speed became really important to
20:53us how fast can someone do that and I
20:58wanted to give you a couple tips and
21:01tricks things that I have seen that
21:02companies do that are generally like not
21:04great number one you're a start-up you
21:08don't have lots of fraud problems you
21:10don't have you probably don't have a lot
21:11of people spamming your service you
21:14probably don't have a lot of like fake
21:15actors in the system trying to sign up
21:17so if you don't have those problems
21:19don't optimize for them until you do and
21:22I just want you to know that II
21:25generally email and text confirmations
21:28have an enormous drop-off like I'm
21:31talking like you get of the hundred
21:34percent of the people that come like
21:35only thirty percent will like basically
21:37go on and like click on those email
21:38confirmations or forty percent you spend
21:41a lot of time trying to optimize that
21:42and then TV make matters even worse like
21:44you have to hope to god that it isn't go
21:45to spam like it's this really horrendous
21:48experience so just be really careful
21:52you'll lose a lot of users that way it's
21:56really important to iterate on your
21:58initial user experience that will be a
22:00never-ending process you will do this
22:03forever even as the press especially as
22:08it becomes really critical to like have
22:10someone on it so at least one
22:11responsible person just thinking about
22:13that all the time and then the last
22:16thing is is not every company not every
22:18product is capable of doing this but to
22:21the extent that you can just let users
22:23into the product it's always better to
22:26do that in mixed fennels case it wasn't
22:27possible because like we needed to get
22:29your data so you have to sign up but we
22:31experimented with like what if someone
22:33just copied and pasted the JavaScript
22:35but they didn't sign up and we just
22:36collected their data and then after that
22:38we like asked them to sign up we did
22:40experiments like that so to the extent
22:42that you can achieve achieve this that's
22:45awesome and good examples of this are
22:47actually like Airbnb and Google where
22:49you just just try it but there are
22:51always exceptions to these rules for
22:52example like Pinterest makes you sign up
22:55and they didn't used to but they do now
22:57they're good reasons for that and then
23:03and then the last one definitely not
23:06least are people coming back to me this
23:10is probably one of the metrics that is
23:12greatly ignored by most startups and
23:17it's often the reason why I've seen
23:19companies even with millions of users
23:21actually dive I've seen probably 15
23:25different companies that grew virally
23:29and then just died the company died and
23:32I want to demonstrate this in a graph I
23:35called this the shark fin effect and the
23:38reason why I call it the shark fin
23:39effect is because well one was because
23:41Mixpanel in in Mixpanel the default line
23:44color for your first like data point is
23:45like blue and so it always to me it
23:47always looked like a fin like in the
23:49water and it looked a lot like this and
23:52what happens is you fought you like
23:55you're slaving away and you find a way
23:57to go viral right maybe you're going
24:00super viral on Facebook or on some or
24:03Instagram or something and you're like
24:06wow I have struck gold I'm acquiring
24:08tons of users my life is awesome
24:11I'm going to be a billionaire is great
24:13right and the problem with this is is
24:16that if it happens too early but you
24:18didn't quite think about if you quite
24:21about retention and becomes really
24:23problematic and Yulin and this hurts
24:25companies so basically what happens is
24:27the app goes viral and then eventually
24:28the rate at which you're losing users
24:30becomes so it becomes high enough that
24:32you can't acquire the new ones
24:35and it turns out that like even though
24:37we like to try to be extremely rational
24:39people it's really hard because like
24:41when the apps going viral you're just
24:43like oh my god I need to like just take
24:44advantage of this moment because what if
24:46I don't get in what if my competitor
24:48finds it so I'm just gonna optimize
24:49optimize optimize but it turns out that
24:51it ends up not being very valuable
24:53because if you're just losing all the
24:55users it's really really hard to
24:57reactivate if you talk to people that
24:59have ever done like a reactivation
25:01campaign your activation campaigns
25:03generally don't go very well you usually
25:05just like lose those users until they
25:07decide that maybe they want to give the
25:09product a second chance so it's not like
25:11like your response is like when this
25:13happens a lot of people's response is
25:14like oh we lost the users because our
25:16app wasn't valuable let's just email
25:18them again and see if they'll come back
25:19and like that's usually not a great idea
25:22so it can be really it's really
25:23important to really think through
25:26retention and you really don't want to
25:28be like this guy the shark fin effect is
25:31really really bad in fact LivingSocial
25:34is a great example a lot of people don't
25:35know that like LivingSocial was
25:37basically this app that was on facebook
25:39that had nothing to do with like like
25:40the Groupon deal cutting thing and they
25:42went super viral on I can't exactly
25:45can't remember exactly what the product
25:46was um and LivingSocial literally had to
25:48pivot the entire company to basically
25:50competing with Groupon because the app
25:53wasn't very retentive um in those days
25:55so there's some pretty basic ways to
25:59measure that people are coming back
26:02the most obvious one that I hope most
26:03people know is that there's just new
26:06users so making sure that you're
26:08tracking whether a brand new user
26:09someone who just signed up what
26:12percentage of those users will come back
26:13like a week later or thirty days later
26:16it's really important to track a longer
26:18period of time because a week later is
26:20just not a harsh enough metric and it's
26:23important to see how fast you'll end up
26:25losing users because you'll have to
26:26figure out a way to read to Rhian gate
26:28to find new ones and that becomes really
26:31really hard at some point if the number
26:33of users you have you
26:34becomes starts to become quite high and
26:37then the second one is is just daily
26:41active users there might be people that
26:44disagree with this I don't think they're
26:46too many but I now think that monthly
26:49active users has become like the new BS
26:52metric that's like very similar to like
26:54just like a number of registered users
26:55like number of registered users like a
26:57very silly metric I remember seven or
26:59eight years ago I think like I want to
27:01say it was like LinkedIn came out and
27:03said we have 200 million registered
27:05users there's like who cares how many
27:07people are using LinkedIn and I now
27:09start starting to think that like Mau is
27:10like becoming quite close to that now
27:13daily active users is really hard to
27:16maintain I mean how many products do you
27:19use every single day like if you just
27:22look at your phone and you go through
27:23all of the possible apps like how many
27:25of those do you actually use every
27:27single day so I think it's a really
27:30harsh metric and we actually found at
27:32Mixpanel so even if you're a b2b company
27:34and you might think well I don't know
27:36I'm selling to businesses so like kind
27:38of really maintain like a daily active
27:39users especially on Saturday and Sunday
27:42and we actually found that there's like
27:44a really really strong correlation with
27:46daily use and rate of churn like rate of
27:50like people like stop not no longer
27:52paying for the product and it didn't
27:54take very many users it was like one
27:57point you know for daily actives would
28:00like reduce churn to like a dollar turn
28:03by like to like I don't know like some
28:0610 percent is incredible and so I really
28:10stress measuring a da you and then the
28:12last thing is a b2b slant again is
28:15something that I think a lot of b2b
28:17companies make an early mistake with
28:19it's actually not just a b2b company
28:20it's actually companies that do
28:22subscription so I don't know if they're
28:24any companies that would ever think that
28:25they may offer a subscription of any
28:26sort you know like if you're a
28:28meditation app or something like that um
28:30and it's just dollar its monthly dollar
28:32turns if you're charging your customers
28:34monthly many businesses charge annually
28:37but usually most startups start with
28:38monthly because an annual commitment
28:40sometimes can be hard for early adopters
28:42is just measuring revenue churn I want
28:45to give you I'm gonna give you a slight
28:48if you have if you have if if you have
28:52revenue churn and your revenue churn is
28:54say 7% per month right you might think
28:58I'm keeping 93 percent of these dollars
29:01every single month this is awesome
29:04I'm doing really really really well the
29:06problem is is it's seven percent monthly
29:08churn and if you extrapolate that out
29:11over 12 months it's actually you're
29:13actually losing fifty fifty eight
29:15percent of your revenue in a year it's
29:18really harsh and a lot of people like
29:20for some reason this like this thing is
29:23like not mathematically intuitive to
29:24most people I don't know why it wasn't
29:26even mathematically intuitive to me
29:27early on and if you think about it like
29:30you're spending all of this energy with
29:32marketing and improving the landing page
29:35and improving the getting starting flow
29:37and you're doing all of that and if you
29:39don't have retention if you're losing
29:40that seven percent every single month
29:43you have to somehow make up for that
29:45fifty eight percent that you lost going
29:48into the new year and if you start
29:49thinking about you're like wow we're
29:50making a million dollars and we're
29:52losing five hundred and eighty thousand
29:53dollars and we're starting the year like
29:56you now you have to do that and grow on
29:59top of that and once your base of
30:01revenue gets really high it becomes
30:03really hard to keep up and you kind of
30:05have this like really long arc of like a
30:08shark fin effect basically so it's
30:11really important to look at your monthly
30:12churn and this will be the single cause
30:15of like death but the problem is it's
30:17like the worst kind of death it's like
30:18death that occurs it's really like it's
30:21really flat growth but it occurs like
30:23eventually in like four or five years if
30:24you're able to acquire users and I
30:27mentioned this not only because you know
30:29it talked to some founders about this
30:31but this is also something that like
30:33really hurt us in the early stages of
30:35Mixpanel and we really had to figure out
30:37a way to fix this and there's all kinds
30:39of ways you can use product you can use
30:41pricing there's all kinds of ways to
30:42experiment with this but this thing
30:44probably gave me the greatest number of
30:47nightmares and lack of sleep I think
30:49ever so I really want to stress that on
30:53all b2b companies here really think
30:55through this number and think about like
30:57almost graph like okay what would our
30:59rate of growth be if churn stayed
31:01really important cool I wanted to go
31:08over two things one I recognize that
31:10there a lot of people here that have
31:11zero users so they're like what the hell
31:14do I measure so what if I have less than
31:1750 users well the truth is I couldn't
31:19really think of a better idea than just
31:20basically you just have to talk to your
31:23users and I thought I'd give you a mini
31:25story about at least that makes the
31:26panel what we did when we had like no
31:28users and I had that page that just said
31:30metrics you'll drool over one trick that
31:33I used is that I had like maybe 10 or 11
31:36or 12 customers and honestly I just I
31:39just put all of them on I am like on
31:41gchat or something I don't know what the
31:43kids use these days maybe like what's
31:45that but but but I honestly just like
31:50would just badger them on like whatsapp
31:52or you know gchat or whatever and then
31:54just ask them and I remember one time we
31:56were trying to redesign our funnel UI
31:58and we used to have a vertical funnel
32:01but we had a competitor that has a
32:03horizontal funnel and it seemed like
32:05that was a better idea it seemed like
32:06that was more intuitive for customers
32:08and so I didn't really know how to like
32:10get the data cuz like how was I really
32:12gonna find out whether they liked one of
32:13those two you eyes and I didn't have the
32:15it was just me and a co-founder so like
32:17how the heck were we gonna like do an
32:19a/b test on that feature there's no real
32:21way to realistic way to do that and so
32:23all I did was I just made like a really
32:25crappy version of the UI that I thought
32:27would be better the horizontal version
32:29of a funnel I was a kind of greyscale
32:31and I mean like a really beautiful
32:32colorful version of like the vertical
32:34one which was kind of our control and
32:37then I just asked like eleven people
32:39which one's better and I want to I want
32:42to impress this thing because Gustav
32:43mentioned that it's the it's the Delta
32:45it's the difference that matters an a/b
32:47test not that this is like a true
32:48perfect AV test but like I asked eleven
32:51people and I am got feedback in a day
32:53and ten out of the eleven said
32:56horizontal funnels and then we just made
32:58it and we built it and we never looked
32:59back and it was totally the right
33:00decision in retrospect so I don't think
33:03there's any way that you can kind of
33:04skirt by talking to customers but here's
33:07the bet here's the benefit of talking to
33:08customers even though it feels maybe
33:10it's more tedious than looking at a
33:11graph you are going to get way more info
33:15and depth from talking to customers then
33:19looking at a data point in a graph no
33:23matter how much you slice and dice and
33:24segment the data no matter how
33:26mathematical you are about it you will
33:28never get as much information and into
33:31in building your own intuition than just
33:34talking to customers and it's really
33:36critical that you do it and the one
33:38thing that I wish I had done back then
33:39was I just wrote I wish I just wrote it
33:41down I wish I wrote down tons of the
33:43feedback because I think that it would
33:45have helped my co-founder would have
33:47helped employees in the company down the
33:49road really go on the journey make them
33:52feel like they went on the journey with
33:53me rather than having our roles kind of
33:57feeling like they were so split up and
33:58then the last thing that I thought I
34:00would bring up is is to impart what I
34:03kind of said at the beginning which is
34:04that I think I think that one of the
34:07biggest mistakes that people make with
34:08analytics is being really
34:11overcomplicated thinking that they need
34:13to be super sophisticated thinking that
34:15they need to track all kinds of crazy
34:17cohorts and they need to have like a
34:19dashboard with like 30 you know panes on
34:22it that all like loads you have like
34:23Mission Control I don't think that is
34:25necessary I had that it didn't work it
34:29was really hard to run the company that
34:31way and in what also happens is is that
34:34when you build your team it will be
34:35really hard you will be confusing for
34:37your team of those 30 panes like which
34:39one do I need to care about and I think
34:42that it's we need things that are humans
34:45just need to simplify things so I'd pick
34:47one Northstar metric and I think in a
34:49Northstar metric I would choose in this
34:50case is like what is a number that
34:52you're willing to bet the company on
34:54right like if that number goes south you
34:58if that number goes up you will like you
35:01will have like made a huge dent in the
35:03universe like what is that what is that
35:05metric and I'm not saying you need to
35:07choose that metric forever but choose it
35:09for six months choose it commit some
35:11time to it if you find out that it's
35:13actually like the wrong metric which it
35:14will be probably the first time you
35:16choose is usually your metrics are wrong
35:18and then you can change it but commit
35:20again to for another six months but
35:22choose that one number and then like if
35:24what you have to do at the beginning is
35:26basically just like print it out and
35:28put it all around the office do it
35:30because people will start to be
35:32maniacally focused on it they'll show up
35:34to meetings and go yeah but that number
35:36is like down what are we gonna do and
35:38where you can get really complex is
35:40discovering why that number is down or
35:42why it's going up that's when you can
35:44get really sophisticated that's when you
35:45can slice and dice it and figure out
35:46what the retention is and things like
35:48that and measure it over a funnel but
35:50keep things really simple and then the
35:52second thing is like don't boil the
35:54ocean like pick three to five other
35:56things I think less is more and just
36:00stay there telling even large companies
36:02most large companies don't do this and
36:05it usually is very discombobulating for
36:07them for the workforce it's totally fine
36:10to have numbers that you wish to monitor
36:12but don't focus on them awesome I think
36:16that's it and then for the people that
36:18were like slightly bored because I went
36:20over the basics I thought I would help
36:22you out by just kind of having some
36:23advanced topics for later that you could
36:25read about really these are things that
36:27like I found really useful my favorite
36:29one is this one the next feature fallacy
36:32by Andrew Chen and it's just this idea
36:35that we always think that like the next
36:37thing that we make will be the thing
36:38that will change the trajectory of the
36:40company a kie like the next features
36:42right around the corner and we're gonna
36:44be huge and I would really recommend
36:46that you read that article it's really
36:48awesome one of my my favorite one so for
36:50advanced stuff these are just people
36:52worth following reading about and can
36:55give you more ideas than what I've given
36:56you today awesome thank you
37:04Jeff Kuhn a cool all right
37:14yeah I want to be fair I don't be like
37:17if like every speaker like choose them
37:18in the front yeah yeah so that's the
37:33problem with that is that um it's soo
37:35it's so yeah oh yes I think someone
37:39asked what are like good benchmarks for
37:43like those given those five steps
37:44visiting signing up you know using the
37:48product what would be good conversion
37:49rates for any one of those steps and the
37:51the chart part with benchmarks like that
37:54is that it's so dependent on the
37:55business you know in b2b like going from
37:59like a visit to a sign up like I think I
38:01got like a four to five percent
38:03conversion rate is pretty good I think I
38:05like swap notes with Stewart at slack
38:07because we kind of had similar landing
38:09pages I think I asked him you know what
38:11was your conversion rate at slack and I
38:13think we both ended up being somewhere
38:14around like four to five percent from
38:15visit to sign up but but that said like
38:18I don't know what that would be for you
38:21know something like Airbnb where it's
38:22like getting to a landing page and then
38:24you know searching for your first place
38:26that you would want a book so I think
38:28it's really dependent I know that I know
38:30that Mixpanel published it on its blog
38:33publishes lots of benchmarks so I know
38:37that I know the marketing team there
38:38that does that and they like publish all
38:40kinds of benchmarks for like what's what
38:42should your retention be if your gaming
38:43company if you're an e-commerce company
38:44if you're a b2b company things like that
38:47social company a video company things
38:49like that so I think that you could
38:51probably go there and find various
38:52benchmarks reports but it's really
38:55depending on the business yeah
38:59so when you're going through the signup
39:01process how do you see numbers vary if
39:03you're large like creative accountant a
39:06and password doing all that kind of
39:07setup versus a single like button login
39:10with like cool guy how does that like
39:14transfer do more people to sign up if
39:16they have that like super click option
39:17do you see where they will drop off
39:19because they have to use one of those so
39:20yeah are you I think the question is um
39:22if what what is what converts better
39:25like those like social buttons where you
39:27can just like quickly authenticate like
39:28Facebook Connect versus just like
39:31standard signup is that right yeah like
39:34all of these questions they require like
39:36a lot of specificity because it's
39:38actually like really unclear and
39:40actually and in that case actually don't
39:41know I truly don't know and so I think
39:44the way to answer this question is to
39:46find friends in tangential industries
39:48that like don't compete with you but
39:50basically do that and then just be like
39:52hey what did what have you what is your
39:54conversion rate have you tried something
39:55I I don't actually think a lot of those
39:57numbers are like secrets in reality
39:59right unless you I mean you can't go to
40:01your competitor and ask that but like
40:02most of these things are not like super
40:05duper secret type stuff and like you
40:09know they could either give you the
40:10information and save you you know the
40:11six months or the or they'll just like
40:13say or you try it out but it's it's just
40:15it's sort of strange to keep it a secret
40:16so I would rather give you ways that you
40:19can discover that I would just ask
40:20friends that have like have a startup or
40:23even if they work at a big company
40:24sometimes they're willing to share it
40:54yeah so question is Airbnb ease landing
40:58page was super simple and how do you
41:00balance like I mean what if a lot of
41:02people just know about Airbnb and then
41:04just know what to use Airbnb for and
41:07then that's why they can keep it kind of
41:08simple versus like having something more
41:09complex a good example of this is like
41:12if you go to like the front page of
41:14Craigslist it's like absolutely nuts
41:15it's in like even Amazon if you go to
41:17amazon.com right like even though Amazon
41:20has like huge brand recognition you go
41:21to amazon.com and it's like whoa there's
41:25like million things going on at once
41:26right so how have these companies found
41:29the balance how have they found how to
41:31deal with this complexity and and the
41:33truth is is that I find that it's it's
41:35important to have like a guiding
41:37principle like what is it that you're
41:39trying to actually optimize for right if
41:42in the case of YouTube
41:43let's take YouTube as a good example
41:44right YouTube could just like be like
41:46Google they just like what do you want
41:48to look what do you want to watch right
41:50they could do that but they don't and so
41:52I think that the question really is is
41:54like what is YouTube
41:55what is the team at YouTube trying to
41:56optimize for are they just trying to get
41:58you to like watch a video are the is
42:01Airbnb just trying to get you to search
42:03for any kind of hotel or a hotel but
42:06Airbnb and so I think that like the team
42:11has to develop first a hypothesis of
42:13like what matter what do we think
42:15matters and then from there it's
42:17important to experiment and try to
42:19figure that out it turns out like it's
42:22it's not - again it's about the Delta of
42:25like what people will actually gravitate
42:27more towards like if you make a really
42:28simple experience but like people are
42:31like really confused
42:32because they don't even bother to do it
42:33or you or you ask them then you kind of
42:36have like new information and then I
42:38think the key is to find the minimum
42:40number of things you need to add to get
42:42someone started and you know and I think
42:44I don't think there's anything you know
42:46I'm sure Amazon has tested that front
42:47page like crazy and they found that it's
42:50like quite optimal yeah
43:03the can it depends on the product I'll
43:05give you I'll give you an example in
43:07mixed panels case oh sorry the case yeah
43:09the question was our video are like one
43:11to two minute video tutorials effective
43:13there yeah so I think that it's it's
43:18complex we did a video tutorial video
43:21and our flow for example and you know
43:25like I would say that I would say that
43:28like a lot of people didn't watch it
43:29truthfully and inner and this is what I
43:33mean by experimentation like a lot of
43:34people didn't watch that video Vittorio
43:36me actually I think eliminated it you
43:39know the first three steps but what we
43:41did find was this interesting thing that
43:43occurred as a result of looking and
43:45measuring things which was that um well
43:47we have a feature in mix panel where you
43:49can like see every action that a user
43:51takes and so I would literally just like
43:53go to every user and then just like
43:54watch like a creepy person just like
43:57sort of watching like everything that
43:58like someone would do in the product and
44:00eventually when you do that like when
44:01you do like hundreds of those and you've
44:03just watched hundreds of those you start
44:05to develop some sense of like okay you
44:07what you develop is a an idea a
44:11hypothesis and all of these things like
44:13all of these things require art and
44:14science and I think that companies that
44:16take it too far they like take take it
44:18too far to science end up getting it
44:20wrong they end up optimizing for like
44:22weird things like you could always make
44:23a button more yellow and bigger and like
44:26but that doesn't meet that doesn't make
44:27users happy eventually and so anyway to
44:30answer your question what I found was
44:31that lots of people were just clicking
44:33randomly on like all the myth side menu
44:35options and my hypothesis was well they
44:39had an integrated mix panel yet so
44:40there's nothing to look at so what the
44:42heck were you doing just like clicking
44:43on like funnels and retention like why
44:45would you do that and my hypothesis was
44:47they were just curious like what the
44:49product did what is it that they did and
44:51so we did this a B test where we like
44:53had a control with like nothing you know
44:55just the regular you know clicking
44:57around seeing like some image and then
44:59we had another one that had like videos
45:01and then we tried to see like what
45:02caused people to like do their
45:04integration their first integration
45:06what was the conversion rate and it
45:08turned out like the video like super one
45:10actually we didn't even a more simple
45:11test than that I think we just emailed
45:13people a video and said watch this video
45:15super basic didn't even bother to
45:17implement it on the website and had an
45:18amazing conversion rate and then we
45:20decided to add videos all to all those
45:22tabs where there wouldn't have been
45:23anything anyway and so that's why I want
45:27to say like I want to stress this it
45:28really depends and what matters is that
45:30you run the experiment to develop a
45:33hypothesis ie the art not the science
45:36and then measure it which is the science
45:38to find out whether the thing that you
45:40thought of was in fact right the a
46:00percent loss in revenue over a year yeah
46:17so what these are slightly two different
46:20things one is one is like user retention
46:24and then one is like dollar churn and so
46:27in the case of dollar turn like the
46:29reason I bring up dollar turn is that
46:30even like you could totally have a
46:31company where nobody uses the product
46:34and the company continues to keep paying
46:36for it right but I find that like most
46:39companies are like pretty incentivized
46:40to cut their spending and employees or
46:42incentivize because they lose it in
46:43their budget and so with respect to
46:46dollar turn you can have lot in a b2b
46:52company for example you can have a lot
46:53of use you can have you can have even
46:55one user using the product but the
46:57company could continue to you continue
46:59to spend money on it and like you might
47:01see like a forty percent retention rate
47:03like might be okay right as long as the
47:05company is like continuing to spend but
47:08losing 40 percent of your dollars in a
47:09year is really hard and not good
47:12retention so make sense so those things
47:13are not exactly linked they're not
47:16directly linked yeah yeah
47:20yeah like to give you to give you kind
47:22of a benchmark like a chat app that's
47:24like often used a lot we'll have like 60
47:2770 percent retention like those are some
47:31of the most retentive things like
47:32something like slack would probably have
47:33like eighty percent retention it's kind
47:35of like unbelievable whereas um you know
47:38like a product like mix panel when we
47:40first started out like the first year I
47:42think we had like 30 40 percent you know
47:45retention and that's not that great but
47:48who's their first few years yeah yeah so
48:02a question is are these metrics can they
48:04be used for hardware startup yeah I
48:07don't I don't see any reason why we
48:09worked with jawbone for a long time and
48:11we work with Fitbit and most hardware
48:15does your cart hardware company have a
48:17software component to it okay so it's a
48:25hardware company with subscription model
48:27yeah yeah absolutely
48:29it may be more challenging as a hardware
48:31company to track but it's like sometimes
48:33but it's totally doable most hardware is
48:36like connected to the internet somehow
48:37so I don't see any reason why like a lot
48:39of the scooter companies for example
48:40they're like all connected to hardware
48:43and they have a software component so
48:44there's really no difference
49:14yeah so question is the first page was
49:18like trying to target like various kinds
49:19of sizes of companies and then like the
49:22second landing page of Mixpanel like in
49:242012 basically didn't kind of disposed
49:28of that and it had something very
49:30different it was marketing like features
49:32and stuff and how do you figure out like
49:35what to do basically from that point to
49:37the next point yeah so I think that the
49:40basic gist is that so you can clearly
49:42see like I had no idea who our customers
49:44were and then after like a few years you
49:48just you talk to users you ask them like
49:50what size of company they are if they
49:52sign up if you want to we didn't do that
49:53but we just spent a lot of time we just
49:55it was so obvious that like everyone
49:57that was signing up for make spell at
49:58that time was a start-up so for a small
50:01in our in our case would be we call that
50:06I think like took like maybe no longer
50:15than six months to figure out that we
50:17were not gonna sign up a large company
50:19yeah we just didn't have the product it
50:23just wasn't there we hadn't worked on it
50:24long enough so the only people that were
50:26willing to basically try it were
50:27companies that could take the risk to
50:29try it and most of those companies
50:30didn't have anything anyway so it was
50:32even easier is it easier to target
50:33companies that had nothing and and had
50:36had we're willing to bear risk and then
50:38most of the time I would always employ
50:40this trick where I would just I would
50:41charge the money for it and then if they
50:43said no I would just give it to them for
50:44free because the feedback was more
50:46important but I always wanted to see if
50:47they would pay for it and so the only
50:49people that are willing to do that are
50:50also small businesses turns out so if
50:53for us it was just like it kind of like
50:55hit us it slapped us in the face it
50:57wasn't like this hard thing to figure
50:59out yeah yeah so we did it to be clear
51:20we weren't necessarily building things
51:21to bring new users on oh sorry the
51:23question is you guys built forgot about
51:26you didn't you just you decided to not
51:28build forgot about password how do you
51:30know when to build quality of life
51:32features versus features that just
51:33acquire new users yeah so I think that
51:36we didn't we didn't we wasn't so much
51:40that we only built features to acquire
51:41new users we built features to acquire
51:43new users and to keep existing users so
51:45we were very focused on like parts of
51:47the product that would actually make a
51:49difference right that would be clearly
51:52differentiating compared to all of the
51:53other Possible's possible options out
51:56there and like one of the things that's
51:57like not different is whether you have
51:59forgot about password right that was
52:01like in the bucket of things that like
52:02maybe don't matter and so I think the
52:04question is like really like when does
52:06it become a problem enough for you that
52:07it's worth building right like if you if
52:10you're finding that like it turns out
52:11that that the number of requests that
52:14you're getting for forgot about password
52:15is like increasing and then manually
52:17doing this process it's like annoying
52:19start then you should just build that
52:21feature because you know that if you
52:23don't build a feature they can't login
52:24then you can't retain them and then you
52:26lose them so it becomes this like
52:28flywheel effect where like the only way
52:29to retain them is to do that and in fact
52:31even though Mixpanel didn't build forgot
52:33about password for like maybe you know
52:3512 to 18 months we went eventually we
52:38found that people like people would try
52:40like multiple times I was looking at
52:43that stream of every action that a user
52:44would take I found that like people were
52:46like going to the login screen like five
52:48times and I was just so confused it's
52:51like why are they going to the login
52:52screen five times I was like oh I think
52:54it's because they can't login like some
52:58people don't even hit forgot about past
52:59well just like try try try and I'm sure
53:01you've all done this like with a bank or
53:02something like I'll just do it tomorrow
53:04you know you don't even bother right and
53:06so it turns out that even that happens
53:08and so we went so far even though we
53:10didn't do that I like to think that like
53:12we really made up for this later on we
53:14went so far as to like we found this
53:16like clever trick that Facebook used
53:18which was that when you failed to log in
53:20to Mixpanel like three times on your
53:22third time we just send you an email
53:24that says you know we just sent you an
53:27email just go to your email click the
53:28button you'll log in right now don't
53:30worry about forgetting it don't worry
53:31about changing your password just like
53:32login and it turned out that like that
53:35had this like great retentive effect
53:36because we were really focused on
53:37retention at that time yeah
53:40okay couple more questions yeah how do
53:55you gauge product market fit how do you
53:57measure product market fit so I think
54:01there's a lot of people that probably
54:03have already will talk about or have
54:04already talked about like you know
54:06people will say very and I think they're
54:07there all right so I'll give you a kind
54:09of I'll give you a quantitative slant
54:10versus like a qualitative slant some
54:12qualitative slants or things like when
54:15people are willing to spread you to
54:16their friends that kind of thing the
54:18quantitative version of this is really
54:21just measuring your your attention like
54:24how many people it's really measuring
54:26like da you it's really measuring the
54:28percentage of people that come back a
54:29week later or 30 days later it's the
54:32number of people that like become almost
54:36like they like need your product or they
54:39like feel like life would be a lot worse
54:41and I think that you can you can totally
54:43measure that by how they use it and so I
54:45think the quantitative measurement is
54:47basically finding like some benchmark
54:49some sense that for your industry maybe
54:52that's like thirty percent and just
54:53remember like that benchmark is probably
54:54like average and your desires to
54:56basically be above average and so if you
54:59can like significantly beat that
55:00benchmark then I think that you've like
55:03really you've you've done better than
55:05probably product market fit I think
55:07that's one thing I think another thing
55:08is like you can try to measure like
55:11repetitive use like frequent use in a
55:14day so like you can make pretty harsh
55:16metrics you can be like for example with
55:18da you you could say I don't want it to
55:20just be daily active user like they did
55:22one thing I want it to be that user has
55:24to have watched five videos and then and
55:26then then and only then or do they count
55:28as a daily active users you can be
55:29particularly harsh about that metric
55:32right someone coming and like just see
55:33that's why I say ma use kind of a BS
55:35metric now because you could come and
55:36watch one video and then never come back
55:38and you that count and that's not like a
55:40very healthy business so I think that
55:42find ways to be harsh about your North
55:45Star and I think that you'll find that
55:46that you'll find that you reach the park
55:48market fit yeah yeah I think with
55:50Mixpanel it was like 30 40 percent
55:53really really horrendous and we totally
55:56didn't have product market fit yet it's
56:09like a Peter teal question what's one
56:10piece of conventional wisdom that you
56:12think is generally wrong about startups
56:17in general gosh that's a hard last
56:26question it's like everything else is
56:28like fine um gosh I think I don't know
56:37I'm not I I need to think about it but I
56:38think that one thing that has like
56:40become pretty clear to me is that I
56:44think I think after having done Mixpanel
56:48for about a decade I kind of figured out
56:51that I tried to impart it on you in in
56:56in the in the talk today which is just
56:58that it's it's much better to just like
57:03I think that we tend to complicate
57:05things as humans and we don't we tend
57:07and that comes that comes through and
57:08like everything for example like we we
57:11tend to have we tend to like make a
57:13version one of a product and then we get
57:16to like version two and we're like okay
57:17we're done we like we did it we solve
57:19the problem and then like often a
57:20founder will like kind of get distracted
57:22and then go and like work on the next
57:24product on the next thing and they
57:27haven't yet figured out like their
57:28retention is probably not great or
57:29they're operating at negative margins or
57:31like all these things that same thing
57:32even happens with numbers like we pick
57:34our three metrics then we continue to
57:36get more complex as a business and then
57:38we add the next 20 things and we all we
57:40do all of these things to like in in
57:43hopes of like feeling like we control
57:45the situation better and I just think
57:47that those are all like distractions and
57:49mistakes and I think it's really hard
57:51for the average founder to actually just
57:54be focused on something boring we tend
57:57to have like a TD or something and like
57:59we just just I decide to get very easily
58:01distracted and I think it's hard to do
58:04something do the same thing for four or
58:08because we don't like feel challenged
58:10anymore and I think that that's a huge
58:12mistake that founders often make and I
58:15don't know that anyone say oh that's
58:17that's that's like totally wrong Suhail
58:19but for whatever reason like every
58:21founder repeats it so there must be
58:25something I don't know something there
58:27must be some kind of wisdom that we all
58:28think this is like the right thing and I
58:29made those mistakes I made new products
58:32that I shouldn't have I met too many
58:33things over optimized things that didn't
58:36need to be optimized so yeah that's
58:39really it cool thanks
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
