00:09hi there my name is Tom blumfield I'm a
00:12group partner at Y combinator and today
00:14we're going to be talking about one of
00:15my favorite topics metrics and why
00:18they're so useful for startups so why
00:21are metrics important first of all it's
00:23pretty obvious that with better metrics
00:25you'll make better decisions it's like
00:28flying an airplane with no instrument
00:30you're Flying Blind you don't know
00:32what's happening to the aircraft and
00:33you're not in control having great
00:35metrics is like having great instruments
00:37in an aircraft it lets you tweak and
00:38iterate and make sure you're really in
00:40control of your startup we often see and
00:43I've seen in even in the last two or
00:45three weeks Founders who've had these
00:47great launches they've launched on
00:49Hacker News on product hunt and they've
00:52had hundreds of people come and use
00:54their service day after day but they
00:56have no idea how many of those news
00:58users are new users or returning users
01:00they don't know if they're daily active
01:01or weekly active they could be churning
01:03off all of their users instantly and
01:05they don't know at all so the first
01:07thing they do after launching blind is
01:10to go back and build metrics in we would
01:13advise you don't do that you should
01:14build basic metrics into a product
01:16before you launch and as an investor
01:19it's really easy to tell Founders who
01:22are in command of their metrics versus
01:24Founders who aren't and it's really
01:26impressive when founders can talk about
01:29what percentage best signups are da or W
01:32or what the annual revenue per user is
01:35and we'll go into some of these in
01:37detail but it's a big differentiator
01:39when a Founder can talk so fluently
01:40about these metrics before we dive in I
01:43want to give a couple of warnings The
01:46Other Extreme is also bad so it's a
01:49Founder who even before they launched
01:51has a dashboard with perhaps 500 metrics
01:54maybe they've been a product manager at
01:56a big tech company or they've just
01:57watched too many YouTube videos but they
01:59want to make every decision in their
02:01startup with metrics and when you have
02:03only have a few hundred or a few
02:05thousand users that's basically
02:07impossible you know they want to split
02:08test everything should this button be
02:10blue or green and frankly it doesn't
02:13matter and you don't have the volume of
02:15users or data to make those kind of
02:17split tests sensibly so what you should
02:20do is certainly split test the really
02:22important decisions you know should the
02:23cost per user be $80 per year or $200
02:27per year that's a really good experiment
02:29to split test but you know making
02:31buttons red or green that's not really
02:33something you have the scale to split
02:34test until you're really at the the size
02:36of Google or Facebook a final warning
02:39don't hide behind your metrics you've
02:41still got to get out of the building and
02:43talk to customers Brian from Airbnb
02:46still hosts Airbnb users in his home
02:49it's an obsession with staying close to
02:51customers so you can't let metrics get
02:53in the way of that so let's get started
02:56you're planning a product launch in
02:58perhaps a week or two and maybe maybe
02:59you've not got any metrics in place yet
03:01what do you do the first thing is to
03:03pick four or five key metrics to track
03:06accurately not 30 or 50 four or five is
03:10fine this number will grow over time
03:12we'll talk about what those key metrics
03:13should be in eight a little bit you
03:15should pick the most straightforward
03:17analytics solution you can operate it
03:19might just be uh your SQL database
03:21making simple SQL queries to count to
03:23the number of signups post hog from
03:26Winter 2020 has a great SQL Analytics
03:29tool you can use on top of pretty much
03:31any SQL database so you should check
03:33that out you should also agree the
03:34definitions of these four or five key
03:36metrics and stick with them so it might
03:39not be the absolute perfect definition
03:41of an active user but constant arguments
03:44about what your key metrics are are even
03:46worse than having no metrics at all so
03:48your whole team has to come together and
03:50agree that an active user is someone who
03:53uses the product every day or at least
03:55once a week or at least five times a
03:57week it honestly matters matters less
04:00the precise definition than you actually
04:02all agree with it I've remember so many
04:06disagreements where the marketing team
04:08said you know we've sent you 2,500 New
04:11Leads this month and the sales team says
04:13no no no they weren't qualified leads
04:15they don't meet our definitions and this
04:16disagreement internally just destroys
04:19the productivity of meetings where
04:21metrics are involved so you really have
04:22to have centralized definitions of
04:24metrics that are written down and
04:26everyone agrees on so say you launched
04:28and perhaps the metrics aren't quite
04:30what you hoped the weekly active users
04:32aren't quite as high as you you
04:34originally wanted in that situation
04:35Founders are often tempted to pick a
04:38different metric or change the
04:40definition of those metrics so instead
04:41of a weekly active let's go for a
04:42monthly active the number looks a bit
04:44better honestly you're only fooling
04:45yourself in this situation it's really
04:48really important that you keep the
04:50definition of your metric consistent
04:51over time to see if you're improving or
04:54not that's why it's so hard to compare
04:56metrics between different companies
04:59definition just vary so a weekly active
05:01user at my company monzo was someone who
05:05transacted who made a financial
05:06transaction at least once a week an
05:09active user at some of our competitors
05:11used different definitions maybe it was
05:12every two weeks or eight weeks and so
05:14this active us account between companies
05:16just became totally meaningless just
05:18important that you keep it internally
05:20consistent so you're keeping a good
05:21track of it so now let's talk about what
05:24those key metrics are those four or five
05:27you should really start tracking from
05:29early on it will vary from every company
05:31back in the early days of the internet
05:33companies like to use metrics like page
05:35views or unique visitors or something
05:38like that because they're really really
05:39big numbers and startup Founders love to
05:41report really big numbers you've
05:43probably heard the term vanity metrics
05:46these are numbers that seem really
05:47really big and perhaps they keep
05:48increasing they're not actually tied to
05:51the success of your company more
05:52recently common vanity metrics are
05:54things like gmv or gross merchandise
05:57value that's the total dollar value of
06:00goods that are sold on eBay rather than
06:02eBay's Revenue itself or gross
06:03transaction value for a fintech like
06:05monzo you could report we're transacting
06:07$50 billion do a year it sounds like a
06:09really really big number but the revenue
06:12that the company makes can be very very
06:13different and so almost always
06:16especially for B2B companies your key
06:18metric should be Revenue if you pick
06:22another number take gross transaction
06:24value you'll find that your employees
06:28and eventually you might might start
06:29optimizing for that number I worked with
06:31a Neo Bank a couple of batches ago in
06:33the Middle East and they were reporting
06:35gross transaction value and they're very
06:36very happy they came to every group
06:38office hours and their gross transaction
06:39value was growing 50% every two weeks it
06:43looked really great and we scratched
06:44beneath the surface a little bit and it
06:46turned out that they were signing up
06:48much much bigger customers who had
06:50higher transaction value but giving them
06:52massive cash back massive rebates to
06:54transact on the platform so whilst gross
06:57transaction value was Rising every two
06:59week weeks Revenue actually was pretty
07:01flat for about the last two months the
07:04founders were tricking themselves they
07:05were fooling themselves into thinking
07:07their company was succeeding when in
07:09fact it was pretty flat in Revenue terms
07:11so revenue is the key metric I would
07:13suggest for most B2B companies and don't
07:16hide if your Revenue isn't good one of
07:18the most impressive Founders I've ever
07:20worked with sent 10 successive monthly
07:23investor update emails with zero with a
07:26big zero as the main metric at the top
07:29of the emailed she kept herself honest
07:31she was honest with investors and it
07:33became clear what they needed to focus
07:35on to fix the company so if you ashamed
07:37of this number you hide it away it's
07:39easy to kid yourself but I think if you
07:41put it up front and Central and pay
07:43attention to it it's the right thing to
07:45focus on so two other key metrics
07:47especially for investor updates
07:48alongside your Revenue please include
07:51burn rate that should be net what that
07:54means is monthly costs minus your
07:57revenues if you're loss making which
07:59most early startups are it's the amount
08:01by which your bank balance decreases
08:04every month that is your burn rate and
08:06your Runway is a function of that so say
08:08you have a million dollar in the bank
08:10and your burn rate is $100,000 a month
08:13that means you have 10 months Runway
08:15that means in 10 months you're going to
08:17run out of money and the the startup
08:18will be bankrupt those three numbers
08:21Revenue burn rate and Runway are
08:23absolutely crucial to include and if
08:26they're not at the top of your investor
08:27updates honestly I always uh assume this
08:30founder has something to hide for
08:32Consumer companies we have a separate
08:34video diving deep into the metrics that
08:37are important for those kind of startups
08:39and often revenue is very important but
08:41for the earliest days of consumer
08:43companies often you're trying to get
08:45some kind of critical mass or network
08:46effect and so growing the active user
08:49base in the early days for a consumer
08:51company might in some cases be more
08:54important than Revenue so we've talked
08:55about the main three metrics that should
08:58be at the top of every investor update
09:00that's Revenue burn rate and Runway now
09:02we're going to dive a little bit deeper
09:04one of the most important metrics for
09:06all startups really is retention this is
09:09the idea that if you sign up a 100
09:12paying customers say you sign them up in
09:14January how many are still paying you in
09:17February March and April two months 3
09:19months four months later that's your
09:21retention rate so it might be 80% or 70%
09:26it's a sign that people love your
09:27product they keep coming back and they
09:29keep paying for it so you can measure it
09:31for all customers who signed up in
09:33January and the subsequent months and
09:34that's your January cohort and then you
09:37measure the same for all of the
09:38customers who signed up in February
09:40that's your February cohort and you can
09:41stack these cohorts on top of each other
09:44and there are a number of different ways
09:45of graphing this you might have a heat
09:47map and some analytics tools do this
09:49really nicely you might have a curve
09:51that sort of decays over time those are
09:53two pretty common ways of graphing it
09:55and we'll show some examples but I'd
09:57like to suggest a third way this is is
09:59when it really clicked for me I'd heard
10:01everyone tell me that Revenue was really
10:03really important but it only really
10:05clicked for me after I worked at a
10:08dating startup that actually had very
10:09bad retention and ultimately failed
10:11sadly but it clicked when I stacked
10:13these cohorts on top of each other so
10:15you see the January cohort at the bottom
10:17and then the February cohort and then
10:18the March cohort and the April cohort
10:20and what happens if you have sticky
10:23cohorts if your retention is really high
10:24you know 80 90 100% is you cohorts stay
10:28really fat over time and you build up
10:30this layer cake so you can imagine what
10:32happens two or three years later if you
10:35have dozens of these monthly cohorts
10:37stacked on top of each other they're all
10:40paying you money they're all
10:41contributing to your Revenue every
10:42single month even three years later and
10:44if you're in a low churn business it
10:46adds up to a layer cake that looks
10:48something like this so this is an
10:5018month graph of monthly cohorts each
10:53month you're adding a new layer on and
10:55after 18 months you've got 18 cohorts
10:58still paying you this was like my first
11:00company go cardless it was a a recurring
11:03payments company very similar to stripe
11:05and with those kind of companies people
11:07Implement a payment solution once they
11:09don't really like to change it every
11:10month there's a lot of effort so the
11:12customers are very very sticky and you
11:14can imagine the team at go cardless goes
11:17on holiday you know for a month after 18
11:19months and the revenue stays pretty
11:22consistent the beauty is actually
11:24expanding Revenue so if those customers
11:27you signed up in January launch grow
11:30with the company they're using goard or
11:32stripe as a payment processor they're
11:34transacting more volume in year two and
11:37year three they're growing their
11:38business the revenue for for strip or go
11:41carders actually increases as well so
11:43the team perhaps goes on holiday or
11:45signs up no new customers the business
11:47still grows Revenue that's the beauty of
11:50this this High retention business it's
11:52sort of growing constantly underneath
11:54you you're adding these layers and
11:56layers and layers of Revenue and
11:57eventually become Unstoppable but that
11:59only happens if your retention flattens
12:01out at some point if these Decay curves
12:04flatten at some point and it almost
12:06matters that they flatten out at any
12:07point as opposed to a high point you
12:09know I I take a 20% retention that
12:11flattens out over a higher retention
12:14initially that goes to zero because if
12:15you sign up 100 people in January and by
12:17month three or month six they've all
12:19churn off they've all stopped paying you
12:22get a very different layer cake that
12:24lovely flat layer cake that builds up
12:26and up over time if your customers all
12:28churn out looks like this so you can see
12:30customers you sign up in month one by
12:32month three have more or less gone and
12:33let's fast forward to month six they've
12:36all gone by month 9 or month 10 and so
12:39rather than building up secure and
12:41steadfast layers month after month
12:43you're actually scrambling to fill up a
12:45leaky bucket you're pouring water into
12:48the top of the bucket and it's leaking
12:50out of the bucket just as fast as you
12:52can fill it up you can imagine this is
12:53an impossible task and so if your
12:56business has customers that that don't
12:58retain where retention goes to zero
13:00you'll reach some natural Plateau where
13:03you're working as hard as you can to
13:05fill up the customers who simply churned
13:07out last month and it's very very hard
13:09to build a big business like that it's a
13:11futile Endeavor so we talked about
13:13overall retention number of customers
13:16for B2B startups people often talk about
13:19net dollar retention this is just a
13:21fancy way of calculating retention
13:23mostly used in B2B SAS companies so
13:25let's take an example we've started an
13:27AI customer service chatbot very uh
13:30invogue at the moment and say we we sign
13:32up 10 paying customers in January in the
13:35first month and they're each paying
13:38$10,000 a month so 10K Mr each you're at
13:42100K of monthly recurring Revenue feels
13:45pretty good right let's fast forward a
13:47year and in each of those subsequent
13:48months you'll have signed up more
13:49customers let's let's ignore them for
13:51now focus on those 10 initial customers
13:53you signed up in January but fast
13:55forward 12 months perhaps two of those
13:57customers have canceled their contracts
13:59at some point in the year so we're down
14:02$80,000 of monthly revenue from that
14:04cohort but you've also upsold three
14:07customers perhaps you've introduced some
14:08new functionality or they've they're
14:10using it more and instead of paying
14:1210,000 they're each paying 20,000
14:15perhaps you do phone chat as well as
14:17text chat so that's $30,000 of
14:20additional Revenue so we've lost 20 but
14:22gained 30 so that's net 10K plus so
14:27$110,000 of monthly revenue from that
14:29January cohort that's why it's called
14:30net dollar retention it's the amount
14:32you've gained that's netted off against
14:34the amount you've lost so that cohort
14:36that was making $100,000 at the start in
14:39January of year one a year later January
14:42of year two is making a
14:44$110,000 that's equivalent to 110% net
14:48dollar retention so a net dollar
14:50retention above 100% means your cohorts
14:53are growing over time if your net dollar
14:55attention is below 100% they're
14:57shrinking over time youve got to pour
14:59more water into the funnel to fill up
15:00the the Leaky buckets we talked about
15:02and that's what gives very sticky
15:04businesses like stripe like go carders
15:06like PayPal this exponential growth it's
15:10adding new customers every month but
15:11having existing customers grow
15:13underneath them as well and that gives
15:15you this exponential growth curve that's
15:17very very impressive the final thing as
15:19a benchmark any early stage B2B SAS
15:23company should be looking at net dollar
15:25retention well above 100% this is for
15:28several reasons first of all you've
15:30probably underpriced your product with
15:32your first launch so you might charge
15:35$10,000 a month for your initial
15:37customers you realize pretty quickly uh
15:39that the product could be sold for 20 or
15:41$30,000 secondly you're adding features
15:44the whole time presumably you're
15:45improving your product and so that makes
15:47it more appealing and customers are
15:49willing to pay more money thirdly you
15:52should be getting it better at sales and
15:53upselling over time as well it' be weird
15:56if you weren't getting better at that
15:57and so those three reasons net dollar
16:00retention for early stage B2B startup
16:02should be 125% 150% would be great even
16:06higher than that for mature companies uh
16:09in the same range 110% 120% is pretty
16:11good net dollar attention if your net
16:14dollar attention is below 100%
16:17especially for Enterprise B2B SAS
16:19something is wrong you are churning off
16:21customers they don't love the product
16:23and I would invest in fixing that
16:25talking to customers and figuring out
16:27why they're turning off rather than
16:29trying to to shove more customers in the
16:31top of the funnel by investing in sales
16:33and marketing for example net dollar
16:35retention is absolutely crucial for B2B
16:37SAS companies okay the second Deep dive
16:40we're going to do on B2B metrics and
16:42this is applicable to Consumer companies
16:44as well is gross margin gross margin is
16:48your Revenue the money you get from
16:50customers minus the cost of goods sold
16:53so you can imagine that if you're a
16:55grocery store that's most obvious you're
16:56selling you're selling sandwiches for
16:58example the cost of good sold is the
17:00cost of the bread and the cost of the
17:01butter and the the filling that goes in
17:03the sandwich that's cost of good sold
17:05for a software company it's any cost
17:09that varies per customer or for each
17:12incremental customer you incur more cost
17:15so let's go back to example we had
17:16earlier we were running a an AI customer
17:19service bot and you're probably using
17:21something like open AI or anthropic to
17:23power the core model behind that and so
17:26the cost the credits that you pay to
17:28open AI or anthropic or someone else is
17:31your cost of goods sold we didn't used
17:33to talk about this very much for B2B SAS
17:34companies because the cost of goods was
17:36very very minimal for Pure software it
17:38might have been your AWS bill or your
17:40bandwidth bill or something like that
17:41it's minimal and so pure B2B SAS
17:44companies in the past might have had
17:45gross margins of 95% you know you sell
17:48$100 worth of software and it's only $5
17:51of cost and so people just assume it's
17:53very very high margin but these days as
17:56software sort of taking over more and
17:58more Industries gross margin has become
18:01more and more important so for AI
18:02companies today the gross margin the
18:04amount they pay to open AI or anthropic
18:07or others for the foundation model is a
18:09really important cost and by the way
18:11just because you're getting free credits
18:13doesn't mean that that's a cost that
18:15doesn't exist it just means you're
18:16hiding it for the moment so companies
18:18that hide behind open AI credits and
18:19claim that they've got these huge huge
18:21gross margins have a nasty shock coming
18:24when those credits run out it's also why
18:27heavily operational businesses are so
18:30tricky and when a company joins YC with
18:33a with something like a grocery delivery
18:35business or really any kind of business
18:38where a lot of humans involved a lot of
18:39operational processes going along you
18:41maybe you paint houses or install heat
18:44pumps or something you have to pay a lot
18:45more attention to the gross margin
18:47because it's very rare that it's as high
18:48as 95% you might be down at 5 10 15%
18:52gross margins which means you have to do
18:54a lot more work you have to get a lot
18:56more customers a lot more Revenue to
18:58generate the same gross margin and that
19:00gross margin is the thing that can then
19:02pay your head office rent your
19:04engineering salaries all of those
19:06remaining costs that don't vary per
19:08customer but still have to be deducted
19:10before you get to profitability and so
19:12for operationally intensive businesses
19:14we often try to work with Founders to
19:16see if there's a software only version
19:19of their business that they can run at a
19:21much higher margin so for example
19:23instead of running a delivery company
19:26where you have Vans and bikes and
19:28and delivery people instead can you take
19:31the software that powers all of that and
19:34sell it to other delivery companies
19:36you're going to have a probably a much
19:38easier life certainly you're going to
19:39have much higher gross margins if you do
19:41that so in the zero interest rate
19:44environment sort of 2010 to 2021 period
19:50companies were scaling negative margin
19:53businesses because Capital was so cheap
19:56famously Uber did this they used capital
19:58as a weapon so they took these
20:00businesses that were initially negative
20:02gross margin that means they're
20:04effectively selling $10 worth of service
20:06but only charging $9 so losing money on
20:09every single order but trying to get to
20:11a sort of a network effect or a Tipping
20:13Point um for Uber famously it was a
20:16certain number of drivers certain
20:17density of drivers and riders in a
20:19certain city that gets the flywheel
20:21going but when they launched in a new
20:22city they didn't have that density and
20:24so they had to subsidize drivers and
20:26subsidize Riders which made it a
20:28negative gross margin business and so
20:30they raised enormous amounts of capital
20:32to expand across the globe before
20:34competitors could catch up but burnt
20:36tens of billions of dollars of invested
20:38money in doing so and that Blitz scaling
20:41approach that that scaling of negative
20:43margin got popular with Founders and so
20:46we saw it in ride sharing then we saw it
20:48in 10-minute grocery delivery we saw it
20:50with electric scooters and honestly
20:52there's like a a whole Wasteland of
20:55startups that tried to do that and then
20:58realize they couldn't continue to raise
21:00money as investors just didn't want to
21:01keep subsidizing these businesses and
21:04certainly now with much higher interest
21:06rates Capital has become much more
21:08expensive investors are really really
21:10loed to invest in negative margin
21:13businesses and it's much much harder to
21:15scale those those negative margins we
21:17did this at monzo so monzo was an online
21:20bank in the UK and for our first half
21:22million customers or so we were losing
21:24money on every customer 30 or 40 per
21:27customer customer we scale to more than
21:29half a million of those customers it
21:31costs a lot but we had a plan to turn it
21:33around so we brought technology in house
21:36we didn't rely so much on external
21:38vendors we introduced charges for
21:40certain things we introduced new
21:42products that customers were happy to
21:43pay for and over time we flipped those
21:46negative unit economics so rather than
21:48losing 30 or 40 pounds per customer we
21:51ended up when I was there making 30 or
21:5340 pound per customer and now three or
21:55four years later monzo is profitable so
21:57if if you start with negative unit
21:59economics you really really have to have
22:02a plan to fix them and I would really
22:04advise you don't scale your customer
22:06base you don't try and grow as quickly
22:08as possible whilst you have negative
22:10unit economics you fix them first and
22:13then you scale pH we covered a lot of
22:16stuff today so as a recap we talked
22:19about revenue and why it's the best core
22:22metric for most B2B companies then we
22:25talked about retention and it's fancy
22:27cousin net dollar retention and why
22:29having a net dollar retention above 100%
22:32is so important for B2B startups and we
22:35finish with gross margin and why it's so
22:37important not to scale businesses with
22:40negative gross margins I wrap up with
22:42some final thoughts make sure you're
22:45tracking your four or five key metrics
22:47before you launch don't launch without
22:49metrics in place it's like Flying Blind
22:51be rigorous in what you track track the
22:54right metrics don't fall for vanity
22:55metrics like gross merchandise value
22:58your impressions or unique users have a
23:01clear definition of each of your metrics
23:03and a central way of measuring them in
23:05your company to avoid those pointless
23:07arguments that derails meetings don't
23:10hide behind your metrics you can't split
23:12everything especially as a small startup
23:14so a lot of these decisions just have to
23:16be made by talking to your users and
23:18using your product intuition you still
23:20have to get out of the building and talk
23:21to customers that's so important so I
23:25hope that helps run your startup with
23:27the right blend of metrics talking to
23:30customers and product intuition those
23:33three are a vital blend thanks for
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
