How to Get Users and Grow - Alex Schultz, VP of Growth at Facebook - Stanford CS183F: Startup School
Stanford Online2017-05-03
Stanford#Facebook#Alex Schultz#Growth#Startup#Y Combinator#CS183F
172K views|7 years ago
💫 Short Summary
The video features discussions on the importance of user retention for growth, analyzing retention curves, network effects, and creating 'Magic Moments' for user engagement. It emphasizes aligning goals, understanding user data, and utilizing data-driven decision-making for successful growth strategies. The speaker also highlights the significance of optimizing notifications, leveraging media channels, and personalizing marketing content. Additionally, the video emphasizes the value of targeting, conversion optimization, and creating seamless user experiences to drive engagement and revenue generation in businesses.
✨ Highlights
📊 Transcript
✦
Importance of retention for growth, focusing on monthly active users.
02:39Significance of having a great product and the role of engagement in revenue generation.
Value of metrics like cohort analysis for tracking user retention.
Impact of growth strategies and the team's approach to tactics for success in a competitive market.
✦
Analyzing user retention curves to determine user engagement levels over time.
05:27Importance of tracking user activity from sign-up through the first 30 days and beyond.
Success with small user samples in accurate retention analysis.
Description of three types of retention curves: flat-lining near zero, stabilizing, and re-accelerating after a decline.
Examples like FarmVille, Minecraft, and World of Warcraft used to illustrate different curve types.
✦
Importance of network effects in the success of social services and marketplaces.
07:09Introducing a new interface, such as an Android app after iOS, can re-engage users.
Adding new categories, like Amazon expanding beyond books, can boost user retention and engagement.
Different growth curves, from asymptoting to zero to steadily increasing, can impact a company's growth potential.
Understanding which growth curve your business aligns with is essential for long-term success.
✦
Focus on net growth, churn, and resurrection in order to drive business growth.
10:45Prioritize reducing churn and increasing retention to achieve growth.
Creating 'Magic Moments' for users, such as reconnecting with old friends or making a first purchase, is key to retention.
Companies like Airbnb emphasize delight and memorable experiences to drive users towards pivotal moments.
Sustained growth and success can be achieved through prioritizing retention and creating memorable experiences for users.
✦
Importance of finding 'magic moments' in a product through understanding user experiences and interactions.
15:07Significance of user retention and correlation between specific actions and long-term usage.
Example of eBay used to show correlation between actions and retention rates.
Emphasis on high retention rates in social networks and messaging apps for long-term success.
Value of identifying and validating key user interactions to build a sustainable business model.
✦
Importance of User Retention in Business Models.
16:11Higher retention is essential for successful business models like eBay or Amazon.
Physicist Geoffrey Taylor used dimensional reasoning to calculate bomb power, highlighting the importance of precise calculations.
Facebook's ad revenue curve shows the link between user registration, activity, and revenue generation.
User behavior significantly impacts revenue outcomes, emphasizing the importance of user retention and engagement.
✦
Importance of analyzing curves for predicting future outcomes based on past data.
21:53Methodology can be applied to various metrics like engagement, retention, and customer lifetime value.
Setting clear goals for growth is crucial, as demonstrated by examples from Facebook's early days.
Aligning the entire company towards a common goal, such as monthly active users, is essential for success.
Ensuring alignment and direction in larger teams or companies can be a challenge.
✦
Importance of clear goals for company metrics.
22:52Conflicting priorities can arise if goals are not aligned among team members.
Focus the entire company on growth rather than having a separate growth team.
Understanding user data and feedback is crucial for identifying opportunities for improvement.
Addressing issues such as retention rates, revenue per user, and acquisition channels.
✦
Importance of Data Analysis in User Engagement Improvement.
25:50Analyzing data is crucial for identifying gaps in user engagement, like push notifications not being received or clicked on.
Tracking user interactions helps in understanding growth trends and potential issues.
Personal experience of data logging at Facebook led to insights that prevented a growth slowdown.
Data is valuable in improving user experience and there is no need to choose between user experience and metrics.
Data helps gain empathy for users, with an example of product optimization for different screen sizes emphasized.
✦
Importance of using data to understand users outside of Silicon Valley.
30:53Data validates product development and predicts user behavior.
Metrics are valuable for optimizing user experience.
Data is powerful in forecasting revenue.
AB tests are recommended for product development, but startups often hesitate to trust results.
✦
Utilize data for faster decision-making and independent testing by team members.
33:12Prioritize goals based on data analysis to facilitate quicker decision-making process.
Initial customers should be friends for honest feedback and payment.
Reach out to desired users personally to avoid spam filters.
Utilize PR, social media, and ads for initial growth, focusing on product-market fit and retention for successful strategies.
✦
Growth tactics in marketing emphasized in the video segment.
35:50Importance of channel targeting, creativity, and conversion highlighted.
Significance of SEO strategies, including keyword research, content targeting, and link building, discussed.
Need for incremental analysis and focusing on marginal ROI over absolute ROI emphasized.
Caution against optimizing products solely for power users and importance of considering broader audience in marketing strategies stressed.
✦
Importance of optimizing notifications for marginal users on platforms like Instagram and eBay.
38:52Focus on email, SMS, push notifications, and offline mail for user engagement.
Value of earned, bought, and owned media, with an emphasis on leveraging owned media like website traffic.
Strategies for on-site merchandising to personalize creative content and trigger relevant suggestions for customers.
Significance of personalization in marketing, with examples from Facebook and opportunities for deep personalization through on-site merchandising.
✦
Importance of Custom Audiences and Look-Alike Targeting on Facebook Ads.
41:49Targeting based on behavior is more important than demographic targeting.
Signals indicating the right ad or content for individuals are key.
Creative content is valuable but not as crucial as targeting and channel selection.
Personalizing and adding a call to action to creative content can boost engagement.
✦
Importance of using the right words, focusing on conversion, and optimizing landing pages for online advertising.
45:55Metrics used at eBay to measure user engagement and the impact of redirecting search traffic to specific landing pages.
Emphasis on fast decision-making, bold actions, and data-driven strategies for company growth.
Prioritizing retention and identifying key metrics as crucial factors for success in business growth.
✦
Importance of creating magic moments in product design for enhancing user experience and driving engagement.
48:11Examples from LinkedIn and eBay show how streamlining the user journey can increase activation and revenue generation.
Focusing on reducing friction and optimizing the user flow can improve conversion rates.
Companies can create a more seamless experience for users by prioritizing user experience design.
00:11Good afternoon everybody.
00:12Today, we have Alex Schultz,
VP of growth at Facebook, and
00:15probably the world expert on growth,
as he can vouch from this weekend,
00:19the person I go to when I have
a really hard growth question.
00:22Many people who work at Facebook, many of
these founders at Facebook in fact, and
00:27very early employees have said hiring
Alex has been one of the secret
00:31successes of Facebook that
is not well understood.
00:33You joined almost ten years ago, and
00:36I think at that point Facebook
had less than 100 million users?
00:39>> Yeah.
>> Way less than 100 million users.
00:42Facebook now has 1.6 billion and
Alex has been after the importance
00:47of having a great product, maybe
the second most important driver of that.
00:49>> Having Naomi,
these very important people.
00:52>> And the team, anyway, thank you
very much for coming to speak to us.
00:54>> Thank you.
00:56I did not pay Sam for that introduction
and I would note that Havi is my boss so,
01:01hi Havi.
01:01But no, Havi and Naomi, Danny,
01:04they are these people who've been in
the growth team for four years and
01:08then people like Blake, and Martin, and
Ray and James Wang who set the team up.
01:14And it's very nice, but I am a VP on
growth, I am not the VP of growth.
01:19So you've had an amazing lecture series.
01:23I just watched all of them yesterday to
make sure I didn't say anything without
01:26actually having seen
what everybody else said.
01:28And it was totally awesome.
01:30I loved what D'Angelo had to say, so
that lecture, if you weren't here, if you
01:34didn't watch it, I strongly recommend you
go back and see what Adam D'Angelo said,
01:38because I could not emphasize his
points on metrics more than enough.
01:42And so what I'm going to try and do for
01:43the first half is go over some of
the points that have been made already and
01:47emphasize what I think is most important
when you're thinking about growth and
01:50what my take is and our take is on
the Facebook growth team on those.
01:54And then in the second half, try and
talk to you a little bit more about
01:57tactics that have been
particularly useful.
02:01The caveat that I think you should take
onboard is what Stewart from Slack and
02:06Flickr said.
02:07Which is to some extent, you do have 100
million monkeys bashing at typewriters,
02:11and I am very, very lucky to be
on the growth team at Facebook
02:14to get to work on a product like Facebook
that has the retention that Facebook does.
02:17That has everyone in the world wanting
to use the product and to say that,
02:22I drove it or any of us drove it beyond
their being great product market fit
02:27is perhaps not true.
02:30I can't tell you what the AB test
is if we didn't even have a growth
02:33team at Facebook.
02:35So what matters most to growth?
02:39The single most important
thing to growth is retention.
02:45Adam commented on this in his talk, and he
showed you cohorts from some real company.
02:50But here is a stylized cohort of
how you should look at retention.
02:54The number of days from
acquisition is on the x axis.
02:57On the y axis is the percent of
people who has still monthly active.
03:01I care a lot about using monthly active,
not daily active, not weekly active.
03:05There are really good arguments for using
daily active that people say all the time.
03:08Because it's about intensity of usage.
03:11But what really, really matters is
that you got skin in the game and
03:13people come back to your
product at least once a month.
03:16And then you build on top
of that engagement revenue,
03:18whatever your key metrics are which
we'll get to in a minute, but firstly,
03:21I believe monthly active is
the most important thing.
03:26The second thing is how do
you rally calculate it?
03:30So I've got a real graph for
revenue in a few slides from Facebook from
03:33the first 250 days of our ads products,
which I'll show you.
03:37But the thing that's really,
really important is to say that,
03:41it's very easy to calculate this curve
even if you have 1,000 or 100 users.
03:47And what you do is you look at all
of your users on the first day
03:51after they signed up.
03:52Were they monthly active?
03:54Yes, clearly, right?
03:55The first 30 days everyone's
going to be monthly active.
03:58I'd prefer if you defined a month as 28
days, by the way, because multiples of
04:01seven is really, really good for
these kind of metrics because of weeks.
04:04But for the first 30 days, everyone
has been active on their first day,
04:08on their second day, on their third
day as a monthly active user,
04:10whether they came back or not.
04:11And then what you do
is you look at day 31.
04:14And obviously, only some of your users
will have been active for 31 days.
04:18Everyone who signed up a day
ago was not active for 31 days.
04:23So what percentage of them
were active on day 31,
04:26what percentage of them
were active on day 32?
04:29And as you go further and further out in
the number of days since registration,
04:33the number of people who have been
on your service that long declines.
04:37Which means that your metric is
going to get noisier, right,
04:40as you get out to the far
right hand side of that curve.
04:44But that doesn't mean you can't get
a handle of where things flatten out very,
04:49very early on with very
small numbers of users.
04:52The number one criticism people give me
when I talk about retention as a metric.
04:55Or even when you look at what Adam
presented in his presentation earlier is
04:59you need lots and lots of users.
05:02So you need millions of users to
calculate that, that is not true.
05:06We did it with thousands of users in the
early days of advertisers for Facebook for
05:10example.
05:12Or you need to have had all of those
users on for a very long time.
05:16Look at Adam's graph it was multiple years
with months, and months, and months.
05:20No, you can do it day by day, and
05:22you can get to a really good place super,
super fast, using this methodology.
05:27So what you're looking for
in a retention curve?
05:30You're looking for it to flat line above
zero, not flat line like a heart monitor.
05:35You want it to flat line at some line
that asymptotes with the x-axis.
05:41There are three types of
these curves that I've seen.
05:43The first type is where
asymptotes to the x-axis.
05:47Like really, really close,
goes down to almost zero.
05:50You hear about a lot of those companies.
05:51You think about most games,
most games are smash hits.
05:55How many people use FarmVille now versus
how many people used FarmVille at its
06:00absolute, please tell me
you know what FarmVille is.
06:01[LAUGH] Yeah, okay, all right.
06:04How many people used farm,
I'm feeling old.
06:05I had hair when I first did one of these.
06:07How many people use FarmVille now
versus how many people used to use it?
06:11FarmVille was a smash hit, then there's
other ones which look like this,
06:15where they stabilize out.
06:16So you think about Minecraft,
right, or World of Warcraft.
06:19These games that somehow have network
effects and people stay in them for
06:22a long time.
06:24The other version for this curve I've seen
is the ones where they go down, they look
06:28like they're going to hit the x-axis and
then they re-accelerate and they go up.
06:33There's a few different things that can
cause that right, so you have this,
06:36this overwhelming thing that is driving
you down to have zero retention and
06:40somehow the numbers go up again.
06:42So things that drive that for
businesses, and
06:44you actually I think
mentioned this in your intro.
06:46One, network effects, so
06:50when all your friends get on Facebook,
you're much more likely to use Facebook.
06:54When all your friends get on Twitter,
06:55you'll more likely use the Twitter,
WhatsApp, Messenger.
06:59Whichever social service
you're talking about,
07:00there are network effects
that really matter.
07:03Network effects also
count in marketplaces.
07:06When you have enough sellers,
you can have enough buyers.
07:09When you have enough traffic,
you get more sellers.
07:11And so as marketplaces go, often what you
find is these old cohorts that signed up
07:15resurrect and come back to the service.
07:18So network effects can drive the curve
up after it's flatlined somewhere or
07:21is heading towards zero.
07:22The second thing that can do this
is opening up a new interface.
07:28So think about it.
07:29A lot of people release
their apps first on iOS.
07:32I actually loved Diane's talk last week
for this, by the point that he made of
07:36starting with Nokia, focusing on Android
because it's what most of the world use,
07:41but in Silicon Valley we typically build
our apps first on iOS more often than not.
07:46And then when someone releases
an Android application, and
07:49this was something I saw with
Instagram years and years ago.
07:53But when you release
an Android application,
07:54people might have tried it on an iPhone
And then traded their iPhone for
07:57an Android and
no longer be able to access your service.
08:01People might've tried your
service on a friend's iPhone and
08:03then when you add an Android they
come back and use your service.
08:06So that's another thing that
can drive you up again.
08:09And then the third thing that I've seen
on this curves that drive this curves up
08:12again is adding categories,
so think about Amazon.
08:16There was a really good discussion,
was that Stuart earlier or?
08:19Someone earlier in this talked about
Amazon, and how Amazon started with books
08:23because there was this large
inventory available of books.
08:28But then they added in another category.
08:30And then they added in
another category over time.
08:33And imagine, there will be people who came
to Amazon to buy that one book, but then
08:37two years down the line Amazon suddenly
had kitchenware, I don't know, or fashion.
08:40And they came back and
then they bought fashion and
08:42they brought them more use cases
to come back more monthly active.
08:46This was true with eBay.
08:47We added motors.
08:48I remember being at eBay and
we had to do all of our reporting on
08:52gross merchandise volume with and
without motors.
08:56And no one had ever believed that you
could buy a car online, and eBay did it.
09:00And it created a whole new use case for
people, many of whom were existing eBay
09:03customers, who now had one more
category they could shop in.
09:07So you want to curve the asymptotes
to a line parallel to the x-axis.
09:12You can do it with far more
limited amounts of data
09:15than most people think you can by using
the technique I outlined earlier, and
09:19I'll show you a real world
example in a few slides.
09:21And there are three types
of curves that I've seen.
09:24Curves that asymptote like this,
awesome, your company will survive.
09:29And how big will it get?
09:30We have another question.
09:32Curves to go straight to zero.
09:34That's okay, you could be in games and
09:35your job is to produce one hit
after another after another.
09:38EA doesn't expect whatever fee for 2012,
09:41they still have lots of users
now if they've released FIFA 27.
09:44So it's okay, there are businesses
where hitting the x axis is okay.
09:48But you need to know you're
in one of those businesses.
09:51And then the third that I've
seen is the curve going up.
09:54And that can be driven by network effects.
09:56That can be driven by being
available on more platforms.
09:59That can be driven by adding categories
to your marketplace or service.
10:05From this falls out a brilliant
way to look at growth, and
10:07this guy Danny Ferrante who was at Yahoo.
10:09Who was one of the first members of
the growth team runs core data science at
10:11Facebook, he's totally awesome.
10:13He's really the brains behind
the operation I'm the mouth.
10:17Danny came up or brought this to Facebook,
and it's a simple way to look at growth.
10:21We were looking at growth as how many
users were we growing by week on week?
10:26We looked at it on an absolute basis.
10:29And looking at it like
that we were like okay so
10:31we're signing up whatever
it was a million a week and
10:33we're growing by whatever it was
800,000 a week I don't know.
10:38Cool, we're churning 200,000 people.
10:40And we never actually
looked at it in this way.
10:42When we change the way we look
at things to say well okay,
10:45the net growth is 800,000 a week,
the new users are 1 million a week.
10:51But what is churn and resurrection?
10:54And what we actually found at the time,
and
10:56I can't remember the absolute numbers so
I'll just make some up.
10:59But imagine you're churning 1.3 million.
11:02So not 200,000, 1.3 million.
11:05You're resurrecting 1.1 million.
11:07So plus 1 minus 1.3 plus 1.1,
11:11you've got net 800,000 in growth.
11:15But that 800,000 in
growth is being driven,
11:18I think I did the math wrong, but
800,000 in growth is being driven by
11:22a different order of magnitude from churn
and resurrection than you realized.
11:28And so that helps you focus on
how do I stop people churning?
11:32How do I get people to come back to my
service who are not coming back to my
11:36service right now?
11:37So once you have your retention curves,
and
11:39once you start to have
a business with retention.
11:42You need to look at
your growth accounting,
11:44not just how many users are we acquiring.
11:47Make sense?
11:49Overall the number one thing
I focus on for growth,
11:52and the one thing that we focus on for
growth is retention.
11:55So what gets you retained
in any given service?
12:00Magic Moment.
12:02What is the moment when you used Airbnb?
12:05What is the moment when used Door Dash,
when you used Slack,
12:08when you used Facebook,
LinkedIn, WhatsApp?
12:11Whatever the product, what is that
moment when you went, yes, I've got it.
12:15Right, on Facebook it's friends.
12:17You see that first friend,
12:18that person who I've got completely
the wrong crowd for this but the person
12:23from high school who you lost contact
with as you grow older and moved away.
12:27But you see that person and you go, wow.
12:29I haven't talked to them in years.
12:31Let's see what's going on in their life.
12:32And they've had a kid and lost their hair
too, and you feel better about yourself.
12:39That is a magic moment on Facebook.
12:41So how quickly can you drive
the user to that magic moment.
12:44You look at Airbnb, right?
12:45They talked about the delight
that they give people.
12:50They talked about these different
screenshots of the stages of the user
12:53journey that they've created
as a map of the user journey.
12:56Getting that first person
reserve your house.
12:59Staying at that first property,
those are our magic moments.
13:02eBay, buying your first item
from a stranger online.
13:06Nowadays that doesn't seem crazy but
back when I was working at eBay in the UK,
13:09that was revolutionary.
13:12And people really, their eyes lit
up when you talk to them about it.
13:15In Germany they had this great ad, tights
by minds that showed the build up of when
13:19you are waiting for the auction to finish
and view to make your first purchase.
13:23That was such an adrenaline
endorphin moment.
13:27But users loved it.
13:28Nowadays, it seems a long time ago.
13:30Those magic moments are what
you need to look for.
13:33So how do you find them?
13:34Well you can find them in
the non scalable way which is
13:37don't they just sound relatively obvious?
13:41You can actually think through what
is the experience on my product?
13:43What do I want people to do?
13:44You can ask your first users, okay,
you're keeping using my product.
13:49Why are you keeping using my,
13:50what is the thing that makes
you happy to use my product?
13:53What was that first moment?
13:55It's non scalable but it generally works.
13:57When you ask people what is
the best thing about Facebook?
14:01The first thing they say is
connecting with friends and family.
14:04In all the research,
why do you like Facebook?
14:06It helps me connect to my friends and
family.
14:09So you can do it the non-scalable way or
you can do it the scalable way.
14:12Look for correlations.
14:14So Marcus talked at
startup school himself and
14:16he said that we focused on getting
people to ten friends in 14 days.
14:22It's a smooth curve, right?
14:24Number of friends on the x axis and
percent retained on the y axis.
14:28It's not like there's a step change at
ten, if you get nine you don't retain.
14:31If you get 11 you do and
it all changes at ten.
14:33No, it's clearly a smooth curve and
we picked a point on the curve.
14:36But there's a strong correlation between
14:39number of friends you've got in the first
14 days and whether you retain or not.
14:43There's a correlation between number
of friends in the first day but
14:45there are correlations for
any product like did you place a bid,
14:49did you do a buy it now or
did you list an item on eBay?
14:51Was the thing that was completely
correlated with retention for eBay.
14:56You can imagine, I don't know, I mean
DoorDash was founded at Stanford, right?
14:59You could imagine for DoorDash.
15:01You've placed your first order and
that order arrives to your door.
15:05And whether you place an order or not and
how many orders you place you place in
15:07the first week, probably they are actually
correlated with whether you retain or not.
15:13So you need to look for the magic moment
of your product by talking to users and
15:17then try and
validate that with correlations.
15:19Makes sense?
15:23Cool, I love this.
15:25I put it in every presentation I could.
15:29So I'm lucky enough to have been trained
in physics at Cambridge in the UK.
15:33And my favorite and my favorite lecture
series, not my favorite lecture but
15:39my favorite lecture series was in this
thing called dimensional reasoning.
15:42And so it's easy for
me to stand up here and say, okay,
15:44you need to get flat retention and
then you have a viable business.
15:48That's not actually true that's
kind of a glib statement.
15:51If you're producing a social network and
you have 5% retention you're
15:55not going to long term have
a tremendous social network.
15:58Or a messaging app like what use
would WhatsApp be if only 5% of
16:01your friends actually used
WhatsApp who signed up for it.
16:04I mean it would be useless, right?
16:05And there's no way we could release 1
billion plus numbers of active users if
16:09that was true for us.
16:11On the other hand, if you have
a marketplace that is selling fashion.
16:16And you have 5 or 10% retention but
16:18those people spend a lot of money
you have a totally viable business.
16:22And so the level of retention
you need varies dramatically
16:25by the category that you're in.
16:26Obviously, the higher the better
is true in everything.
16:29If you can have an Ebay
type business model or
16:31Amazon type business
model with 90% retention.
16:34You probably end up bigger than Amazon and
that would be incredible.
16:38But I think the point I like to make is,
you can figure out and
16:42I love this as an inspirational story.
16:43So Geoffrey Taylor was
this British physicist,
16:46he was on the edge of
the Manhattan Project,
16:48he wasn't in the corp, he ended up winning
a Noble Prize later for a different area.
16:53And the US and
16:54Russian governments were releasing these
images to say this is how big our bomb is.
16:58But they weren't releasing the actual
power because that was a closely guarded
17:01state secret.
17:03And so what Geoffrey Taylor did he
used this technique called dimensional
17:05reasoning where you look at the units in
the metric you were trying to achieve.
17:10So joules, right, for energy, it's joules.
17:13Kilograms times meters
squared over seconds squared,
17:16that's what joules decomposes into.
17:19In this case, kilograms,
what are you going to use?
17:21The volume of the air times the density
of the air, which handily, is one.
17:25So you've now got meters cubed times
meters squared over seconds squared.
17:29So meters [INAUDIBLE] over
seconds times seconds minus 2.
17:33The time is on here, the diameter,
the radius of the sphere is on here.
17:40You just plug the radius in as m,
you plug the diameter of the time in as
17:46seconds, and you end up getting
25 kilotons as the output.
17:51You get 10 to the 14,
I think is what it works out as.
17:54And it turns out the bomb was 21 kilotons
as measured by the US government.
17:58He could take one photo and
17:59figure out the most closely guarded state
secret in the United States at the time.
18:03You can figure out what retention you
need to have if he can figure that out.
18:07>> [LAUGH]
>> So
18:10here is what I promised,
this is an actual curve.
18:12I presented a few different events, so
18:14I taken off the numbers because
I'm worried they're State secrets.
18:17But this is from 2007 to 2008.
18:20I joined the week we started
our ads business at Facebook,
18:23our self-service ad business by us.
18:26The pink speaks to the right-hand axis,
the blue speaks to the left-hand axis.
18:31What the blue is is how many advertisers
had been in the system at least one day,
18:35two days,
18:36three days because the x-axis is number
of days from acquisition make sense.
18:40So the thing has only
been around 250 days, so
18:430 people have been
advertisers 251 days ago.
18:47And when they turned 51st date and
all the people have been active on day 0.
18:53And then each of those pink data points is
the total revenue achieved from every user
18:58who had registered.
19:00Who was using the service
on their 30th day, or
19:03their 31st day divided by how many
users signed up 30 or 31 days ago.
19:10Not how many were active, right?
19:11How many signed up.
19:13So total in the cohort,
who signed up 30 or 31 days ago.
19:19You divide the revenue they made in
the day by that number of users.
19:22And that gives you that shape
of curve that's up there.
19:26I stumbled over my words a bit there.
19:28I promise you if you look over
it again it makes perfect sense.
19:30I did this with Danny back in 2008.
19:33The really important things I want you to
take away from looking at this curve is,
19:37look how flat the pink line is.
19:42Like I've been talking about
curves flattening out,
19:43this is an actual curve flattening out.
19:46And look how quickly it's flattened out.
19:48By 100 days you can pretty much predict
what the line is going to be at 200 days.
19:53I've seen this over and over.
19:54I've seen this with engagement.
19:56So if you divide time spent
by the same denominator.
20:00I've seen it actually for sentiment.
20:02If you have large enough
sentiment survey and
20:05you divide it by that same denominator.
20:07I have seen it for posts.
20:09I have seen it for
photos uploaded, messages sent.
20:13You can use the same techniques
that I talked about for
20:16determining how much retention you have
and whether your business is viable,
20:21to figure out what is the lifetime
value of one of your customers.
20:25If I was to acquire the total
addressable market for
20:29my business today,
what revenue will I be making in 100 days?
20:33That's what this curve a allows you to do.
20:37You can predict it for posts.
20:39You can predict it for page views.
20:40Time spent, whatever you want.
20:43The same methodology works for
engagement, that works for retention.
20:48And, actually, retention is often
the biggest driver of this metric.
20:52And we did this with very little data.
20:54Later on, there's a stat we are okay
releasing, which was end of 2012,
20:58we had about 700,000 active advertisers.
21:01Beginning we had about
300,000 active advertisers.
21:04And this was five years before that.
21:08So you can imagine there was a very small
data set that was behind calculating this.
21:12The last thing is I said things
get noisy the further up you go.
21:15You can literally see that on this graph,
right?
21:16The variance in the numbers the further
to the right you go on the graph.
21:20So now, how do you operate for growth?
21:23Most important thing is have a clear goal.
21:27Brian, who is awesome on
growth marketing on Facebook,
21:29great guy, he always says, if you are a
growth team your product is a number.
21:35One of the most brilliant things I think
that Mark did in the early days of
21:39Facebook was he aligned the entire company
on monthly active people as our goal.
21:44Now why does this matter?
21:46It matters for a bunch of reasons,
but the biggest and
21:48most important reason is I
think it aligns everyone.
21:53When you have a company of more than
two or three people, or by the way,
21:57a team of more than two or three people,
certainly when it gets to the hundreds.
22:01You can't control what anyone does.
22:03I mean, you could probably
control what one person does, but
22:06then the other 199 would do
whatever the hell they wanted.
22:12So what do you want to do?
22:13You want to make sure everyone is
pointing in the same direction, and
22:16that direction is generally right.
22:18So let's think back to
the early days of any company.
22:21Whether it's Facebook,
whether it's Air BnB, LinkedIn,
22:25whatever you want to look at.
22:26It would be completely reasonable
to have a goal of revenue, right?
22:31You want your company to stay alive,
you want to make enough money that your
22:34company can still be viable, that you can
employ people, that valuation goes up.
22:39So a reasonable person could say,
I think we need to drive towards revenue.
22:44At the same time,
22:44you could have a metric that was like,
if you're Air BnB number of hosts.
22:49If you're Facebook, number of posts.
22:52If your DoorDash number of orders.
22:54I don't even know what this company's
metrics are beyond Facebook's.
22:56You could easily have a production
metric or a consumption metric.
22:59You could have a time spent
in our service metric, and
23:02every one of those metrics would be a
completely reasonable metric to drive for.
23:06So you've got ten reasonable people,
23:10maybe five of them think you
should have monthly active people.
23:13One is really focused on revenue,
two are looking at time spent and
23:16two are looking at number of posts.
23:19Everyone is pushing in a reasonable
direction that is vaguely aligned but
23:23probably will have conflict.
23:25If you want to get ad in front of
users as quickly as you possibly can
23:28you may not necessarily want to put
friends in front of them immediately.
23:32If you want to get them to post
you may dial up the composer and
23:37push down the ads.
23:38I don't know.
These are trade offs that you could
23:40be making.
23:41And if the whole team isn't clear what
the goal is, they will be in conflict.
23:47So having a clear goal that your
company looks to drive is really,
23:51really important, and
23:53having that goal be MAP in my opinion is
the best metric that you can drive for.
23:58Because it is that once
a month visitation,
24:01that base level you can drive off of.
24:02If you're going to do
a consumer internet company.
24:05But fundamentally,
the most important thing is pick a goal,
24:08your team's product is a goal,
focus on that goal.
24:11Align the company around that goal and
24:14then you are going to
operate correctly for growth.
24:17The other thing I always say is
you don't need a growth team.
24:20If you're a small company, if you're
even series A, don't have a growth team.
24:24Don't hire someone like me.
24:26The whole company's job should be growth.
24:29I'm very happy at Facebook,
so I don't want a job offer.
24:32But the whole company's
goal should be growing.
24:36The purpose of a start up is to grow.
24:39So, what is the number you want to grow?
24:40And align the whole company
against growing that number.
24:44Does that make sense?
24:46Now, Naomi is amazing and Naomi is
someone people don't know that well
24:49out of Facebook, but she runs product and
engineering for growth.
24:52She's brilliant,
she's been at Facebook 12 years.
24:55So, when you say I'm original,
I feel not original everyday.
24:58She sits like three feet from me,
she's amazing.
25:00Anyway, she has this approach that
we use to iterate through our growth
25:05planning cycles,
which I think works really well.
25:07Understand, first of all
understand what is going on.
25:11Look at all the data you have,
talk to your users.
25:15Ask your users what they think.
25:17Understand everything that is going
on with your product that you can
25:20possibly understand.
25:21What matters?
25:22Retention.
25:23Are you churning too many?
25:24Do you have the retention?
25:25Is the revenue going up
on a per user basis?
25:27What are the acquisition
channels you're getting?
25:29Which channel are people
resurrecting through?
25:31Are they resurrecting through email?
25:32Is it push notifications?
25:33What's going on?
25:34Understand what's going on for
your system.
25:36Have that all tracked.
25:40Then identify, look through all of those
different metrics, all of that different
25:45research you get in, and
identify where there are opportunities.
25:50Where do you see big drop-offs?
25:52Do you have a lot of people with
push notifications turned on
25:55who are not receiving push notifications?
25:57Do you have a lot of people who
are receiving push notifications who
26:00are not clicking on them?
26:01Do you have loads and loads of social
media traffic, but no search traffic?
26:06Are your email viral invites working but
your SMS ones not, and
26:11what is the difference between those?
26:13Go and look at the data and identify
where there are gaps, and then try and
26:18double down on fixing those
gaps in the execution phase.
26:22Now to this day, this is still
how we run growth at Facebook and
26:25you can even see a a talk Naomi and I
gave online nine years ago I think at FA.
26:31Eight or nine years ago, where we
talked about using this approach and
26:34we have kept it going.
26:36Understand what's going on, identify
the opportunities in that data you've
26:40understood, and then execute against it.
26:43Another really interesting thing I think
was said earlier is data debt sucks and
26:48you're going to regret it later.
26:49And your intuition is going
to run out at some point and
26:52then you're going to want data.
26:53In 2009,
26:54we actually shut down all of the work
the growth team was doing during January.
26:59And we only spent the time logging, and
27:01making sure every critical
flow on Facebook.
27:03Registration, at the time we
didn't log clicks on emails.
27:08So, click some emails, it seems
obvious now but we didn't have it.
27:11Clicks on the emails,
27:12did you then actually log in or
did you fail in the password entry.
27:17All of these different things,
we logged them so
27:20that we could understand
what was going on.
27:22And that was super critical because
actually in March that year,
27:25we had a massive growth slow down.
27:26And we would not have been able to
understand what was going on if we hadn't
27:30done the logging in January.
27:32Because what went on was
actually super new ones and
27:36without that logging in place,
we wouldn't have known.
27:39Get the data logged so
you can run through this process.
27:44So speaking of data, I am very,
very lucky at Facebook.
27:48I do growth marketing, I've run
the awesome internationalization team.
27:52But I also run the analytics team for
the company and
27:55that's been a couple years now.
27:57When I talk about data
to people internally,
28:00I like to say these three points.
28:03There's a meme in the valley that you
operate either for user experience or for
28:08metrics.
28:09And I really want to bust that meme.
28:12Data when used in the right way
gives you empathy about your users.
28:18One example I've talked about in
the past is we launched a product on
28:22Facebook that you may have seen
on your big 42 inch monitors.
28:27No one has a big 42 inch monitor in here,
but I'm sure some of you do at home.
28:31That has on the right-hand side all the
friends you can chat with on Facebook, but
28:35at the top it has this list of stories
that's ticking over called ticker.
28:40When you're on a small screen,
28:41that product used to collapse into
the right-hand column of Facebook and
28:45push things down so that you could
still see these stories ticking over.
28:50That product worked great for
all of us who had
28:53huge numbers of friends who are creating
a lot of content on big screens.
28:57But if you went to the Philippines where
there was huge usage in Internet cafes and
29:02people run narrow screens,
29:05the majority that amended up having
that in the right-hand column.
29:08And the majority of them were
low engagement with Facebook and
29:11couldn't see people you
may know below the fold.
29:15And so, we actually lost friending amongst
users who are very different to ourselves.
29:19And we only figured out what was
happening by looking at the data and
29:23saying, why is this bad on small
screens and awesome on big screens?
29:27Why is it not so great for low engagement
users and super for power users?
29:31It was because we used the data to
understand these users who are different
29:36to us here in Silicon Valley.
29:37Now think about it,
I presume I don't know.
29:39I'm seeing a lot of Apple products here.
29:42I have a Lenovo at the moment.
29:44But most of you probably have a iPhones.
29:46Who has a iPhone?
29:49Who has an Android?
29:51You're the best.
29:52Most people in the world
are using Androids.
29:56But everyone in this room has an iPhone,
first and foremost.
29:59You don't understand.
30:01I'm actually forgetting
how to use iOS now,
30:03which actually is a problem in my job.
30:04But I've got an Android because
the majority, maybe not the majority,
30:08a vast number of our users use Android,
as Jan said in the last talk.
30:13And I need to understand what
the Android operating system offers.
30:17But I can't understand Android and
iOS well as a user at the same time.
30:20So, I used data to understand
what's going on on iOS.
30:25You need to gain empathy for your users.
30:27And research is great, and
talking to your customers is great.
30:30But if you listen to the Twitch guy,
30:35Emmit, talk to your users then go away and
build the product to serve them.
30:43And look at the data, in my opinion,
30:44would be the next thing to validate
if you built the right thing.
30:47If you're doing the right thing,
30:49if your research actually ties up with
what your users have said to you.
30:53So, data gives you empathy.
30:55It's not a trade-off between metrics and
users.
30:58If you build the right metrics,
you are optimizing for the users.
31:02Two, data predicts the future.
31:06Over and over and over again,
I love this quote.
31:07It's a science fiction outright thing.
31:09But like the future is here,
it's just not evenly distributed yet.
31:14So, those curves that
show you the retention.
31:17When I said you can look at your
total addressable market and
31:20say, if I sign them all up,
how much revenue would I get?
31:23Or you just look at who
you signed up today.
31:25Let's say back in those days, we signed
up a thousand advertisers in a day.
31:28I could tell you how much
revenue they would contribute
31:30hundred days out because of that curve.
31:33Now, also you're users
are going to ramp up.
31:35So, a user who shops in one category
if you're eBay or if you're Amazon and
31:39then buys in a second category and
31:40a third category is clearly
going to get more valuable, right?
31:43Shopping the shop, driving up same store
sales is a common technique in e-commerce.
31:48Okay, so
31:49how many of your users are increasing the
number of categories they're shopping in?
31:52You see someone sign up.
31:55What likelihood is it that they're
going to add extra categories, and
31:58what is your revenue
going to be in the future?
32:00You can predict that with data.
32:04AB tests are another great thing.
32:06See all these people walking in
the science march believing in science.
32:10And then when you apply the scientific
method to their product and
32:13run an AB test, they ship the thing
that looked bad in the AB test
32:17because they didn't believe the AB test.
32:20I see that with all kinds
of startups I talk to.
32:26AB tests tell you what's going to
happen when you ship them to 100%,
32:30use them it works.
32:34Finally, and this is something Mark
actually added recently for me.
32:39Data helps you make decisions faster.
32:44If you can not as the CEO, have to have
a conversation over something where you
32:48could just run a test, your company
is going to be able to move quicker,
32:52because every decision
won't have to come to you.
32:56People on the team,
32:57if they're clear on what that goal is,
that north star metric is.
33:00They can run the test,
see if they moved the metric and
33:02then only come to you when they know
if they've moved the metric or not.
33:07Decisions like that help you move faster.
33:09Knowing what your most important priority
is because you've looked at the data.
33:12That helps you move faster.
33:16Make sense?
33:17Cool.
33:19So now you want me to push as much of
the time as possible so I'm pushing for
33:23as much as time as possible.
33:24So let's talk about tactics.
33:26First 100 users.
33:27Sam talked about this brilliantly.
33:29If you actually listen I thought
Tracy in the previous conversation
33:32was just fantastic.
33:33When she talked about getting 30 of her
friends in the construction industry to
33:37try Plangrid, and then asking them
to pay and 29 of them kept paying.
33:40Like ask your friends first for
your first 100 customers.
33:44Get people who will give
you honest feedback, and
33:46I love the point about make them pay,
if it's a paid product.
33:52Second, research and then reach out to
who you'd like to have as your users.
33:57Totally makes sense, right?
33:59Like, you can do how Oscar was
discovered by the Lob Team right.
34:04Just email a bunch of people,
don't get caught in a spam filter.
34:07Don't just auto generate emails or you'll
definitely get caught in a spam filter.
34:11But send personal emails,
34:12get introductions,
see which friend you have in common.
34:16I emphasize by the way, I have never start
up, so I might be entirely the wrong
34:20person to tell you about first 100 users,
but Sam asked me to.
34:23Third, social media and
PR, PR is unscalable, but
34:26it can give you amazing bumps.
34:28Amazing bumps.
34:31And in the early days I think
PR is a good growth leader.
34:33Later on, I think companies don't realize
when PRs stop being a growth lever,
34:37because they have these other metrics.
34:38But you can get these
great one-time bumps.
34:41Just know you can't get them every single
week, and sometimes they are a one off.
34:45And finally, by ads.
34:46I am not as against ads as Sam is,
as a great method for growing users.
34:50I think that buying ads can
be absolutely fantastic.
34:53I built my career buying ads, two of
the most valuable internet companies,
34:57Google and Facebook make all their
revenue from selling ads like ads
35:00are actually a good thing.
35:01If you get the targeting right and we're
going to talk about that in a second.
35:06If you're having a growth team, if you're
focused on growth tactics what people
35:10talk about is growth hacking,
though I hate the term If you pour fuel.
35:13This is a picture from a guy
called Nick on my team, Nick.
35:17I did not tell him to hold his hand into
the flames, which you'll see in a second.
35:21If you pour lighter fluid onto dead
coals with no spark, you get nothing.
35:26And if there's one thing to
take away from this lecture,
35:29you need to have product
market fit to drive growth.
35:32You need retention to drive growth.
35:34Otherwise, every growth tactic,
35:36every acquisition tactic you could
possibly run doesn't matter.
35:42But if you have a spark there.
35:45If you have a glowing ember you can pour
fuel on the fire of your product and
35:50that is what growth tactics does.
35:52So I look at things based on channel.
35:55Targeting creative and then conversion.
35:57So you think about channel,
I'm going to do some greatest hits here.
36:00If ou look at SEO, you think,
I have to think about keyword research.
36:04Which keywords do I want to show up for
on Google on the other search engines?
36:08And make sure you have content
targeted against those keywords.
36:11You need to get links,
links from the internet to your site.
36:14If you're a Silicon Valley startup you
technically get a lot of links off high
36:17value sites so you usually
don't have to worry about that.
36:19But then you need to
link inside your site.
36:21Many people will tell you
links don't matter anymore,
36:23in my experience that's not true.
36:26I don't know why it's not true, but
it's not been true in my experience.
36:29And then finally, there are a bunch
of basics out there on SEO,
36:31you can look them up.
36:32There are a load of websites
that talk about them.
36:34STO book,
36:35MOOs, there are a bunch of websites out
there you can read about SCO tactics.
36:39Four page search.
36:40The biggest thing I see people do wrong
is they dont think about incrementality.
36:43They by page search and
they don't necessarily say, well,
36:47would I have got those customers anyway?
36:49So look really hard.
36:50If you're going to buy page search as
a startup you should see a step change
36:53in your line.
36:54Your marketing team,
36:55if you've reached the stage where you have
a marketing guy should show you a line
36:58that is flat and then they turn
on page search and it goes up.
37:02That's the same of any ads
if you are a small startup.
37:04If your a big company incrementality
analysis there's this great Berkeley
37:08professor Steve Tedelis whose done a deep
study in it with Ebay that I love and
37:12I think he's really smart.
37:14Big companies, you have to be very,
very smart about getting incrementally.
37:17You're a small startup.
37:18Show me a step change.
37:21Do think about marginal not absolute ROI.
37:23So people always look at I spent
this much I got this much.
37:27How valuable is it?
37:28That's not what you want to look at.
37:29You want to look at
the last dollar you spent.
37:31How much did you get for that?
37:32Not overall does that make sense?
37:35Not as many knots.
37:37I'm going to blast through that.
37:39Look at margin ROI, look at diminishing
return curves and economics.
37:43And then again,
you need to do keyword research.
37:47So email and SMS push notifications.
37:49This is something every
startup gets wrong.
37:51We always optimize for ourselves.
37:54Have you seen the Instagram thing on the
internet where someone shows a phone of
37:58an Instagram user with a million
followers when they posted something and
38:01they have push notifications on.
38:03And the string just goes [SOUND] and
38:04all the notifications
are coming in at high speed?
38:07That's what we're like for
our own products.
38:10For our products we are the power
users and by nature
38:13every start up I have gone and talked
to has always optimized for themselves.
38:17Has always optimized for the power user.
38:21And so by optimizing for
the power user, what you do?
38:23Well, we shouldn't put down by default
because it's going to completely expand my
38:26whole screen.
38:27Power users are smart.
38:29If you give them the options,
they know how to turn off notifications.
38:33It's your marginal users you should
care about with notifications.
38:37That person for whom they received
their first Like on Instagram or
38:42the host that gets their
first notification on Airbnb.
38:45Or the seller that gets
their first bid on Ebay or
38:48the buyer who's just been
out bid in their first item.
38:52That's the person that needs
to get the notification.
38:55And needs to respond to it, and
you need to optimize for in your head.
38:59Marginal user is the number
one thing in email, SMS,
39:02push, notifications, and even offline
mail which Rob was talking about and
39:06is still very important for
many companies.
39:08People get wrong,
they optimize for themselves.
39:11Beyond that you need to make sure it
gets delivered, opened, clicked on, etc.
39:19There's so much more here and
39:21if I push through all of it I'm
not going to get to any questions.
39:24Do you want me to get to questions?
39:27>> This is pretty important.
39:28How about this and no questions.
39:29>> This and no questions,
is everybody okay with that?
39:31Everybody for next week.
39:33>> So think about on site merchandising.
39:35You own a lot of media.
39:36There's this idea of earned,
bought, and owned media.
39:39Does that make sense?
39:40So if you post on Facebook and
39:41it gets loads of likes you've
earned that media, right.
39:44If you bought an ad on Facebook or google
or somewhere, you've bought that media.
39:48And then if they're on your own website,
you own that media.
39:52People underestimate
the value of owned media.
39:56If you have a lot of traffic on your site,
ask them to invite their friends.
40:01Ask them to complete that
next action in your product.
40:04Think about your on-site merchandising
as an extension of your product, but
40:08understand that on-site merchandising
is a marketing problem.
40:12So how can you trigger the right thing?
40:16Someone has just bought an item for
you, immediately like Amazon does.
40:20Trigger for them a set of suggestions
like why don't you buy this and this, so
40:24the customers who bought this also
bought this, works incredibly well.
40:29What about creative?
40:32Creative is really important.
40:34It should be personalized, because they're
on your site, you know their name.
40:39Hey Alex, why don't you buy these items
that other similar customers have bought.
40:43Performs way better than buy these other
items that similar customers have bought.
40:47That's one simple personalization.
40:50But instead of just saying this
customer bought this item,
40:52you should do a prediction.
40:53This person, all the items they've
ever bought let me predict for
40:57them in a personalized way
what item they should buy.
41:01Look at Facebook and how often we say
five of your friends have done x.
41:04Social context, that's personalization.
41:07Use people's own profile picture.
41:09We have a great personalization example
where when you're on a Facebook page,
41:15we show you what your ad would
look like if you bought it,
41:18with a Buy This button underneath it.
41:20It could not be more personalized.
41:22On-site merchandising gives you so
many opportunities to deeply,
41:26deeply personalize the creative.
41:28And finally, paid and organic social.
41:31I do work for Facebook,
we do make all of are money from ads.
41:34And I personally think that I am not the
best person to speak about buying ads on
41:38Facebook, because I'm the guy who buys ads
off Facebook to drive traffic to Facebook.
41:41But what I'll say is if I left
Facebook today, and I was a start-up
41:45I would be crazy not to use our custom
audiences and look-alike targeting.
41:49So I highly, highly encourage you if you
think about nothing else on Facebook ads
41:53to look into those.
41:53It's a great way for
you to say these are my best customers.
41:57Upload the list to Facebook.
41:59How do I find more customers like those?
42:01It works incredibly well for
a lot of companies.
42:03And the other thing across all social
media is to a great extent you're
42:06dealing with interesting squared.
42:08Because the click-through rate
on your thing, the like rate or
42:11whatever on your thing will be
related to how interesting it is, and
42:14also the distribution, whether it's being
shared via a messaging app, reshared on
42:18Twitter, or on Facebook, will be driven
by whether people found it interesting.
42:22Make sense?
42:24Cool.
42:24Okay, targeting.
42:27Behavioral is most important.
42:28So what you did Is much more
important than who you are.
42:34There are 50 year old,
60 year old parents and
42:36grandparents who don't need two
friends to be active on Facebook.
42:39And there are some who need 500.
42:41And there are teenagers who need
500 friends and some who need two,
42:45and you're much better to figure out like
what is the behavior that tells you that.
42:49Then say on a demographic basis,
I'm going to do this versus this.
42:53People who buying Nike's, Men's Nike
trainers typically are men, but sometimes
42:58people buy for their boyfriend, or
their husband, or their dad or whatever.
43:02And so if you just target women
with women's trainers and
43:05men with men's trainers
you are making a mistake.
43:08What you should do is look at what
signals you to show the right ad or
43:12the right content to the right
person at the right time.
43:15Targeting matters.
43:19I've talked about this before.
43:20Okay, creative.
43:22Creative is important but
43:23it's nowhere near as important as the
above two unless you have a 10x creative.
43:28If you produce
the Dollar Shave Club video,
43:30probably it's very very worth it but
most people don't produce
43:33a piece of creative that
amazing that changes the world.
43:36Typically I find targeting comes before
creative in every single way, and
43:41channel comes before that.
43:42Does that make sense?
43:44So creative is awesome and really
important and can give you a big uplift,
43:47but think about your channel first and
your targeting second.
43:51If you have creative though,
you should put it in context,
43:53you should personalize it,
you give it a call to action.
43:55On call to action,
this is a great example.
43:58Where on every ad back in the day on
Facebook we had a little link at the top
44:02that said advertise.
44:04We translated it wrong
by mistake in French.
44:07We found this with the data in French,
we translated to clear an ad.
44:12Create an ad.
44:13And we had significantly more,
44:15actually 40% more acquisition
from that channel that
44:17was at the time our single most important
channel for advertiser acquisition.
44:21We took the French translation and
44:22we reverse translated it and
changed the whole world.
44:25Instead of saying the kind of passive
advertise to the active create an ad.
44:28And we achieved a 40% bump in that
most important channel for us for
44:33advertiser acquisition.
44:34That is the call to action,
like what you say matters.
44:38So when I say creative doesn't
matter as much as channel, and
44:40targeting, it still matters a lot.
44:43And the words you use matter a lot.
44:47This is the example I said
about in context with intent.
44:50If you are on your own page,
this used to be ad we would show to you,
44:54that say hey this is what your ad
would look like elsewhere on the site.
44:57And you can see that by hour this
was the hour that we doubled
45:01advertiser acquisition for
Facebook with the weekly push.
45:05If you want to see more about this, Brian
Hale does a great talk about this at F8,
45:09and it's online.
45:10And then the last thing, yes.
45:14The last thing is conversion.
45:16Conversion is really really important, but
45:18it's not important until you've thought
about all the other things first.
45:21So things to think about with conversion.
45:23The landing page people land on
will determine what your conversion
45:26rate is going to be.
45:28If you can, don't have one button
on it that they then have to click
45:32to go to the registration form.
45:33Put the registration
form in the landing page.
45:37It's important to know what your
most important metric is though.
45:40Back at eBay, what we had was this
original metric confirmed registered
45:44users, and a new metric activated
confirmed registered users.
45:48And an activated confirmed registered
user was someone who bought an item,
45:52bid on an item, listed an item.
45:55And a confirmed registered user was
someone who registered and confirmed and
45:58that was it.
46:00To get confirmed registered users we
wanted to pump all of our search traffic,
46:03this was back in 2004 we were
spending huge amounts on search.
46:06We wanted to pump all of our search
traffic at the registration page.
46:10To get activated confirmed registered
users we actually wanted to land it
46:14on the search term that
you were looking for.
46:16So if you were looking for
46:17a trampoline, show you the trampolines
you could buy on eBay.
46:22That change,
46:22and we changed all of our advertising
to change the landing page, dropped
46:27confirmed registered users significantly
that we were acquiring for eBay.
46:31But it increased activated
confirmed registered users.
46:36It increased revenue and
lifetime value per registered user.
46:40And it increased total revenue per day
from the people we were acquiring.
46:45So although I think you can focus on
the registration form really hard.
46:49Any commas, sometimes it's really,
46:51really important to land people on
what they are searching for, so
46:54that they will activate and
they will buy on your service.
46:57So it's not as simple as just put
the registration form online.
47:01But we did do that on Facebook.
47:04So the final thing I like to
say is Facebook moves fast.
47:07And move fast is an absolute
core value for us as a company.
47:11Like core, core, core value and we still
do bold things like taking messenger out
47:15of Facebook,
that was challenging at the time.
47:17But I think now people are seeing
how valuable a decision that was.
47:21And that was a project that I was
deeply involved with and ran.
47:24We're willing to move fast, we're willing
to be bold because fundamentally a good
47:29plan violently executed on today is
better than a perfect plan next week.
47:35Data helps you move fast.
47:37These growth tactics help you move fast.
47:39Best of talks about making decisions with
70% of the data not 90% of the data.
47:43And I deeply believe in that.
47:45So move fast, focus on retention and
47:49have one key metric
that you care about for
47:55your company's growth.
47:58Thank you.
>> [APPLAUSE]
47:59>> Okay, apparently we have time for
48:00one question.
48:00I'm sorry.
48:01Go for it.
48:02>> So you mentioned that [INAUDIBLE].
48:08How do you think that creating the.
48:11>> So the question was for the magic
moments, you can kind of tell how you can
48:14figure out went they happened and
that how you can create them.
48:18I mean honestly if you look at LinkedIn,
48:21Facebook, Twitter every social
network in the known universe.
48:24What we do is, we immediately
ask you to import your contacts.
48:27So once we know that friends is
the thing that we need to do for
48:30you, we focus on what is the way to
remove friction from getting you friends.
48:34If you look at the example I used with
eBay, we hadn't really figured out that
48:38activating you was important to
generating the revenue long term.
48:41So we shifted the landing page and the
registration form to the search results.
48:45So we got you to the item you wanted
to buy as quickly as possible.
48:49And that caused you to activate and
get to that magic moment faster.
48:51So those are the kind of
things once you identify it
48:54I actually think the tactics
fall out very logically.
48:57My boss Javi likes to say that common
sense is the least common of all senses
49:01and I agree with him.
49:02But if you step back and just look okay,
if I want more people to buy an item on
49:05eBay should I land them on
the registration page or
49:08the search results page?
49:09You'd probably land them on
the search result page and
49:12that would get them to
that magic moment faster.
49:14>> Thank you.
49:14>> Thank you, okay.
49:15>> [APPLAUSE]
🎥 Related Videos
![Welp, Here's what the Fed *Just* Said [Fed FOMC Minutes]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fl8u23oq1E9E%2Fhqdefault.jpg&w=3840&q=75)
🔥 Recently Summarized Examples
4 Steps to Master Any Complex Skill (quickly)
40 Years of Fitness Experience in Less Than 11 Minutes.
Gun Controlling Media Makes FATAL Mistake... They Have Tied Their Fate To Biden's & Gun Rights Win
GET READY! Palantir Is Officially The Next Nvidia.
Abundant Thinking: The Hidden Key to Get Everything You Want (Audiobook)