00:05today Sarah and I are joined by Emily
00:07glassbook Sans who's the head of
00:08information at stripe which includes
00:10data science growth machine learning
00:12infra business applications and
00:14corporate technology Emily was
00:16previously the VP of data science at
00:18corsera where she led development of AI
00:20powered products to have personalized
00:22learning scalable teaching skill
00:24measurement and more we're excited to
00:26talk with Emily today about stripe Ai
00:28fintech and education
00:30Emily welcome to New priors thanks so
00:32much for having me oh yeah thanks so
00:34much for joining so you now leave the
00:36information or get striped can you tell
00:37us a little bit more about what the
00:39organization does how it's evolved under
00:41your tenure and what are some of the
00:42span of responsibilities that you're
00:44focused on yeah so I joined stripe uh
00:47back in 2021 originally actually to lead
00:50data science and David Singleton striped
00:54CTO reached out I didn't know a ton
00:56about stripe but I knew millions of
00:58businesses were using it to collect
00:59payments which had to mean really
01:01interesting data on those businesses and
01:03on a large SWAT to the economy Stripes
01:06clearly helping companies run more
01:08effectively and also in a position to
01:10learn from its data what kind of
01:12interventions significantly improve
01:14company's long-term success uh and in
01:16some cases to actually action those
01:18today I wear two hats so uh the first is
01:21I support a bunch of different teams
01:23that are together tasks with enabling
01:25the effective use of data across stripe
01:27and this includes you know from
01:28decision-making and internally to uh
01:30building data powered products we've
01:32been investing a bunch in foundations
01:35which includes building out our ml
01:36infrastructure and better organizing our
01:38data you know the really sexy stuff um
01:41but also in applications like seing a
01:42bunch of new gen bets and and getting
01:45them out to our users so that's kind of
01:46hat one and then second I'm accountable
01:49for our self-served business so a huge
01:51number of smbs and startups come to
01:53stripe directly um to get started they
01:56self- serve through the website and
01:57we're really focused on understanding
01:59who those users are um getting them the
02:01right shape of integration efficiently
02:03um Building Product experiences um that
02:05meet their needs including as they grow
02:08um and growing the portfolio of products
02:10they use so um for many of our users
02:12it's not just payments but invoicing or
02:15subscriptions or billing or tax or rev
02:18um depending on on what their business
02:20model demands yeah and I guess stripe
02:22for a long time has been doing different
02:23things in ml in terms of traditional ml
02:25you know I think fraud detection and the
02:27fraud detection API that you all have as
02:29one examp example of that but you were
02:31actually quite early in terms of
02:32adopting llms and sort of early
02:35generative AI models could you tell us a
02:36little bit more about how that came
02:37about how the interest was sparked and
02:39how adoption really took off I mean I
02:42think it's fair to say that stripe isn't
02:44first and foremost an AI company as you
02:46knowe fex including Stripe have long
02:49used traditional ml in many contexts
02:52including sort of Fraud and risk but
02:54first and foremost we're building
02:55Financial infrastructure for the
02:56internet so stripe got started by
02:58enabling first really digitally native
03:01startups to accept online payments and
03:03then over time millions of companies
03:04started relying on Stripes Financial
03:06infrastructure for a bunch of different
03:08needs whether that's reducing fraud or
03:10managing money flows or unifying online
03:13or offline Commerce all the way to
03:14launching embedded Financial offerings
03:17um and so as not a kind of first and
03:19foremost AI company uh we probably like
03:22a lot of people listening to this
03:24podcast had kind of our like hey what
03:26the heck are we going to do moment uh a
03:28year or so ago when llms really broke
03:32through the Zeitgeist and we were
03:33looking at the technical breakthroughs
03:34and the product launches all over the
03:37ecosystem um with awe but also honestly
03:39a little bit of overwhelm the sense of
03:42well there's very clearly a real
03:43opportunity here to better serve our
03:45users but what is it exactly and how do
03:48we get it off the ground quickly and
03:50safely so it starts with a story of
03:53three Engineers who hacked together in 3
03:55weeks an internal beta for an llm
03:57Explorer and the basic idea of llm
03:59explorer was hey let's get a chat GPT
04:03like interface in the hands of the 7,000
04:06talented stripe employees and really let
04:08them figure out how to apply it to their
04:10work our leaders all the way up to John
04:12and Patrick have intentionally crafted
04:14this strong culture of kind of Bottoms
04:16Up experimentation and we think a lot
04:19about sustaining it internally as we
04:21grow and with llms it was no different
04:23right and so where we started was let's
04:25quickly unlock internal experimentation
04:26let's get llms safely in the hand of all
04:30employees at stripe the enthusiasm was
04:31palpable you know at stripe as it was
04:33across industry we knew the
04:36experimentation um was going to happen
04:38and so we really wanted to make sure
04:39that we enabled it to happen well and
04:41safely it feels to me like when people
04:43start adopting llms they tend to do it
04:45in sort of three areas as an Enterprise
04:48there's what are you doing in terms of
04:49external products and how do you
04:50incorporate it there's how do you use it
04:52for internal tools or use cases and then
04:55there's what are your vendors doing you
04:58know if you're using intercom zendesk
05:00are they adding it and if so how do you
05:01think about that as a company the third
05:03one seems to be each team kind of deals
05:05with it as our vendors bring it up um I
05:07found in in general people have tended
05:09to follow the pattern that you mentioned
05:10which is they start off kind of thinking
05:12hey what should we do externally and
05:13then they immediately collapse into
05:14doing something internally just so that
05:16people get their hands on it they try it
05:18out they kind of see what it does and
05:19how it works and they get some internal
05:21efficiencies and then they start
05:22thinking about the external product side
05:24of it um you know was there anything you
05:27did specifically to start to promote
05:29that internal usage did you do a big
05:31internal hackathon did you try other
05:33ways Beyond sort of the the some of the
05:37things that you mentioned in terms of
05:38adoption internally just so that you
05:39start spreading the thinking and
05:40knowledge about it I think you're spot
05:42on that a lot of companies you know the
05:44first place they go in their mind is how
05:46can this manifest in our product how can
05:47we help our users and then they realize
05:50hey any one person can actually answer
05:52that question we need to be putting this
05:54in the hands of folks with a range of
05:57different backgrounds and expertise
05:59thinking about different parts of our
06:00product and business to really apply it
06:03and that's exactly what kind of this
06:04built-in 3 weeks beta did within days a
06:08third of Stripes were using it and you
06:10can think of llm Explorer basically as a
06:13front end that supports multiple models
06:15in the back end so we started just with
06:17GPT 3.5 and GPT 4 but today we serve
06:20over a half dozen models through the
06:21tool we knew it needed to have certain
06:23security features dripping pii and
06:25rehydrating Etc straight from the start
06:27and we spun up yes a slack Channel and a
06:31hackathon and more to help kind of build
06:33momentum we didn't actually need to do
06:35much to build um momentum within days a
06:39third of stripes we using it and so from
06:41there we started to look at okay what
06:42are they using it for what do we see in
06:44the logs and the answer was Stripes were
06:46using it for all sorts of things
06:48honestly um but there was this
06:50opportunity to create more community and
06:53sharing in the tool directly so that
06:55they could build on each other's work
06:56and weren't sort of doing that in in in
06:58Sid flag so um a simple example but
07:01shortly after we launched the original
07:02tool we set up this little functionality
07:04called presets and basically just lets
07:05you save and share your prompt
07:07engineering um maybe this exists if not
07:10some startups should go build it for
07:11everybody and then everyone else at
07:12stripe can like search and up vote and
07:14you see what bubbles to the top and
07:16basically overnight we had like 300 of
07:19these reusable llm interaction patterns
07:22and they ran the gamut but you know um
07:24just an example like thousands of
07:25Stripes still today use the stripe style
07:27guide which basically you know I don't
07:29care if you're a product marketer
07:30writing copy for the website or a sales
07:32development rep writing a cold email or
07:34like you're an exact who's preparing for
07:35a meeting you run your copy or talk
07:37track or whatever through this style
07:39guide and it returns back to you the
07:42same content in stripe tone the
07:43enthusiasm was palpable and we had to
07:45figure out ways to harness it and build
07:46more of a community around it um the
07:49weekly active user count of this LM
07:51Explorer is still at almost 3,000 which
07:53is just shy of half the company using it
07:55every single week um and yeah for sure
07:58Engineers but also ton of sales people
07:59and marketers and folks so I think
08:02there's a lot that the technology can
08:03actually do to create the community and
08:05then the next step is okay how do you
08:07get it from this like internal
08:10prototyping to actually like enabling
08:13also more production grade
08:15Solutions how did you begin to look at
08:18that data or go from explore to exploit
08:21here right uh so one of the uh um
08:25Stripes I talked to said that I should
08:27ask you about applied ml accelerator
08:29teams I don't know if that's like here
08:30or further along in the funnel of like
08:33you know your your plan to get um these
08:35new AI capabilities distributed across
08:37the company in real ways we can talk
08:40about it here and we can talk about it
08:41later so the idea of accelerators is
08:46fencing one to two Pizza teams and
08:51to get new AI bets seated and one of the
08:57accelerators was actually what produced
09:00this llm Explorer so it's very hard to
09:03just pull three Engineers off of you
09:05know their work building raadar to build
09:08an llm Explorer so we have this sort of
09:11um experimental bets funding internally
09:13it's run out of um you know by David
09:15Singleton so out of our CTO office and
09:18it'll basically be like hey we'd like to
09:20build a one pizza team and we we want to
09:23fund it for 6 months so relatively
09:25durable and here's roughly the charter
09:27and here's roughly the mile stones but
09:30we're going to learn and iterate as we
09:32go and so actually this infrastructure
09:33is an example of an output from the
09:36accelerator um we have other
09:37accelerators that are working on the
09:39applied side so hey you know we know
09:43that given the advances and llms in
09:46particular there's way more we can do
09:48for our support experience both user
09:50facing and also internally for our Ops
09:52agents um and to your comment earlier a
09:54lot for sure there's third party
09:55Solutions we can buy but is there some
09:58you know home grown solution that can
10:00actually be used across a variety of
10:02internal applications um and can we just
10:05go build that so that's that's another
10:06example of the kind of thing um that our
10:08applied accelerators build and I think
10:11you know the applied accelerators aren't
10:12you staff a one you know you fund a one
10:15pizza team and you go hire these people
10:17there are actually opportunities for
10:19growth and development for internal
10:21Talent so the vast vast majority of
10:23folks who join the accelerator have been
10:25at straight for many years um they're
10:27doing a rotation onto the accelerator
10:29it's likely to become their permanent
10:31home but that's up to them you mentioned
10:33you have some favorite applications that
10:35have already like these sort of
10:36assistant capabilities can you talk
10:38about some of them the primary ways
10:40we're finding um llms useful today at
10:42stripe in user facing applications is
10:45first automating the writing of code um
10:48and then second accelerating information
10:50retrieval and both are proving really
10:52powerful for our user so on automating
10:54code um radar assistant and sigma
10:57assistant are two new products that are
10:59in beta and rolling out to all users
11:02soon radar assistant is really about
11:05generating custom fraud rules from
11:07natural language so most folks listening
11:09probably have heard of stripe radar it
11:11was one of our first non-payments
11:13products it's an ml powered product it
11:14helps identify and block fraudulent
11:17transactions um but then in addition to
11:19the core radar product which works
11:21generally under the hood without any
11:23user provided direction we have radar
11:25for fraud teams which is about letting
11:27users write C rules so maybe you know
11:30you don't have any customers in a given
11:32country and you want to block any
11:33transactions um from IP addresses in
11:36that goo to generate these rules
11:38employees that are users used to have to
11:41code up the rules themselves but radar
11:43assistant lets them use natural language
11:45to write those rules so um it's a little
11:47thing but speed matters a bunch in
11:49fighting fraud you have to work faster
11:50than the frauders and with radar
11:52assistant a whole range of people in an
11:54organization from fraud analysts all the
11:57way to less technical folks can
11:59Implement rules quickly and directly
12:01without having to work through a
12:02developer I actually think that's um and
12:04I do want to hear about Sigma assistant
12:06I think that's a really interesting
12:07pattern that applies Beyond perhaps the
12:10fraud world because there are so many um
12:13let's say like just decision engines
12:15today that are some combination of uh
12:18heuristics and then machine learning
12:21together um and uh I I think that will
12:24continue and the ability to um take
12:27natural language explicit described
12:29policy and have that work really well
12:31with less engineering assistance I think
12:33is going to be useful in like other
12:35domains like you know could be
12:36underwriting could be fraud could be
12:38other choices totally and you know I
12:40think for some of our customers um this
12:43is opening the appature in terms of
12:45which employees can use Solutions like
12:48custom radar rules but for a lot of our
12:50customers it's allowing them to use
12:52these solutions for the first time right
12:53so think about the non-technical small
12:56businesses on stripe a bunch of them you
12:58don't have to be technical to get
12:59started on stripe you can use our no
13:00code Integrations you can use payment
13:01links you can use hosted invoices these
13:04are companies who like wouldn't dream of
13:06coding up custom fraud rules and so not
13:09having to have they just don't have the
13:11developer skills on hand and so just not
13:13needing to being able to use these tools
13:15um with just plain English I think is is
13:18really powerful and more broadly I
13:20really love that democratizing power of
13:22generative Ai and it's very much aligned
13:24with with our founding
13:26ethos you're also going to talk about
13:28Sigma assistant Sigma assistant is
13:30similar in that it generates code from
13:32natural language but it's in a pretty
13:34different context it's actually applied
13:35to generating business insights so Sigma
13:38is our sql-based reporting product um it
13:41lets businesses analyze and get insights
13:44directly from their stripe data and
13:45stripe data is as we've talked about
13:47pretty interesting for most of our users
13:49it's all of their Revenue data so which
13:52customers where are buying what for how
13:54much who's retaining who's churning um
13:56pretty Central to a bunch of different
13:58DEC decisions The Firm has to make and
14:00sigma assistant is all about making sure
14:03our customers employees don't have to
14:05speak SQL to get access to those
14:06business insights they can just use
14:08natural language to ask questions um of
14:11the stripe data um some of the folks in
14:12the beta are asking you know really
14:15interesting questions and and getting
14:16them answered you know from the very
14:18basic how much revenue did we generate
14:20in December um to you know what types of
14:23customers tend to be most delayed with
14:25their payments so we're excited to be
14:27rolling that out broadly later this year
14:29where do you hope all this technology to
14:30be in one or two years like how do you
14:32think generative AI will impact your
14:34business your customers the way you do
14:36things when I step back and ask where
14:38should we be in kind of three years five
14:42years I think the vision the opportunity
14:45is is much bigger than what we could do
14:48in in a year with a fintech lens in
14:50general I think the current sort of gen
14:54advances beg the question of what does
14:56it actually mean to apply generative of
14:58AI to the economy at large you could
15:00start with payments optimization I think
15:03folks know that we do a bunch of backend
15:05and front-end optimizations for payments
15:07is there some actually New Foundation
15:09model built on financial data that would
15:11blow the existing conversion and off and
15:14fraud and cost optimization models out
15:17of the water um you know we can do
15:19incremental model improvements today and
15:21quarter over quarter they drive
15:23meaningful bips of uplift but it doesn't
15:26feel crazy to think that a good
15:27foundation model could outperform more
15:29traditional approaches by I don't know
15:32100 bips 200 bips so I think just in
15:34payments optimization alone we can ask
15:36the question of what might Foundation
15:38model look like in that context and then
15:40I think where it gets really interesting
15:43um with generative AI on all this
15:44payments data is can we actually become
15:46more of the economic operating system
15:49for our users and you can imagine all
15:52sorts of ways this could be productized
15:54everything from a dashboard of insights
15:55and recommendations to like an API you
15:57hit to get customer level predictions to
16:00like you know turning important business
16:02model and personalization decisions so
16:04pricing recommendations discounting kind
16:06of on autopilot with stripe we know we
16:08can abstract away a bunch of the need
16:11for our users to worry about payments
16:12and refunds and disputes but you could
16:14imagine also starting to tackle those
16:16sort of higher order tasks understanding
16:18the value of users and setting the right
16:19price and determining the geost strategy
16:22and then this is more at a macro level
16:24but businesses rely on all sorts of
16:27economic signals CPI for tracking
16:29inflation or small business index for
16:31tracking the health of the sector and
16:32those are very useful for steering
16:34business decisions based on macr Trends
16:37but they tend to be quite lagging and so
16:39this question of can real-time data
16:41speed the time to Insight and thus
16:43response I think is interesting so you
16:45know those are all more sort of futur
16:47looking but I'm very bullish on a world
16:50where we're able to really holistically
16:52help users grow their businesses um well
16:54beyond payments but built on payments
16:56data rewinding all the way back to now
16:59you are a year into the exploration how
17:02do you decide to invest Beyond a one
17:04pizza team like does that happen
17:06organically in all of the product
17:08engineering teams you have does it
17:09happen where you like at some Cadence
17:12look at the usage data and say like oh
17:14these like you know top down bottom up
17:15are things we care about like do you
17:17need to restructure the or to make that
17:20happen yeah it's a great question and I
17:22don't think we 100% have the answer of
17:26what's the right operating model but
17:29we've been very conscious and iterative
17:33um as we're learning and so so far the
17:36answer is both like you know it's not
17:39one one piece of team or two piece of
17:41teams it's four of them today and should
17:44it be six or should it be eight or
17:46should it be 10 and then in parallel
17:49where can we really support the vertical
17:51teams or the core product organization
17:54in adopting llms or generative AI more
17:57broadly directly there are a couple of
17:59examples but at stripe we very focused
18:01on leveraging AI so that non-technical
18:03folks that are users can do things that
18:06they couldn't do before and then also so
18:09that technical folks can move an order
18:11of magnitude faster and there are some
18:13pretty obvious industry standard ways
18:15that we're finding llms can automate the
18:17WR writing of code and accelerate
18:19information retrieval and we're building
18:21those both out of the existing vertical
18:24teams and out of the accelerators that
18:26makes sense you mentioned um that
18:28you know I think it was at least six
18:30different models you're using internally
18:32how do you think about what models to
18:33use for what and do you focus on rag
18:36fine tuning open source closed Source
18:39time to First token inference I'm sort
18:41of curious like what that Matrix of
18:43decisions is relative to specific use
18:44cases and how you ended up with this
18:47sort of proliferation of models because
18:48I feel like the more sophisticated
18:49people get the more they tend to have
18:51this proliferation happen internally so
18:55we do have a proliferation of models but
18:57we are not centrally for example like
19:00within our ml infrastructure group super
19:04prescriptive about what model individual
19:07applications need to use so I talked
19:10about llm Explorer and the presets and
19:13sort of that was back in March and we
19:15very quickly turn that into building an
19:16internal API for more programmatic use
19:18of llms right we wanted it to be equally
19:21easy and safer Stripes to build
19:23production grades systems and services
19:25there are 60 applications built on that
19:28now a bunch internal but also several
19:30external and I'm happy to talk about a
19:33couple of them that's what planted the
19:34seeds for a lot of the product
19:35initiatives we now investing in more
19:37heavily um we have default models uh
19:41based on the use cases but we also give
19:44individual teams agency to choose based
19:46on cost considerations latency
19:48considerations um there's obviously like
19:51depending on the application different
19:52performance requirements um and then you
19:55know there also is this very real
19:57question of costs so so you know we're
19:58running this infrastructure centrally um
20:01but we found that for um the most
20:04expensive applications you know we do
20:06Bild them to the local teams and so we
20:09work with them very closely to
20:10understand what makes sense given the
20:12unit economics product um the importance
20:14of quality at this stage how they're
20:16thinking about scaling what the latency
20:17requirements are and so on um we have
20:20heard that previously um it was a little
20:23bit overwhelming for individual teams to
20:25figure out what model to use but also to
20:28the Enterprise agreement and get the
20:30infrastructure up and running and so
20:33centralizing a lot of that I do think
20:35has um sort of economies of scale um but
20:39again we're not prescriptive and we we
20:41do leave agency um to the individual
20:43applications to make those trade-offs
20:45what other infrastructure do you decide
20:47to build centrally right so another
20:49strip told me that I should ask you
20:50about your internal experimentation and
20:53um sort of testing infrastructure and so
20:56love to hear about anything new you've
20:57built in order to um like enable teams
21:00from you know your org or a central
21:02org yeah so you know I think it's always
21:06a combination of buy and build and you
21:10we recognize that um there are a lot of
21:13great companies building a lot of great
21:15ml infrastructure and experimentation
21:17Solutions and some of them are very
21:20point and some of them are very general
21:22and um you know we stitch together where
21:25there's a clear external solution and we
21:26build internally where we feel our need
21:29is more unique or somehow very important
21:32and not currently Satisfied by the
21:34market our experimentation platform is
21:36one that we've built internally we run a
21:39lot of charge level experiments and
21:42latency and reliability requirements for
21:44charge level experiments are very very
21:47high and so building and running that
21:49internally has been worthwhile but there
21:52are lots of cases flight weights and
21:55biases there's lots of third party
21:58solutions that we that we lead on as
22:00well when you think forward on the the
22:03directions that the overall Financial
22:05Services industry is going and let's put
22:07stripe aside for a second because I
22:09think stripe um is obviously a Core
22:12Company to sort of the internet economy
22:14and it touches so many different pieces
22:15of fintech and things like that but
22:18where do you think outside of strip the
22:19biggest white space for fintech's
22:22is like from a startup perspective or
22:24even an incumbent perspective like where
22:26do you think this sort of Technology
22:28impact it's a great question um I don't
22:32know exactly what others will do I think
22:38um having a really robust understanding
22:43identity who businesses are what they're
22:47selling um has always been important and
22:50you know I think often in Industry we
22:52think it's important for marketing or
22:53sales or sort of go to market motions
22:56but it's also super important in fintech
22:59um yeah it's important for credit
23:00lending decisions but it's also
23:03supportability um decisions and
23:07understanding where you know the
23:09business does or does not meet the
23:11requirements of a given card network uh
23:14or a given bin sponsor um and so I think
23:17that that identity piece like who is
23:20this Merchant are they who they say they
23:22are um but also what are they what's
23:25their business what are they selling and
23:27how does that matter up to this pretty
23:30complicated regulatory environment um is
23:33a really interesting and hard problem
23:36that lots of folks are solving in their
23:39own ways but uh is is likely um an
23:43opportunity I think there's almost
23:46certainly an opportunity to you know
23:49whether stripe does it or somebody else
23:50does it to make um sort of financial
23:54Integrations way more seamless um strip
23:57has a whole Suite of no code products so
24:00you can use uh you know payment links or
24:03no code invoicing but how does one
24:05actually build a really robust um
24:09specific to the user integration without
24:12needing um you know a substantial number
24:16of payments Engineers or um any
24:19complicated developer work llms are
24:22proving that they can be very good at
24:24writing code um we have a couple cases
24:26actually where we're already seeing it
24:28work but as the as the decisions get
24:31more and more complicated I think
24:32there's still a lot of work to do um to
24:34build the right integration and to build
24:36it well um in an automated way and then
24:40I think as I mentioned before some of
24:42this layer on top of the payments data
24:45of like okay you could build solutions
24:49that make payments work better but
24:52payments actually allows you to really
24:53deeply understand and improve the
24:58is pretty fascinating and you'd have to
25:00think about like is it a startup that
25:02does that or is it an incumbent that
25:04does that and what's the what's the
25:07business model um what's the business
25:09model there but you know if I think
25:11about the case of stripe
25:14um you know sort stripe has the
25:16opportunity to be beneficent right
25:20incentives are super aligned the more
25:21stripe can help its users businesses
25:23grow the more stripe grows and the more
25:25the economy grows and so
25:28whether it's stripe or someone else
25:30using financial data to help businesses
25:33be more successful to grow the pi to
25:34grow the GDP um I think is is really
25:37powerful it's a really unique data set
25:40is there something in that data um the
25:43obvious example to me that comes up is
25:45radar but uh otherwise like leveraging
25:47that data and giving it back to
25:48merchants in some useful way already
25:51yeah so radar is a great example I think
25:53you also see it throughout our payments
25:55product so maybe the most sing to a
25:58consumer like an end user not our
26:01customer but our customer's customer
26:02would be something like the optimized
26:04checkout Suite so it's this bundle of
26:06front-end payments optimizations and
26:09it's a lot of little things honestly
26:10like dynamically presenting payment
26:13methods in the order that are most
26:14relevant for the customer that really
26:17add up in terms of driving efficient
26:18checkout experiences for end users and
26:20in turn driving up revenue for our
26:21customers and and growing the internet
26:23economy um less Salient to to the end
26:27user is this whole host of backend
26:30payments optimization so um for example
26:33we use ml to optimize authorization
26:36requests for issuers basically
26:38identifying the optimized retry
26:40messaging and routing combinations uh to
26:43recover a big chunk of false declines
26:46about 10% so billions of of dollars
26:48globally um and there are very similar
26:51applications um across a range of our
26:54products so for example for recurring
26:56charges in our billing product we use
26:57Smart Dunning to reduce declines U it
27:00actually reduces declines by about 30%
27:02you basically identify the optimal day
27:04and time to retry a payment um for
27:06transactions that are declined for
27:08example due to insufficient funds it's
27:09really easy to know at what day and time
27:11sufficient funds will pop in and the
27:13list goes on you know stripe radar which
27:15you mentioned you know considers a
27:17thousand characteristics of a
27:18transaction and figures out in less than
27:20100 milliseconds if if each of the you
27:22know billions of legitimate payments um
27:25made on stripe um can go can go through
27:27so you know those are all payments or
27:29payments adjacent optimizations but
27:31conversion off fraud we don't really
27:33talk about cost optimization that's
27:34another one are all places where having
27:37that scale of data allows us to create a
27:40better experience for the end user
27:43create more revenue for the business and
27:46grow the economy so I know that your
27:49background in um like labor economics
27:51has influenced both your career
27:53decisions like joining Corsair at stripe
27:55and your approach to data science can
27:57you like talk a little bit more about
27:59like how you think that shapes you as a
28:01leader or even Stripes approach to like
28:04understanding like macro Trends and
28:05macro data the through line in my career
28:07both in Academia and then in industry
28:09has been using data to understand how
28:11individuals and firms make decisions and
28:13in particular to help those decisions be
28:15higher quality and so you know you
28:18mentioned labor economics I've long been
28:19fascinated by who gets access to
28:21opportunity and why so it started all
28:24the way back in college I met this
28:25playwright in New York she Tau me lesson
28:27fth of Productions on us stages were
28:29written by women and asked if I could
28:30help figure out why and as part of that
28:32I read an audit study so you know some
28:34excellent play rights donated for
28:36never-before scen scripts I sent them
28:38out to hundreds of theaters and asked
28:40them um whether they wanted to put it on
28:42stage why are why not and I just varied
28:44the pen name like is this written by
28:45Mary Walker or Michael Walker um and
28:48briefly you know basically I found that
28:50when purportedly written by a woman the
28:52exact same script was less likely to be
28:53produced but more importantly the
28:55theater Community cared like like they
28:57wanted the best plays in production and
28:59so the study spurred awareness and
29:01overtime change and today half of
29:03Productions on us stages are written by
29:04women and I think that early experience
29:07showed me how powerful data especially
29:09when you use kind of robust econometrics
29:12and causal inference and actually are
29:13getting to the root of the drivers um
29:17can be in understanding and improving
29:19decision-making and it's why I pursued a
29:21PhD in economics it's what took me out
29:23of Academia um to corsera corsera at the
29:26time had only 40 people but it already
29:29showed the potential to dramatically
29:31expand access to world-class learning
29:33and done right also Downstream labor
29:35market opportunities um and that's also
29:38a lot of what led me to stripe you know
29:41well before me stripe was operating as
29:44kind of a beneficent player in the
29:46ecosystem and has been very interested
29:48in genuinely helping businesses on
29:51stripe grow and using data to do that
29:54and sometimes we um help by guiding them
29:58and sometimes we help by actually just
30:00building the product for them but that's
30:02been kind of a through line in in my
30:04journey and a lot of what I I love about
30:07stripe how much does stripe think about
30:09macroeconomics so you have this amazing
30:12view into the global economy through all
30:15the Commerce transactions that are
30:16happening on your platform across so
30:17many different Industries how does that
30:19data inform how stripe thinks about
30:22different aspects of its business so for
30:23example my sense is Google through
30:26AdWords and other AD related products
30:28similarly has a pulse on where is been
30:30happening or not happening does it look
30:32like we're tipping into a recession that
30:34impacts hiring decisions or other things
30:36for them I'm just sort of curious if
30:37some more things translate for stripe
30:38over time yes certainly um we do get
30:42rich insight into where the econom is
30:45headed and we use it to guide our
30:47internal decision making I think there's
30:50an interesting question we're exploring
30:52on is there a version of this that we
30:55can actually be providing to our users
30:58to help them make decisions and help
31:00them grow the example of like the CPI or
31:04small business index and like can we get
31:05that in users hands six months earlier
31:09so that it's way more actionable is a
31:11really interesting question and honestly
31:14we're early days there but I think as
31:16part of thinking about how might we
31:18become more of the economic operating
31:20system for our users it's not just the
31:23micro components of you know how do you
31:26price or how do you personalize it is
31:28also the macro components of how do you
31:31think about the ecosystem that you're
31:33operating in and how can we help you
31:36operate more effectively given the macro
31:39trends that you're operating in that
31:41makes sense you don't look at the one
31:42pizza team and say no pepperoni this
31:43month you know she's only no no no no no
31:46I mean I think you know joh and Patrick
31:48pretty well they as yeah no that would
31:50be Whiplash stripe is in a um very
31:53fortunate position to really be in
31:56charge of Our Own own destiny and be
31:58able to take a very long-sighted view in
32:01choosing where and how to invest in the
32:03business and so no from the from the
32:05perspective of like do we add 10 people
32:08or 100 people or a thousand people we're
32:10not micromanaging at that level that's
32:12much less driven Honestly by the macro
32:15on average and much more driven by where
32:18do we see opportunities to serve users
32:21given what's happening and I mean we can
32:22even talk about AI users right AI users
32:25actually have there's this whole wave of
32:26AI startup and they have fundamentally
32:28different needs than a bunch of the
32:30waves of startups before them and so
32:32that actually begs the question of where
32:34do we invest more now to get ahead of
32:37those needs um because we know because
32:40we know there's demand yeah it makes a
32:41lot of sense I guess U the last area
32:43that we had sort of questions about
32:44given your um background and all the
32:47amazing things you've worked on over
32:48time is you know you spent a lot of time
32:50at corsera which is really focused on
32:52how do you bring different forms of
32:54online learning and knowledge to the
32:55world and one of the areas that a lot of
32:57people have talked about from a global
32:59Equity perspective in Ai and its impact
33:01is education yeah and so we're really
33:03curious to get your thoughts on how you
33:06view AI impacting education but also
33:08importantly where will that first
33:10substantiate is that a us-based thing is
33:13it certain countries or markets is it K3
33:1612 is it college is it post col learning
33:18like we're just a little bit curious how
33:19you think about you know Ai and
33:21education and where is it going to be
33:23most important in the short run versus
33:24long run I was at corsera for about
33:27years I Grew From an IC to Leading the
33:29endtoend data team and through that
33:31journey I was increasingly motivated by
33:33Building Products that were only
33:34possible because of the data and the
33:37first places we started were in the
33:39obvious places oh you personalize
33:41discovery of content you personalize The
33:42Learning Experience you do more to scale
33:45the teaching experience but where we
33:48moved to relatively quickly was what you
33:51would think of as less education and
33:55more labor market which which is how can
33:58we use Education data to help Learners
34:01and companies measure and close skills
34:04gaps and get folks into the jobs that
34:07best fit their skill profiles and so you
34:10know that's not at all to downplay the
34:12opportunity of that we have in AI to
34:16make meaningful advances in how you know
34:20Elementary School students learn and
34:22make that learning really customized to
34:25them and make sure that there is high
34:27quality instruction in lots of pockets
34:29of the world that wouldn't otherwise
34:31have it but I also think there's this
34:33important pull through to the labor
34:35market and you know I'm a labor
34:37Economist by training you people get
34:39education for two reasons they get it to
34:41develop um skills but they also get it
34:44to be rewarded for those skills in the
34:45labor market and that first piece is
34:47like how you develop skills the learning
34:49and that's really important in AI can
34:51definitely help but the second piece is
34:52like how you signal that learning out in
34:54the market how you build a credential
34:57and you know I I was on um um some World
35:02economic forums and we were working a
35:05bunch on hey could we make with data
35:07skills more the currency of the labor
35:09market and corsera substantially moved
35:11that direction including in their
35:12Enterprise product and I hope many
35:14others will move that direction too um
35:17not instead of or as a substitute for
35:19using it in the learning experience but
35:21just recognizing that so much of what
35:23individuals need from education is that
35:26signaling is that credential and I think
35:28the best way most Equitable fairest way
35:31to do that uh is through skill
35:32measurement great that makes it ton of
35:34sense as discussed earlier stripe has
35:35this amazing Vantage Point into all
35:38sorts of different online businesses and
35:39how they're evolving over time what are
35:41the differences between some of these
35:43AIC Centric sort of nextg companies that
35:45stripe serves as customers versus what
35:48you've seen traditionally in e-commerce
35:49or SAS or other areas it's a great
35:51question I mean we've worked hand inand
35:53with the Builders of a bunch of
35:55different Technology Way waves to make
35:57sure they have the financial
35:59infrastructure they need some of the
36:00earliest waves were marketplaces infl
36:03platforms social media um think kind of
36:05the young door Dash or instacart or
36:07Postmates or twilio and you know those
36:09grew up to become some of the the
36:10largest companies today we've we've
36:13grown up with them there's also as you
36:14noted kind of the SAS wave and the
36:16current wave is AI and in terms of the
36:19unique needs of AI startups um probably
36:22four notable differences versus the
36:24prior waves you know the first is just
36:27at a basic level unlike a bunch of the
36:29past generations of software startups
36:31we're seeing AI startups have
36:33substantial compute costs right out of
36:35the gate and that that's putting a bunch
36:36of pressure to build monetization
36:38engines faster um the second thing we're
36:42seeing is a lot of these startups are
36:43seeing Global demand for their products
36:46straight out of the gate right they're
36:47making digital art or music or all sorts
36:51of borderless things and they want to
36:54get that across borders from day one
36:57third I would say is a lot of
36:58subscription businesses and obviously we
37:00see subscription businesses in a bunch
37:01of different context but especially sort
37:03of the AI startups that are consumer
37:05facing heavily skewed towards
37:07subscription business models and then I
37:09just say fourth as a COR layer of the
37:10first maybe obvious because the startups
37:13are generally monetizing at a much
37:15earlier stage they're in an interesting
37:17spot where you know with very lean teams
37:20they need to operate financially uh like
37:23very real businesses right they need to
37:24grow up uh a little faster than they're
37:27than they're sometimes ready and so uh
37:29we're seeing you know a bunch of uh
37:32adoption of our revenue and financial
37:34automation Suite uh to deal with to deal
37:37differences that's pretty amazing it
37:39actually reminds me a little bit of the
37:4070s original uh vesting schedules were
37:44four years because companies would go
37:45public within four years and so that's
37:47where the fouryear vest comes from for
37:49stock and so I think historically
37:52companies used to grow up really fast
37:53and then you look at the initial
37:54internet wave and Yahoo and eBay and a
37:56variet of companies became profitable
37:57within a few years and so it feels like
38:00this AI wave is exhibiting a lot of the
38:01same characteristics and that may just
38:03be reflective or indicative of um real
38:06product Market fit an enormous user man
38:08that's almost pent up I feel like
38:09whenever you have one of those waves
38:10that's when you see this rapid
38:12monetization it's happening so fast I
38:14mean we saw a massive spike in the
38:15number of generative AI companies on
38:17stripe over the last year you know and a
38:19bunch of them were two person teams
38:20you've likely never heard of all the way
38:22well maybe you've heard of but most
38:24people haven't all the way to kind of
38:25hypers scaling startups with millions of
38:27users like otter Ai and and mid Journey
38:30you we're looking at the list of top 50
38:32AI companies put out by Forbes last year
38:33and notice over half were using stripe
38:36and you know oftentimes you have top
38:38startups and a bunch of them aren't even
38:40really monetizing yet and it's striking
38:42what share of these AI companies are
38:43monetizing and monetizing early um and
38:46monetizing fast at the foundation layer
38:48yes open Ai and mistrall um but also a
38:50bunch of companies at the application
38:52layer um moon beam for writing assistant
38:54or runway for video editing which is
38:56pretty remarkable Emily this is a great
38:58conversation thanks for do joining with
38:59us thank you so much for having me yeah
39:01thanks for joining very good to see you
39:05usual find us on Twitter at no prior pod
39:08subscribe to our YouTube channel if you
39:10want to see our faces follow the show on
39:12Apple podcasts Spotify or wherever you
39:14listen that way you get a new episode
39:16every week and sign up for emails or
39:18find transcripts for every episode at
