00:05today I no priors we're having a special
00:09episode of Sarah and me just talking
00:12hello Sarah how are you hey alot what's
00:15going on I see you a lot not much good
00:17to see it let's talk about models what's
00:18going on in the model world yeah um I
00:21guess there's a lot of hand models that
00:22are emerging so I was thinking of maybe
00:24trying to do that eventually it's almost
00:28as good of a business as as investing I
00:30know right um yeah so there's been a lot
00:32that's happened in the model world uh
00:33recently obviously Google launched
00:35Gemini which I think had a few
00:36interesting characteristics both in
00:37terms of uh performance but also the
00:39huge context window right it was Million
00:41token context window uh companies like
00:43magic I think in the past have actually
00:44put out like a 5 million token context
00:46window model and things like that but
00:47it's really exciting to see that and I
00:49think for certain application areas like
00:51biology longer context Windows actually
00:53seem to be quite important and so for
00:55example if you're doing your protein
00:56folding model and you have a short
00:59context window you're often actually not
01:01encapsulating much of the protein right
01:03the average uh protein is I think
01:06something like 300 amino acids long at
01:08least in the human genome but there are
01:09things that are dramatically larger than
01:11that and so you just can't capture it in
01:13some of the context window is being used
01:14for biological models and so I do think
01:15this is going to be one of those areas
01:17that will end up being more important
01:18than people think at least in the short
01:19run um but Gemini 1.5 seems to have some
01:22really interesting performance
01:23characteristics there's obviously Sora
01:25from open AI which was um the video
01:27model that uh you know is is beautiful
01:30to watch you know there's other model
01:31companies like Pika and others that I
01:33think are doing exciting things as well
01:35and then um mrr or liiz launched uh Le
01:40chat which is really the name of the
01:42product Le big model Le big model Le Big
01:45Mac I believe they call it mistol large
01:49lar M large they launched that and the
01:54thing that's really really impressed me
01:55about mrst is the velocity of
01:58shipping it's incredible impressive they
02:00went from basically starting the company
02:03to almost GPT 4 level in less than a
02:06year right n months y it's amazing and
02:08they have uh you know small performant
02:11models they have the Big Mac or you know
02:14the large model they have chat they have
02:17multiple languages it's just it's very
02:19impressive execution so and then I think
02:21the other thing that they just launched
02:22or announced was that deal with
02:24Microsoft where you know they're they're
02:26now being licensed onto
02:28Azure and so I I think the main models
02:30in Azure now are open AI llama mstr and
02:32then some of the Microsoft models so
02:34again that's striking as well so uh just
02:36very impressive progress by that company
02:38so far I think the design space for what
02:41you actually want from Models is
02:43certainly going to include
02:44state-of-the-art capability and mrr is
02:46very much going up after that and
02:48they've said so but I I I think like
02:51from the beginning the company has
02:53talked about um efficiency uh and
02:57latency and the ability to serve
02:58different use cases with that and um and
03:02also you know being long-term proponents
03:04of retrieval right like one of the big
03:06debates in the research world right now
03:09I don't know how much of it is a debate
03:10but people are talking about it I'm on
03:12one side of this is that um like Rag and
03:15retrieval is dead with sufficient
03:17context and uh curious what you think
03:20here but I'm I'm more of the belief that
03:22it just opens up the set of trade-offs
03:24you can make between um retrieval more
03:27sophisticated retrieval and model
03:28reasoning by having a larger context
03:30window versus saying like we don't need
03:32any um ability to uh work with a
03:37specific data set versus just retrain or
03:39um stuff something into context yeah
03:41we're going to have both in my opinion
03:42the other thing I think that's very
03:43underd discussed and this could lead
03:44into agent stuff but I'd like to um also
03:47spend a little bit time on Gemini before
03:48I move to agents is if you look at a lot
03:50of the optim that are done for um areas
03:54where you had uh human related sort of
03:57reasoning or other components pre- llm
03:59based reasoning uh a lot of it was
04:01happening at infer time right so when
04:03you were doing when you're trying to
04:04build a better poker AI a lot of what
04:08you did was um you know certain types of
04:11tre searches or other things when you
04:12hit inference time right you built the
04:14model but at inference it did a lot of
04:16work and I think that's also a little
04:18bit underd discussed in terms of
04:21probably a lot of what's going to happen
04:22in the future particularly we get into
04:24agents and reasoning is stuff that's
04:26happening at that point of inference and
04:28then it's used to sort of Fe back over
04:30and um sort of continuously train or
04:33retrain a model over time because I
04:34think that's the other piece of it is
04:35you know from a model perspective
04:38you uh spin up a giant Data Center and
04:41you spend $100 million over 12 months
04:45overall between all the different works
04:46that you do and everything to launch
04:47your next model and then you have a file
04:50and then you use that file sry for the
04:52next year as you train the next model
04:54versus saying you're going to do some
04:55sort of continuous upgrading or training
04:56and so all these things are going to
04:58shift over time I think it's early in
05:00the technology cycle and so all these
05:01things are going to happen um you know
05:04one of the companies that has a lot of
05:05capabilities to do interesting things
05:06over time of course is Google so I'm a
05:08little bit curious if Gemini has changed
05:10your opinion of sort of the AI model
05:12race and what role Google plays in the
05:15future you know has it not changed your
05:17mind much I think the question on like
05:21whether or not uh Google has the ability
05:24to do the research work to have a
05:27competitive product uh has been answered
05:30right Gemini is a very impressive model
05:33I think the um the capabilities that
05:36they have internally that they haven't
05:37released yet around um additional like
05:40function calling and multimodality are
05:42also really really impressive and so the
05:44questions around Google are less about
05:47do they have like they have all these
05:49extraordinary advantages and you're
05:52you're the exg googler like I want to
05:53hear your opinion but they have the
05:55distribution they have the C the
05:57consumer Behavior they have all the data
05:59on like what the search behavior is they
06:02have the data on what queries are
06:04valuable and which they would peel away
06:06and turn into like an answer um uh they
06:12like advertising auction systems and
06:16they have a great research team and
06:18enough gpus and um and the model
06:21capabilities you think it was
06:22Progressive enough though do I think the
06:25models are progressive enough yeah one
06:27might actually ask if they're perhaps a
06:28little too far in that direction right
06:31um and and so I I think like the
06:34question is actually can they steer
06:37Google to like focus on being
06:39competitive versus the many other
06:42demands from their employee base um and
06:45like different missions that are not you
06:48know brokering the world's information
06:50and like market cap yeah it's
06:53interesting because um the launch of 1.5
06:56has made me more bullish on Google uh
06:58and I I was always actually quite
06:59positive on them right like um I think I
07:01read a blog post a year a year and a
07:03half ago basically about the model world
07:04and one of the things that I mentioned
07:06at the time was I felt like Google was
07:07kind of a sleeping giant and once once
07:09it awoke you know um it could really
07:11make enormous progress quickly and just
07:14as mrr's executed from scratch as a
07:16startup which is extremely hard to do
07:18right you're literally building
07:19everything from the ground up um
07:21although obviously there's open source
07:22to support you and all these other
07:23things but fundamentally you're just
07:24building an entire company it's pretty
07:25amazing right um uh Google has uh really
07:29accelerated its efforts and it's had a
07:32series of launches over the last two
07:33three months that have been quite
07:34impressive in terms of the Velocity from
07:36cold start to to having things that are
07:40accessible and they have all the
07:42resources that one would need in order
07:43to do extremely well in AI right they
07:47have the compute they have unique
07:49proprietary data as well as all the data
07:51from the web all the data from YouTube
07:54um they have specialized data that you
07:55could potentially opt into like you know
07:57all your emails and your Google Docs and
08:00you know they have this immense Corpus
08:01of really valuable information um and
08:04then they have amazing
08:05talent and so really I think the the
08:08thing that was um lacking until recently
08:10was the will and it seems like now
08:12because of the competitive Dynamic the
08:14will has been reborn right um and so it
08:18it really feels to me like they are
08:20going to make really big strides going
08:23forward and um you know it's always
08:25possible that the velocity only
08:26increases from here for them if think
08:29about the domains in which these um
08:33General llms are still not as capable I
08:38mean it's every domain but um in
08:40particular not as capable as we want um
08:44like two of the areas one you already
08:47um uh that I I'm I'm excited about
08:50include like biology and then robotics
08:53right so maybe let talk about that for a
08:55second as a as a task for example if you
08:58ask chat GPT to design a DNA sequence
09:00that can express crisper cast 9 it can't
09:04do that yet right and if we think about
09:06cell design protein design protein
09:08optimization a lot of these um are areas
09:12where you have researchers showing like
09:15really exciting progress in use of
09:17Transformers and diffusion models to um
09:20get to much better predictions for for
09:24example um drug Discovery and um Target
09:28identification so I think you know I've
09:31seen a number of companies in this area
09:33of better understanding of biology that
09:35really feels like a different type of
09:36reasoning a different type of data set
09:38and as you said even um like specific
09:40context window constraints and so I
09:43think that's an interesting one and then
09:45on the um I don't know if you wanted to
09:47mention the robotic side or if that's
09:48something you've been looking at too the
09:50robotic stuff seems super interesting
09:51it's a little bit earlier on than some
09:54of the other models and and part dat to
09:55data constraints but it seems like
09:56there's pretty reasonable ways to
09:57generate some of that data now so so um
10:00it seems like the you know in general I
10:02wouldn't be surprised if 2024 and 2025
10:05is the year of proliferation of
10:07models um where we're going to start to
10:10see an expansion in terms of the
10:11different types that are covered you
10:12know chemistry Material Sciences etc etc
10:15robotics will be part of that biology
10:16will be part of that maybe physics and
10:18math um I think maybe the last thing
10:21that is happening from a model
10:23perspective is I think the last few
10:26weeks have seen um a lot of different
10:28sort of agent companies get up and
10:31running and um I think that's been an a
10:34really interesting wave and some of them
10:37again are taking very different
10:38approaches from the traditional let's
10:39just build a giant llm and they're
10:41looking at things like alphago or some
10:45Centric uh work that have been done in
10:47the past you know how do you build a
10:49better Pro poker paper how do you build
10:51diplomacy how do you build
10:54go and there you have a very strong
10:56notion of acting sequential based on
11:01information you have some forms of
11:03what's known as selfplay you know you
11:04you have the machine play itself a
11:06billion times ago and it learns new
11:08patterns based on that you have really
11:10interesting approaches in euristic and
11:12algorithms at time of inference versus
11:15training and so I think that that
11:18purpose of knowledge is about to hit the
11:20world in the context of new products and
11:23it'll take time for those products to
11:24emerge you know six months 12 months a
11:26year but um it does feel like that's
11:30another wave this coming where you're
11:32fundamentally uh different approach to
11:35re uh that involves reinforcement
11:37learning but is just different in terms
11:38of how you think about what you're
11:40actually doing in architecting and what
11:41you're influencing and all the rest so
11:43that that's the one other area on the
11:44model side that I think is very exciting
11:46one thing that I've seen um here is that
11:48people are getting much smarter about
11:51agents as part of systems versus
11:54expecting to um uh simply like construct
11:59an agent and have it work with
12:01compounding failure across a bunch of
12:03tasks right uh in a gen General
12:07environment across any type of software
12:09right and so if it is operating in an
12:11environment that supports reinforcement
12:14well like a game environment or even a a
12:17web application environment um but one
12:21that is constrained to particular tasks
12:23or working agents working in domains
12:27support a sampling and validation like
12:30code generation like I'm really excited
12:31about that and I I feel like I've begun
12:33to see the glimpse of some of those
12:36things work whereas a very real question
12:38you could have asked in Q3 Q4 of last
12:41year would be like does is any of this
12:43stuff useful right is it anything and I
12:46think now it's like it is yeah yeah
12:48people went too broad to early versus
12:50just saying I'm just going to focus on a
12:51handful of targeted use cases or domains
12:54and I'm going to figure out how do you
12:56create feedback loops in those domains
12:57so I can actually train effect
12:58effectively and so you know the very ear
13:01early versions of this even predating
13:03this llm wave was um you know hey we're
13:05going to have a browser plugin and it'll
13:07watch everything you do and then it'll
13:09do which is a very different from
13:11Problem from saying hey we're going to
13:13make RPA better we're going to make code
13:15better we're going to make customer
13:16support better we're going to make you
13:17know XYZ thing better um so I think the
13:20targeted approach makes a lot of sense
13:21yeah and I think some of the teams
13:23working on this have also they've
13:25actually experimented with Post train
13:29in environments where you can pay for um
13:34for uh human feedback data right and if
13:38you do that then you actually understand
13:40like the um the distribution of data you
13:44need the scale of data you might pay for
13:46and that's very exciting because it
13:47turns it like the agent problem from one
13:51that is um like open-ended untenable to
13:54just like how much is it going to cost
13:56to make a particular task work and I'm
13:58massive oversimplifying here but that is
14:00a very different proposition when scoped
14:03than like as you described the initial
14:05set of Fay into agents which is like you
14:08know we'll try to do anything yeah that
14:10makes sense I think we'll still get
14:11there but there is um like rapid success
14:18front Nvidia everybody's talking about
14:21earnings what do you make of it I think
14:23earning money is an excellent idea how
14:26about you I think Jensen understands
14:28this better than everybody else I think
14:30one thing that people have been talking
14:32about is whether or not this was a like
14:35short-term phenomenon right like if um
14:38there was only so much demand and once
14:42the supply chain caught up a little bit
14:45um there would be less insane growth and
14:48I think now people are pretty confident
14:50especially hearing Jensen's comment that
14:53they expect to continue to be Supply
14:55constrainted through the rest of the
14:56year demand is just like much much much
14:59larger than I think most people expect
15:01on the capex side um and I think it's
15:04like worth understanding the upgrade
15:06cycle that drives that right because
15:09there's this huge efficiency incentive
15:10to upgrade from A1 100s to h100s to h200
15:15b100s I was talking to one of my
15:18portfolio companies that's buying in the
15:20tens of thousands of GPU size and is
15:23skipping to b100s because they described
15:26it as like free money in terms of
15:28training efficiency it's funny when
15:31somebody describes spending hundreds of
15:32millions of dollars as free money but
15:34free money in terms of training
15:35efficiency if you can actually get
15:37access to a cluster of a certain size
15:39and so if if others feel that way it it
15:43is um wild how much this expand like
15:47expands the the server Market yeah it's
15:49probably a good time to run a hedge fund
15:51I think in general um one thing that's a
15:54little bit under discussed is a lot of
15:55the emphasis on startups and startup
15:56rounds and know look the startup race
15:58100 million or whatever and the reality
16:00is a lot of the spend is the big
16:02hyperscalers and then other clouds that
16:03are building out right now and then I
16:06think the other thing is that if you
16:08were to look at at least Enterprise
16:10adoption of AI it's still really really
16:12really early days and despite that if
16:14you look at Microsoft Azure Revenue in
16:17the last quarter they mentioned that um
16:19Revenue grew by 5% from AI related
16:22products which if I'm doing the math
16:24right if it's a $25 billion a
16:27quarter uh Azure sort of um Revenue then
16:31that means they're adding something like
16:32one one and a half billion a quarter in
16:36new spend due to AI right so that's five
16:38or six billion annualized and so um you
16:42know one thing that is a little bit uh
16:45uh perhaps not talked about is there's a
16:47lot more stuff coming and over the next
16:50two years three years Etc as Enterprises
16:52really adopt this at scale we should
16:54anticipate as well that um you know the
16:57need for compute will continue to grow
16:59so it's really interesting to see but
17:00this replacement cycle you're talking
17:02about the massive spend by big Tac on um
17:06llms because they're driving most of the
17:08spend on llms because they're they're
17:10the big rounds right the big rounds
17:12aren't Venture capitalists investing
17:14billions of dollars it's the big tech
17:15companies it's Amazon and Google and
17:17Microsoft and and uh Salesforce in
17:20Nvidia actually right um and then
17:23there's the Enterprise adoption which is
17:25still TBD so yeah there's a lot going on
17:28on this point if you look back um a
17:30month you know AI years are like dog
17:32years so a year to The Meta earnings be
17:36at the end of January did you did you
17:38see this article uh that David Khan
17:41wrote at Sequoia the 200 like ai's 200
17:44billion dollar question yeah was this
17:46where he basically said based on the
17:47spend if you think of the ROI you need
17:49then you need to generate hundreds of
17:51billions of dollars in return yeah in
17:53order to justify all the yeah all the
17:54spend that you had yeah yeah very succin
17:56summary and um I was like okay okay yeah
17:59that is the question and I feel like The
18:01Meta earnings beat was the like one day
18:06answer to that question right so to your
18:09point they're one of the large Spenders
18:11um uh they said they're going to spend
18:1330 to 37 billion doll on uh capex in
18:182024 driven by like AI driven by servers
18:22right um Mark has this great like quote
18:25where he's talking about 600k uh h100
18:28equivalent units of compute and saying
18:30like there's no room for other people
18:32but the response to all of the
18:34investment that has um in in capex for
18:40um training and inference at meta over
18:42the prior years has been like a huge
18:45earnings beat from better targeting
18:47leading to better conversion better
18:49recommendations leading to better
18:50engagement better advertising tools
18:52leading a better Roi um as well as like
18:54the cost controls that the rest of the
18:56industry is doing and so they have had
18:58this one day I thought it was really
18:59nice that the number was exactly this
19:01too they this one day ad of 197 billion
19:04of market cap biggest single session ad
19:06before Nvidia I forget where Nvidia
19:08ended up Landing after their beat uh but
19:11like that's the answer right like you
19:13know 197 billion um of increase in
19:20on 25 30 billion of capex like you
19:24should keep doing it yeah yeah it's kind
19:26of amazing it's kind of a related
19:27question I remember Yuri Milner showed
19:29me this chart which basically he looked
19:32at the aggregate increase in startup
19:34market cap and the aggregate increase of
19:37what at the time was like Fang market
19:38cap and obviously now there's like the
19:39magnificent seven or eight or whatever
19:41it is um and so if you looked at the top
19:44tech companies of the time they added
19:46like I don't remember it was five or 10
19:48times the market cap of all the startup
19:49ecosystem combined during the same
19:51time and to some extent You could argue
19:54we're going into the same thing at least
19:55in the short run for AI and we still
19:57haven't seen the monster AI companies
19:59emerge from scratch and in dly those
20:00will exist um but at least for the next
20:03few years it seems like where we're
20:04going to see that really huge market cap
20:07incremental ad um maybe companies like
20:10openi and some of the model companies
20:12but also it seems like increasingly it's
20:14just going to be existing companies
20:16adding huge amounts of uh uh revenue and
20:18earnings and um compute and everything
20:22else along the way so it's back to like
20:24maybe the right thing to do right now is
20:25just start a hedge fund I think that
20:29also begs a question of
20:33um how to think about like all of the
20:37other companies like Tech and not in
20:39terms of um amount of impact from AI
20:42actually think it would be like a really
20:43fun lens to run a HED from um uh with
20:47because you can take a you can take a
20:49very long-term view of something that
20:51feels very secular just classify
20:53companies this way and long short like
20:55take that strategy as the only lens um
20:58because like I I do think that there are
20:59a number of services companies that are
21:03um squarely in the sites of things that
21:08you will be able to significantly
21:10automate and the only question is which
21:13of these management teams is going to
21:15have the um investment capability
21:18technical Talent guts conviction to
21:21invest the way Mark did through you know
21:24people were really mad about the cap
21:25expend for a few years at
21:29right and I think the answer is mostly
21:31especially some of these Services firms
21:32like um maybe they partnered to get
21:35there but they mostly will not make the
21:37transition I think the other thing it
21:39isn't really discussed is the impact
21:40it's already having on some businesses
21:41so obviously service now had like a
21:43blowout quarter and part due to AI so
21:45we're starting to see a little bit of
21:46Enterprise adoption um one of the folks
21:48from Clara posted uh today that they
21:51built an AI assistant that's powered by
21:53openai that in its first four weeks
21:56handled 2.3 million in customer service
21:58chats for them and so I ended up
22:01handling 2third of all their customer
22:03service inquiries it was onar with
22:05humans in terms of customer satisfaction
22:08it was higher accuracy so it led to a
22:1025% reduction in repeat queries um
22:13customers resolved their errands in 2
22:15minutes versus 11 minutes it's live 24/7
22:18in over 23 markets communicating in over
22:2035 languages and it performed the
22:23equivalent job of 700 full-time agents
22:27and so BAS basically Clara in you know a
22:30few months or a year or however long it
22:31took him to build this built this
22:34customer service chat product and it
22:36replace 700 people's worth and they say
22:38that at this point they have something
22:40like 3,000 full-time agents and so it
22:43cut the agents needed to BU about 25%
22:46right and so uh it's this really
22:49interesting post from clarner where they
22:50announced this and then one of the
22:52things they announc is part of that is
22:54you know longer term Society needs to
22:56think about what this means for Society
22:59uh because this technology seems to be
23:00so good for certain human level tasks
23:03and this is back to that point of AI
23:05adoption in the Enterprise is just
23:06starting but how many years is it before
23:09every Enterprise realizes that they can
23:10cut customer support dramatically at
23:12least for certain types of products just
23:14through just through adding you know
23:16simple apps you know and so uh I think
23:20that's the other thing that is kind of
23:21happening in the background that isn't
23:23talked about that much but you know is
23:24already starting to really show its face
23:26in in pretty interesting way yeah well I
23:29I do think you're going to get this
23:30accelerated adoption that goes use Case
23:33by use case right where like in in any
23:37Market you have early adopters that
23:39build it in house or go get these
23:41Solutions and are willing to take the
23:43risk when you don't actually know like
23:44what the impact will be how well it will
23:47work but as soon as one payments company
23:50does that and it's a better experience
23:52for the customer or it has real like
23:55impact on operating cost I think like
23:57you switch very quickly over to the
23:59entire sector being like we have to
24:01adopt it in order to be competitive on
24:04both fronts yeah yeah this stuff tends
24:06to happen slowly and then suddenly all
24:08at once and I think we're in the slowly
24:10phase right now and um I actually had my
24:12team go and take um uh Global Services
24:16and look at that right and so if you
24:18look at uh spend on software in the US
24:21right now it's about half trillion
24:22dollar in software spend a year if you
24:25look at uh human Services just payroll
24:29for things where gen can probably impact
24:31things it's 3 and a half to 5 trillion
24:33so if you convert just 10% of that
24:38Revenue you've effectively recreated the
24:41entire US market software industry in
24:44market cap right and so these are huge
24:46trends that are coming and you can kind
24:49of Imagine vertical by vertical what are
24:50those things going to be and then you
24:52can ask is it going to be built as
24:53internal tools for companies is it going
24:56to be uh new company that emerges that
24:59serves these things or is it going to be
25:01an incumbent who figures it out and adds
25:02it and so this sort of customer support
25:05chatbot thing you know you would have
25:07thought that there's a company doing
25:08this for everyone and it looks like in
25:10this case they're um they just did it
25:14in-house uh but you could also Imagine
25:17an existing company like a zendesk or
25:18somebody adopting to this and the real
25:20question is which of those three
25:22scenarios is going to happen at least
25:23from a startup perspective but from a
25:25technology wave perspective this is
25:28and you can build in the feedback loops
25:30really easily for this type of product
25:31right because you can have the customer
25:32rated or thumbs up thumbs down at the
25:34end of a session Etc so you have a
25:36really good sort of um rhf or some sort
25:38of training set for it as well so it
25:40it's a it's a product that should get
25:42better and better and better over time
25:43as you use it yeah I think one of the
25:46things that is an indicator of uh like
25:50where that Services spend might be that
25:53gets externalized is actually like the
25:55big tech companies actually have you
25:57know they're tech companies but they
25:58have broader businesses than um I think
26:02sometimes they're given credit for right
26:04like Facebook meta interacts with smbs
26:07as advertisers if you look at anybody
26:09who has this like large Commerce um type
26:13customer base so as you just mentioned
26:15Clara or Square or meta or Shopify like
26:20they've all done this now and it's
26:22working right and and so I I think the
26:24fact that these are the companies that
26:26have the technical teams that are
26:27capable of doing it in house is a nice
26:30indicator for like well if it's that
26:32effective everybody else should too and
26:34the question is I think not every
26:36segment of customer like retailers with
26:40um enough of a technical team to build
26:42an e-commerce presence may not build
26:44this themselves then it's a more likely
26:46uh scenario that either an incumbent or
26:48a new company be it Sierra or something
26:51else ends up owning that customer
26:52service segment yeah 100% yeah we we
26:55have a long list internally of like the
26:58companies that I think should exist in
26:59the space right because there's there's
27:03ones and very few companies exist for
27:07most of them if any companies and so I
27:10think it's it's back to this idea that
27:11there are these human capital waves
27:12happening in Ai and the very first wave
27:15we saw was researchers and they built
27:17the early model companies and they built
27:19some of the early applications like
27:20perplexity and Harvey and all these
27:22things were actually started by people
27:23who were working on models initially and
27:26they were just Clos as the technology so
27:27they knew to do and then the second wave
27:29of human capital was like infra people
27:30because it was the second closest to
27:32llms and then the third wave of course
27:34is going to end up being application
27:36Builders but many of them were not aware
27:38that any of this stuff was important
27:39until chat GPT came out 15 months ago
27:41and they're just starting to show up
27:42right it takes some 9 months stick with
27:43their job and a few months to figure out
27:46what to do and find a co-founder and a
27:48few months to build prototype and so we
27:50haven't seen anything yet really in the
27:51app wve you know all the apps or many of
27:54the apps so far were started by people
27:56who are very close to the research
27:57community and then it's kind of
27:58permeated into other areas with with
28:00some growing really fast right there's
28:01like half a dozen medical scribing apps
28:03that all seem to be growing at a pretty
28:04good Pace or there's um a few other
28:07application areas where it seems like
28:08there's a number of people working but
28:10then there's lots and lots of spaces
28:11where it seems like nobody's doing
28:12anything which is which is kind of weird
28:15honestly yeah there's a joke that the
28:17foundation model companies um are here
28:21to replace all the jobs but they don't
28:22understand what any of the jobs are and
28:24I think there's like a little a little
28:26bit of Truth in the sort of exposure to
28:29uh what happens in you know a a broad
28:32range of companies in terms of functions
28:34and Outsource services and so I think
28:37that is the opportunity right like now
28:39it's a race for people who are just
28:42great Engineers smart about a domain to
28:44go experiment On The Fringe of that and
28:46I I still think there's opportunities
28:48around like you and I have talked about
28:50um the uh domain areas where you might
28:53want specific models or verticalized
28:56companies still and we should we should
28:57talk about that but I uh I my team and I
29:00just gave a presentation at this AI in
29:02production conference about how if 2023
29:05was the year of infrastructure like 24
29:07is the year we begin to see applications
29:09so I think we're pretty aligned there I
29:10do want to ask you like one thing before
29:13we move away from all of the earnings
29:15stuff which is um the most obvious Place
29:19somebody's already making money is
29:21either like Cloud providers inference
29:24providers or um just Nvidia as a
29:27what would it take to compete to have
29:29like a second source with Nvidia I think
29:32there's a few different approaches right
29:33I mean uh fundamentally if you look at
29:35what people claim as a defensibility in
29:37part of Nvidia it's a mix of Chip
29:39performance Cuda and interconnect um you
29:42know Nvidia bought uh melanic back in
29:452019 it was an Israeli company uh to
29:48basically provide the interconnect side
29:49I think that was like a $5 billion
29:50acquisition so was quite large relative
29:52to n's market cap at the time um and
29:55then obviously Cuda has been developed
29:58uh uh and then obviously they they've
30:00iterated really well on these sort of
30:02different generations of chips um so
30:04minimally you at least need the you need
30:06some form of silicon this performant and
30:08then you need to make sure that it's
30:09actually able you're you're able to um
30:12use it effectively and then you're able
30:14to scale it which is sort of the
30:16side um and there's the incumbent side
30:19of it right AMD is obviously working on
30:20this in trying to Etc and then there's
30:23the um the startup side of it where
30:24we've seen things like rock uh merge
30:27where they have very fast inference for
30:29open source models as well as language
30:30models which is pretty striking you have
30:32sras which has taken a fundamentally
30:34different approach to the chip side as
30:35well so you know there's a few startups
30:37that I think have some interesting early
30:40hardware and there's some new companies
30:41like ads that have talked publicly about
30:43how they're really focused on
30:44Transformer base models and
30:46architectures for chips that they're
30:47building so um there will be this
30:49potential wave of second sourcing over
30:51time but uh you know in general if you
30:54look at many of the most advanced chip
30:56markets historically at least there's
30:59tended to be a winner or I should say a
31:01leader and then there's been there's
31:02tended to be a second place party and
31:05that was you know during the
31:06microprocessor world that was Intel and
31:08then AMD was number two and um you know
31:12in uh mile it kind of morphed a little
31:14bit right you had Qualcomm and arm doing
31:16different things but both quite
31:18successfully but I think Qualcomm was
31:19always at least for a period of time
31:20with the bigger company although arm is
31:22much larger now I should actually check
31:24that in terms of market
31:28yeah Qualcomm is 176 billion and then
31:32is 140 billion so they're pretty close
31:34actually now um there used to be a
31:36pretty big disparity uh between the two
31:38in part that's because arm is being used
31:40now in sort of broader ways uh so you
31:42know you you kind of tend to see these
31:44Market structures and semiconductors
31:46where there's a leader and then a second
31:48place and um I think part of that is
31:52traditional mors law chip generation
31:53related stuff I don't know how that will
31:56hold up or how that'll more of an AI I
31:58don't know if you have an opinion on
32:00that yeah the um you know the way Jensen
32:05has described advancements in chip
32:07performance tend to be more um uh memory
32:11management and new techniques versus
32:13just like transistors fitting um on a
32:16particular die size and I think somebody
32:18else said Nvidia called it Jensen's law
32:20of like ability to get performance from
32:21full system but the the only thing I'd
32:24add to your description of um
32:27competitiveness here is also like
32:30manufacturing even for these fabulous
32:32chip design companies is a big deal
32:35right like so you got to do what you
32:36said design something better including
32:38interconnect design an entire like build
32:39an entire software ecosystem C has been
32:42around since 2006 but after that you
32:43have to go get capacity at tsmc right
32:46and then you need to get yield up and
32:48then you need to all to be competitive
32:49in terms of pricing I think the desire
32:52like the economic pressure given to
32:55trillion of market cap and
32:57more demand than Nvidia can support is
33:00higher than ever but I think the moat is
33:02actually really really deep and so um
33:04when I think about like what could be
33:07what could be enough to go disrupt that
33:10I've seen I'm sure you've seen many of
33:12these companies but I've seen a few
33:13different um approaches it could be a um
33:18chip and system designed for like
33:20specifically very much around latency um
33:23it but the other thing that you said
33:26right like something for example
33:28optimized two Transformers as an
33:29architecture you're taking a bet around
33:32how much stability there is around a
33:33particular architectural approach and I
33:36think that's felt like a um a quite good
33:40uh bet for a while now but for the first
33:43time in a long time there is some
33:44interest in in things like State space
33:47models with companies like cartisia and
33:50um some some Alternatives right um if
33:54you're a really big company with your
33:56own use case right if you're meta or
33:59you're Google and you all you you know
34:01either have like the entire ad system
34:04recommendation serving spam Etc um or
34:07all that like search and your own cloud
34:10then you don't need to make everything
34:11work on the software ecosystem side you
34:13just need to make one application work
34:16and you know these companies also
34:17acquired teams in but that's how you end
34:19up with like tpus and trainum and all
34:21that uh but I I would love to meet
34:25companies in this area and still haven't
34:27haven't seen something that's gotten me
34:30um over the edge even in a place that is
34:31so obviously economically fertile yeah I
34:35think one thing you pointed out which
34:36was interesting to expand a little bit
34:38tsmc and the whole F Faba semiconductor
34:42world where you're basically you know uh
34:44Outsourcing the development or the
34:46manufacturing of the chip to a handful
34:47of players tsmc being the bigger but
34:49there's the biggest but there's there's
34:50a there's one or two others that are big
34:52enough to at least handle some volume
34:55and you know there's been this push to
34:58repatriate semiconductor manufacturing
35:00to the US and has run into all sorts of
35:01obstacles that are pretty avoidable
35:03environmental reviews that go on
35:04endlessly or other things that have
35:06prevented people actually starting to
35:08build these things that take many years
35:10to build um and it's been interesting to
35:12watch that in Japan they're starting to
35:14actually have really interesting uh
35:17development of Fabs specifically for
35:20this purpose and so I'm increasingly
35:21wondering whether Japan emerges as sort
35:23of a Second Source location and part of
35:25geopolitically hedge Taiwan
35:27um but I think that's something also to
35:29kind of watch in terms of where are you
35:30actually seeing Fabs go out and how do
35:33you think about that geographic
35:34distribution but also why is the US in
35:36some sense getting in its own way for
35:37something that has pretty broad-based
35:39atretic importance on multiple levels
35:42you know including National Security
35:43ones so if you listen to the um tsmc CEO
35:48about this he talks as much about um uh
35:53about like the human capital and the
35:56culture cultural elements of human
35:58capital required to make a place like
36:00tsmc work as the capex spend right and
36:06the um the access to equipment and the
36:09need to actually build the Fab um I
36:11think that's pretty interesting because
36:12like you know we we can invest a great
36:15deal um but it's it's very hard to
36:17change culture and so I I I do think
36:21that there's um there's one version of
36:24like maybe you have Fabs in
36:27um Japan or Mexico or southeast Asia or
36:30a um uh like a just a broader Global
36:33supply chain for chip production or
36:36maybe you have robots making chips yeah
36:38I mean that's all true but the flip side
36:40of it is Intel has manufactured chips in
36:42the US for a long time TI did
36:44historically right but Intel still does
36:46so I don't think there's a complete lack
36:48of human capital obviously it's
36:49concentrated in part in Taiwan and
36:51secondary extend in um Korea right now
36:54but I I do think there's there's the
36:56capab ability to do it and I think again
36:58we're there there are other things that
37:00are getting in the way I think even
37:01before that can you can you even break
37:04round on the plant maybe step one right
37:07we should start with the basics and then
37:08we can deal with culture when we
37:09actually have a Fab yeah well and I'm
37:12I'm I'm uh I guess very willing to
37:13believe that these companies and
37:15industries didn't exist in the place
37:16they places they do without like great
37:19leaders for tmcmc or otherwise and so
37:23like maybe it's not a solvable problem
37:27like I'd be curious if you believe in
37:28the Intel Fab business that they're um
37:31uh trying to push and push to other
37:33customers now but it to me it's not
37:35binary it's like of course we can like
37:37make chips in America the question is
37:40can we make them without the um turn and
37:44with the yield and cost to make them
37:46competitive but maybe it's so important
37:48like you don't need them to be
37:49competitive for some period of time yeah
37:51and also my point is um we're already
37:53doing that for Intel right Intel's fat
37:56business isn't in the US not the not the
37:58The Fabulous TMCC sty business just
38:00making their own chips they've been
38:01doing it for decades in the US it's been
38:03fine it's been high yield you know yeah
38:06it's been it's been fine but it's also
38:08been um behind in terms of uh process
38:11Technologies right but maybe maybe
38:13that's not a human capital issue then
38:14maybe it's other other issues that Intel
38:16yeah it seems like it's a other issue
38:18yeah I think my general take on the H
38:19Market is the more I learn the less I
38:22know in Ai and it's the opposite of
38:24every other field I've ever been in US
38:26the more you learn about something yeah
38:28usually the more you learn about
38:29something the more you can create sort
38:32of these straight line hypotheses or you
38:34know what you know kind of compounds and
38:37it's static and I feel in the AI world
38:40like every week there's like so many new
38:43things that your entire world model
38:44shifts in like a fun
38:48way yes it's it's uh fun to be exhausted
38:52but I think um you know there there's
38:54just so much going on in the pace of in
38:56it really feels like you know that that
38:59early slope into the you know the
39:01exponent that is a singularity or
39:03however you want to phrase it but it
39:04really feels like this uh
39:06self-reinforcing loop of new stuff and
39:08honestly a lot of it was kind of held
39:11back in the larger tech companies and
39:14now it's kind of flourishing externally
39:15and that's creating competitive pressure
39:17on the larger companies and the larger
39:19companies are reacting and that's
39:20spawning more startups and it's just
39:21this really interesting virtuous cycle
39:24uh and to some extent the big tech
39:25companies are help fuel it all by then
39:27funding the companies that are working
39:29at very late stages with huge rounds and
39:32they're funding a lot of the compute in
39:33the industry in a way that's you know at
39:35least in order of magnitude maybe two
39:37orders of magnitude more than what the
39:39Venture Community is doing and so it's
39:41this really interesting brous cycle of
39:44startups come out that accelerates big
39:46Tech doing stuff that causes some people
39:48to leave big Tech to do some interesting
39:49things externally they then get funded
39:51by big Tech and that accelerates both
39:53themselves and big Tech and you have
39:54this kind of interesting cycle happening
39:56right now so it's very exciting days
39:59yeah I drew um I drew slide that has
40:01like as you might hope like a bunch of
40:03reinforcing Cycles it's very fancy and
40:06the one I would add to that is like what
40:08we started talking about which is when
40:10something begins to work if it is
40:12actually valuable like the Clara thing
40:14that you describe like at some point if
40:17it's valuable and it moves the needle in
40:18the business you have to do it like as a
40:20part of the competitor set and so I
40:23think like we started with this like
40:25narrative drift thing where um you know
40:28CEOs would say that they're going to do
40:30AI because like the markets believed
40:32that was the future and it was very
40:34generic and you see that show up in the
40:36spending numbers or at least the
40:38expectations around spend right I was
40:40looking at this um survey from one of
40:43the investment banks that says like
40:44Fortune 1,000 um uh it budgets go to 5
40:48to 8% this year instead of 3 to 5%
40:51generally and it's all because of AI
40:52like that's pretty big right that's like
40:55two x and like if that's true then
40:57that's also part of the reinforcement
41:00cycle here because if the companies
41:02start to work then they get to continue
41:03building these products BCS you know or
41:06investors like us will will keep keep
41:07trying so I think it's pretty exciting
41:10yeah it's rlpa F yeah
41:12rlpa rules right off the tongue
41:15reinforcement learning through product
41:18adoption feedback you're welcome well
41:22I'm just gonna plug that in to Chachi BT
41:24and have it write the paper but um I
41:26will be sponsoring author if you'll be
41:30author yeah I'll see if I include
41:38violence find us on Twitter at no prior
41:41pod subscribe to our YouTube channel if
41:43you want to see our faces follow the
41:45show on Apple podcasts Spotify or
41:47wherever you listen that way you get a
41:49new episode every week and sign up for
41:51emails or find transcripts for every
41:53episode at no- pri.com
