00:05hey everyone welcome to no priors I'm
00:08Sarah Gua I'm a lot go this week on no
00:10priors we're back with another episode
00:12where we answer your questions about
00:13tech Ai and everything in between
00:16I think we have a lot of different
00:17questions that people are brought up
00:18this week that they were hoping we could
00:19cover and some topics that we thought
00:20were would be kind of interesting I want
00:22to go to one of our listener questions
00:25and I think a topic that's really
00:27popular with many of the companies that
00:29you and I work with in terms of access
00:31to Computing for a much smaller scale
00:34experiments what uh what's going on with
00:36the GPU crunch yeah the companies that
00:39you and I work with many of them are
00:41companies that you know they need to use
00:44very specific infrastructure to train
00:48and serve large models right these work
00:51on gpus and the structure of the
00:55um it's just not very robust right so
00:58you have a very small number of
01:01producers Nvidia and amb AMD generally
01:04and then Nvidia is very far ahead on the
01:08high-end processors that are most
01:10efficient for large scale training in
01:12inference then you have the pandemic
01:14Supply disruption which we haven't fully
01:16covered for if you actually look at the
01:18supply chain you go from the actual
01:22designers to you know the Reliance on a
01:25few major foundries like tsmc
01:29um you know expansion of this capacity
01:31is not easy right new Fabs are billions
01:34of dollars yield is a very complicated
01:36thing you can think of it as a massive
01:39Precision manufacturing problem where
01:41temperature pressure chemical
01:42concentration tool imperfections new
01:45processes materials issues like anything
01:48can make production have lower yield or
01:50lower quality right and and so like if
01:53you think about the speed with which the
01:58industry driven by both large and small
02:00players has decided that they want to do
02:04AI uh like the physical processes cannot
02:08keep up with that demand it's as if you
02:10know half the companies in the world
02:12over a year-long period decided like
02:16yeah we need super computers not
02:18superconductors but gigantic networked
02:22gpus so like what is the actual Gap so
02:25you're to your plane it sounds like much
02:27of the AI world is dependent on gpus in
02:29order to train and then do inference on
02:34and the big suppliers are basically
02:36Nvidia AMD and then there's like a long
02:39tail of smaller folks what is the Delta
02:41between the amount of capacity that
02:43exists today and that's needed are we
02:45off by 2x 10x some other number it's
02:48hard to say because uh right now there's
02:51no way to explore like the price
02:54elasticity of these things right
02:57um so you know just very specifically
03:00like the industry is kind of looking at
03:02deliveries in small quantity in
03:04September larger quantities in December
03:06January most of the large Cloud
03:08providers are sold out for any scale for
03:12at least through April of next year and
03:16so you have like really interesting
03:17Dynamics like large Cloud players who
03:21you know are the biggest consumers of of
03:23these gpus already like a Microsoft
03:26going and buying from other providers
03:29for near-term near-term Supply right so
03:32I think one question that I ask you is
03:34like do you think this is a long-term
03:35thing do you think it's a very
03:36short-term thing but I I think it just
03:39goes back to like the the fundamental
03:41Dynamics are do you expect the demand
03:44for these chips to continue increasing
03:48at a pace that outcreases the ability to
03:50scale a very physical like real world
03:53process right just to even be more
03:56specific one of the challenges like I
03:58was talking to Jensen about this and a
04:01bonder like not part of the GPU itself
04:03but like a critical tool in the
04:05manufacturing and assembly of gpus is
04:08very specialized and so the ability to
04:10build any of these tools as well to
04:13enable these processes is is a blocker
04:15if you look at the demand from large
04:19Labs today to continue increasing model
04:23scale and training Time by magnitudes I
04:27think it's hard to see that Dynamic
04:28going away what do you think I feel like
04:30there's a couple different sort of
04:32second order implications of the fact
04:33that we're seeing this giant GPU
04:36I think the first one is that we're
04:38seeing new sort of models that are
04:40dependent on GPU access or ownership is
04:44ways to create all sorts of really
04:45interesting monetization and potentially
04:47eventually cloud services so that's
04:49things like Coral weave
04:51or Foundry ml or other companies that
04:53are basically providing now gpus in
04:54different ways in some cases through
04:56aggregation or federating different
04:58sources of gpus in some cases it's just
05:00having these large GPU clouds and being
05:02able to use them in really interesting
05:04ways and one of the interesting I think
05:06side notes is that gpus used to be very
05:09heavily used for crypto mining and while
05:11crypto is down it may actually be more
05:13economic to just use them for to rent
05:15out for AI training purposes or
05:17inference purposes so I think that's one
05:19really interesting almost like sectoral
05:20shift in terms of existing GPU capacity
05:23the second is that a lot of the
05:24different players that are startups
05:26who've built their own semiconductors
05:29specifically for AI training I think are
05:31starting to see a lot of really strong
05:32pull so for example cerebras and I think
05:35we're going to have Andrew from
05:36cerebrasan um our podcast in a couple
05:38weeks they just signed a 100 million
05:41dollar deal with UAE for building nine
05:45super computers using their chips which
05:47are optimized for AI amazing and so I
05:49think they and grok and other sort of
05:51semiconductor providers are going to
05:54find really strong pull during this
05:56period where people are desperate for
05:57any solution and they're willing to do
06:00take the extra steps to really be able
06:02to utilize other forms of silicon and so
06:04I think it creates a bit of an opening
06:06for other players in the market and so
06:08it does seem like it's going to have
06:09these really interesting sort of
06:10cascading effects on members of the
06:12startup ecosystem and you know new
06:14players that are working against all
06:15this two um sort of second order things
06:19are like what do you do when scaling is
06:21blocked on capacity like you try to be
06:23more efficient it's not been an area of
06:25massive Focus to date because people
06:28have been chasing the state of the art
06:30following chinchilla scaling as the
06:33simplest path forward but there are
06:35really interesting lines of research
06:37that are undervalued today unless the
06:40hardware supply crunch continues
06:42including in dynamically figuring out or
06:46routing to efficient models so think of
06:49like The Frugal GPT work or generally
06:52like distillation or even just a more
06:57intelligent choice of data for your
06:59pre-training or your fine-tuning
07:01training mix so you can use less compute
07:04right for for the same or for for
07:06improved quality and I I think like
07:09everybody's been on this one path and an
07:11interesting second order effect is like
07:13does it spread people out into lots of
07:14different directions in terms of chasing
07:18I personally don't think the um Supply
07:20crunch goes away immediately and like a
07:22part of the dynamic is just you know how
07:25much more people want to scale and
07:29another part is like you know
07:32if this stuff is actually useful then
07:34inference like inference already
07:36dominates open AI compute usage right
07:38and so that demand will continue to go
07:40up yeah I do think demand will only
07:44um rock it from here at least in the
07:45short run and so the real question is
07:47the degree to which the semiconductor
07:49industry adjusts to that and the reality
07:52is that people really view nvidia's
07:55chips as the most advanced on the market
07:57and so that means that a lot of it is
08:00just a bottleneck and how much
08:01can Nvidia scale up manufacturing and
08:04there's other players like AMD there's
08:06the startups we mentioned cerebrask and
08:08others but a lot of the capacity is just
08:10going to be how much can it can Nvidia
08:12and maybe AMD scale up in the short run
08:14at least and so that may just cause some
08:16ongoing bottlenecks assuming again that
08:18we continue to see this very rapid
08:19growth and an AI and AI applications I'm
08:23working on a blog post right now
08:25actually about this because it feels to
08:26me that we're still in the very very
08:29of this wave of AI adoption right it's
08:31not a Continuum where we had cnns and
08:33rnns and that's something we have
08:34Transformers Transformers created a
08:36whole new capability set and we're only
08:39you know eight months since chat GPT in
08:42a few months since five months I think
08:43since uh gpt4 and so the only people
08:47who've really adopted this technology
08:48yet are the AI native companies like
08:50open Ai and the journey and a few other
08:53folks and then we had the first wave of
08:55startups come the perplexities and
08:57Harvey's and characters of the world as
08:59well as the first wave of incumbents
09:01adopting it notion and zapier and sort
09:03of very very early founder-driven
09:06and so we've had zero real Enterprise
09:08adoption in terms of real products at
09:10scale or close to zero
09:12and you know most Enterprises big
09:14businesses take six months nine months
09:17is to do their planning Cycles
09:19and then they'll spend a year
09:21and then finally they'll launch these AI
09:22apps and so we're probably a year or two
09:25years before we really start to see
09:26large-scale AI applications by existing
09:29incumbent Enterprises in real products
09:34so from a ramp perspective one can
09:35imagine that a lot of the future ramp
09:38in about two years or you know one to
09:40two years something like that so there's
09:42still a lot of um room I think for the
09:44hype cycle for increasing ongoing
09:47excitement sometimes irrationally so
09:50and then also for sort of adoption of
09:52semiconductors and other underlying
09:54infrastructure so there's still a lot to
09:55come it feels like I agree with you and
09:57I still think we're really early in
10:00let's say like the collective
10:01exploration of applications and
10:04constraints right like you had the
10:07people who were bleeding edge of just
10:10personal interest uh like I think Chachi
10:13BT was is looked at correctly as the
10:16starting gun for people to begin
10:18developing these AI applications
10:20generally but if you think about how
10:24long it takes to ship actual interesting
10:26products to Market and then the build up
10:29of some collective understanding of like
10:31how to make these models more useful in
10:34different applications and then you know
10:36turn them into workflows and then
10:39Advance the state of the art given a
10:41particular workflow if you have a
10:43hypothesis on value like that'll that
10:45all takes time so I think we're in
10:47inning one yeah it's all been demos so
10:49far yeah so I guess related to that a
10:51lot of the interest and excitement right
10:53now is around to agents
10:55you know I spoke recently there's a
10:56group um called the AGI house which you
10:58know Jose's different hackathons in the
11:00Bay Area and stuff like that and they
11:02help kick off like a agent hackathon
11:05they had and things like that what do
11:07you think happens in the ancient world
11:08like what form does that take and is it
11:10a handful of very broad Asians is a
11:13highly specialized ones like what do you
11:14think has come in there
11:15yeah uh it's such a um like powerful
11:19broad idea that I think both will happen
11:24um and and so like the the overall idea
11:26is you know you you don't just talk to a
11:29chat bot or or query an interface you
11:32have some sort of planning mechanism
11:34that is model driven that allows you to
11:37take asks autonomously take actions
11:42um and like complete a more
11:44sophisticated task often using other
11:46tools and then return that result or
11:49report back on your work to an end user
11:52right and so you know I think that is
11:57um the pure consumer applications so
11:59things like inflection which is going to
12:02you know have personalized that do more
12:05for you minion which is working on like
12:08web agents uh and and then you know I
12:11think like there's been very recently
12:14more attention or just more
12:16understanding of how powerful it is to
12:18have agents that in some way write
12:21executable code right um because you can
12:24programmatically use many more tools you
12:26can call apis and I think if that is uh
12:30do a task that is not a single query but
12:33requires multiple steps in uh in
12:36analytics or an Enterprise automation
12:39um or even you know within like you know
12:42companies that we work with uh like
12:44Harvey like a single legal task is
12:47actually a composition of thoughts
12:50planning attempts of research like
12:52writing that a an associate might do and
12:55so I think it's going to be a pretty
12:58yeah it's kind of interesting because if
12:59you look at past technology waves and
13:01you ask about specialization versus sort
13:04of broadness you know are you building a
13:06broad-based platform that you can use
13:07for anything or a vertical application
13:10that really helps you with one or two
13:12most of the things that really work are
13:14these vertical applications that help
13:16you really well now some of them broaden
13:18and grow into the broad-based platform
13:20for everything right even in consumer
13:22that's true like Facebook started off as
13:25and in fact it started with like five
13:27colleges and they added all colleges and
13:28then later they added the ability to add
13:30your work email as a way to register and
13:33then they open it up to everybody and
13:34then they start building the platforms
13:35on top of it in gaming and other things
13:37right but it kind of happened
13:40and there's counter examples to that you
13:42know Google would be a very broad-based
13:43thing from day one it helped you
13:45discover information on the web right
13:46you needed a tool for that
13:48but it feels like in the agent World a
13:50lot of the people that I hear talking
13:51about ideas have these very Broad
13:54sort of abstract ideas and so an idea
13:57um I'm going to build an agent that is
14:00going to be your assistant
14:01and you're like okay well what is it
14:02going to help me with and they say
14:04it's going to make you happy and you say
14:06well I'm you know I'd love to be happy
14:08but at the same time you know starting
14:10with a very targeted focused initial use
14:13case tends to be the best way to build
14:14product a because you know who you're
14:16building it for B you can really nail
14:17the use case and there's the old sort of
14:19ycism which I think which is really good
14:21which is better to Delight a small
14:24number of people than to have a very
14:25large number of people indifferent to
14:28and so I think my my bias for the agent
14:31world is if you're building an agent
14:33start with something really targeted
14:35if it's a assistant to help you what
14:37exactly does the assistant do does it do
14:39background information searches on all
14:40the meetings you have that day
14:42does it specifically help with certain
14:45does it help with other aspects of your
14:47day planning or synthesis of what you've
14:49done or follow-up action items or
14:50whatever it may be about choose one or
14:53and do them very well versus do
14:56and then eventually you may build a
14:58thing that you start off that does one
14:59thing very well but then broadens into
15:00everything but usually starting with
15:02everything means you're not really doing
15:03anything deeply or well and so I think
15:06that to me is one of the main patterns
15:09at least in terms of Prior waves of
15:10Technology development
15:12I very much feel like this is like a
15:14very classic tension between
15:16um what I consider to be like uh I don't
15:19know the like infrastructure platform
15:21engineering like even research agenda
15:24driven approach that is like oh you
15:27don't understand like the technology is
15:30General we don't want to be taken off
15:32the research path that pollutes our uh
15:36our data mix in a way that it is not a
15:38general purpose technology anymore right
15:42um or you know it can do anything why
15:45um or even getting feedback from users
15:47because you release this stuff it is
15:48broadly capable that they're doing
15:49everything with it something's much more
15:52successfully than others and I think
15:54more of a like a product engineering
15:56like traditional like startup mindset
15:58that is like actually complete the task
16:01right and I I definitely think um uh the
16:04overall exploration has been skewed to
16:06one side not as productively today
16:10um and one of the like even if you think
16:11from the research agenda one of the
16:15interesting to think about the like uh
16:19the you know have more Focus everybody's
16:21thinking about but have more focus on
16:23accomplishing the specific task is like
16:26you want to be happy a lot all I want to
16:28do is like never write boilerplate code
16:30again right and so if you think about
16:32that's how I define happiness
16:34okay great then we're still the same
16:37um but uh like if you think about like
16:40okay let's like complete one task if I
16:43uh want to ask um you know an agent to
16:46just like fix all the bugs in my
16:50um then my uh ability to like
16:53successfully complete that task includes
16:55a lot of like bug fixing specific
16:59techniques right like you could do test
17:01time search and then see if all of the
17:03different things that you uh generated
17:05actually execute as one very simplistic
17:08example right and so like I think there
17:10are a lot of ways to advance in
17:13um the research in very specific tasks
17:16that are much more attractable but maybe
17:18I'm not thinking big enough that makes
17:20sense I think um I would add one-third
17:21piece to that framework you have which
17:23is the research driven versus product
17:24driven I think there's a third approach
17:26which is infrastructure tooling driven
17:28and that's where you're like I'm not
17:30going to build that agents but I'm going
17:31to build the infrastructure that allows
17:32anybody else to build them rapidly now
17:34sometimes those types of businesses or
17:36approaches work really well and
17:38sometimes those things are solely an
17:40outgrowth of a vertical product that
17:42works really well that you then open up
17:44the infrastructure for everybody else to
17:46and it's very Case by case dependent
17:48it's the difference between stripe
17:50where it's just like we need to build
17:52payments for everybody everybody keeps
17:53building it over and over again and the
17:55Facebook all the platform which only
17:57existed because you got to hundreds of
17:58millions of users you could open up
18:00office like a third party service and so
18:02I think as people think through that
18:03third angle of building an
18:05infrastructure for others they need to
18:07understand whether that infrastructure
18:08will be an outgrowth of an existing
18:09product area and benefit from the
18:12characteristics of the the market
18:13liquidity of that product
18:15or whether it's just a piece of
18:16infrastructure everybody keeps building
18:17over and over and therefore it's a
18:19really good thing that just provide to
18:20the world so I think it's kind of an
18:22interesting future topic
18:24we are on a you know couple month bull
18:27run at this point 2024 Tech markets
18:30what's coming like will people be able
18:32to fundraise will funds be able to
18:34fundraise our customers purchasing
18:37you know I think there's going to be
18:40um four markets next year in some sense
18:42one market is just Ai and I think AI
18:44will continue to run in different ways
18:46and it'll look very expensive at the
18:48time and a handful of companies will
18:49look really cheap in hindsight just like
18:51with every other technology wave and I
18:53think that's separable from the rest of
18:54tech that existed prior to the AI wave
18:56for companies that fundraised in 2021
19:00prior to being like AI companies a
19:03subset of them I think if I were to sort
19:05of divvy up that pie of those companies
19:07sort of mid to late stage private tech
19:09companies not an AI and what's going to
19:11happen to them next year and in 2025 I
19:13think a third of them are just going to
19:15or a third of I should say unicorns
19:18we'll eventually just go under be fire
19:21they won't be able to ever raise money
19:22again a third will be at the highest
19:25valuation they'll ever be at ever in the
19:27lifetime of the company they'll reach
19:28their terminal value
19:30and those examples from 2014 of
19:32companies that went through that same
19:33wave they've you know raised in 2014
19:35they went public a few years later and
19:36then they never surpassed their their
19:38market cap again and then I think lastly
19:40there'll be a third of companies that
19:42grow past it and so I do think there's
19:44going to be a lot of Carnage next year
19:45and a lot of companies going under and
19:47as those companies go under three things
19:49will happen number one
19:51it'll be much easier to hire people and
19:53people are already seeing that at
19:54startups it's easier to hire again
19:56second it should have follow-on effects
19:58and ramifications for commercial real
20:00estate and we'll see a second shoe drop
20:02and then third The Venture Capital
20:04Community will be impacted because a lot
20:06of the things that they've been using to
20:07fundraise new funds or do other things
20:09with Will suddenly go to zero
20:11they're big unicorn success will go from
20:14a multi-billion dollar or billion dollar
20:16company to basically a company that
20:17isn't worth anything
20:19and so I think that's going to have
20:21knock-on effects to the Venture
20:23ecosystem but I think that'll take like
20:25two three years to play out because all
20:27these things are a bit time delayed
20:29um but yeah I think that other shoe
20:31still hasn't dropped in private Tech
20:34and a lot of it is just companies raise
20:36so much money in 2021 they still have
20:37lots of money so everything still feels
20:39like it's continuing to go but at some
20:42point that money's going to run out so I
20:43think it's going to be a pretty bumpy
20:452024 and 2025 potentially
20:49yeah my advice to companies that you
20:52know raised a very healthy evaluations
20:54during that period of time and then are
20:57actually building businesses is to try
21:00to completely disassociate from that
21:04um because people will put themselves
21:06into all sorts of contortions to do a
21:09flat or up round to evaluation that
21:12makes no sense right yeah and if you
21:15don't have the historical context of
21:17that making no sense it's an extremely
21:20um sort of realization to have but if
21:23you look at there's this um one analysis
21:26of uh actually the very best technology
21:29companies and the ones that endured from
21:33the internet bubble and how long it took
21:36those companies to reach the evaluations
21:39they were at before the bubble burst and
21:41it's a decade right and it's like
21:43startups don't have a decade to try to
21:45you know get to at par evaluations yeah
21:49I'm actually less worried about
21:50valuation I think valuation is ephemeral
21:52right effectively every or roughly every
21:55tech company in public markets did a
21:56Down Round over the last year and a half
21:58right they all lost or many many
22:00companies lost 30 to 90 percent of their
22:02value right effectively they just did it
22:03down around in public markets because
22:05every day you're repricing a public
22:06stock I'm more worried about the people
22:08who burn tons of cash
22:11and they don't have a lot of Revenue to
22:13show for it and then when they're going
22:14to go out to raise more money people say
22:15well you burnt 50 million dollars you
22:17burn 100 million dollars you generate
22:18five or ten million dollars of Revenue
22:20and so the issue isn't that your
22:22valuation is off we can always reset
22:23valuation it's the fact that you burned
22:26all this money and you don't have
22:27anything much to show for it right and
22:28that's where I think the real issues
22:30will happen because you can always
22:31reprice things and people will be forced
22:33to and you know it'll just happen but I
22:35think it's the underlying business case
22:37and business model that's going to be
22:39yeah I guess like the the unforced error
22:42there for companies who actually have
22:44the time to make the decision is
22:47um the thing you want to avoid is like
22:49not adjusting your cost profile or you
22:52know holding on to that valuation until
22:53it's too late yeah or just deciding it's
22:55the wrong business and it's not working
22:57and you know the most important precious
22:59thing for you as a Founder is your time
23:02and I think people forget that you have
23:03this golden period in your life where
23:06hopefully a lot of other complications
23:09in terms of sick family members or
23:12School related issues or whatever it is
23:15and you can take risk and you have a
23:18low-cost basis and you can do all these
23:19things and that's the moment when you
23:21can best take risks to start a company
23:23for many people not for all and you know
23:27you're really giving up the best years
23:28of your life working on things that
23:31thanks for the discussion it's a lot of
23:35thanks to everyone who sent us your
23:37questions find us on Twitter at no
23:39Pryor's pod subscribe to our YouTube
23:41channel if you want to see our faces
23:43follow the show on Apple podcasts
23:45Spotify or wherever you listen that way
23:47you get a new episode every week and
23:49sign up for emails or find transcripts
23:50for every episode at no dashpriars.com
