00:05hi no pryors listeners time for a host
00:07only episode this week a lot and I talk
00:10about the path to better model quality
00:12from here the potential of fine-tuning
00:14rhf rlaif rag and retrieval systems
00:18generally met a sponsorship of the open
00:20source model ecosystem and finally the
00:23beginning of a new set of consumer
00:25applications and social networks thanks
00:26for tuning in so one thing everybody is
00:29thinking about is what it takes to get
00:32to 10x or 100x better AI systems like I
00:36think be useful just sort of enumerate
00:37the the elements to sort of Step
00:40function better A lot what do you think
00:42yeah you know it's interesting because
00:43there's there's a few different aspects
00:44of that that people always talk about
00:45their scalability of data sets and
00:47compute and parameters and all these
00:48things but the reality is I think a lot
00:50of people believe that in order to 10x
00:52or even 100x use cases and usages for AI
00:55outside of that there's things that
00:57could just be done on existing models
00:58today so you don't need to wait for gpt7
01:01you could start with gpt4 or GPT 3.5 and
01:04add these things and I think they are
01:05kind of bucketed into five or six areas
01:07number one is multi-modality so that
01:10means being able to use text or Voice or
01:12images or video is both input and output
01:14so you should be able to talk to a model
01:15type to it upload an image and ask about
01:17the image and then it could output
01:19anything from code to a short video for
01:23um second is long context windows so
01:25basically when you prompt a model you
01:28basically are feeding a data or commands
01:30or other things and everybody realizes
01:33that you need longer and longer and
01:34longer context windows so magic for
01:35example is doing that for code you
01:37should be able to dump it entire code
01:39into a coding model instead of having to
01:42third which we're going to talk about
01:44today is model customization
01:46so that's things like fine-tuning
01:48something known as rag there's data
01:51cleaning there's labeling there's a
01:52bunch of stuff that just makes models
01:54work better for you fourth is some form
01:57of memory so the AI actually remembers
02:00fifth is some form of recursion so
02:02looping back and reusing models and then
02:04six which is related is potentially a
02:06bunch of small models that are very
02:08specialized being orchestrated by a
02:10central model or sort of AI router this
02:11as well for this for this specific
02:13tasker use case I'm going to route
02:16The Prompt or the data or the output
02:18into this other model that's doing this
02:19other thing which is basically how the
02:20human brain works right you
02:22process visual information through your
02:25visual cortex but then you use other
02:26parts of your brain to make decisions
02:27right and so it's very similar to what
02:29Evolutions where I decided was an
02:31optimal approach but I think it's really
02:33interesting because I think many people
02:34in the field know that these five or six
02:36things are absolutely coming
02:38and they can dramatically improve the
02:40performance on existing systems again
02:4310x 100x better for certain things and
02:46so it's more just a matter of when right
02:48it's not really an if anymore a bunch of
02:50people are working on different aspects
02:52and you know I think it's all coming
02:53really fast and so what you know there's
02:55sort of two things that came out in the
02:56last week or two that are really
02:58relevant to this it'd be great to get
02:59your thoughts on one is open AI
03:01announcing that they're not going to
03:02allow people to fine-tune models and
03:04then second is Google
03:05where they looked at human generated
03:07feedback versus AI generated feedback
03:09for models and sort of fine-tuning
03:11models that way so and if you want to
03:12tell people a bit more about what
03:14happened with openai and why that's
03:15important yeah so fine-tuning as a
03:18capability has been offered by openai
03:21for several years right but they've made
03:24like a a specific investment in allowing
03:27people to do that with more
03:28sophisticated models in particular like
03:30three five and also making it possible
03:33for more Enterprise use cases right and
03:36if you think about sort of like why that
03:38matters at all as you said like you know
03:41you have a bunch of these Labs who are
03:42working on General capability and
03:45working on this sort of direction of
03:46scaling laws like Transformers
03:48predictably improve with scale data and
03:50compute but I think what's really
03:52interesting is like the way every the
03:55way these models end up being used in
03:57many business or even consumer
03:58application contacts is against a
04:00specific task right and so we've talked
04:02a lot about like where research effort
04:05work put or compute is being spent in
04:09the industry right now and there's a
04:10really I think there's really
04:11interesting question of we don't even
04:12know how good models can be at certain
04:15scale right at 70 or 30 or 100 billion
04:19parameters or more but not a gpt4 Scale
04:22based on really high quality data and
04:25curation of that data because it hasn't
04:27hasn't been explored and so
04:29I think we should talk about some of the
04:31different ways you get these models to
04:33actually operate against a specific task
04:36uh with either fine-tuning with rohf uh
04:41against you know the reward for your
04:43task or with with rag as you said in
04:46terms of retrieving from a data set that
04:49you've specified right and there's
04:50reasons you would do all three of these
04:52but I think it's actually a pretty big
04:54step for openai to enable this because I
04:59think there was at certain points in the
05:01in the research world there's been a
05:03narrative that like fine-tuning doesn't
05:05really matter right the general model
05:07matters and I'd be curious if you think
05:10that's a change in research point of
05:12view or just a commercial decision in
05:14terms of labs wanting to make money or
05:16that being more important than ever yeah
05:18I think everybody realized that
05:20fine-tuning works really well when
05:21Chachi PT came out because what Chachi
05:24PT is is they took this model GPT 3.5
05:27which existed at the time and that
05:29wasn't seeing as much usage at least
05:30from you know people just going in and
05:32querying it unless they were really good
05:33at prompts and they basically hired a
05:36bunch of people and the people ranked
05:38the output of the model and they
05:39effectively fine-tune the model against
05:41that feedback from the people who are
05:42assessing is this the answer that I
05:44wanted based on the prompt that I put in
05:45right and so fine-tuning really just
05:47means you create a lot of feedback
05:49usually at least today through people
05:52responding to output and saying is it
05:54good or bad and it created a dramatic
05:58step function and the utility of GPT 3.5
06:01for end consumers or end users or
06:04students or lawyers or all sorts of
06:06different types of people and it really
06:08helped it was kind of the starting gun
06:10for this whole AI Revolution right now
06:11because everybody suddenly realized how
06:13powerful these models were and the model
06:15underlying it fundamentally hadn't
06:16really changed that much what they've
06:17done is they fine-tuned it without with
06:19reinforcement learning through human
06:20feedback or rhf and so I think that
06:23created this uh Viewpoint that these
06:26these types of fine tunings or you know
06:28we can talk about rag in a minute love
06:30to get your thoughts on that can
06:31fundamentally change the user affinity
06:36and so you could imagine in an
06:37Enterprise you say well I really want to
06:38fine-tune this model so that it reflects
06:40medical data that I have this
06:41proprietary that could help make a
06:43better doctor assistant or I want to
06:46fine-tune it against this you know set
06:48of HR responses that are unique to my
06:50company so that if I have a uh an
06:54employee who really wants to get
06:55answered a question that can get a
06:56really good answer back and so it really
06:57gets into those sorts of things where
06:58you can dramatically improve the output
07:01of a model against something that you
07:02specialize do you want to talk about how
07:04rag ties into that because I think
07:05that's a really key component of it too
07:07I think the sort of basic premise with
07:10rag that everybody should understand is
07:13you want to retrieve against a specific
07:16Corpus right and so you can you're still
07:19going to reason you might have a
07:21generation or an answer based on that
07:23Corpus but if you pick a set of
07:25documents it could be legal cases it
07:27could be internal company documents it
07:29could be medical information as you said
07:31right so you still want the reasoning
07:33capabilities of the model right a
07:35diagnosis requires reasoning but you
07:38want it to come from a specific set of
07:40data versus like let's say all of the
07:43pre-training data of you know random
07:45information on the internet about
07:46whether or not you have this disease
07:48right and every piece of forum
07:51conversation about this disease and ever
07:53happen so you know I think of the um the
07:56core driver as like trustworthiness
07:59right citation control of information
08:02source and and so now you have this
08:04architecture where people are using
08:07um think of it as like traditional
08:08information retrieval techniques and
08:10search in combination with these models
08:12I think the other sort of driver besides
08:15trustworthiness on these rag approaches
08:17is two things one is cost and the other
08:21is like freshness right so every time
08:25uh retrain a model or even fine-tuna
08:28model like there is compute involved see
08:30the idea that you know being able to
08:33incorporate new information without
08:35retraining and just using the reasoning
08:37capabilities of the models I think it's
08:38very attractive to people and very
08:40that's also related to the freshness
08:42point of view which is like you actually
08:44want the most recent medical research or
08:46the cases from this past year I think
08:48that's that's sort of a set of the
08:50drivers behind people being excited to
08:53take this approach and use it against
08:55their private data sets yeah and that
08:57actually helps a lot with uh
08:58hallucinations right and so I think it's
09:00important to sort of explicitly point
09:01that out because one of the knocks on
09:03the current set of AI Technologies is
09:04while it may hallucinate or say and you
09:07know say things that aren't necessarily
09:09true or cite a legal case that doesn't
09:11exist and by using rag you can actually
09:13help say okay I'm only going to use
09:14things that that I know exist or I'm
09:17going to filter for things that are
09:18going to be um answers that fit well
09:20with you know the the current set of
09:23knowledge that people have relative with
09:25these sets of issu use so to your point
09:27on trustworthiness I think it's really
09:28important to call out hallucinations
09:29explicitly since that's something people
09:31people keep bringing up is sort of
09:32naysayers oh my gosh what if it
09:34hallucinates and some terrible
09:36misinterpretation happens and therefore
09:37we need to regulate this thing right so
09:39uh it's kind of interesting you know I
09:41guess related to that there's this
09:43reinforcement learning through human
09:44feedback versus AI feedback
09:47and Google just came out with a really
09:48interesting paper on that where you know
09:51they showed that you can have an AI
09:53similarly provide feedback to whether
09:55the AI itself is generating good output
09:57and for certain use cases that works as
09:59well as people and so suddenly instead
10:01of having to hire an army of people to
10:03go and help fine-tune these models you
10:05can actually have an AI help fine-tune
10:07this model and I think the the early
10:09signs that that was going to be true was
10:11actually medpalm 2 where Google showed
10:14that they trained a model specifically
10:16on medical data and the output from the
10:19model tended to be more correct than
10:21human physician experts
10:24and so for certain use cases we are
10:25already seeing AI provide more accurate
10:27answers than Specialists experts right
10:31and in our aif you're trying to sort of
10:34generalize that and say what are all the
10:35different ways that instead of using
10:37expensive people to do this we can use
10:39really cheap AI models to provide that
10:42same feedback and sort of train things
10:43and so there's all these techniques and
10:44technologies that are coming now as part
10:46of this sort of list of six six big
10:48innovations that are part of the future
10:50AI 10x or 100x redmap that are starting
10:53to fall into place I think it's a very
10:55exciting time and I think you know in
10:56the next year we'll keep seeing stuff
10:58like that so there's a few other
11:00announcements that have come out related
11:01to this in terms of using different data
11:03sets or different models but coming from
11:06social networks so for example Twitter
11:08or I guess now we should call it X said
11:11it will train ml models off of Twitter
11:13data and that may have really
11:14interesting consumer applications or
11:16outcomes and then meta is really now
11:19emerging as a primary sponsor for open
11:23source models llama and llama2 have
11:25really taken off and sort of the
11:27developer and Enterprise ecosystem
11:29around the llms so it'd be great to hear
11:32what you think in terms of why are they
11:33doing this you know why why are they
11:35becoming the primary sponsor for open
11:36source Ai and how do you think they're
11:38going to apply it within their own
11:40company I really draw a analogy from the
11:46current sponsorship of meta and Zuck of
11:49you know llama and the open source model
11:51ecosystem to like MySQL right so for
11:55those of us who remember like what
11:58happened with these open source database
12:00companies moscale ended up being
12:01originally made by this guy Monty
12:04wardenius and some Swedish company
12:06became partisan became part of Oracle
12:08and in the early days like MySQL would
12:12crash and corrupt data and there were
12:14some early internet scale companies like
12:17Facebook who wanted to use it wanted to
12:19not be beholden to commercial database
12:21vendors made at scale made it more
12:24robust and contributed back right and I
12:27think like it's a reasonable analogy in
12:29terms of like some core technology to
12:31your company where you don't want to
12:34have a vendor uh you don't see it as
12:36part of your core business model but you
12:39want there to be open source options
12:41right and so I have a lot of admiration
12:43for what meta is doing and I think like
12:45I think it is very likely to be a big
12:49mover in the ecosystem because if they
12:51sponsor some baseline of models that are
12:54big enough to be valuable high quality
12:56enough to be valuable with Facebook AI
12:58research and then enough people find
13:00these models useful and strategic and
13:03they create a developer ecosystem it's
13:05hard for me to picture them not being
13:07sustained as an important ecosystem an
13:09alternative to these you know research
13:12Labs that in many ways compete with
13:14Facebook or meta in different ways and
13:17are very expensive to maintain but if
13:20you look at the history of Open Source
13:21is that really true so say for example
13:22you look at Linux right and Linux in
13:25part was very much sponsored by IBM
13:27throughout the late 90s to the tune of
13:28in some in some years a billion dollars
13:30a year and so even these external
13:32ecosystems tend to get quite expensive
13:34you know and the the reason that IBM
13:37sponsored Linux was to provide a real
13:39offset to Microsoft right they basically
13:40said Microsoft is dominant on the
13:42desktop they're really getting
13:44aggressive on sort of the server and
13:45infrastructure world and so therefore
13:47let's fund this offset for open source
13:50how do you think that analog applies it
13:52to meta or does it or do you just think
13:54it's a different reason in terms of why
13:57um well I think they're pursuing it
13:58because they want to use it and they
14:00don't want to be trapped right oh sure
14:03but they don't have to open source it
14:04right they could just continue to
14:05develop it like they have been and so
14:07why open source it one piece of it is
14:09like wanting to offset the development
14:12costs and the compute costs at some
14:14point right and and just like that's
14:16sort of one of the core premises of Open
14:18Source they've also done like other
14:20really related things like the open
14:21compute project but you know if you
14:24think about why that analogy does or
14:25doesn't apply right like one is does
14:27meta want to make money off of this in
14:28some sort of like B2B way if they keep
14:31open sourcing it the answer is no right
14:33they want to use it in their core
14:34consumer businesses and then two like
14:37for for this to work I think one of the
14:38ways the analogy breaks down is very
14:42um like the need for centralized
14:43training today right it's a complicating
14:45Factor like can you really coordinate
14:47that with the politics and slow decision
14:50making of Open Source communities I
14:52don't know I think that's challenging
14:53they're um there are interesting folks
14:55working on at least these sort of like
14:57technical coordination of of this as
15:00well right like Foundry and together
15:03um but if you just to like make explicit
15:05like why might they care my guess is
15:07like the ability to use these models it
15:09applies in sort of thing more
15:11traditional ways like we can use them to
15:13make the data center like more energy
15:15efficient we can and there's been
15:17publishing about this we can use these
15:22um like ad serving right like lots of
15:24things that matter to the core meta
15:26business but it's also just one of the
15:27most interesting things to happen in
15:29consumer in a long time right you have
15:31things like character inflection
15:33mid-journey Pica experiments like can of
15:36soup like these things they have caught
15:38the attention of consumers in a way that
15:41few things have over the last few years
15:43and so I think it's known that their
15:45Instagram chat Bots being tested right
15:48and so if this is a path to Consumer
15:50engagement and then therefore ads and
15:52it's going to be a really important
15:54element I think they just want to have
15:56access to it without being to hold into
15:59a sponsor what's your view yeah I mean I
16:01think it's amazing that metas decided to
16:02make this move and I think it's really
16:04beneficial to the ecosystem overall so
16:06you know at this point I think llama 2
16:08is really emerging as a model that a lot
16:10of people are rallying around and
16:11obviously that may change over time but
16:12for now I think it's one of the primary
16:14models people are using on the open
16:16source side and the people view is quite
16:18um so I think it's super impressive I
16:20think more broadly in Social and AI it's
16:23kind of striking that the last large
16:25social network in some sense was Tick
16:27Tock which was launched seven years ago
16:28now so it's been a while since we've
16:30seen a major shift and part of that is
16:32because large-scale social products have
16:34already been established and so now you
16:36need to sort of pry users away from
16:37existing products which is much harder
16:39than just filling time otherwise I
16:41remember talking once with Jack the
16:43founder of wikiHow which was like a how
16:45to you know Community Driven website and
16:48he said that the main way that they lost
16:50people who were contributing to wikiHow
16:52was they went to social gaming they were
16:53just playing games instead right so it
16:55was sort of this time and attention
16:56shift 10 years ago and when you
16:59mentioned this to me right and so number
17:01one is you have to despise other people
17:04um number two you know a lot of the
17:07Innovation and social kind of stagnated
17:09a little bit for startups right it
17:11became a lot more let's do Twitter but
17:13more woke or more right wing
17:16um or let's do early Facebook again as a
17:20versus hey we're going to reinvent the
17:22modality or we're going to reinvent the
17:25use case or the communication Channel
17:27whatever maybe and it feels like
17:28generative AI is the first thing in many
17:31years to sort of create that new window
17:34and I think the big social networks like
17:35meta and Twitter and others may actually
17:37be the biggest beneficiaries
17:39of this new way but there also should be
17:40room for startups and there's some new
17:42things you know can of soup was in the
17:43recent YC batch and they're doing kind
17:45of interesting things and I think it's
17:46almost like asking what's the Gen AI
17:49modality and use case and typically when
17:52you look at Social products you used to
17:53have this two by two or some people had
17:55like a two by three of you know is it
17:58broadcast versus Mutual follows in terms
18:01of network structure what's the modality
18:03is it images as a video Etc and then
18:05what's the length and Persistence of it
18:07is it long form is it ephemeral et
18:09cetera so for example Snapchat started
18:12um you know short form uh broadcast and
18:15one-on-one that was ephemeral right and
18:18so uh you could kind of map out the
18:20whole social World against those
18:21dimensions and now there's this new
18:23interesting thing of you know new forms
18:25of content creation potentially upending
18:27one or two of those quadrants so it
18:28seems like a very exciting time overall
18:30yeah yeah I had a a you know long time
18:34obsession with hotel and Tick Tock and
18:38some of the Chinese social companies
18:39that really started as like AI native
18:41content aggregators right if you think
18:46um they really figured out this like
18:47cold start problem in terms of they like
18:51total originally they aggregated
18:54um news content from other places and
18:56then bootstrapped your preferences they
18:58didn't require explicit user input to
19:00say like I am interested in these topics
19:02they analyzed your social profile for
19:05your interests they collected like
19:06location and demo and analyzed articles
19:08for like quality and topics so they had
19:10these like Rich per user models of
19:12Engagement it based on interaction data
19:14and then you have this magical
19:15experience of like a better content feed
19:18that then drove the iteration around
19:19better labeling and I think exactly as
19:23if those companies figured out like the
19:25cold start on relevance
19:27um maybe the opportunity I think one of
19:29the potential opportunities in in this
19:32generation of social is like cold start
19:34on the content itself right like you've
19:38um other amazing companies like
19:41like the Instagrams of the worlds right
19:43they they create tools for Content
19:46creation for like magically compelling
19:48assets that are much easier and then
19:51like turn it into a social network and
19:52so generation feels like a
19:55um a really compelling answer in terms
19:58of like how to have a Content feed that
20:00is both like really engaging for you and
20:02then giving people creation superpowers
20:04yeah and I think uh mid journey and PK
20:06are two great examples of that to the
20:08point earlier and then character is sort
20:10of a form of that if you decide to
20:11create your own character or sort of
20:13interact with something that's more
20:14customized there so it does seem like
20:16there are these really interesting uh
20:19shifts that are happening and then the
20:20question is is it more for creation and
20:23sharing or does it become a new social
20:25product or a new communication product
20:27in other words is it giphy or is it
20:29you know uh Facebook right and the lenza
20:33was a good example of giphy right it was
20:35used to basically create content that
20:36you share on other social networks and
20:38the question is what are going to be the
20:40big consumer apps that sort of emerge on
20:41top of that and again it may just be
20:43meta again right but I think it's a
20:45super interesting question and uh and
20:47probably the most exciting time in
20:48Social for a very long time and it's
20:51almost ignored area from uh
20:53entrepreneurship and founder perspective
20:55right now everybody's rushing at the
20:58Enterprise staff and the infrastructure
21:00and you know that whole stack and it's
21:03almost like the generation of people who
21:04are going to start social products all
21:06did them five years ago and did the you
21:08know let's do Twitter again
21:10and the generation that's really focused
21:12now kind of grew up where SAS was sort
21:14of opportunity or SAS and Dev tools were
21:16the opportunities that everybody was
21:17mining against so it'll be interesting
21:18to see whether or not that shifts back
21:20in any meaningful way
21:22um the one other thing I think is kind
21:23of interesting just related to
21:24entrepreneurship and AI right now and I
21:26was talking to a Founder about this
21:27where they were trying to do something
21:31and um by really hard I mean addressing
21:34a really hard Market but using gen AI
21:36and early in markets like when a new
21:38technology shifts and disrupts the whole
21:40Market you actually want to just do the
21:43right why do the hard stuff there's so
21:44much low-hanging fruit why don't you
21:46just go after this stuff is super easy
21:48and my my sort of advice to Founders
21:50generically on this stuff is like don't
21:51do the hard stuff right now or if it's
21:53hard do something that's technically
21:55hard that enables a giant breakthrough
21:56in terms of use case
21:58but don't actually do the hard Market
21:59because there's so many easy markets
22:01right now you should just you should
22:02just go for the easy stuff and if you're
22:03grinding and grinding and grinding and
22:05not getting customer attention don't
22:06spend more time on it it's just not
22:08worth it right now now five years from
22:10when the use of these Technologies are a
22:13bit more saturated that's when you have
22:14to go do the hard stuff right but you
22:16know it's kind of interesting to to
22:18think about you know prior technology
22:20waves and when should you do the easy
22:21versus hard yeah it's actually just
22:23talking to some of the founders that are
22:24in our accelerator right now that come
22:26from like really great Technical and
22:29research backgrounds and they were
22:31reaching for a problem broadly in the
22:33engineering and code generation space
22:36that was very ambitious right and I
22:39could see kind of a a solve it all type
22:43um and it's not that it's not valuable
22:44it's just that there is so much you
22:46could do that is as you point out easier
22:51um and like requires pushing the balance
22:53of research but you have far higher
22:56likelihood of having something that's
22:58useful to give to customers this year
23:01with far less risk and I don't mean to
23:04constrain people's Ambitions but the
23:07ability to give yourself multiple at
23:09bats with the wind at your your back in
23:11terms of the entire field progressing
23:13versus trying to get out in front of
23:15everyone else with a a multi-year
23:18research goal when there's like it's
23:20just gold hang out everywhere you know
23:22my my orientation is is I think similar
23:24here yeah it's no GPU before product
23:26Market fit I think that's the takeaway
23:27the loud slogan of the Year okay awesome
23:30uh fun to hang out and talk about the
23:34find us on Twitter at no Pryor's pod
23:37subscribe to our YouTube channel if you
23:39want to see our faces follow the show on
23:41Apple podcast Spotify or wherever you
23:43listen that way you get a new episode
23:45every week and sign up for emails or
23:47find transcripts for every episode at no
