00:05a blockchain operating system might just
00:07be the key to a democratized web3 in
00:10fact more than 25 million users are
00:12already getting a taste of this thanks
00:14this week ilad and I are joined by Ilya
00:17palosuken the co-founder of near and
00:19co-author of the landmark Transformers
00:21paper to discuss the interaction of
00:23blockchain and AI Technologies what we
00:25should expect from AI agents how to
00:28handle the content authenticity problem
00:29and why the alignment problem in AI is
00:32really a human problem Ilya welcome to
00:34no priors thanks for doing this thanks
00:36for inviting you are one of the authors
00:39of the original Transformers paper we've
00:42also had gnome and Jacob on how did you
00:44get involved with that seminal work in
00:46AI I worked on a team on natural
00:48language understanding it's focused on
00:50question answering and the state of the
00:53art at this time was lstm's recurrent
00:57which you cannot launch in production at
00:59all because they're too slow and take a
01:03fair bit of time to process as documents
01:06scale so Jacob at the time was using
01:10attention for query similarity and he
01:13had this idea like using attention for
01:15encoder decoder type
01:18um I kind of jumped into it and with the
01:21shoes were playing around with can we
01:24actually get it to train and understand
01:25the order of words and do translation
01:28just based on you know attention so yeah
01:32it was pretty cool to explore that and
01:34obviously grew into something very
01:36interesting and awesome
01:38you originally co-founded near in I
01:41think 2018 meaning for it to be an AI
01:43focused company what was that initial
01:46Mission and how did it become a
01:49yeah so we started this idea that we
01:51wanted to teach machines to code you
01:53know we have Transformers coming out
01:54there was a lot of kind of really
01:56interesting push in 17 16 17 around Ai
02:01and so our expectation was we kind of
02:04would ride the exponential growth of AI
02:08which has happened in this year we
02:11thought it will happen in 1718 and so
02:14with that we got a really interesting
02:16data set around language to code but
02:19more interestingly we had a whole
02:21community of developers mostly students
02:23who were doing crowdsourcing for us so
02:26we would give them code they would write
02:27descriptions we would give them
02:28descriptions they would write code for
02:30them write tests like all kinds of tasks
02:33and we actually faced the challenge of
02:35paying them because a lot of them were
02:37in China in Eastern Europe and kind of
02:40other countries where there's monetary
02:42control problems people don't have bank
02:43accounts and so we started looking into
02:46blockchain just like to solve our own
02:48problem the eye kind of uh expansion
02:51explosion didn't happen at the time and
02:53so we saw an opportunity of we can
02:56actually build a blockchain that we
02:57would use to solve this first and focus
03:00on that uh while kind of waiting out the
03:03AI thing to really happen and as you go
03:07into the blockchain rabbit hole you
03:09realize there's a lot more that meets
03:11the eye yeah yeah ended up being a
03:13pretty big mission exactly so you call
03:16near a blockchain operating system for
03:18any of our listeners who haven't used it
03:21like what does that mean
03:22so the idea is that we want to kind of
03:26go abstract right we want
03:29kind of an environment where you can
03:31discover and use web3 experiences you
03:34know benefit from them and not need to
03:36think about the low level you know
03:38implementations and quote-unquote
03:40Hardware that runs under it right so
03:42similarly how operating systems on your
03:44phone you know kind of abstracts out all
03:46the complexity of you know networking
03:48and payments and everything you just use
03:50it and you have apps that developers can
03:52build and so that's really what we're
03:54trying to achieve and kind of build this
03:56framework and platform for everybody to
03:59build their applications in web3 and
04:02really deliver it to the user to
04:03Consumer where do you see a lot of that
04:05overlap coming in terms of web3 and AI
04:08you've thought very deeply about both I
04:10remember when I first met you you were
04:12just switching from sort of nears
04:14original Mission into the blockchain
04:16and you know you were known as a team
04:18that could literally build anything
04:19right like you had yourself and Alex and
04:22Pie guy and all these like amazing
04:23people and you went down the direction
04:25of building blockchain in part I think
04:27originally around this data labeling
04:29kind of mission and the ability to do
04:30payments and things like that and now I
04:32know you've been thinking a lot again
04:33about how these two worlds interact or
04:35intersect where do you think are going
04:37to be the biggest places of overlap
04:38between Ai and blockchain or web3
04:41there's few levels of interesting
04:44intersections I think the the most
04:46obvious one that everybody talks about
04:47is various marketplaces for resources
04:50right be that compute model or data
04:53right so data crowdsourcing so those are
04:56pretty obvious right web 3 is really
04:57good at Market creating marketplaces
04:59creating traceability and uh providing
05:01like an equitable place for everyone to
05:04now the more interesting ones is where
05:07AI kind of Agents right which you know
05:10we've seen like initial versions of but
05:12obviously they're going to continue
05:14if we you equip them with a blockchain
05:18account right they are now becoming an
05:20economic agent that is able to pay other
05:23people and pay other AIS to do work
05:25right and they can communicate right and
05:27I think one of the things that a lot of
05:29people who are oh like just language
05:31models are just the same advancements
05:33like as everything before missing the
05:36point that this is the first time that a
05:40machine is able to communicate with
05:41people in the same way right there's no
05:43more need in an intermediate human that
05:46interprets data and then tells it to
05:48other people now machine can communicate
05:50directly to people and so it can task
05:52them with work it can provide them
05:54context and so I really think one of the
05:57most interesting cases is organizations
06:00that are run completely by AI right
06:02where quote unquote CEO role is taken by
06:04AI agent who is tasked by you know by
06:07Community or board of directors or
06:09whatever is oversight governance is to
06:12you know hit specific apis and follow
06:14specific Mission they can even give
06:16specific feedback with training data
06:18when they don't think it's doing the
06:20right job but what it does is like
06:22creates this kind of a new layer of
06:23management that potentially removes a
06:25lot of middle management right now which
06:27is like transforming information and
06:29context for each individual person and
06:31giving them specific area of work and
06:33then gather like kind of harnessing
06:35their creativity and putting it back
06:36together right I think that's is very
06:39interesting use case that kind of really
06:41melds blockchain AI together
06:44why like you have a traditional biotech
06:47cancer research commercial entity like
06:50why blockchain and why AI for that
06:53I use this example right we want to you
06:55know continue making progress on solving
06:58cancer right and it's a very complex
06:59problem right there's a lot of like
07:01specific sub cancers that you know need
07:03research and so all of this and like
07:05coordinating people doing experiments
07:07propagating information recruiting you
07:09know people recruiting the candidates
07:11right all of this requires like somebody
07:13to do this work and kind of organize the
07:15process and really set up a lot of
07:18Pipeline and you know funding and all
07:19those things and right now there's so
07:21much overhead around everything from you
07:23know how grant funding is allocated from
07:25the non-profits that collect money for
07:27research how you know like experiments
07:29are set up the information sharing like
07:31all of those pieces are really kind of
07:33broken and so you can actually have you
07:36know like recording coordinated effort
07:38that is designed just to do that and it
07:41can consume all this information and
07:43kind of specifically task you know who
07:45is the best person at doing the
07:47experimental which lab is the best at
07:49doing this specific sets of experiments
07:50you know fund them for this you know
07:53amount of money you know over oversee
07:55their delivery and then kind of iterate
07:57and you know if if it thinks this lab is
07:59not doing a good job fire them without
08:00having like extra you know personal
08:03affiliations that you know people do
08:06have I'm actually excited about some
08:08folks are already building some examples
08:10of this in like a simpler uh forms but I
08:13think we'll see you know first
08:15organizations like this probably even
08:18potentially it was a simpler missions
08:20and kind of more straightforward like
08:21kpi metrics but where kind of this
08:25information propagation and onboarding
08:26of people happens already through a kind
08:30of uh language model AI agent a simpler
08:33version of this that I've heard people
08:34talk about and it may be the first step
08:36towards it is actually providing
08:38on-the-job feedback via an AI versus
08:41like a human manager with the idea that
08:43it depersonalizes the feedback right so
08:45if you have a agent or an AI providing
08:48feedback some surveys at least have
08:50suggested that the average employee may
08:52be more comfortable with that because it
08:53feels more objective it feels
08:54depersonalized it feels like it can be
08:56provided in a directive way and it seems
08:58like that's one aspect of sort of this
09:00Ai and CEO concept that you're
09:02describing do you think the first place
09:03that it'll show up is Dows or do you
09:05think it'll show up in a different part
09:07yeah I think Dallas is and especially
09:09what happened to his dad's there was a
09:11lot of people who were really excited
09:13about Dao's kind of as a concept and so
09:15they put a lot of time running them but
09:18it's actually a very like not
09:19interesting job right it's like in young
09:21board new members you explain to them
09:23all the same thing you know you answer
09:24to their questions and so that's the
09:27part which like you can already automate
09:28right you can like have a Discord bot
09:30that is like have all the context about
09:32the Dallas you know interactions and
09:34kind of onboard new people and it gives
09:36them like new you know tasks to start
09:39with and kind of coordinate them so I
09:41think that will be the first place where
09:42this kind of starts showing up and as
09:45well because you have like payments kind
09:46of very like there and you don't have
09:49any social constraints that usually you
09:51have in like regular organizations like
09:54you know I I a lot of people will Revolt
09:56if you like tomorrow say hey by the way
09:58your new boss is the say I models
10:00yeah yeah how do you think about AI in
10:03the context or I should say blockchain
10:04and AI in the context of things like
10:06alignment yeah so I think this is a very
10:08interesting topic so I have this view
10:10that we need human alignment instead of
10:12AI alignment so right now kind of when
10:15we talk about you know hey we need to
10:17align AIS with like human values but the
10:19reality is that you know all the
10:21problems that exist they all exist
10:25because of humans doing things and and
10:28they've existed before I actually like
10:30to use the Byzantine fault tolerance
10:33problem right which is basis for
10:35blockchain but the its roots are in you
10:39know history where there was people
10:41propagating misinformation and you were
10:43trying to like figure out how to prevent
10:46misinformation in the Army right so this
10:48is like a really old problem of
10:49misinformation and kind of
10:51um like how to work around that and so I
10:54think what we need to start doing is
10:56figuring out how do we build a society
10:58that is actually able to deal with uh
11:01kind of effective misinformation at
11:03scale right so like we've kind of built
11:05like a lot of our society has started
11:07building up tolerance to misinformation
11:08around you know TV and mass media but we
11:12don't have like a system and framework
11:14around dealing with it at scale and
11:16that's what AI brings brings just scale
11:18to the same problem and so this is where
11:21reputation identity and kind of systems
11:25around our social code operating system
11:27that powers our community kind of
11:30communities is really important like how
11:32do all these pieces work together and
11:34how do they actually operate when there
11:37is malicious actors who potentially are
11:39able to you know in Mass create like
11:41very personalized misinformation or
11:43create you know fake political actor
11:45that is you know convincing every
11:47individual exactly in what uh they think
11:51you know that government should do to
11:53get elected and the this is where web3
11:56comes in as like a set of Primitives
11:58right we have cryptography to
12:00authenticate content and create uh a
12:03path everything from you know you take a
12:05picture of his camera some of them
12:06already have a secure Enclave that can
12:09sign the image that's taken and so as
12:13that image gets processed we can
12:15actually propagate that information and
12:16have a proof that it came from you know
12:19specific time and place and then being
12:21processed by specific set of filters
12:23right so that can give you like an
12:26anchor then you still need to know kind
12:29of who is publishing what right like
12:30recording this podcast you know people
12:32listening to it it could have been
12:33completely generated at this point right
12:35but if for example we all sign the you
12:39know the final podcast and say hey yes
12:41we've recorded it and this is valve
12:42content now when somebody's listening to
12:45it they can check that indeed hey this
12:47content is signed by us now the question
12:49of us comes in right so this is where
12:51kind of identity and reputation is
12:55this is where uh kind of Unchained
12:58identity becomes your kind of
13:01coalescence of all of the content and
13:03all the interactions that you do and
13:05then that links to kind of you know
13:07reputation in different communities uh
13:09and uh provide context for people who
13:12are watching for this content to be able
13:15to understand you know who is this
13:17person who's talking or where they're
13:19coming from and what are the information
13:20values they have so I think like it's it
13:23needs to be a kind of systematic
13:24approach and it'll start with pieces
13:26right I think one of the important
13:28pieces will be kind of a green lock
13:30similar to SSL transition on the content
13:34right like as you go to YouTube as you
13:35go to uh you know New York Times you
13:38actually will see that like hey this
13:39content been signed by this party and
13:42this party is in some trust root or
13:44trust Community like uh graph of
13:47communities that you are following right
13:49so that's probably like one important
13:51piece and again blockchain and
13:52cryptography is just like tools to
13:54enable that product variants and then
13:57from there you know we need similar
13:58things on the government level right
14:00when you file paperwork when you file
14:02you know your identity the fact that
14:03your SSN is a you know number that you
14:07give to everyone which is like supposed
14:08to be secret is like for example
14:10ridiculous right so things like that is
14:12like all of this needs to improve and
14:13kind of upgrade to this new level where
14:16like a massive amount of kind of at
14:19scale of things that have been happening
14:21now are possible what do you think is
14:23the most likely form of blockchain based
14:25identity because you know the blockchain
14:27really has been the earliest place where
14:29you've had programmatic actors
14:30interacting around economic and other
14:33utility functions right it really is
14:34money as code and effectively smart
14:37contracts are ways to programmatically
14:39interact with that right so you you had
14:41almost like the execution layer without
14:43the intelligence and now we're adding
14:45the intelligence you have the
14:46cryptography but you're missing a real
14:47sense of identity which is needed if you
14:49have an agent or bot representing you
14:51interacting with another agent which is
14:52probably where a lot of things will work
14:54in the future online what do you think
14:56is a most likely form of identity on the
14:57blockchain and why hasn't happened yet
14:59it has happened to some extent right we
15:01have you know like millions of people
15:05actually using blockchain right now and
15:07they're using it more for financial use
15:08cases and kind of that's their financial
15:10identity the wallet is identity kind of
15:12thing yeah well it has became an
15:14identity right and the reality is like
15:16your quote unquote private keys are your
15:18identity but that's just too hard of a
15:20concept for people to actually work with
15:21right and so on near we actually changed
15:24that we you know you have a properly
15:26named account so like mine is root.near
15:29which can have lots of different private
15:31Keys accessing it with different
15:32permissions right I can give a key and
15:35in a way permissions to an agent to for
15:37example interact on behalf of this or I
15:40can withdraw it right I can give it to
15:41specific application Etc so like a more
15:44extensive model is needed that's one we
15:48need to have more social interactions
15:50kind of being spawned from this and so
15:53this is again blockchain operating
15:54system is powering actually social
15:55interactions and kind of communication
15:57we actually have a project working on
15:59chat and other ways of using now this
16:02identity it in more places it's mostly
16:05because we didn't have a critical mass
16:07of this applications that are using this
16:10identity so to for it to really become
16:13kind of the core and if it's not the
16:15core it's not as useful because nobody
16:17you know like hey you don't have it so
16:18like we're not going to use it as a
16:20default thing everywhere so like we
16:22really need to kind of go over like
16:24again I think SSL is really good example
16:26of something that's like it it delivers
16:29value it's clearly valuable but it was
16:32such a like uphill battle to get it
16:34there right and so I think like until
16:37you have this critical mass of like
16:39kind of website switched and browser
16:41support it didn't become a default right
16:43so we kind of need like the same here to
16:46happen like we'll need to have a
16:47critical mass of applications using you
16:50know identity and then uh then we kind
16:54of seize it like in browsers or wallets
16:56or whatever like applications to hold it
16:58and then we'll see a transition function
17:00happen where like hey oh you don't have
17:02it like you should get it because it's
17:03actually easier and better to use it and
17:05it gives you like more Financial Freedom
17:07as well and more upside
17:09where do you think the most likely
17:11failures like system-wide are are to be
17:14like with um you know growing
17:16capabilities in AI like where where do
17:19these mid against in terms of uh
17:20reputation systems with blockchain or
17:23like content Providence are are likely
17:25to how is that going to manifest in ways
17:29yeah I think there will be probably next
17:32year will be very interesting in U.S
17:33because I think this this will be a
17:36place where everybody will just take
17:37whatever their toys they have in toolbox
17:39and do it even Just for kicks right even
17:42if it's not malicious although some
17:44players will be malicious and I think
17:46that what we'll see everything from like
17:48completely fake narrative candidates
17:51uh to like I would be very interested to
17:55see like a web page where you land and
17:58you know you log in and it literally
18:00generates specifically for this user
18:03based on their interest a agenda for
18:05this candidate right so like hyper
18:08focused you know marketing for
18:10candidates based on like who's this
18:13um voter is right so things like that
18:15like we'll have all those possible
18:17things where the media will kind of be
18:19flooded with like you know you can spin
18:21up New Media right now and just generate
18:23content about your candidate like that
18:25you want uh and then Market that so like
18:27you can have like all kinds of things
18:30now just exploding without any way of
18:33like framing it on the user side if like
18:36does this have history is this coming
18:38from the right sources has it been
18:40validated right and so I think that's
18:42going to be a really
18:44um uh important I think the other side
18:46actually is law enforcement and this is
18:48sadly already happening the people are
18:51using these tools now in very malicious
18:53ways right now and law enforcement don't
18:55have a like really good ways to deal
18:58with this and so I think everything from
19:00this like on camera like signing we need
19:04this now like they really have no way to
19:06like kind of identify uh if the image
19:09was generated or not and similarly Like
19:12For You Know audio recordings and things
19:14like that like there needs to be kind of
19:16additional kind of levels of uh
19:19verification and this goes into actually
19:21like video calls and voice calls because
19:24right now somebody can call you on the
19:26phone and play a recorded record like
19:29generated audio of somebody that
19:31recorded 30 seconds off right and this
19:34can be this very nefarious means right
19:36it's a huge Consumer Fraud problem
19:37already well it's huge consumer but it's
19:40also like beyond that is becoming like a
19:42real criminal problem like criminals are
19:44be able to use these tools now and it's
19:46like the barrier of Entry there is like
19:48very low and so uh this is where like
19:51you really need like you know the phone
19:52calls the kind of all of this like you
19:54need more information identification and
19:56like kind of cryptography embedded into
19:59the system otherwise it's completely
20:01going sideways really quickly yeah this
20:04is where people would be using apis like
20:05element or lfg or 11 labs to create a
20:10void snippet right where they'll upload
20:12to your point 30 seconds of voice train
20:13a model and then the output sounds like
20:16close enough to the person that you
20:18could fool uh financial advisor or a
20:20bank or somebody else to you know do
20:22transactions on your behalf or things
20:24like that yeah or and you like swipe
20:26their phone and and now you're able to
20:28like impersonate completely right so
20:30yeah so this is like a real problem and
20:32like having kind of authenticated passes
20:36to really establish and like we have
20:38actually like the phones are actually
20:39have so much already like we have face
20:42ID and fingerprints we have you know
20:44there's secure enclaves that sign things
20:46that are like haven't been hacked as far
20:48as I know like so there's like a lot of
20:50the pieces are there now we just need
20:53like a product stack that actually
20:54pushes it uh to the user and and like to
20:58yeah that makes sense I guess one other
21:00area where some people have talked about
21:01overlap between the blockchain world and
21:05the AI world is around training and
21:08there's almost like two or three
21:09different forms of that one form of that
21:11is there's a lot of GPU capacity that
21:14was purchased for mining on the crypto
21:16side and given how valuable GPU is now
21:21on the training side there's all sorts
21:22of sort of models to aggregate gpus
21:25specifically for training in different
21:27ways you know aggregating access
21:28capacity and then separate from that
21:30there's ideas around well can you train
21:32a model in a distributed way across a
21:35blockchain more generally do you think
21:37either of those things are Concepts that
21:39will work or how do you think about them
21:40relative to the Future
21:42yeah I mean it's interesting because it
21:45it sounds like such a no-brainer that
21:47hey let's grab those gpus that for
21:49example ethereum just moved from proof
21:51of work to proof of stake let's grab
21:53those and start using them the challenge
21:55is the gpus there are like not the ones
21:58that AI folks want to use right uh like
22:02kind of old AI is really zeroed in on
22:05like how do I get a100 so h100s and the
22:08gpus that like folks used for ethereum
22:10mining and like similar
22:13um is like older ones like uh that are
22:16not also focused on like floating Point
22:18arithmetic for example as much and so
22:21the challenge was more around like
22:23people who did did that like core weave
22:26is probably a good example right they
22:27were Mining Company like it's more that
22:29they had a know-how how to build data
22:31centers and they can like get access to
22:33massive like talk to Nvidia and like get
22:36massive access to that versus like
22:38repurposing the same gpus although I
22:40mean obviously like for smaller models
22:42for some specific uh maybe in for
22:45instance there's there's maybe a
22:46transition there's a question of
22:48decentralized training right uh in
22:50general right like hey we have like lots
22:52of gpus everywhere can we train it and
22:55the reality right now that the
22:57requirements on bandwidth right like
22:59people who are training these models
23:01right now they have like a you know 800
23:04gigabit connect right between the gpus
23:07right so maybe you have 100 megabits on
23:10between this usually not and you need to
23:12like replay and like uh work around
23:14problems for decentralized so I think
23:17decentralized training right now is like
23:18still not as realistic although there's
23:20some research people are trying I think
23:23an inference is really interesting
23:25because we do need so much more compute
23:28for inference than we need for trading
23:29right like it's it's a very interesting
23:31like economy of scale you train once
23:33like llama trained once and then
23:35everybody runs it everywhere and so the
23:38inferences where I think there's a lot
23:42cases one is you want it to be private
23:45right right now if you're doing
23:47inference uh you need to send it to some
23:49service and that service may or may not
23:51record it and uh okay both input and
23:58you want large capacity at like that can
24:01scale with more usage right tomorrow I
24:03have you know 10x more users I want to
24:06be able to scale with that and so this
24:08is where I think using some of this
24:10Hardware that exists as well as kind of
24:13leveraging maybe new methods of privacy
24:16and coordination that can again crypto
24:19has like NPC like multi-party
24:21computation there's zero knowledge
24:23proofs Etc like they can be leveraged to
24:26uh achieve that and have kind of uh
24:29secure like secure decentralized
24:31inference so I think that's way more
24:34realistic than training and also many
24:37and then I guess one of the really early
24:40applications that Nero was thinking
24:42about was Data labeling and to your
24:44point the ability to pay people who are
24:45doing data labeling for AI purposes
24:48right and since that time I think a
24:50number of companies have really grown
24:52out in terms of the data labeling World
24:54in a centralized way there's scale.ai
24:56there's serves there's a few others do
24:58you think the best solution in the long
25:00run is still a decentralized model where
25:01you're using tokens to pay effectively
25:04for labeling do you think things will
25:06stay in the centralized world like how
25:08do you view all that evolving over time
25:10yeah I think decentralized kind of the
25:12web stream Marketplace is a more
25:15effective way to do this and it kind of
25:17provides few interesting benefits one of
25:19them is that it opens up kind of the
25:23market right where you don't need to set
25:25up like a local office and kind of hire
25:28people and train them Etc like you can
25:30just open up Global Market anybody can
25:32join and you have a very specific rules
25:35right that if they follow they get paid
25:37right so I've used Mechanical Turk
25:39before for example and you can actually
25:42as a client you can just decline them
25:44paying them right so people in
25:46Mechanical Turk like the workers have
25:48very low kind of way to push back if if
25:52I say at the same time they don't have
25:54any like quality and knowledge
25:56assessment on the platform right so so I
25:59think having quality knowledge and this
26:01kind of escrow model all embedded into
26:04one Marketplace that opens up for
26:06everyone and get you know anybody
26:07everywhere can get paid at any time like
26:10offering that both the people who doing
26:12this work want because they kind of are
26:14more protected actually and it's like
26:16fair game and then the people who want
26:18to give tasks they can actually get
26:19access to like way larger uh Workforce
26:22they can like specify specific
26:24parameters they can you know price it at
26:25whatever level they want that's going to
26:28be the kind of future of it
26:30can you talk a little bit about what
26:32makes the quality control problem for
26:34annotation hard here right because one
26:37thing that I've seen with significant
26:38research Labs is like still continued uh
26:41in sourcing of annotators
26:45um for both pre-training sets and lhf
26:47because some of the external services
26:50and marketplaces can't get to the level
26:53of quality that they're looking for in
26:55particular domains so can you just
26:56describe the Dynamics there
26:58yeah so I think there's two parts one is
27:00like domain knowledge right
27:02um that generally like heart like it's
27:06hard to tap in in into like a very
27:09specific centralized service right
27:10because they need to kind of like for
27:12them to do payments do all those things
27:14they need to set up a subsidiary in
27:16whatever country they have the workers
27:18so you need to train them they need to
27:19hire them maybe it's contracts but like
27:21they need a lot of overhead that they do
27:23that for example developers let's
27:25imagine you know you're building a new
27:27really cool developer platform uh which
27:31uses you know language funnels and you
27:33want to fine-tune on code right well the
27:36existing platforms like them hiring a
27:39bunch of developers uh to actually do
27:41this right and you know if they're doing
27:42this full-time that's like super
27:44complicated then uh kind of building out
27:47the validation tooling for how to like
27:49cross-validate that the work has been
27:52done now on Webster Marketplace you know
27:55any student can join and like do do this
27:57right they don't need like you know join
28:00it like get a contract with a specific
28:01company they don't need to have the
28:03company in the local region to work with
28:07um and like students you know for coding
28:09for example are really interested in
28:10doing this because they are usually
28:12don't have much money and this is a way
28:14for them to practice their uh work
28:16anyway and then as a task Giver you can
28:19actually specify the specific way you
28:21want the cross-validation to happen and
28:23uh one of the things we've done uh it's
28:25like honeypots right where you actually
28:27specify specific types of incorrect
28:29answers that people need to Mark as
28:31Incorrect and otherwise they actually
28:33lose uh the buy-in and so there's like
28:37clear like economic Game Theory where
28:39people have buy-ins they uh they lose
28:43them if they like do for quality of work
28:49like way more incentive to do this
28:51versus like let's say if you're working
28:52on a contract there's like way more
28:54leeway usually uh if you're not doing
28:56your work right so it's like just way
28:57higher kind of uh self evaluation as
29:01well that happens and so I mean there's
29:03a lot of pieces that needs to come
29:04together for this to be like high
29:06quality but again it just opens up this
29:08Marketplace and makes it effective and
29:10it in a way removes a lot of the human
29:12part as well one thing that I think is
29:14really neat about how near approaches
29:15Innovation is you do both internal sort
29:18of near Road mapping and product
29:19development and then you also have a
29:21series of things that you either spin
29:23out or spin up or you're sort of
29:24involved with sort of these ancillary
29:25companies or projects or efforts what
29:28areas are you most excited about over
29:30the next coming year in terms of either
29:31nearer some of these other efforts that
29:33you're involved with so we do actually
29:34have a project uh in this web 3 AI data
29:39Marketplace that we are spinning out
29:43um to focus on not now like they build a
29:46product they have all the pieces now
29:47it's like ready to actually go to market
29:51I think the the really interesting area
29:53is kind of partnering with existing kind
29:57of I'd already website enabled or
29:59interested in web3 teams who want to
30:02give access to more functionality to
30:04their uh users right we have for example
30:07sweatcoin which is really good example
30:09of like it was a web 2 project that had
30:11120 million installs that had a ton of
30:14people using it every day kind of for a
30:16very specific use case right kind of
30:18tracking their steps and you know maybe
30:19getting a discount on their next shoes
30:21but now as they transform into F3
30:24they're kind of opening up right and you
30:26can now participate in economic activity
30:28you can you know learn about new kind of
30:31innovations that happen in the ecosystem
30:33you can now you know but like as they
30:35integrate more into block Library System
30:37I can potentially interact with like on
30:40the social side do the tasks and gigs
30:42and so like you kind of really open up
30:44the what before was like a very limited
30:46kind of economy to really do this like
30:49you know composable Open Lab I think
30:51that's really exciting and like we will
30:53see probably more and more examples of
30:54that uh and finally I'm really
30:57interested in kind of as I mentioned
30:59like because we have now open the web
31:00and social wear the kind of what I call
31:03future of SAS so I think a lot of
31:06between web3 and AI a lot of SAS will
31:09actually start being uh replaced because
31:12right now what SAS is is like one
31:14database with a specific UI for a
31:17specific problem the database is the
31:20same between CRM the hiring tool
31:23marketing Tool uh even some of the
31:26project management tools right the
31:28database underlying is like not that
31:29different and it's been just like the
31:31front end and like interconnecting all
31:34of those databases is like ton of work
31:36it always breaks right
31:38um but now you can have like the
31:41database you own right so using kind of
31:44step three tags and then you can build
31:46all of this front ends on top either
31:48through kind of block sharability system
31:50shared components or even through
31:52describing with natural language some of
31:54the interfaces and business processes
31:55you want to have right so the way people
31:58will interact is like kind of their
32:00business operations and all the tooling
32:02they need will start to change and I'm
32:05so I'm really excited about this space
32:06and like we have one company that is
32:08kind of you know starting uh to build
32:11out some of the things in this space and
32:12over next year we'll see kind of that
32:15evolving do you think that moves to an
32:17agent-driven World in other words when
32:18you imagine the interfaces on top of
32:21this that are sort of driving these
32:22business processes for future SAS
32:25to view them as sort of traditional uis
32:27or do you view them as agents that are
32:29interacting programmatically or some
32:31hybrid it will be a hybrid so I like in
32:34my imagination right now at least I
32:36expect like you can describe a business
32:38process which is like hey you know when
32:39we have a new creative from like
32:41marketing department spin up a Twitter
32:43campaign and create me a dashboard that
32:46tracks the conversions on our product
32:48right and so what it does it like
32:50creates you know the pipeline of those
32:52things and then it also creates a page
32:54where I can see like normal user
32:56interface of like analytics so it might
32:59be more generated Dynamic UI exactly
33:01yeah and it's like adjusted for specific
33:03use case you need and probably there's
33:06like a bunch of templates that is like
33:07you know fine tunes for your specific
33:09problem like and this is possible right
33:12yeah I guess it kind of moves you um
33:14down the path of what you were talking
33:15about in terms of like AI CEO or AI is
33:17project manager where you're kind of
33:19morphing into a world where you're
33:21delegating to an AI to drive a bunch of
33:23activities and then come back to you
33:24with the results like you would an
33:26employee or a co-worker which is very
33:28different from the world of UI today
33:30where you just go to the same spot to
33:32see analytics you go to the same spot
33:34for communication you're good which is
33:35your email you go to the same spot for
33:37you know interacting with the workflow
33:39and you're saying this should be more of
33:40a dynamic world where things get brought
33:42back to you based on a series of tasks
33:44that you provide out yeah and there's
33:45like probably a shared environment as
33:47well where you know we probably will
33:48co-work on a business process and you
33:50know we'll share one display but then
33:52we'll maybe Fork it because I'm more
33:54interested in conversion and you're more
33:56interested in retention for example and
33:58so so that's kind of the dynamism right
34:00now that also doesn't exist where like
34:02we all look at the same you know jira
34:04task management and I'm like I don't
34:07really care about half of this stuff
34:08right but it's not a filter problem it's
34:10like I want different information showed
34:14author of the paper that changed the
34:16world here we are in 2023 is it bigger
34:19Transformers all the way are there other
34:21architectural directions that are worth
34:22thinking about that you're paying
34:25I think there's definitely something
34:26around like how do we get these models
34:29to have the capacity to
34:31like let themself think before
34:33outputting or like kind of uh process
34:36more and I I think it's like still
34:39within the Transformer structure and it
34:42can be like Advanced but I haven't seen
34:44anything that's like really matches my
34:46intuition around us but I think the like
34:50the Simplicity of this architecture and
34:52like indeed like the the amount of
34:54optimization that's going into this
34:56right now is just it'll be really hard
34:58to match uh and kind of you know
35:01there's enough exclusivity you can
35:03express any function so like it's not
35:04this is not a problem at this point of
35:06like hey we don't have an exclusivity
35:07right it's more around how do we how to
35:11compose a data set that's you know
35:13cleaner better or add some you know
35:15self-critique and understanding of like
35:18is this content correct or I need more
35:20time to think what versus you know hey
35:22I'm forcing you to Output next token
35:24even if you don't have an answer yet so
35:26I think that that parts we really need
35:28and and I think uh they kind of fit in
35:30the architecture but
35:32um just require more engineering and
35:34more different types of tasks as well
35:37for training I think like you know the
35:40fact that we're just using a big
35:41language model is kind of interesting
35:43because this is not the task you would
35:45expect uh everything to be able to you
35:49know just predict next token so like you
35:50know starting to obviously at all of H
35:52being already helpful but like starting
35:54to like hey can you critics the center
35:56what would be the balance Etc that is a
36:00training or fine-tuning thing or do you
36:02that as an inference thing I mean it's
36:04going to be like a combination right so
36:06I think we just need an architecture
36:08that at training time your enable to so
36:12like I mean this the simplest thing is
36:14like instead of outputting a token in
36:16the next right you can actually give it
36:19like you know empty token for example
36:21for some period of time and then when it
36:23says like okay I'm ready give a child to
36:25the next token right and so this way you
36:27can train into like think more before
36:30outputting and then at inference time
36:32you can vary it right like hey I'll give
36:34you more time to think you know uh or
36:36like no you have no time to think but
36:38then you can train it to like actually
36:40be able to like dynamically to uh so
36:42again this is like a very simple thing
36:44but like you can keep expanding on this
36:46you know output it and then feed it back
36:49and like is this the right answer like
36:51Etc so there's a few different models
36:53but I think the toyakov's point like the
36:56the fact that this model is like doing a
36:58really effective search in kind of this
37:00knowledge space means that probably like
37:03pushing more into that concept is more
37:06useful than doing more searches at
37:08inference time because like it means you
37:10already lost all the semantics if you're
37:11doing searches the first time I think
37:13you made a really interesting point
37:14where it's possible that Transformer
37:16architecture increasingly is getting
37:19and there's two components of that one
37:21is it just seems to run really well on
37:22the main silicon that we're using right
37:24now for AI which are gpus and then
37:27secondly there's so much optimization
37:31and so much being built around it that
37:33it effectively creates
37:35optimization that just won't happen for
37:36any models anytime soon and so you
37:39effectively end up with this interesting
37:40feedback loop or lock-in effect for this
37:44do you think that we're in a spot now
37:46where this is just kind of the future
37:47for the next five years or 10 years or
37:49something or what do you think is the
37:51likelihood that other approaches or
37:52architectures will emerge anytime soon I
37:55mean there might be an another
37:56architecture that like reasonably fits
37:59with the same silicon I think that
38:00there's an interesting question example
38:02of there's a company that built an
38:04alternative right silicon that is kind
38:07of allows to process things in pipelines
38:10and so like the chips are actually like
38:13kind of smaller compute chips but they
38:16kind of all uh like in a grid and the
38:19data flows from one side to another
38:20right so the example there is on one
38:22side it's like a really interesting
38:24architecture you can build really cool
38:25things with this but it doesn't fit
38:27Transformers very well right like you
38:30can do Transformers with it but it
38:31doesn't fit very well and like your your
38:33cost like you get like you know cost to
38:37to Output ratio is not that interesting
38:39and so in comparison to you know you're
38:42just optimizing on gpus or using some of
38:45the new hardware accelerators and so
38:47this is where exam like I mean I'm not
38:50to speculate here on specific company
38:51but you know I wouldn't expect they will
38:54have like a ton of people lining up
38:56because like there is ton of
38:58Alternatives flow Transformers with
39:00coming in and like somebody would need
39:02to like go in and develop a lot of new
39:04architectures that fit better uh this
39:07model and so uh it'll be really hard for
39:11them to like be a viable business and
39:14kind of have the economy of scale the
39:15Nvidia is having right now just kind of
39:18continue optimizing and building best
39:20state-of-the-art chips right so unless
39:22somebody's like really investing in this
39:24I think it will be more around like what
39:26what else we can do with current silicon
39:28right and kind of combinations of this
39:30and then I mean maybe there's something
39:33new will come out yeah but with things
39:35lock in technologically they actually
39:36tend to lock in pretty strongly until
39:38there's a really big sea change or sort
39:40of the optimization of those things hit
39:42a asymptote and it's interesting because
39:44I think a prior example of this kind of
39:46um chip plus software reinforcement Loop
39:49was really the windows and Intel
39:51monopolies of the 90s you know they used
39:54to call it Wintel for Windows and Intel
39:55because it was such a strong Mutual
39:57lock-in effect where you had chips they
40:00were optimized for Windows and windows
40:01optimized for the chipset and it just
40:03kind of kept going from there and so
40:04it's interesting this is I feel like a
40:06stronger version of that in some sense
40:07where you have the underlying compute
40:10architecture and the most important
40:11model reinforcing each other in a way
40:13that kind of locks both of them in yeah
40:15and what what changed that is pretty
40:17much come of mobile right and creation
40:20of irm devices I run chips that are kind
40:23of optimized for mobile and then came
40:24back into PCS right so yeah so and
40:26unless there's like a completely new
40:28form factor which hard to predict right
40:31but also it's like that's a lot of
40:32investment to go from not just software
40:36not just Hardware but like full stack
40:38right Innovation yeah I think it's
40:40unclear if this is a strong enough
40:42Market Force but the short-term you know
40:46demand Supply imbalance around gpus with
40:49all of the growth of applications
40:51especially as like you think any of
40:52these applications work like inference
40:54needs grow right your ability to build
40:57enough for NVIDIA really to build enough
40:59gpus to service the demand is like it's
41:03blocking a lot of companies right and I
41:06think the question is like there is more
41:08incentive to make heterogeneous Hardware
41:10work than there ever has been and it's
41:13like can that catch up with the full
41:15stack optimization that you described
41:17the Cuda like investment that Nvidia has
41:21it's super unclear but I think like
41:23there's been no reason to chase that
41:25until you know this past 18 months and I
41:27think now there is yeah but at the same
41:30time we have like every single you know
41:32large companies doing their own Hardware
41:34accelerator as well as you know a bunch
41:36of folks who are kind of spun out of
41:38those and so like we're going to have a
41:40you know a market full of Hardware
41:42accelerators which are still optimized
41:44for Transformers or at least like
41:45similar structured architectures hitting
41:48the market like this year in the next
41:50year yeah Elliott this is great I hope
41:53you will uh after a lot and I work
41:55through all of the Transformers authors
41:57like Pokemon style gotta catch them all
41:59I hope you'll come back for a reunion
42:00episode but thank you for doing this
42:02yeah thanks for jumping on for sure
42:06find us on Twitter at no Pryor's pod
42:08subscribe to our YouTube channel if you
42:10want to see our faces follow the show on
42:12Apple podcast Spotify or wherever you
42:15listen that way you get a new episode
42:16every week and sign up for emails or
42:18find transcripts for every episode at no
