00:00hi everyone welcome to the a 6 and Z
00:02podcast I'm sonal given all the ongoing
00:05excitement around artificial
00:06intelligence deep learning and machine
00:07learning especially with the nips
00:08conference this coming week today we're
00:10talking about what happens when we go
00:12from so-called toy problems to practical
00:15AI in production the conversation is
00:17also part of our ongoing series on AI
00:19and practice you can find other pass on
00:20upcoming episodes on our website under
00:22that tag but joining us for this episode
00:24we have Jose B sack who leads strategic
00:26and programmatic partnerships for Amazon
00:28Web Services so has a front row seat on
00:31what's happening with a bunch of
00:32companies interested in AI and machine
00:33learning we have Scott Clark who's a CEO
00:36and co-founder of Sagat which provides
00:38optimization as a service and then we
00:40have general partner martine casado
00:42the discussion covers everything from
00:43taxonomy of startups and methods for AI
00:46to a brief debate about whether AI means
00:48the end of theory or not and we also
00:50discuss the problems of data and
00:51optimization as well as the pros and
00:53cons of machine learning as a service
00:55and touch on the theme of the API
00:57economy but we begin by quickly
00:59reflecting on where we are right now
01:01what are we seeing what companies
01:02adopting AI beyond R and D the first
01:05voice you'll hear is Scott followed by
01:06Joe why no so I think AI is kind of this
01:10in this unique position that it hasn't
01:11been in historically before all the
01:13pieces are coming together people have
01:15the datasets now they have the tooling
01:18and of the open-source community it's
01:19been huge in that with tools like MX net
01:21being widely adopted and production
01:23alized and now they have the
01:25infrastructure readily available with
01:27things like AWS and all these new Nvidia
01:29chips in addition to a whole bunch of
01:31api's to make a lot of the hiccup and
01:33like difficult parts of the system
01:34easier and easier and so the combination
01:36of all these things together means that
01:39instead of spending a decade in the R&D
01:41lab to try to come up with something now
01:43a couple of data scientists can make
01:45real business impact almost immediately
01:47with the AI go to market I mean as AWS I
01:50think we have more than 2 million
01:51customers now on our platform you can
01:54imagine all the inbound that we get
01:55where all these customers want to get
01:56into AI it's today's mobile first right
01:59I think sundar Pichai actually even in
02:01one of his big talks of the State of the
02:03Union at Google called Google and AI
02:05first company yeah which was quite a big
02:07shift totally Microsoft has switched
02:09from a mobile first over two and a half
02:11first now as well so I I think
02:13one sees there's actual business value
02:15the funny thing that I've seen is a lot
02:17of this what we were calling AI today
02:19was really just you know statistical
02:20predictions or like you know using basic
02:23regression techniques a little bit and
02:26is such a buzzword like financial firms
02:28they use really basic techniques and
02:30they call it AI so we see financial
02:32services getting disrupted we see
02:34healthcare life sciences preventive
02:36maintenance is looking at all the the
02:37sensor data on big machinery on
02:39airplanes on all kinds of equipment and
02:42trying to predict failures in the future
02:44this is actually one of the top use
02:45cases that I'm seeing recently why does
02:47AI uniquely help in that context or in
02:49this specific ml deep learning because
02:51of all these sensors now sprinkled all
02:53over these be machine all your airplanes
02:55your vehicle now is full of sensors you
02:57could take all that data you can
02:58actually predict a lot further into time
03:00or learn a lot more from the data over
03:03longer time series series of time it's
03:05one of those industries that is catching
03:08you up there they're kind of sitting on
03:10a goldmine of data but data doesn't
03:12equal AI I have a lot of those kind of
03:13larger customers they come to me and
03:15they say I have all this data what
03:16should I do now how do I do this AI
03:18think I get that so much and you know we
03:20have to even step back and say okay
03:22let's go build a day delay because you
03:23have to spare it data sources you know
03:25let's talk about the problem you're
03:27like actually start with the question
03:30hell are we doing with you know what
03:32what is it we're trying to do here you
03:34talk about hyper parameter tuning are
03:35you want you could talk about you know
03:36cleansing data and prepping data and
03:37entertain data and deploying it at scale
03:39and on IFT devices but if you're not
03:41actually understanding the problem you
03:43really want to solve when you see
03:44business value and a derivative problem
03:46that is really the ROI like how do you
03:48define the ROI and which problems to
03:49solve because there's problem that can
03:52be discovered I'm tracking something
03:53like five or six hundred use cases
03:55internally there are sales folks are
03:57coming to us and saying hey I got this
03:59problem I get this problem I got this
04:00problem I get this problem which are the
04:02ones that are salient enough to do
04:03driving ROI and this is something that
04:05comes up all the time with us but for
04:06you to even apply an optimization
04:08algorithm you need to know what you're
04:10aiming for yeah right
04:11so it needs to be tied to business value
04:13you need to be able to articulate like
04:15maybe if I'm building a fraud detection
04:16system maybe naively accuracy is the
04:19most important thing you could think of
04:21but if you catch all the $1 fraudulent
04:23transactions and miss the million dollar
04:25ones that's actually bad for the
04:27so it requires a lot of domain expertise
04:28and I think this goes into needing
04:30specialized datasets for every
04:32individual application but also unique
04:35targets and goals to shoot for and then
04:37once you have this complicated system
04:39and you have a target that you're
04:40shooting for then it becomes an
04:42optimization problem but if you don't
04:44have that data and you don't have that
04:45target you need to figure out what it is
04:47you're even trying to achieve well
04:49what's interesting hearing you're
04:50talking about is like hey you know every
04:52time you have like a really hot frothy
04:54space even the most basic questions or
04:56an answer like something as simple as
04:58like what is AI good for what can you
04:59apply to etc and I think every time
05:01you've got these kind of new buzzy
05:02things like people treat it like magic
05:04they're like you know I have like
05:06standard retro thing I add magic and
05:09then I get something amazing hey I in a
05:11box and boom you're up and running
05:12so I actually categorize startups that
05:14come in in one of four buckets and the
05:17AI spectrum from like kind of the most
05:19basic to the most science fiction and
05:21here the following so there are
05:23companies that come in that have been
05:24doing you know hardcore ml stuff for a
05:27long time but they haven't called it AI
05:30they're probably older techniques
05:31probably nothing the kind of latest DNN
05:33stuff or whatever and then they start
05:34calling it AI because they know that
05:36that AAS profit the second one and the
05:38one that I tend to focus the most on
05:39they actually understand what you can
05:40apply AI to so they're like you know
05:42it's good for these things to solve
05:44these problems they're taking that that
05:45are playing into an existing problem of
05:46doing an you startup so those I spend
05:48the most time with just because they
05:49understand the technology they normally
05:50have the core team they understand the
05:52the third ones are really interesting to
05:54me and I'm getting more and more
05:55interested in them but like it takes a
05:56little bit of a region I call them the
05:58end of theory so interesting so what
06:00they do is they basically believe that
06:02you can apply AI to a problems where you
06:05don't have to have a theory beforehand
06:06you don't need to know what you're
06:07looking for so for example let's say
06:08you've got a bunch of security data you
06:11don't know what to look for it in there
06:12will tell you what to look at or maybe
06:13you've got a bunch of marketing data you
06:15don't really know if there's something
06:17there but will tell you what to look at
06:18so you don't have to have a theory of
06:19what we're looking for but we'll apply
06:21it and give you a theory and then the
06:22fourth the most science fictiony these
06:24are the ones I don't give a lot of
06:25credibility to they basically want a to
06:27solve their product market fit problem
06:29they basically say I don't really know
06:31what company to build you know so what
06:33I'm gonna do is I'm gonna enter a space
06:34I'm gonna add AI and then like that like
06:36basically tell me what company or
06:37product to build and those I think
06:39that's mostly just kind of which will
06:40thinking I don't think it's gonna
06:41actually solve like what company to
06:42build I love that taxonomy and it's
06:44funny because what you described as the
06:46end of theory which by the way was a
06:48cover Chris Anderson wrote for Wired
06:50making this argument that in the age of
06:52you don't need theory because you have
06:54so much data you can essentially mine it
06:56to learn what you don't know and yet you
06:58have this chicken egg problem that
07:00you're describing where the ideal case
07:01for the companies you might work with is
07:03that they have a goal or something
07:05they're trying to do so how do you see
07:07people actually navigate so I think a
07:08lot of times in machine learning and
07:10artificial intelligence you can kind of
07:11break it into two camps there's the
07:13completely like supervised learning
07:14algorithm where you know what you're
07:16going for and you just want an algorithm
07:17that can do that better than anything
07:18and those generally have big data sets
07:20they're discovering themselves versus
07:22and unsupervised which is the contrast
07:24exactly so the idea is I have a bunch of
07:28fraud data and I just want to minimize
07:29fraud and so I can come up with some
07:31sort of metric that I care about this is
07:34correlated with business value and I
07:35just want to maximize that metric then
07:37you have unsupervised learning
07:38algorithms and this can fall into things
07:40around like anomaly detection or it's
07:42just like here's a big soup or lake of
07:44data show me interesting things well
07:46they're they're contexts that I was
07:48thinking of when I think of unsupervised
07:49I also think of cases like the recent
07:51news about alpha zero and the algorithm
07:53kind of learning on its own from no data
07:55that was a reinforcement learning case
07:57is that technically unsupervised well so
07:59then I guess you could say there's a
08:05reinforcement learning yeah and I mean
08:07it may have started with no data but
08:09it's generating it as it goes along
08:10she's kind of going cumulative so that's
08:13a third category and it would one-shot
08:14cuz remember there's a big move for
08:15awhile with like one-shot learning type
08:17of algorithms would that also go in that
08:18category what I'm trying to get at is a
08:20difference between small data and big
08:22data basically and where those fit in
08:23your taxonomy yeah so the nice thing
08:25about something like unsupervised
08:26learning and the nice thing in general
08:27with all of these is you can kind of
08:29composite them together depending on
08:30what you're doing so for something like
08:32a natural language processing task if
08:34you maybe have some end goal like you
08:37want to have some sort of conversational
08:38AI system or you want to be able to do
08:40question-and-answer over some blank
08:41large corpus you're very carefully
08:43avoiding the word bot which I love last
08:49year but basically what you can do there
08:51is take an unsupervised learning
08:53to kind of learn the features of the
08:56language itself and then apply a
08:58supervised learning algorithm on top of
09:00that feature representation that you've
09:02learned and so this is the nice thing
09:04about like you don't have to have the
09:05Machine necessarily know English to
09:07start because you feed it in all of
09:09these different examples and it learns
09:11ways to represent that which then can be
09:14aimed at a specific goal like answering
09:16these questions traditionally those
09:17systems are thought of as independent
09:19though and they fall into these kind of
09:20three categories and machine learning
09:22but what's really interesting when we're
09:24seeing a lot of customers do is start to
09:26treat that like an entire pipeline this
09:28does make it a lot more complicated it
09:30makes it more computationally intensive
09:32so leveraging some of the new
09:33technologies is incredibly important for
09:35doing that but it also makes it a harder
09:37optimization problem as well because
09:39you've taken a system with a lot of
09:41knobs and levers that's normally been
09:43optimized be a trial and error and now
09:44you're making it twice as large you're
09:46making it combin entirely more complex
09:47yeah that complexity grows exponentially
09:49and so some of the standard techniques
09:51that people do like trying to solve this
09:53this tuning problem in their head or via
09:55brute force just completely fall flat
09:57Scott do you think AI is the end of
09:59theory like do we have to like stop
10:00knowing the way we think about it is one
10:05of these aspects of machine learning is
10:06unsupervised learning you just throw a
10:09bunch of data at the problem and try to
10:10have the data learn on its own you're
10:13not aiming for a specific goal that's
10:15the end of the review yeah this goes in
10:17with it and of theory in the sense that
10:18you're not asking a question you're not
10:20shooting for a specific objective you're
10:22just trying to create these patterns and
10:24formulate some sort of maybe it's
10:27anomaly detection or you're just trying
10:28to look for something interesting yeah I
10:31know I'm doing exactly and it's a
10:33clustering algorithms it's all these
10:34sorts of things that are incredibly
10:36useful and now we have large enough
10:39datasets that it's becoming incredibly
10:40necessary tool because you can't
10:43necessarily ask every single question
10:45and hope to get an answer or you don't
10:47want to do like what some of the like
10:49psychology research is suffering with
10:50right now is this like p-value hacking
10:52it of like if you ask enough questions
10:53than just like statistically you might
10:56get a spurious answer it's actually a
10:58false positive context exactly and so
11:01the idea is if I can just give it a
11:03bunch of data and then it comes up with
11:06for me or I can go back and say oh this
11:08is really interesting this actually I
11:11know how I'm gonna leverage this to then
11:13ask this question that I would have
11:14never asked before I think that's
11:16becoming extremely powerful you'd
11:17painted a three level taxonomy of
11:19supervised unsupervised and
11:22reinforcement learning so where are you
11:23then on the end of theory I think
11:25there's gonna be need for all of it to
11:26be honest when it comes down to solving
11:28a very specific business problem like
11:30fraud detection you don't want the
11:32algorithm to learn on its own just let a
11:35lot of fraud through as you slowly come
11:36up with many of what the world looks
11:38like you want to solve this very
11:39specific supervised learning algorithm
11:41or when you're training a car or
11:43something like that how to drive a car
11:44you don't necessarily want it just to
11:46like go get into a million accidents as
11:48it slowly learns like what does steering
11:49even mean it needs to be somewhat more
11:52directed that being said in the security
11:54space if it's more like anomaly
11:56detection that might be something
11:58different because you don't necessarily
11:59know what all different breaches could
12:02look like and so you need to kind of do
12:04this more unsupervised like clustering
12:06based approach I do want to quickly ask
12:08you to define what is optimization
12:09because when I hear that word I think of
12:11like the McKinsey word like optimization
12:13of the workforce I know you mean it in
12:14the context of algorithmic optimization
12:17but could you break it down friend yeah
12:18we think of optimization for the more
12:20mathematical perspective
12:21so there's inputs to some system and one
12:24or more outputs that you want to
12:25maximize so you have an AI fraud
12:28detection system and there's lots of
12:29configuration parameters that make that
12:31actually work yeah and so we think of
12:34optimization in the sense of how do you
12:35set all those configuration parameters
12:37those hyper parameters and architectural
12:39parameters of a deep learning system in
12:41order to get the most accurate fraud
12:43detection and a lot of that by
12:45definition has always been trial and
12:46error yeah and so typically that's how
12:48people solve this problem
12:50this is why Google will pay like a
12:52million dollars for someone with ten
12:53years of deep learning experiences is
12:55that intuition that's built up but just
12:57like how unique datasets are helpful for
13:00solving unique problems unique
13:01algorithms and kind of unique
13:03configurations can get you quite a bit
13:05better than the one-size-fits-all
13:06approach a lot of times that hard-won
13:09intuition doesn't actually transfer to a
13:11completely new type of problem you could
13:13know everything in the world about a DNN
13:15and then you apply it to
13:17a recurrent neural network or whatever
13:18it may be and it becomes much more
13:20difficult to kind of you have to start
13:23it doesn't say practically speaking very
13:25few startups can afford that type of a
13:2610 million dollar ten ten years of
13:29experience type of hire yeah exactly so
13:32this goes a little bit also to what you
13:33were saying earlier Joe about how
13:35because when you talk about optimization
13:36and the inputs that you were putting in
13:39and then you're tuning your hyper
13:40parameters and getting something out
13:42that's when you were talking earlier
13:43about this fact that all these companies
13:45have these data sets and the biggest
13:46question is how to actually clean and
13:48process their data set because obviously
13:49it's garbage in garbage out oh that's
13:51right you really have to get the data in
13:52shape yeah that is a foundational prob
13:54data is a foundational problem today
13:56it's so easy to go grab a jupiter
13:57notebook but web frontend that you can
13:59execute code in cells so i can basically
14:01have a jupiter notebook full of Python
14:03and heavily annotated most of the
14:05classes these days and deep learning or
14:06using jupiter notebooks and today
14:08there's just so many jupiter notebooks
14:09out there that people have built you
14:11know different solutions or or tutorials
14:13on there's this explosion of tutorials
14:16and code that's out there but these
14:17largely are all toy examples and if you
14:20want to get serious if you actually
14:21wanna optimize these for actual real
14:23world usage there's probably two really
14:25big pieces that someone just can't
14:28automate or really bring kind of a
14:30pre-canned one-size-fits-all solution
14:32we're kind of entering that golden age
14:33of applied applied machine learning
14:35applied AI you know five years ago when
14:37I was hanging out at Berkeley and seeing
14:39the talks from Jitendra Alex Peter veal
14:41and and the folks there and I would see
14:43some of these these breakthroughs in
14:44computer vision and Jitendra has that
14:46that image that he always shows every
14:48year about I think there's a beggar on
14:50the street right he's got a cup hanging
14:51out and there's someone walking by and
14:53it's the quotation is always is this guy
14:55gonna and put money in his cup or not
14:57and the algorithm is supposed to predict
14:58that so there's a really cool things
15:00that were happening 5 or 6 years ago
15:01those are being operationalized now it's
15:03scale so you're basically saying that
15:04we're at a moment because I've actually
15:05heard the opposite I think we're both
15:07right though which is that a lot of the
15:09work and the buzz is all algorithms in
15:11academia and actually translating it
15:13into practice is and it hasn't been
15:15operationalized to you I think is been
15:17operationalized by big companies like
15:19Amazon and Facebook and obviously the
15:21larger deep tech companies I think we're
15:24at the precipice here of having all of
15:26the the kind of foundational pieces
15:28automated to the point where I think
15:31that you want to solve in mind is
15:32obviously important having the data in a
15:34place where it you know can be trained
15:35it's clean it's annotated especially
15:38supervised when I say we're on the
15:39precipice I believe it's the supervised
15:41you know machine learning world is just
15:43exploding with applications obviously
15:45unsupervised deep reinforcement learning
15:46it's still in the bleeding edge when you
15:48talk about having RL applied and salic
15:51autonomous driving having a car driving
15:53around and learned how to drive you know
15:55by crashing a million times isn't
15:56tractable as a you know is an algorithm
15:58so it's you know be able to do all that
16:01simulation in the cloud and then
16:02transfer domains it's still not a solved
16:04problem you said there were two areas at
16:06startups so the parameter problem and
16:08data I think the data engineering aspect
16:10of things is understated for machine
16:12learning AI I see like a lot of our our
16:14partners that have been helping our
16:15customers over the last number of years
16:17there are a lot of big data size and
16:19they've been using Hadoop and spark and
16:21they've been dabbling in machine
16:22learning and advanced analytics over
16:24that time they've actually built up a
16:25good amount of data engineering skills
16:27so I think those are still really
16:28valuable and you think those will
16:30transfer to this type of I think so I
16:32the thing is they're not experts in the
16:33algorithms and an optimization for
16:36example but I also think that deployment
16:38is getting to the point now where it's
16:39almost push-button with a lot of these
16:40API so we're talking about you can
16:42deploy to an endpoint and do new
16:43predictions you don't on any mobile
16:45device it can kind of just bolt on top
16:47and be this value add as opposed to
16:50something that's rip and replace I'm
16:51completely agnostic that a framework
16:53you're using the infrastructure and the
16:57objective that you're shooting for when
16:59it comes to AI being practical I think
17:00the two bookends are the following and
17:02the most commonly thought of one bookend
17:04is it's still academic it's not useful
17:06it's not applicable and then the other
17:07book in is a is magic and like the a is
17:10magic book and basically says you know
17:12it's about data and these magical
17:13algorithms so whoever has the data and
17:17the algorithm wins and then like
17:18basically everything's automated and the
17:20reality of practicality is in the middle
17:22which is like yeah it's not it's not you
17:24can just like throw data at the problem
17:26and throw one algorithms problem and
17:28like QED you're done you've got a
17:29company you have to have the right data
17:31it's very domain-specific you have to
17:33have the right algorithm and
17:34optimizations like every use case of AI
17:37requires specific tweaking in a massive
17:40massive problem space to get a useful
17:42solution and so it's
17:44magic it is absolutely practical so it's
17:46not you know academic but it requires
17:48you know at some level I feel like the
17:50complexity has moved not necessarily
17:52disappeared right it's like you've moved
17:54complexity from Upland writing code to
17:56now it's basically an optimization
17:57problem in a data problem how much is
17:59like the tweaking and the data unique
18:02per problem is it per vertical is it per
18:04like how do you even think about that
18:06there's so much work involved with
18:08actually getting the data ready pointing
18:10at the right direction and things like
18:11that and over the last two years we've
18:13actually seen this huge transformation
18:15where it used to be this toy problem
18:18where it was like we want to do deep
18:19learning and whatever that means we just
18:21want to do something and optimization
18:23actually isn't super important there
18:24because it's just like can I stand
18:25something up it was a coding problem
18:27before but once you actually start to
18:29apply it to something then it's how do I
18:31extract as much value as possible out of
18:33this how do i scale this as quickly as
18:35possible how do I deploy this and have
18:37it be a reliable system and so some of
18:39these bottlenecks that were historically
18:41in making practical a I have started to
18:44shift into these more deployment
18:46optimization I it's actually really good
18:48to hear the rings is not just all hype
18:50and it's all like quite here yet we're
18:52actually a very exciting middle point of
18:54the Fortune 100 two years ago had fun
18:57playing to kind of accelerate the R&D
18:59phase of certain things but now those
19:01same systems are in production and every
19:04little piece of optimization matters and
19:06for every single problem it's a
19:09completely different type of
19:10optimization you can take a problem
19:13that's really good at classifying Street
19:16like the Google Street View data set
19:18where it's like pictures of houses and
19:19you want to be able to read the address
19:21off of it and that's a kind of classic
19:23computer science computer vision problem
19:25and then you want to do a different type
19:27of classification you might need to have
19:29a completely different architecture for
19:31your neural net so you're saying just a
19:33pause on that for a minute that while
19:34some of the skills may transfer and the
19:37mindsets may transfer and even some of
19:39the way you might think of the models
19:40may transfer the optimization tricks you
19:43use our custom and special to each of
19:45these cases well yeah the intuition for
19:47how to configure these systems does not
19:49transfer which is why you need to retune
19:52reoptimize and reconfigure these systems
19:56to make sure they're maximizing that
19:57a little bit more about why it doesn't
19:59transfer yeah so I mean back to a
20:01Martines point like to a certain aspect
20:03some of these deep learning systems are
20:05kind of magical in the way that they
20:07work they're very difficult to explain
20:09what's actually happening under the hood
20:11a black box problem exactly but the the
20:13problem with the black box is if you
20:15have a black box with 20 different knobs
20:17and levers in it and you're trying to
20:18get some result out of the end when you
20:20completely change what's being fed into
20:23that black box if a completely different
20:24data set or you're shooting for a
20:25completely different target like all of
20:28that intuition on how you set those
20:30knobs and levers is now completely
20:32worthless and so you need a new way to
20:33very efficiently and automatically set
20:36that for that new problem and that's
20:37true every time the data set changes
20:39every time you add a new data set maybe
20:42you have an unsupervised learning
20:43algorithm learning feature
20:44representation every time you add a new
20:46feature pointed at a new problem and one
20:49thing that we found is that like an
20:51untuned sophisticated system will
20:54underperform a tuned simple system you
20:58can take a simple machine learning
20:59algorithm like a random forest and it's
21:01always going to give you like a b-
21:03answer you can take a sophisticated deep
21:05learning algorithm and if you don't tune
21:07it properly it's going to give you a
21:08terrible random answer but if you tuned
21:10it properly and train it properly it's
21:12gonna beat a human in a practical
21:14podcast can I indulge a philosophical
21:16question is actually pretty lay about
21:20the technology behind this so I'm gonna
21:21start with what seems to be a probably
21:23an entirely different metaphor but
21:24imagine like you're cleaning your house
21:26and you're trying to get rid of dust
21:28right so to get rid of the dust one
21:30thing you do is you can open the door
21:31and you can sweep the dust out of the
21:32house and the dusk on another thing you
21:33can do is you can basically just move
21:34dust around and you're like oh it looks
21:36better under the bed I sometimes think
21:37about complexity this way which is like
21:39are we just moving complexity from like
21:41basically writing coding algorithms to
21:44manipulating data and optimization but
21:48there's still the same amount of
21:49complexity or have we reduced complexity
21:51with AI do you see what I'm saying like
21:53I was getting just out of the house are
21:56we just moving it around into a separate
21:57problem domain I love that question
22:00I'd say socially a combination of the
22:01two because it practically when you are
22:03removing the dust from your house you do
22:05get it into these little piles and then
22:07you move it out of the house some of
22:10these more sophisticated
22:11algorithms are making it better because
22:13the distributed dust problem is much
22:15harder than cleaning up a pile of
22:17beautiful so so basically you're
22:19creating like the mounds of dust and
22:20then you can kind of focus on getting
22:22the dust of it exactly so the two-step
22:24process there is a reduction in
22:26complexity to get to the goal it's not
22:27like you're just moving around the dust
22:28in the house what you're gonna see is a
22:30lot of the complexity we talked about
22:31get automated I mean you'll see us to
22:33try and drive those piles out of the
22:35door as well automated dust collection
22:38and one of my all-time favorite books is
22:42Philip Pullman's His Dark Materials
22:43trilogy and I always think of dust in
22:46that context so this is like only dust
22:49beautiful okay so I have a question for
22:51you guys then especially Joe given that
22:53you work at AWS and Scott from your
22:55vantage point is this gonna come about
22:57because one thing I always hear about is
22:59debates about ml as a service and AI as
23:01a service and whether that's gonna be
23:03the way that these services are gonna be
23:05delivered and there seems to be a lot of
23:06hype around that in and of itself I'd
23:08love to hear you guys have thoughts on
23:09that I mean I guess you know the the
23:11diplomatic answer is we have to to
23:13support them all I mean we see you know
23:15layers of abstraction that are valuable
23:18to two different types of users
23:19researchers aren't gonna use an API
23:21obviously because it's you know they
23:24can't do anything with it there's no
23:25knobs there's nothing to break and do
23:26anything with data scientists are not
23:28experts so they need some level of
23:30automation or or helping a helping hand
23:33on things like you know optimization
23:34give me the end of the metagame I'm
23:36diplomatic answer is then there's a
23:37whole bunch of guys who don't know what
23:38they're doing and they need some type of
23:40API but it needs to be flexible enough
23:42where they could start to bring their
23:43own data in because even today you know
23:45just being the self deprecating amazon
23:49guy we have services that aren't they're
23:51not very flexible like our image
23:53recognition service is really cool and
23:54it does a lot of great things but I
23:56can't actually bring my own data to it
23:57and I can't actually optimize and
23:59customize for my problem well in your
24:01defense I will say that when you are
24:02delivering a service to people there are
24:04expectations and consistencies and
24:06things you have to do to scale but
24:07clearly all the experiments that people
24:09have to make you can't actually have
24:11this one size fits all when you build a
24:12service like that you build it for the
24:13lowest common denominator and it's being
24:15used by c-span it's being used by you
24:17know travel sites when it comes down to
24:19if I wanted to apply this for biometric
24:22security in my corporation I want to
24:24a whole bunch of data the fingerprint
24:27daters pictures of sub Scott it's not
24:37good enough to probably be visualizing
24:38your data and looking at all the pretty
24:40pictures anymore so business
24:41intelligence is kind of moving into data
24:43science you need insights that's like a
24:44big theme I don't want to say bi is dead
24:46but it's you know it's on its last leg I
24:48think everyone wants to move to two more
24:49predictions actionable prescriptive
24:52analytics and you can't do that when
24:54you're just kind of looking at pretty
24:55pictures so I think we're seeing just
24:57this mass transition from bi you know
24:59analysts over to data scientists yeah
25:01they don't know a whole lot about
25:02machine learning I think they're gonna
25:04get to the point where they can push a
25:05button and they can use an extra boost
25:06algorithm and then optimize my boom I'm
25:09getting a great result I know the domain
25:11that I'm in I know the data I know the
25:13problem I'm trying to solve whether it's
25:15an extra boost or a deep neural net I
25:16really don't care as long as my end
25:17predictions are are accurate you're
25:19getting the answers you need what's your
25:22view on the AI is a service debate I
25:23actually think that there's room for
25:25both so there's kind of machine learning
25:27as a service that this kind of generic
25:28one size fits all and then there's these
25:30more specialized tools and the way that
25:32we like to think about it is like in the
25:34early 90s when the web was coming online
25:36and everything like that if you didn't
25:38have a website like a one size fits all
25:41just like get me an e-commerce shop
25:43online or something like that that zero
25:46to one was very transformative for
25:47people but I would still argue it's
25:49still true for small and medium-sized
25:50businesses that are like on Shopify and
25:52yeah so we think it Shopify is more of
25:54the like machine learning as a service
25:55option where it's just like I need
25:57something and I don't really know what
25:58I'm doing but I just need something and
26:00that zero to one can be incredibly
26:01powerful yeah that being says as
26:03businesses start to differentiate
26:05themselves on their AI strategy as they
26:08start to hire data scientists and doing
26:10this bespoke knowledge and these custom
26:12data sets and things like that now you
26:14don't want the one-size-fits-all
26:15solution you want to build a company
26:16like Amazon where you can really
26:18optimize every aspect of that website to
26:21really make the most out of it takes
26:23them from that one to two but I still
26:25think there's this need especially in
26:26the immediate term to help people go
26:28from zero to one one thing which is
26:30different than IT industry in the past
26:34is it seems pretty clear because the
26:36value is in data and because
26:38value is an optimization that the
26:40infrastructure layer will be free or
26:43maybe not free but lows come to matter
26:44what I mean by that is in the past if I
26:46were gonna give you a computing
26:48infrastructure I would charge you for
26:50that computing infrastructure and like
26:51the tooling and everything else around
26:53that but it seems like the tooling is
26:55something that people are happy to build
26:58and offer for free attending layer of
27:00abstraction whether it's a service or
27:02not and so I think from an industry
27:04perspective that's a very different
27:05horizontal ization than we've seen in
27:07the past so what does it mean when you
27:08have horizontal versus vertical AI
27:11layered so in the past like a lot of
27:12times tooling was something you could
27:13monetize like purify was a billion
27:15dollar company that basically sold the
27:17debugger right all the tooling and all
27:18the infrastructure in order to build
27:19application was very much monetizable
27:21for AI because there's so much value in
27:24optimization there's so much value in
27:25data it's almost like this tooling
27:27infrastructure layer is something that
27:28you know is being offered or given or
27:31you know many players are just offering
27:33for free and there now if there is
27:34libraries and things on top of services
27:36and then the actual value is the
27:38vertical application of those two
27:40whatever so for example if I look across
27:42a I startups the ones that tend to be
27:45getting the most traction have taken AI
27:47and applied it to a vertical problem
27:48they have access to a proprietary data
27:49set or they've done a specific sort of
27:51optimization and now there's a vertical
27:53focus towards something as opposed to
27:55I've got this very horizontal kind of
27:57generic AI later I think that's the game
27:59of the big players like the Amazons or
28:01the googles and so I think from an
28:03industry-wide in the startup perspective
28:05I really think vertical focus is how
28:07we're gonna see the gains of the
28:09enterprise as opposed to what we've seen
28:10in the past in computer science which is
28:13the more horizontal I completely agree
28:14you know over the last three or four
28:15years I've seen a number of startups
28:16come to me whether I was at Intel and
28:19and working with Intel capital or not
28:20with Amazon and they would say hey I can
28:22scale up training of deep neural nets so
28:25much better than everyone else and I
28:26provide their algorithms you know come
28:28work with us or come by us or whatever
28:29they were looking for and I think no
28:32longer your point about vertical ization
28:33is absolutely valid and I see the ones
28:36that are successful are the ones that
28:37have a really nice mixture of kind of AI
28:40research the ones that that have kind of
28:42one foot in research on the algorithms
28:44have really deep expertise but they also
28:47are mixed with true domain experts in
28:50the vertical that they're trying to work
28:52so for example if it's a medical imaging
28:53startup if you don't have a hospital
28:56you're working with to provide you data
28:58if you don't have doctors if you don't
28:59have clinicians that you're working with
29:01or have on staff frankly I don't see a
29:04whole lot of legitimacy to what you're
29:05doing I've seen startups that overfit to
29:07a public data set when I was at in town
29:09they say this is fantastic look at us
29:10we're getting 99% accuracy on this
29:12managing debt it said we're worth you
29:14know $190 come by our invest in us if
29:19that's me wasn't a value prop so I
29:21completely see the mixture of domain
29:23expertise in a vertical along with the
29:25AI expertise and illustrative compared
29:29back to how it used to be so I listen I
29:30used to have a friend I went to college
29:32with he built applications if you'd go
29:34after different verticals but he was
29:35like a domain in specific application
29:37developers he built like booking systems
29:39for like kennels and veterinarians and
29:42then lodging systems he didn't have to
29:43know he'd go and talk I don't see what
29:44they wanted yeah they need build an
29:46application so like you know like he
29:47could build basically a horizontal
29:49company that was selling to these
29:50different verticals but if you look at
29:51AI like like in order to add something
29:53of value to these domains you have to
29:55understand the data you have to
29:56understand the use and you have to
29:58understand the optimization is almost
29:59like these companies are becoming these
30:00very vertically focused companies are
30:02the ones that are successful whereas
30:04like you know kind of IT folks were
30:05normally used to thinking of these as
30:06horizontally we've made the argument in
30:08our own podcast on weird the machine
30:10learning edge for startups will be and
30:12how they can compete with the googles
30:14and the other folks would have these
30:15huge in-house teams and it is
30:16essentially along these lines of this
30:18vertical part but I love it also in the
30:19big picture of that this idea of
30:21augmentation and having all these tools
30:23give you the superpowers but you still
30:24have this human skill in how's that
30:26that's the domain expert that's right
30:27you're talking about brutalization is
30:29going to be a huge part of that and
30:30going after you have very specific
30:32problems with very specific datasets but
30:34specialization along that horizontal
30:36layer and I think is going to be key
30:38like I don't think there's going to be a
30:39one-size-fits-all everything but if
30:42there's specific parts of that journey
30:45to getting to practical productionize
30:49day.i system that humans are bad at or
30:52that can be automated if you are laser
30:55focused on that specific specialization
30:57like optimization data collection would
30:59be another data version would be another
31:01like like general things that are
31:02scaffolding exactly to
31:05do the AI problem scaffoldings are the
31:07perfect way to put it but then every
31:09building that the scaffolding is wrapped
31:11around is unique isn't it exactly
31:13do people know the difference between if
31:15it's a data problem or an AI problem and
31:17how do they know because if you have
31:18like people who have legacy skills who
31:20are coming up to speed like is it
31:22obvious I think the data problem is the
31:23first like layer and maslov's hierarchy
31:26of AI like you need to actually have the
31:28data then you need to be able to
31:31understand the business context of what
31:33you're aiming for and do a lot of the
31:34data engineering to make sure that you
31:36can actually leverage that data in some
31:38way that it's not just in filing
31:39cabinets somewhere yeah
31:41then there's the tooling and the
31:43infrastructure for actually like
31:44training some of these more
31:45sophisticated algorithms and then
31:47actually optimization just sits at the
31:48very top that's for it to be amazing
31:49when I think of the Maslow's hierarchy
31:51of needs in the psychological sense it's
31:53really about like the basics and the
31:55survival things and then the
31:56aspirational stuff at the top but
31:58optimization is a means to an end it's
31:59not the end in and of itself but the way
32:01I think about Muslims hierarchy is like
32:02once you get to the top then you kind of
32:04have everything in order and you're like
32:05doing it right and then you can go to
32:07getting bands and the optimization is
32:09about like that last mile of now I'm
32:11applying it to a business problem how do
32:13I make as much money as possible as
32:14efficiently as possible I mean there are
32:16the number of partners that provide
32:18foundational api's for example for a LP
32:20for a natural language processing you
32:22can take your text throw it into their
32:24API get sentiment get named entities you
32:27know get you know parts of speech out
32:29basic things for NLP and and integrate
32:31that into a larger workflow and not have
32:33to go and find a corpus of data annotate
32:35it clean it figure out what algorithm to
32:37use train and figure how to deploy it
32:39all that is basically a restful api away
32:41from most api call from all these that's
32:44a case where you're using the api to
32:45like pull in different data streams but
32:48i also think of companies like ship oh
32:49and they give shipping as a service yeah
32:51yeah essentially through an api and I
32:53think is really interesting because what
32:55I love about this is about
32:56democratization of all these things
32:58across ecosystem because you essentially
33:00have all these superpowers you're
33:01pulling on these api's are giving you
33:02this super power and that super power
33:04you're consuming it that way in order to
33:06do whatever the hell you want as a
33:08company and what you're buying with that
33:09API is really data is the is the data
33:12that they've collected perhaps an
33:13annotated trained on which is you know a
33:16really great thing for most companies
33:19gonna go build a system like and and the
33:20way that we think about this is if
33:22you're in business there's something
33:24that you're good at there's some way
33:25that you differentiate yourselves from
33:27your competitors and you should
33:28outsource everything else focus on what
33:30you're great at and then bolt api's
33:32around them to supercharge it if I'm in
33:35a medical imaging vertical you know
33:37focus startup I want to hire guys that
33:39are focused on that vertical you might
33:40want to hire the radiology I once
33:41exactly the early days of web I remember
33:44there was this like move of talking
33:46about tech innovation as like
33:48combinatorial innovation and I think is
33:50kind of a buzzy word but I actually
33:52think it makes sense in this context one
33:53of my favorite writers is Brian Arthur
33:55who wrote the nature of technology and
33:56how it evolves and he did a lot of
33:58fundamental work in complexity economics
34:00anyway it's really interesting but this
34:02idea that you can take all these
34:04different pieces and kind of we combine
34:05them in ways that are creating entirely
34:07new things that's always our innovation
34:09has happened and now it's happening on a
34:10grander scale with these things which i
34:12think is amazing and beautiful a single
34:14person can do now it would have taken a
34:16team of researchers a decade ago well
34:18thank you guys for joining the a6 and
34:20say bye guys thank you thank you thank
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
