00:00 This is the Internet of Bugs, my
00:01 name is Carl, and that is a lie.
00:05 So this video is in three parts.
00:08 First, we're going to talk about
00:10 We're going to talk about what
00:12 should have been done.
00:14 What Devin actually did and how it
00:17 did it and how well it did it.
00:18 I have been a software professional
00:22 I am not anti-AI, but I really am
00:26 anti-hype and that's why I'm doing
00:30 Devin was intro'd not quite a month
00:32 And it was touted as the world's
00:36 "first AI software engineer."
00:38 And I don't believe that it's the
00:40 first software engineer and I
00:41 already made a video about that.
00:43 I'll put the links in the
00:44 But today is about the specific
00:47 that's the first line of the video
00:49 description, which says "watch Devin
00:51 make money taking on messy Upwork
00:54 That statement is a lie.
00:56 You cannot watch that in the video.
00:59 It does not happen in the video.
01:00 It does not happen.
01:01 What's worse though is that the
01:04 hype and the fear, uncertainty and
01:06 doubt from people repeating and
01:08 embellishing on that claim because
01:09 they're trying to get clicks or
01:10 they're trying to go viral or they
01:12 just want to be part of the zeitgeist.
01:13 The hype around Devin in general is
01:16 And that statement seems to be what
01:18 a lot of it is is pinned on. For the
01:19 record, personally,
01:20 I think generative AI is cool.
01:22 I use GitHub co-pilot on a regular
01:25 I use ChatGPT, llama 2, Stable
01:28 All that kind of stuff is cool, but
01:30 lying about what these tools can do
01:32 does everyone a disservice.
01:34 So Devin does some impressive
01:36 And I wish the company had just
01:38 been truthful and just taken the
01:41 win, but they didn't.
01:43 And they had to pretend that it did
01:45 a lot more than it actually did.
01:47 Now, I don't want to take anything
01:49 away from the engineers that
01:51 actually built Devin.
01:52 I think Devin is impressive in many
01:53 ways and I'm especially not trying
01:55 to pick on the guy that's in the
01:58 The lies are not in the video
02:00 They're in the description and they're
02:02 in the tweets that the company made
02:04 And then they're in a lot of places
02:06 and people that have repeated that
02:08 lie over and over again.
02:09 It shouldn't be okay.
02:11 Companies should just not be
02:12 allowed to lie without getting
02:14 And people shouldn't repeat things
02:17 they heard on the Internet without
02:19 checking for themselves.
02:20 I realize that's tilting at windmills,
02:23 but I'm going to die on that hill.
02:25 Since nobody else that I've seen
02:28 seems to be explaining why this is
02:31 I guess if it's going to get done,
02:34 I'm going to have to do it.
02:36 So here I go. Before you think this
02:39 understand this kind of lie does
02:41 You're watching this.
02:43 You're probably at least somewhat
02:45 Keep in mind that there are a lot
02:46 of people out there that see
02:48 headlines don't read the articles
02:50 that are not technical.
02:52 And what these lies do is they
02:54 cause non-technical people to
02:57 believe that AI is far more capable
02:59 than it is at the moment.
03:00 And that causes all kinds of
03:03 People end up being a lot less
03:05 skeptical of AI than they should be.
03:08 They're a lot less skeptical of the
03:10 output of AI than they really
03:12 And taking AI at face value these
03:14 days is getting a lot of people in
03:16 Just Google "AI lawyer fake cases" or
03:19 "AI fake scientific papers."
03:22 And those are just the prominent ones.
03:23 And this hurts real software
03:25 professionals too, because there are
03:27 going to be folks that are going to
03:29 trust the code that AIs generate.
03:30 And that just means more bugs on
03:32 the Internet and there are already
03:34 way too many already.
03:35 It's already a mess.
03:37 They're already too many exploits.
03:38 They're already too many hacks.
03:40 And the more bad code that gets out
03:42 there, the worse the ecosystem
03:44 becomes for everyone.
03:45 Enough of that. On to section two.
03:47 What was the job that Devin was
03:48 supposed to have done?
03:49 So this is the beginning of the
03:51 video or early in the video.
03:53 Note that in the bottom left hand
03:54 corner of your screen, I have stuck
03:57 the time code of every frame that I'm
03:59 going to be breaking down for you.
04:01 So this is 2.936 seconds into the
04:06 So you can go look yourself if you're
04:08 curious about any particular thing
04:09 or want to know the context around
04:10 something that I'm talking about.
04:12 This is the job that Devin
04:13 supposedly did on Upwork.
04:15 We'll talk about it in a minute.
04:17 First off, look at the left of your
04:19 Notice that they searched for this.
04:21 So this is not some random job.
04:23 This is not "Devin can do any job on
04:26 They cherry picked this.
04:27 That isn't deceptive necessarily.
04:31 You would kind of expect them to.
04:33 But keep in mind that what that
04:34 means is chances are Devin is
04:37 actually worse at most jobs than
04:39 Devin turned out to be on this one,
04:41 which wasn't great.
04:44 So zooming into that particular
04:47 There at the bottom, that's what
04:52 the customer actually wanted.
04:55 "I want to make inferences with this
04:59 "Your deliverable is detailed
05:01 I'm not going to talk about the
05:02 estimate to complete the job thing.
05:04 Devin didn't do that.
05:07 I'm not worried about that.
05:10 This is what Devin was actually
05:14 This is what was copied and pasted
05:15 "I'm looking to make inferences with
05:17 this model in the repository.
05:19 Here's the repository.
05:20 Please figure it out."
05:21 Okay, back to the job.
05:24 "Your deliverable will be detailed
05:26 instructions on how to do it in EC2
05:29 "Please figure it out" is not the
05:31 same as "detailed instructions on how
05:33 to do it in an EC2 instance in AWS."
05:35 For the record, this at the end of
05:38 the video is the report that Devin
05:40 There is nothing in that at all
05:42 about what the customer was
05:44 actually asking for.
05:45 So what should the results of this
05:47 job actually look like?
05:50 To start with, this is what you
05:53 really need to know in order to be
05:55 able to figure out how to do this.
05:57 You're going to have to have some
05:58 kind of instance in the cloud.
05:59 You need to figure out what size,
06:00 type, how much memory, all that
06:02 You need to find out from the
06:04 Would you rather have one that runs
06:06 faster and is more expensive?
06:08 Or would you rather one that's
06:09 cheaper that runs slower?
06:10 Is this going to be something that's
06:11 always going to be up and you can
06:13 just throw stuff at it whenever and
06:14 have it give you an answer?
06:15 Or are you going to launch it, run
06:16 it and then turn it off to make to
06:18 How are you going to get the stuff
06:20 you want to make inferences on?
06:22 How are you going to take the
06:23 images that you want to analyze?
06:25 How are you going to get that onto
06:27 You want to do a web interface for
06:29 You can put them in S3 bucket.
06:31 You know, how are you going to get
06:32 access to the output of that?
06:33 These are all questions that you
06:34 need to know, right?
06:35 This is going back to another video
06:38 that I made, the part of the job of
06:40 a software developer that the AIs
06:44 The hard part, the important part,
06:46 the difficult part, the time
06:48 consuming part of being a software
06:49 engineer is communication with the
06:52 customer, with your boss, with the
06:54 Figuring out what actually needs to
06:56 get done, going back and forth,
06:57 saying, "okay, this would be a lot
07:00 How about we do that?"
07:01 Those are the kinds of things that
07:02 AI just isn't capable of doing, and
07:03 those are some of the most
07:04 important things that we do.
07:06 This just starts right off as AI
07:08 doing the wrong thing.
07:09 Unfortunately, this is Upwork.
07:11 So just for those of you that
07:12 actually are ever going to be in
07:14 this situation, Requests For
07:16 Proposals like this are are bad.
07:19 If you can avoid doing them, avoid
07:21 it. Competent Request For Proposals
07:22 process is going to have a Q&A
07:25 So they tell you "This is what we
07:26 You send them questions other
07:27 vendors send them questions.
07:29 They answer all the questions, they
07:30 send out the answers to everybody,
07:31 and then the bidding happens.
07:32 Since we can't do that in Upwork
07:34 because it's not set up that way,
07:35 the next best thing, which isn't
07:36 actually a good thing, but the next
07:38 best thing is you write down your
07:39 You pick the answer that will cause
07:42 the cheapest amount of work, right?
07:44 The least amount of work for you.
07:46 Then at the top of your proposal,
07:47 you say, "okay, here are all the
07:48 assumptions I'm making.
07:49 If any of these assumptions turn
07:51 out not to be true, that's negotiable,
07:53 but it means that the cost is going
07:54 to go up." Because you want to bid as
07:56 low as you can, but you want to
07:57 make sure that the customer
07:59 that you're bidding that value with
08:02 And if any of those assumptions,
08:03 they want it done differently, they're
08:05 going to have to pay more.
08:05 It's not a good bidding process,
08:06 but if you're going to have to do
08:07 that kind of bidding process, that's
08:09 So, a deliverable for this
08:11 particular job should contain what
08:13 kind of cloud instance type to use,
08:15 what kind of operating system and
08:18 How do you set up the install
08:20 So CUDA, Apex, PyTorch, don't worry
08:22 about if you don't know what any of
08:23 It's not really important for this
08:26 How to install that repo, so that's
08:28 a four year old repo.
08:30 You're either going to need to
08:31 update that repo for modern Python
08:33 and modern libraries, or you're
08:35 going to have to explain how to
08:37 install a four year old or an older
08:39 One of those two things is going to
08:41 You're going to have to explain to
08:42 the customer how the data should be
08:43 got onto the instance, how they're
08:45 going to get their output off the
08:46 instance, all that kind of stuff.
08:47 I actually reproduced what Devin
08:50 We'll talk more about that later.
08:51 This is the actual instance size
08:56 I used a company called Vultr
08:57 instead of AWS because AWS's
08:59 interface is a mess and it wouldn't
09:02 And on top of that, by the time
09:03 this video got edited and uploaded,
09:05 probably the new version of
09:07 something would have been released
09:09 and I would have the numbers wrong.
09:10 So this is just it's a lot more
09:12 It's easier for this job for the
09:15 I would have actually done it on
09:16 There's no... we have no idea what
09:18 kind of image Devin used.
09:19 They didn't tell us anything about
09:22 If you are a masochist, there is a
09:24 link for the whole and I'll put it
09:26 now in the description for the
09:27 whole uncut version of me spending
09:30 35 minutes and 55 seconds or
09:32 however long it took actually
09:34 reproducing what Devin ended up
09:37 So if you have no life, you're
09:40 welcome to watch that.
09:41 I think transparency is important.
09:44 It's really boring to watch, but it's
09:46 important and I wish that the
09:47 company that made Devin and anybody
09:49 else that's making these kinds of
09:50 claims on the Internet would
09:52 "Here's the raw footage of what
09:54 actually happened" so that we can
09:55 verify their claims if we need to.
09:57 All right, so on the next section
09:59 given that we know that Devin didn't
10:02 do what the customer asked and
10:05 Devin's report did not have any of
10:06 the stuff that the customer wanted
10:08 and that Devin didn't actually get
10:09 paid for any of this.
10:11 What did Devin actually do if it
10:13 didn't make money, what did it make
10:15 and how good a job of that did it
10:17 So here's a screenshot from the
10:20 This is the repo in question.
10:22 We'll come back to screens like
10:25 This is the first thing that Devin
10:30 So there's a thing called a
10:31 requirements.txt file.
10:32 It determines what version of
10:34 dependent libraries your code is
10:37 And it had to change some things
10:39 because the the libraries that this
10:41 repo originally used from four
10:44 years ago, some of them aren't
10:45 downloadable anymore because they're
10:48 So something had to change.
10:50 Here it says that Devin is actually
10:54 I guess that's kind of arguably
10:58 I would say it's more a
10:59 configuration file than changing
11:01 the code, but I'll allow it.
11:04 It is really cool that Devin can do
11:06 this if what the tool did was just
11:08 change all of the requirements so
11:11 that would be something that would
11:12 So that would be a cool thing to do.
11:14 So it's good that you can do this.
11:17 I don't know that I'd call it code,
11:18 but it's a very, very small part of
11:20 what actually needs to get done
11:21 instead of what the customer asked
11:23 for, which is basically "I want to
11:24 be able to make my own inferences.""
11:26 Devin was told just using the
11:29 sample data is fine.
11:30 So that's what I did on my reproducing
11:34 Normally it should be more
11:35 complicated than that, but that's
11:37 what we're going to show that Devin
11:40 Okay, so Devin is fairly early on
11:45 I did not hit this error and you'll
11:46 see why in a second.
11:48 So zooming in, here's this command
11:52 So here at the top.
11:54 We have this error with
12:03 "file not found no such file
12:05 So this error is in a code file
12:08 called "visualize_detections.py"
12:10 and the reason that I didn't run
12:11 into this problem is because there
12:13 is no file called visualize_detections.py
12:17 in that repository.
12:18 I don't know where that file came
12:21 from, but more about that in a sec.
12:23 So back to that command line.
12:25 If you zoom in on the other part of
12:26 that window, you see this.
12:28 So Devin is echoing a bunch of
12:30 stuff into a file called inspect
12:33 and then it's running Python on it
12:34 and it's getting a syntax error.
12:36 You can't put backslash 'n' in a
12:40 It doesn't work that way.
12:42 Echo doesn't work that way.
12:43 None of this works that way.
12:45 This is just this is just
12:47 This is the kind of thing that you
12:48 might do as a human because you're
12:51 not paying attention.
12:51 And then you go, oh, yeah, I need
12:53 to change the way I did that.
12:55 But what seems to be happening is
12:58 Devin is creating files that have
13:02 and then it's fixing the errors.
13:03 So here the video says that Devin
13:05 is actually "doing print line
13:06 debugging"" and that's cool.
13:08 That's something a lot of us do.
13:09 You know, there are always times
13:10 that printf debugging or print
13:11 line debugging ends up being useful.
13:13 So it's cool that Devin can do that
13:15 in at least some circumstances.
13:17 But here's another error I didn't
13:18 see and Devin is coming in trying
13:20 to figure this out.
13:21 The commentary here says "Devin is
13:23 adding statements to track down
13:24 these data flows until Devin
13:26 Now, I'm okay with that.
13:28 I don't know if the word
13:29 "understands" there is technically
13:31 I don't know that Devin actually
13:32 "understands" anything.
13:33 I would doubt it, but we anthropomorphize
13:36 stuff like that all the time and it's
13:38 a handy way of using language.
13:39 So I'm not going to give them a
13:41 hard time for that.
13:42 But that said, let's look at what
13:43 Devin's actually doing here.
13:45 So zooming in on this, we've got
13:48 this weird loop that it's doing.
13:51 It's going through this file and
13:52 reading stuff into a buffer.
13:53 So this is the update_image_ids.py
13:56 And again, this file does not exist
14:00 anywhere in the repository that the
14:02 customer wanted us to use.
14:03 In fact, I searched all of GitHub
14:05 and there are only two places that
14:08 a file that this name exists at all.
14:09 The reason there are three on the
14:10 screen there is because one of them
14:11 is a fork of the other.
14:12 And none of them look anything like
14:14 the one that Devin is using.
14:15 So I don't know where this came
14:17 We don't have any idea.
14:19 But the problem is Devin is here
14:24 debugging a file and that file it
14:27 created and it's not in the repo at
14:30 This is pretty insidious.
14:32 So this gives the person who's
14:33 viewing the video who's not paying
14:35 that much attention who didn't have
14:36 time or take the effort to look at
14:38 It gives that viewer the impression
14:41 that Devin is finding errors in the
14:43 repository that the Upwork user
14:46 asked us to look at.
14:49 And fixing the errors in the
14:51 That's not the case.
14:54 Devin is generating its own errors
14:56 and then debugging and fixing the
14:59 errors that it made itself.
15:01 That's not what it seems like Devin
15:05 It's not what Devin is implied to
15:07 It's not what many people who have
15:10 written articles and posted videos
15:14 about Devin have thought Devin was
15:18 But in fact, Devin isn't fixing
15:20 code that it found on the Internet.
15:22 Devin isn't fixing code that a
15:24 customer asked it to fix.
15:26 Devin is fixing code that it
15:28 generated with errors in it.
15:29 And that's not at all what most of
15:32 the people who watch this video
15:34 will think that it's doing.
15:37 What's worse is that there's no
15:40 This is the README file from that
15:43 I told you we'd come back to this
15:45 There is a file called infer.py
15:47 that is in that repo and it does
15:50 exactly what Devin does in this
15:53 The README tells you that it
15:56 It tells you how to use it.
15:58 There on the right.
15:59 there's even a little button that
16:01 you can click on where you can copy
16:03 the whole command line and paste it
16:05 in your window and hit return.
16:06 And if you watch the long video
16:08 where I reproduce the result, that's
16:10 exactly what I did.
16:11 I copied and pasted things, changed
16:12 the path names and hit return and it
16:14 I don't think the person that wrote
16:16 this repository, detecting road
16:18 I don't think the person that wrote
16:20 that could have made it any easier
16:21 to understand how we were supposed
16:23 But Devin didn't seem to be able to
16:26 And so Devin had to create this
16:29 other thing that was a mess.
16:31 This code right here, this reading
16:33 into a buffer thing.
16:38 Right? This is the way we had to
16:40 read files in decade ago in 'C' and
16:42 really lower level languages.
16:44 Python has much better ways to
16:46 As Devin is figuring out, this kind
16:48 of thing is hard to debug.
16:50 It's difficult, easy to get off by
16:52 a little bit, which is I think what
16:54 Devin is trying to debug here.
16:55 I'm not exactly sure what was going
16:56 wrong, but that's what it seems
16:58 is it got off by some characters
17:00 and so the JSON didn't parse right?
17:02 But I mean, this is not how you
17:04 would do it these days.
17:05 This is not how you would do it in
17:07 This is not something that I would
17:09 accept in a code review from a
17:11 This is causing more problems than
17:13 it actually solves.
17:16 In addition, there is a real error
17:19 in the repo and Devin didn't find
17:23 Devin just created a bunch of other
17:25 So like I said, I replicated Devin's
17:29 Again, it'll be in the description.
17:31 I used torch 2.2.2, which is a much
17:34 more current version than the one
17:37 If you go back to that requirements.txt
17:39 file, the hard part of what I did
17:44 was getting a software package
17:45 called Apex installed with the
17:47 right version of CUDA, which is
17:49 NVIDIA's driver stuff.
17:51 I ended up having to build it from
17:52 source, which took about 16 minutes
17:53 of the 36 minutes that I was
17:54 working on the thing.
17:55 So there probably might have been
17:57 an easier way to do it, but for a
17:58 16 minute build time, that just
18:00 seemed to be the most expedient way.
18:04 I did remove the hard coding from
18:05 the requirements.txt file. Devin
18:06 just changed some of the numbers.
18:08 I think my way is better, but
18:09 either way, technically is okay.
18:11 See in the next slides, there is
18:12 actually one error that needed to
18:15 And I'll show you what it is.
18:15 It took me about 36 minutes, 35
18:18 minutes and 55 seconds, I think to
18:20 actually do what I did.
18:21 That will come important later when
18:22 we talk about how long Devin took.
18:24 Okay, so this is a screenshot from
18:26 that long video that I posted.
18:28 It's unlisted, but I gave you a
18:30 link to it if you want to watch the
18:31 whole thing zooming in.
18:32 So this is where the actual error
18:34 It's in a file called dataset.py
18:38 And the error is that the module
18:40 called torch has no attribute
18:43 I did a Google search.
18:46 I found this comment on a GitHub
18:48 I changed the line of that code the
18:50 way that issue told me that would
18:55 I put in a link to show where it
18:56 was that I got the idea to do that.
18:59 Because I'm not an expert in
19:01 exactly how Apex works.
19:02 It was good that I found somebody
19:04 on the Internet entire time on task
19:06 that it took me to do that was like
19:08 a minute and seven seconds or
19:10 something like that is all it took
19:11 me to fix that error.
19:12 It was a quick Google search.
19:14 So here is the change that I made
19:17 So this is a diff between what I
19:19 started with and what I ended up
19:21 This is a diff of the requirements
19:25 So the torch 1.4.0 is what it
19:28 I use the most recent version of
19:30 torch, which is 2.2.2 or at least a
19:32 relatively recent one.
19:34 There might have been a new one
19:35 released in the last hour for all
19:36 I know, a more recent one.
19:38 And then here is, on the right, one
19:40 of the last screens from Devin's
19:41 video and on the left,
19:43 there is my video, the final output.
19:46 They were both more or less the
19:50 I don't know which one might be
19:51 better or worse, but it only took
19:54 Devin took slightly longer than
19:56 So here is the early part of the
19:59 There's a timestamp at 3:25 PM on
20:05 Later in the video, you see a
20:08 timestamp from 9:41 PM on March the
20:11 So we're looking at six hours and
20:14 I have no idea what would have been
20:16 happening for six hours and 20
20:18 Hopefully like Devin was waiting on
20:20 people for a while from that
20:21 because it doesn't make any sense
20:23 that it would take that long.
20:24 That's just crazy because it like I
20:26 said, it took me a little over half
20:27 There's another one and I'm
20:29 assuming this is just like they
20:32 and then came back to it or
20:33 But there's another one from the
20:35 next day from 6 PM and hopefully it
20:39 over that whole time.
20:40 So I'm assuming it just took six
20:41 hours, but it could have taken, you
20:42 know, day and two hours.
20:44 That's just... I don't know why it
20:45 would have taken that long.
20:46 It's not efficient.
20:48 It's not what I would call
20:49 competent. A little weird command line
20:51 use popped up in one of the screens
20:53 when you frame by frame it.
20:54 So here's a weird error.
20:55 Let me zoom in on that head -n 5
20:57 5 results.json | tail -n 5
20:59 So what that says is take the first
21:02 five lines of this JSON file and
21:04 then take the last five lines
21:07 of the first five lines.
21:08 There's no reason to do that.
21:10 No human would do that.
21:12 And it's the kind of thing that AI
21:13 does that just doesn't make any
21:15 sense that when you come around
21:16 later and you look at it and you're
21:17 like, "OK, you're trying to debug
21:20 And there's all this extraneous
21:21 stuff all over the place and it
21:22 makes it really, really
21:23 hard to figure out what the point
21:25 In fact, the right way to do this
21:27 is `head -5 results.json`.
21:29 The `-n` is redundant.
21:30 You can just say `-5`. That
21:32 extra stuff in there is for no good
21:34 And it's the kind of thing that
21:36 just makes it way more complicated
21:38 when AI generates stuff right now.
21:40 Hopefully that will get better.
21:41 But at the moment AI generates a
21:43 lot of stupid stuff.
21:45 It does things in Python the way
21:46 you would do it in 'C' when no one
21:49 would do it that way in Python
21:51 Even when it gets things to work
21:52 right now, the state of the art of
21:55 is it just does a bad, complicated,
21:58 convoluted job that just makes more
22:01 work for everybody else
22:02 if you're ever going to try to
22:03 maintain it or fix a bug in it or
22:05 update it to a new version
22:07 or anything like that any time in
22:08 Let's look at the list of things
22:10 that Devin thought it needed to do.
22:13 If you look at the left there,
22:15 there's like this series of checkboxes.
22:17 I'm going to run through some pages.
22:18 Exactly what they are.
22:18 Isn't really important, but just
22:19 look how many there are.
22:20 This list of checkboxes gives the
22:22 impression that Devin did something
22:25 complicated or difficult.
22:26 And when you're watching the video
22:28 and you see all this scroll by, you're
22:29 like, you know, wow,
22:30 Devin must have done a bunch of
22:31 All you needed to do, all I had to
22:33 do to replicate Devin's results was
22:36 get an environment set up
22:37 on a cloud instance with the right
22:38 hardware and run literally two
22:40 commands with the right paths.
22:42 All of this stuff makes it look
22:44 like Devin did a bunch of work.
22:45 It makes it look like Devin
22:47 accomplished a lot of stuff.
22:48 And really, all you had to do was
22:50 run two commands once you set the
22:52 None of those code fixes are
22:53 relevant at all because it's all
22:54 code that Devin generated itself.
22:56 And at the end, the person that was
22:59 narrating the video says, "Good Job,
23:01 Now, what Devin actually got done
23:05 was kind of cool for an AI.
23:08 If you had asked me a couple of
23:09 months ago, what an AI would have
23:12 done given that problem.
23:15 I would have guessed an output that's
23:17 worse than what Devin actually did.
23:20 So it is honestly, as far as I'm
23:21 concerned, kind of impressive.
23:23 But in the context of what an Upwork
23:26 job should have been, and
23:28 especially in the context of a
23:31 of people saying that Devin is
23:33 "taking jobs off of Upwork and doing
23:35 them,"" and especially in the
23:36 context of the company saying that
23:38 this video will let us watch Devin
23:41 get paid for doing work,
23:42 which is, again, just a lie.
23:44 I don't know that saying "Good Job."
23:46 I don't know that I would agree
23:48 So look, if you make AI products,
23:54 I want it to get better.
23:55 Please make products.
23:58 Just please tell people the truth
24:01 If you're a journalist or a blogger
24:04 or an influencer, just please don't
24:08 amplify things that people say on
24:10 the Internet, things that you read
24:12 on the Internet without
24:15 doing some due diligence, without
24:16 looking to see if they're actually
24:18 If you don't understand if they're
24:19 true, if you can't figure out on
24:21 your own if they're
24:21 true, ask someone or just don't
24:24 Because there are a lot of people
24:25 that are never going to look at the
24:28 They're just going to see the
24:28 headline and they're going to think
24:30 That's unfortunate, but that's just
24:32 And if you're just someone who's
24:34 using the Internet now, please, for
24:36 the love of all that's
24:37 holy, be skeptical of everything
24:39 you see on the Internet or anything
24:41 you see on the news,
24:42 especially anything that might
24:44 possibly be AI related.
24:45 There's so much hype out there and
24:47 there's so much stuff that people
24:49 are bouncing around and
24:51 saying to each other is true.
24:52 That's just not true.
24:53 So please just don't forget to be
24:58 Okay, so that's what I have for
25:00 this video. Until next time,
25:02 Always keep in mind that the
25:04 Internet is full of Bugs and anyone
25:06 who says differently is trying
25:07 to sell you something.
25:08 Have a good one, everybody.
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
