00:07 hi guys so today's session is on aws
00:09 sagemaker but before proceeding with
00:12 this topic i would request you guys to
00:14 like our video also subscribe to our
00:16 channel you can find the link just below
00:18 at the right side of this video
00:20 let's look into what we have in our
00:23 so what's in it for you we would be
00:24 covering what is aws
00:27 why do we need aws sage maker what is
00:30 aws sage maker services
00:32 what are the benefits of using the aws
00:36 machine learning with aws sagemaker how
00:38 to train a model with aws stage maker
00:40 how to validate a model with
00:42 aws and the companies that are using aws
00:45 shmaker along with that we will be
00:47 covering up one live demo on the aws
00:52 now let's understand what is aws so
00:55 what is aws it's an amazon web services
01:00 most widely used public cloud platform
01:04 offered by amazon it provides services
01:08 aw services can be used to build monitor
01:11 and deploy any type of application in
01:14 aws also uses the subscription pricing
01:16 model that means you only pay for
01:18 whatever the services you use for
01:22 now why do we need aws sagemaker let's
01:26 let's consider an example of one of the
01:28 company that is proquest
01:30 now before aws age maker
01:33 the proquest is a global information
01:35 content and technology company
01:38 that provides valuable content such as
01:40 ebooks newspapers etc to the users
01:46 the proquest requirement was to have a
01:48 better user experience maximum relevant
01:50 search results now after aws sagemaker
01:53 they were able to achieve those
01:57 so they achieved more appealing video
01:59 user experience they achieved more
02:00 relevant search results for the users
02:02 now what do we mean by aws age maker why
02:05 this service is primarily used
02:07 so amazon sage maker is a cloud machine
02:11 that helps users in building training
02:13 tuning and deploying machine learning
02:15 models in a production ready hosted
02:18 so it's kind of a machine learning
02:20 service which is already hosted on the
02:22 aws platform now what are the benefits
02:24 of using aws sagemaker
02:27 the key benefits of using aws sagemaker
02:30 it reduces machine learning data cost so
02:33 you can do the cost optimization
02:37 this particular service on the aws
02:39 all ml components are stored in a
02:41 particular place in a dashboard so they
02:44 can be managed together
02:46 highly scalable so it can be scalable on
02:51 this particular service on the fly
02:53 it trains the models quite faster
02:56 maintains the uptime so you can be
02:59 assured that your workloads will be
03:01 running all the time it will be
03:03 available all the time
03:05 high data security so security becomes a
03:07 major concern on the cloud platforms and
03:09 it ensures that you have the high data
03:12 security along with that you can do a
03:15 data transfer to different aws services
03:17 like s3 bucket and all
03:19 with these simple data transfer
03:22 now machine learning with aws sagemaker
03:24 let us look into it
03:26 so machine learning with aws sagemaker
03:28 is a three-step function
03:31 so one is to build second is to test and
03:34 tune the model and third is to deploy
03:37 the model now with the build it provides
03:40 more than 15 widely used ml algorithms
03:43 for training purpose
03:45 now to build a model you can collect and
03:47 prepare training data or you can select
03:50 from the amazon s3 bucket also choose
03:52 and optimize the required algorithm so
03:54 some of the algorithms that you can
03:56 select are k-means linear regressions
03:59 logistic regression sagemaker helps
04:01 developers to customize ml instances
04:05 with the jupiter notebook interface
04:08 in the test and tune you have to set up
04:09 and manage the environment
04:11 for training so you would need some
04:13 sample data to train the model
04:16 so train and tune a model
04:18 with the amazon sage maker sagemaker
04:21 implements hyper parameter tuning
04:24 by adding a suitable combination of
04:26 algorithm parameters also it divides the
04:29 training data and stores that in the
04:31 amazon s3 s3 is a simple storage service
04:34 which is primarily used for storing the
04:36 objects and the data hence it is used
04:38 for storing and recovering data over the
04:42 below you can see that aws sagemaker
04:44 uses amazon s3 to store data as it's
04:47 safe and secure also it divides the
04:49 training data and stores in amazon s3
04:52 where the training algorithm code is
04:54 stored in the ecr ecr stands for elastic
04:57 container registry which is primarily
04:59 used for containers and dockers ecr
05:01 helps users to save monitor and deploy
05:04 docker and the containers later
05:06 sagemaker sets up a cluster for the
05:09 trains it and stores it in the amazon s3
05:12 itself so this is done by the sagemaker
05:14 itself after that you need to deploy it
05:17 so suppose you want to predict limited
05:20 you use amazon sagemaker hosting
05:22 services for that okay but if you want
05:24 to get prediction for an entire data set
05:27 prefer using amazon sage maker batch
05:29 transform now the last step that is to
05:35 so once tuning is done models can be
05:37 deployed to sagemaker endpoints
05:40 and in the end point real time
05:41 prediction is performed so you would
05:45 which you would reserve and
05:47 validate your model whether it is
05:49 working correctly or not now evaluate
05:51 your model and determine whether you
05:53 have achieved your business goals
05:56 other aspect is how we can train a model
05:58 with aws stage maker
06:01 so this is basically a flow
06:04 diagram which shows you how to train a
06:06 model with the aws sagemaker and
06:09 here we have used couple of services of
06:11 an aws to get that done
06:15 so model training in aws age maker is
06:17 done on machine learning compute
06:19 instances and here we can see there are
06:22 two machine learning compute instances
06:23 used as helper code and the training
06:27 along with that we are using 2s3 buckets
06:29 and the ecr for the container registry
06:34 let's look into what are the ways to
06:35 train the model as per the slides
06:39 so below are the following requirements
06:41 to train a model so here in the diagram
06:43 you can see these are the following
06:44 requirements to train a model
06:47 the url of an amazon s3 bucket where the
06:50 training data is stored that is
06:53 the compute resources on machine
06:55 learning compute instances so these are
06:58 all your machine learning compute
07:02 then the url of an amazon s3 bucket
07:04 where the output will be stored
07:08 and the path of aws elastic container
07:12 where the code data is safe the
07:14 inference code image lies in the elastic
07:16 container registry now what are these
07:18 calls these are called as the training
07:23 now when a user trains a model in amazon
07:25 sage maker he she creates a training job
07:27 so we need to first create a training
07:28 job and then the input data is fetched
07:31 from the specified amazon s3 bucket
07:33 once the training job is built amazon
07:35 sage maker launches the ml compute
07:37 instances so these compute instances
07:39 will be launched once the training job
07:41 is built then it trains the model with
07:43 the training code and the data set and
07:46 it shows the output and model rt graphs
07:48 in the aws s3 bucket so this is done
07:50 automatically now here the helper code
07:53 performs a task when the training code
07:55 fails the interference code which is in
07:58 the elastic container registry
08:01 multiple linear sequence containers
08:04 that process the request for inference
08:08 the ec2 container registry is a
08:10 container registry that helps users to
08:12 save monitor and deploy container images
08:15 whereas container images are the ready
08:19 once the data is trained the output is
08:21 stored in the specified amazon s3 bucket
08:24 so here you can see the output will be
08:26 stored here to prevent your algorithm
08:28 being deleted save the data in amazon
08:30 sagemaker critical system
08:33 which can process you on your ml compute
08:35 instances now how to validate a model
08:38 let's look into it so you can evaluate
08:41 using offline or using the historical
08:44 data so first thing is that you can do
08:46 the offline testing to validate a model
08:48 you can do an online testing with the
08:50 live data so if you have a live data
08:52 coming or real time streams coming you
08:54 can validate a model from there itself
08:56 you can validate using a holdout set and
09:00 also you can validate using the k-fold
09:02 validation now use historical data to
09:05 send requests to the model
09:07 through the jupiter notebook in amazon
09:09 sagemaker for the evaluation online
09:11 testing with livedata deploys multiple
09:13 models into the endpoints of amazon
09:16 sagemaker and directs live traffic to
09:18 the model for validation validating
09:21 using a holdout set
09:22 is part of the data is set aside
09:25 where which is called holdout set so
09:29 the part of the data is left
09:31 which is basically called as the holdout
09:34 this data is not used for the model
09:37 so later when the model is trained
09:39 with the remaining input data and
09:41 generalize the data based on what is
09:42 learnt initially so whatever the data
09:45 will be used for validating a model
09:48 because we have not used that data
09:50 while training a model the k-fold
09:54 is the input data is split into two
09:57 one part is called k
09:59 which is the validation data for testing
10:01 the model and the other part is k minus
10:04 which is used as a training data
10:07 now based on the input data the machine
10:09 learning model evaluates the
10:13 the companies that are using aws
10:15 sagemaker one is the adb so you must be
10:25 and the intuit now let's look into the
10:28 demo that how we can actually run the
10:32 so we'll use the r algorithm and then
10:35 package the algorithm as a container for
10:38 building training and deploying a model
10:40 we are going to use the jupyter notebook
10:42 for that for model building for model
10:44 training for model deployment
10:46 and the code for the demo is in the
10:48 below link so you can see here that from
10:51 this link you can get the code for the
10:56 do a demo on the aws
10:58 now i would be using a link which is
11:01 provided by amazon to build train and
11:03 deploy the machine learning model on the
11:05 sagemaker as you can see on my screen
11:07 and in this tutorial you would have some
11:09 steps where you can put those steps and
11:12 the code python codes into your aws
11:15 sagemaker jupiter lab
11:16 so in this tutorial you will learn how
11:18 to use amazon sagemaker to build train
11:20 and deploy a machine learning model and
11:22 for that we will use the popular xt
11:24 boost ml algorithm for this
11:27 so first of all what you need to do is
11:29 you have to go to the aws console and
11:31 there you have to create a notebook
11:33 so in this tutorial you will be creating
11:35 a notebook instance you will prepare the
11:37 data train the model to learn from the
11:39 data deploy the model evaluate your ml
11:43 model's performance and once all those
11:44 activities are done then we will see how
11:46 we can actually remove all the resources
11:48 in order to prevent the extra costing
11:52 now the first step is we have to enter
11:54 to the amazon stagemaker console
11:56 so here you can see i am already logged
11:58 in into the sagemaker console you can
12:00 click on the services
12:02 search for the sage maker here
12:05 and here you get the amazon sagemaker
12:10 now the next step is
12:12 that we have to create a notebook
12:16 we will select the notebook instance
12:17 from the sagemaker service
12:20 after the notebook instance is selected
12:22 we put a name to our instance
12:24 and we'll create a new im role for that
12:28 so let's wait for the sage maker studio
12:31 so here you can see the studio is open
12:34 and you just have to click on the
12:38 and here you have to create a notebook
12:40 instance so here you can see couple of
12:41 notebook instances have already been
12:43 created one of them is in service so
12:46 this is the notebook instance that we
12:47 are going to use for
12:52 i'll show you how you can create a
12:53 notebook instance you just have to click
12:55 on create notebook instance button
12:58 and put your notebook instance name so
13:01 you can put something like demo dash
13:03 sage maker 987 or we can put it as model
13:09 we'll go with notebook instance type as
13:11 default which is mlt2.medium
13:14 and in the permission and encryptions
13:16 under the im role we will click on
13:19 create a new i am role
13:22 now why we are creating a new im rule so
13:24 that we can allow the sage maker
13:26 to access any s3 bucket that has been
13:28 created on our account
13:31 just click on create a role and here you
13:35 that the new im role will be created
13:38 with the set of permissions
13:40 then rest of the things will keep it as
13:42 default and then you just have to click
13:44 on create a notebook
13:46 instance the notebook instance creation
13:49 takes some time so you just have to wait
13:50 for a couple of minutes to get that in
13:52 service we already have one of the
13:55 notebook instance that has been created
13:57 so we will be using
13:59 that to create a demo
14:02 now going back to the steps so these are
14:05 the steps that we have already performed
14:07 now once the notebook instance is
14:09 created then we have to prepare the data
14:13 in this step we will be using the
14:16 sagemaker notebook instance to
14:17 pre-process the data that would require
14:20 to train the machine learning model
14:23 and for that we would be opening up the
14:27 and then we have to select an
14:29 environment a kernel environment in the
14:31 jupyter notebook that would be conda
14:34 underscore python 3.
14:39 go back to the sage maker click on the
14:44 select the running notebook instance and
14:47 here you would select the open jupiter
14:51 now here you would see that the
14:53 sagemaker would try to open up the
14:58 and we would be performing all our
15:00 inputs into that jupiter notebook
15:04 and executing the results there itself
15:06 so just wait for the notebook to open
15:10 now here you can see the jupiter lab
15:12 notebook has been open so i would be
15:14 selecting one of the notebook that has
15:17 so this one so likewise you can create
15:19 your own notebook also
15:21 how you can do that
15:23 first of all let me select the kernel
15:26 environment so i would be selecting
15:28 underscore python 3
15:30 and just click on select so how you can
15:32 create your own notebook just have to
15:34 click on file click on new and here you
15:37 can select the notebook just name your
15:40 notebook select the environment conda
15:43 underscore python 3 to run this demo
15:47 so i have my notebook open so in the
15:49 tabs i would be putting up the python
15:52 codes and i would be executing those
15:54 codes to get the output directly
15:57 so the next step is
16:00 to prepare the data train the ml model
16:02 and deploy it we will need to import
16:05 and define a few environment variables
16:08 in the jupiter notebook environment
16:10 so i would be copying this code
16:13 which you can see that would try to
16:15 import numpy pandas these are all
16:18 required to run the python
16:28 and paste it into your
16:34 right so once you do that execute your
16:38 and here you can see
16:41 you get the output which is
16:43 that it has imported all the necessary
16:46 libraries that have been defined in the
16:49 now the next step is we would create an
16:53 into the s3 service and for that you
16:56 have to copy this python code
16:58 just that you have to edit it so you
17:02 bucket name that you want to get created
17:08 provide the bucket name which should be
17:13 so something like sage maker
17:23 is the name that i have selected for the
17:24 bucket and now you have to
17:29 it says that the s3 bucket has been
17:31 created successfully with the name sage
17:33 maker dash demo 9876 so this is
17:35 something which you can verify so you
17:37 can go to the s3 service and there you
17:39 can verify whether the bucket has been
17:42 now the next task is that we need to
17:47 to the aws sagemaker instance and load
17:50 it into the data frame and for that we
17:54 follow this url so from this url
17:59 build train deploy machine learning
18:00 model would have a data in the form of
18:03 bank underscore clean.csv
18:06 and this will be deployed onto our
18:10 we'll copy this code
18:15 and execute the code
18:18 so it says that it has successfully
18:20 downloaded bank underscore clean.csv
18:23 which is which has the data inside it
18:25 and that has been loaded into the sage
18:27 maker data frame successfully
18:30 now we have a data to
18:33 train our machine learning model
18:36 so what we are going to do we are going
18:37 to shuffle the data and we are going to
18:39 split it one into the training data set
18:42 and the other one into test data set
18:44 so for the training data set we are
18:46 going to use 70 of the customers that
18:48 are listed in the csv file and 30 of the
18:52 customers in the csv file data we will
18:55 be using it as a test data to train the
18:59 so we'll copy the following code into a
19:01 new code cell and then we are going to
19:10 for training the data so that we can
19:15 for building the model and 30 for
19:24 run the execution and here you can see
19:26 that we got the output successfully
19:33 we have to train the model from that
19:34 data so how we are going to train that
19:36 model and for that we will use sagemaker
19:39 prebuilt xgboost model which is an
19:43 so you will need to reformat the header
19:45 and first column of the training data
19:46 and load the data from the s3 bucket so
19:49 what i'll do is i'll copy
20:08 so it has the trained data it would
20:14 click on run execution
20:18 now it is changing the s3 input class
20:22 renamed to training input because now we
20:24 are training the model with the training
20:27 so we just have to wait for some time
20:30 till it gets executed completely
20:34 now the next thing is that we need to
20:36 set up the amazon sagemaker session to
20:38 create an instance of the xgboost model
20:42 so here we are going to create
20:45 this sagemaker session say we are going
20:47 to create an instance of the xt boost
20:50 model which is an estimator
21:03 and here you can see that
21:06 it has basically changed the parameter
21:11 image underscore uri in this sagemaker
21:17 now we will follow the next step that is
21:19 with the data loaded in the xt boost
21:20 estimator we will set up train the model
21:23 using gradient optimization
21:28 we will copy the following code and that
21:30 would actually start the training of
21:38 this would actually
21:40 start training the model
21:42 using our input data that we have
21:45 reserved 70 of that data that we have
21:47 reserved for training the model
21:53 again initiate the execution and it will
21:56 start the training job
21:59 now we'll deploy the model and for that
22:01 i would copy the deploy code put that in
22:17 it says parameter image will be renamed
22:19 to image uri and using already existing
22:22 models so hd boost was deployed already
22:25 if you have not done that
22:27 if you're doing it the first time so it
22:28 will initiate another xd boost instance
22:31 so where you can find your xt boost
22:33 endpoints created you just have to
22:35 scroll down and here under the inference
22:38 click on the end points
22:40 and you should find the extreme boost
22:42 endpoints defined here
22:44 so here you can see that today i have
22:47 created one xt boost
22:51 and that is now in process of creating
22:58 still created it is going to take some
23:00 time to get that in service
23:03 now our endpoint is in service state so
23:05 now we can use it so
23:07 going forward with the next steps we
23:09 will try to predict whether the customer
23:11 in the test data enroll for the bank
23:12 product or not for that we are going to
23:14 copy this code put that in the jupiter
23:22 so here it gives you the output that it
23:25 has actually evaluated and the same
23:28 output we got in the screenshot
23:31 of the demo as well
23:33 now we are going to evaluate the model
23:35 performance so what we are going to do
23:37 we are going to get the prediction done
23:40 so based on the prediction we can
23:42 conclude that you predicted a customer
23:44 that will enroll for a certificate of
23:46 deposit accurately for 90 of the
23:48 customers in the test data with the
23:50 precision of 65 percent for enrolled and
23:54 which are which haven't enrolled for it
23:57 so for that we are going to copy this
24:02 and execute it here in the cell
24:06 so if it is predicted correctly that
24:08 means our model is working absolutely
24:11 fine so here you can say the overall
24:13 classification rate is 89.5 percent and
24:17 there is the accurate
24:19 prediction that has been made by the
24:22 model and that's what the output we can
24:25 in the screenshot of a model so that
24:28 means our model is absolutely working
24:30 fine it has been built deployed and
24:34 now the next thing is that once you are
24:36 done with that you terminate your
24:37 resources and for that you just have to
24:41 uh this code and put that in the cell
24:44 function so that the additional
24:45 resources and the endpoints and the
24:47 buckets that have been created by the
24:49 jupiter notepad should be
24:52 terminated so that you would not be
24:55 incurred with the extra costing so just
24:58 execute it and here you would see that
25:01 it has tried to it would try to
25:04 terminate all the additional resources
25:06 that we have created from the jupyter
25:09 that was with respect to how we can use
25:11 the aws sagemaker demo guys watch out
25:14 for more space with respect to the
25:16 upcoming videos thank you bye for now
25:22 hi there if you like this video
25:24 subscribe to the simply learn youtube
25:26 channel and click here to watch similar
25:28 videos turn it up and get certified
![Welp, Here's what the Fed *Just* Said [Fed FOMC Minutes]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fl8u23oq1E9E%2Fhqdefault.jpg&w=3840&q=75)
