00:00can you describe the type of failure
00:02scenarios that you might see
00:08hey everyone i'm super excited to be
00:10here today with adam
00:12to discuss a tpm mock interview we're
00:15going to be talking about data centers
00:17but before we jump into it adam as usual
00:20could you share a little bit more about
00:22absolutely yeah i have experience in
00:24data center consolidation
00:26and infrastructures based on my
00:28experience working active duty in the
00:30also migrations and accenture capacity
00:32planning at google and most recently as
00:34consumer of aws at airbnb awesome well
00:38let's jump right into it
00:39so um the first question that i just
00:41want to ask you right off the bat
00:43is can you describe the type of failure
00:46scenarios that you might see in the data
00:49great question there's a lot of
00:50different ways that we see
00:52failures all the time in data centers
00:55i think it seems like a pretty far away
00:57concept but these are
00:59these data centers are hosting data for
01:00all the applications that we use and
01:02every day uh the first i'd like to
01:04mention is just component and server
01:07so just think about what actually a
01:09server is it's a machine
01:10with the cpu ram disk and actually
01:14whatever application we're trying to
01:15access there are failure rates
01:17of those rams and cpu and at google
01:21scale when i was working at youtube
01:22in the hundreds of thousands or even
01:24millions of servers we'd see
01:26failures every day in fact there's a
01:29card hardware operators who are riding
01:32around on scooters and swapping out
01:33components on daily basis
01:36another somewhat common uh failure
01:38scenario was site power outages
01:40uh this can happen you know it's not so
01:42much on a daily basis
01:44but if there's a failure in the power
01:45grid uh local fire or disaster going on
01:49um or even sometimes cases with diesel
01:52you know people who are familiar with
01:54data centers also know that they have
01:56um in the form of batteries and
01:58flywheels so definitely a lot of
02:00different things that can go wrong
02:01uh those type of uh power sources though
02:04are primarily used to move from
02:07when the power goes out and then trying
02:09to get the diesel generators on
02:11and i have to mention with site power
02:12outages being very real
02:14um think about the seasonal hurricane
02:16season in the u.s specifically the south
02:19east and that every year we've sort of
02:22seen exercise of moving
02:24our jobs and loads from one site to
02:27trying to be ready and anticipate for
02:30of course there's the least popular but
02:33uh you know number one reason why we see
02:36uh downtime or failures which is
02:38changes in software or configurations uh
02:40you know unfortunately this can happen
02:42if a source app uh someone puts out
02:45something there wasn't proper
02:47qa or testing done on it or if you
02:50have uh the situation where you're with
02:52a third party cloud provider
02:54an employee over there could make a
02:55policy change revoking
02:57revoking all of the access on accident
03:00so these type of things definitely
03:01happen totally um thanks for coming
03:04those scenarios um i kind of want to
03:06change gears in the interview
03:07just a little bit to ask you a follow-up
03:09question that's a little bit more of
03:10estimation question and and so sometimes
03:13in the interviews we'll sort of uh
03:14push you around in different ways um but
03:16the question that i have for you today
03:18is um you know could you estimate
03:20something like how much power or
03:21maybe how many wind turbines that it
03:23might take to power the global data
03:26footprint of a company like google got
03:28it uh so just i understand correctly
03:30and i'll give you a little idea of how
03:32i'm going to approach this i'll first
03:34try and estimate how much power the data
03:37and then how much they generate and
03:39you're sort of looking for a number of
03:41sort of instantaneously or today how
03:44that's right yep okay great um yeah so
03:47i would say just on the number of data
03:50centers alone there's
03:5120 and there's actually publicly
03:52available information
03:54uh for google uh 12 are in the us five
03:57are in europe two in asia and one is in
04:01um off the top of my head i remember
04:03that data centers consume about
04:0415 megawatts or more of power but i want
04:07to spend a little bit of time
04:09and validate that is that okay with you
04:12all right yeah so a server consumes
04:16power and when you have multiple servers
04:20a rack is just like a cabinet in the
04:23about 50 servers can go into this rack
04:26so if you multiply those two numbers out
04:28that ends up being about 10 000 watts
04:30or 10 kilowatts per rack uh so now i
04:34have to think how many racks there are
04:36uh you can depend on the size of the
04:38data center but i'm going to use
04:39somewhere around 2000 racks just based
04:42seeing having seen this number and if i
04:44multiply those two out that ends up
04:47kilowatts or if i go and translate over
04:50the units 20 megawatts
04:52um so this is how much a single data
04:56but it is google and so we have to think
04:58that things are a little bit
04:59bigger i'd like to add a scale-up factor
05:03uh just because there's multiple data
05:05center facilities on
05:07each of these given campuses that we're
05:10uh so i'll use 2.5 multiplier on top of
05:14megawatts that would tell me that each
05:16site is using about 50 megawatts
05:18and so if i go back i told you that
05:20there were 20 different sites
05:22each consuming 50 megawatts that will
05:26about a thousand megawatts if it
05:28translates over that's one
05:30gigawatt of power um
05:33yeah so now that i have the power
05:36of the data centers i'll move into the
05:39second part where you mentioned the wind
05:41power generation like off the top of my
05:44head i actually don't know how much
05:46power wind turbine can create but i
05:49remember from my graduate research
05:50that solar powers about one by one
05:54meter can generate one kilowatt of power
05:57so i'm just gonna assume
05:58at this point that one wind turbine can
06:02kilowatt and so because i estimated one
06:05gigawatt that ends up being uh one
06:09wind turbines to power google's data
06:12got it that's a lot of wind turbines um
06:15do we feel like that was an overestimate
06:18or what would you change if you had more
06:20time to kind of analyze this
06:22answer yeah yeah definitely that that's
06:24uh that's a big number
06:25um but also using it and operating at
06:29i think this is pretty appropriate um
06:32i think out of the framework that i laid
06:34out you know we can sort of go and
06:35change some variables so
06:37if a wind turbine didn't use or couldn't
06:40generate one kilowatt
06:41or if there are slightly different power
06:43calculations for servers or racks
06:44we can go back and adjust these things
06:46but i feel pretty confident and good
06:50cool yeah that's awesome um well we can
06:53debrief or or you know pause the
06:55interview i would just love to hear yeah
06:57anything else you wanted to share
06:58with our audience or any other high
07:00level thoughts on how that interview was
07:03uh yeah i mean this is a challenging
07:05question and i certainly think
07:07you know not having worked in this role
07:09before i probably wouldn't have been
07:12so i didn't learn how to do this
07:13overnight i definitely took practice
07:15and working with these different domain
07:17areas really the takeaway for folks is
07:20you know this question involved a lot of
07:22power and consumption and also just
07:24understand the space and size of a data
07:26center but that might not be your area
07:28of expertise and that's okay
07:30when you receive these type of questions
07:31it's important to focus on the process
07:34and sort of be able to regenerate and
07:36recreate uh the scenario that you did to
07:38create your number again um so yeah i
07:42also i want to harp on you know really
07:45introducing those different factors and
07:47being able to defend them
07:48and using this process you can probably
07:50go forward and tackle
07:52any question because the type of
07:54interview questions you get
07:55they'll be looking for you and how
07:57assessing how you tackle
07:59ambiguity totally um well i thought you
08:02did a great job answering and
08:04very thoughtful and you went through it
08:07obviously in more time digging deeper
08:10into some of those things around
08:11how you might have changed your answer
08:12or trade-offs or scenarios where you
08:16are always really valuable but overall i
08:18thought you did an excellent job
08:19and it's always a pleasure having you
08:21here adam so um thank you again for
08:23being on the show to provide your answer
08:25all right thank you very much
08:28thanks so much for watching don't forget
08:29to hit the like and subscribe buttons
08:31to let us know that this video is
08:33valuable for you and of course
08:35check out hundreds more videos just like
08:39exponent dot com thanks for watching and
08:41good luck on your upcoming interview
