00:00hi everyone welcome to day 6 and Z
00:02podcast I'm sonal so one of our favorite
00:04themes to talk about in this podcast is
00:06how software changes organizations and
00:08the nature of the firm and today's
00:10episode takes a different angle on the
00:12topic by drawing on the research of one
00:14of the largest large-scale studies of
00:16software and organizational performance
00:18out there from the authors of the new
00:20book accelerate the science of lean
00:22software and DevOps building and scaling
00:25high-performing technology organizations
00:27by nichole force grin Jays humble and
00:29Jean Kim joining us to have this
00:31conversation we have Nicole who did her
00:32PhD research trying to answer the
00:34elusive eternal questions around how to
00:36measure this especially given past
00:38debates around us IT matter she's now
00:40the CEO of Dora DevOps research and
00:42assessment which also puts out with
00:44puppet the annual state of DevOps report
00:46and then we have jazz humble who is CTO
00:48at Dora and is also the co-author of the
00:50books the DevOps handbook Lean
00:52Enterprise and continuous delivery they
00:54share the latest findings about
00:55high-performing companies of all kinds
00:57even those who may not think they're
00:58tech companies and answer whether
01:00there's an ideal organizational type in
01:02size architecture culture or people that
01:05lends itself to success here but we
01:07begin this episode by briefly talking
01:08about the history of DevOps and where it
01:10fits in the broader landscape of related
01:12software movements so I started as a
01:15software engineer at IBM I did hardware
01:17and software performance and then I took
01:19a bit of a detour into academia because
01:21I wanted to understand how to really
01:23measure and look at performance that
01:26would be generalizable to several teams
01:27in predictable ways and in predictive
01:30ways and so I was looking at and
01:33investigating how to develop and deliver
01:36software in ways that were impactful to
01:38individuals teams and organizations and
01:41then I pivoted back into industry
01:43because I realized this movement had
01:46gained so much momentum and so much
01:48traction an industry was desperate to
01:51really understand what types of things
01:53are really driving performance outcomes
01:56and would even say this movement this
01:58movement that now we call DevOps so the
02:01ability to leverage software to deliver
02:05value to customers to organizations to
02:09stakeholders and I think from a
02:11historical point of view the best way to
02:14it's a bunch of people who had to solve
02:16this problem of how do we build large
02:18distributed systems that were secure and
02:22scalable and be able to change them
02:24really rapidly and evolve them no node
02:26had that problem before certainly at the
02:27scale of companies like Amazon and
02:29Google and that really is where the
02:31DevOps movement came from trying to
02:32solve that problem and you can make an
02:34analogy to what agile it was about since
02:36that kind of software crisis of the
02:381960s and people trying to build these
02:41defense systems at large scale the
02:44invention of software engineering as a
02:46field Margaret Hamilton her work MIT on
02:49the Apollo program what happened in the
02:51decades after that was everything became
02:53kind of encased in concrete in these
02:55very complex processes this is how you
02:59develop software and agile was kind of a
03:00reaction to that same you can develop
03:02software much more quickly with much
03:04smaller teams in a much more lightweight
03:06way so we didn't call it DevOps back
03:08then but it's also more agile can you
03:10guys break down the texana me for a
03:11moment because when I think of DevOps I
03:13think of it in the context of the
03:14containerization of code and
03:16virtualization I think of it in the
03:18context of microservices and being able
03:20to do modular teams around different
03:23things there's an organizational element
03:24there's a software element there's an
03:26infrastructure component like paint the
03:28big picture for me or those building
03:29blocks on how they all kind of fit
03:30together well I can give you a very
03:31personal story which was my first job
03:33after college was in 2000 in London
03:35working at startup where I was one of
03:37two technical people in the startup and
03:38I would deploy to production by FTP and
03:42code from my laptop into production and
03:44if I went as a roll back I'd say hey
03:45Ronnie can you FTP your copy of this
03:47file to production and that was our
03:48rollback process and then I went to work
03:50in consultancy where we were on these
03:52huge teams and deploying to production
03:54there was a whole team with a Gantt
03:56chart which puts together the plan to
03:57deploy to production and I'm like this
03:58is crazy unfortunately I was working
04:01with a bunch of other people who also
04:02thought it was crazy and we came up with
04:04these ideas around deployment automation
04:06and scripting and stuff like that and
04:08suddenly we saw the same ideas that
04:10popped up everywhere basically
04:11I mean it's realizing that if you're
04:12working in a large complex organization
04:14agile is gonna hit a brick wall because
04:17unlike the things we were building in
04:19the 60s product development means that
04:21things are changing and evolving all the
04:22time so it's not good enough to get to
04:24production the first time you've got to
04:25be able to keep getting there or not
04:26and that really is where DevOps comes in
04:28it's a well as well we've got a way to
04:30build nivel products but how do we keep
04:32deploying to production and running the
04:34systems in production in a stable
04:36reliable way particular in a distributed
04:38context so if I phrase it another way
04:40sometimes there's a joke that says day
04:41one is short and day two is long what
04:43does that mean right so day one is one
04:45we like crazy by the way that you have
04:47to explain the joke to me no no no
04:49because so day one is when we create all
04:51of these systems and they two is when we
04:53deploy to production we have to deploy
04:56and maintain for ever and ever and ever
04:59and ever so day two is an infinite day
05:02yeah for successful products hopefully
05:05we hope that day two is really really
05:07long and we're fond of saying agile
05:10doesn't scale and sometimes I'll say
05:12this in people shoot laser beams out of
05:13their eyes but when we think about it
05:15agile was meant for development just
05:17like Jezz said it speeds up development
05:19but then you have to hand it over and
05:21especially infrastructure and IT
05:23operations what happens when we get
05:26so DevOps was sort of born out of this
05:28movement and it was originally called
05:30agile system administration and so then
05:33DevOps sort of came out of development
05:35and operations and it's not just Devon
05:38Ops but if we think about it that's sort
05:40of the bookends of this entire process
05:42it's actually like day one and day two
05:43combined in one way to the way I think
05:46about this is I remember the stories of
05:48like Microsoft in the early days and the
05:50waterfall cascading model of development
05:54once wrote a piece for me about why
05:56software should be developed like houses
05:58cuz you need a blueprint and I'm not a
06:00software developer but it felt like a
06:02very kind of old way of looking at the
06:04world of code I hate that metaphor tell
06:06me why if the thing you're building has
06:09well-understood characteristics it makes
06:10sense so if you're building a trust
06:11bridge for example there's well-known
06:13understood models of building trust
06:14bridges you plug the parameters into the
06:16model and then you get a truss bridge
06:18and it stays up have you been to Sagrada
06:20família in bus I love Gaudi okay so if
06:23you go into the Crypt of the Sagrada
06:25família you'll see his workshop and
06:26there's a picture in fact a model that
06:28he built of the Sagrada família but
06:30upside down with the weights simulating
06:33the stresses and so he would build all
06:34these prototypes and small pressure
06:36sites because he was fundamentally
06:37designing a new way of building all
06:40designs were hyperbolic curves and
06:42parabolic curves and no one had used
06:43that before that had never been pressure
06:45tested right let's say in that case
06:47exactly he didn't want them to fall down
06:49so he built all these prototypes and did
06:51all this stuff he built his blueprint as
06:53he went by building and trying it out
06:54which is a very rapid prototyping kind
06:56of model absolutely so in the situation
06:58where the thing you're building has
07:00known characteristics and it's been done
07:01before yes or we can take a very phased
07:04and you know if of designing these kind
07:06of protocols that have to work in a
07:08distributed context and you can actually
07:10do formal proofs of them again
07:11that makes sense but when we're building
07:13products and services we're particularly
07:15we don't know what customers actually
07:18want and what uses actually one it
07:20doesn't make sense to do that because
07:21you'll build something that no one wants
07:23you can't predict and we're particularly
07:25bad at that by the way even companies
07:27like Microsoft where they are very good
07:30at understanding what their customer
07:32base looks like they have a very mature
07:34product line Roni Kavi has done studies
07:37there and only about 1/3 of the
07:39well-designed features deliver value
07:42that's actually a really important point
07:45the mere question of does this work is
07:47something that people really clearly
07:48don't pause to ask but I do have a
07:50question for you guys to push back which
07:52is is this a little bit of the cult oh
07:53my god it's like so developer centric
07:55let's be agile let's do it fast our way
07:57you know to pizza is that's the ideal
07:59size of a software team and you know I'm
08:02not trying to mock it I'm just saying
08:03that isn't there an element of actual
08:05practical realities like technical debt
08:08and accruing a mess underneath all your
08:11code and a system that you may be there
08:13for two or three years and you can go
08:14after the next startup but okay
08:16someone else has to clean up your mess
08:17tell me about how this fits into that
08:18big picture this is what enables all of
08:21that oh right so so it's not actually
08:23just creating a problem cuz that's how
08:24I'm doing it no absolutely so you still
08:26need development you still need tests
08:28you still need QA you still need
08:30operations you still need to deal with
08:32technical debt you still need to deal
08:34with re architecting really difficult
08:36large monolithic code bases what this
08:39enables you to do is to find the
08:41problems address them quickly move
08:43forward I think that the problem that a
08:45lot of people have is that we're so used
08:47to couching these things as trade offs
08:50and as dichotomies the idea that if
08:52you're going to move fast you're gonna
08:54one thing which I always say is if you
08:56take one thing away from DevOps is this
08:58high-performing companies don't make
09:00those trade-offs they're not going fast
09:02and breaking things they're going fast
09:03and making more stable more high-quality
09:05systems and this is when the key results
09:07in the book in our research is this fact
09:10the high performers do better everything
09:11because the capabilities that enable
09:13high performance in one field if done
09:15right enable it in other fields so if
09:18you're using version control the
09:19software you should also be using
09:21version control for your production
09:22infrastructure if there's a problem in
09:24production we can reproduce the state of
09:26the production environment in a disaster
09:28recovery scenario again in a predictable
09:30way that's repeatable I think it's
09:32important to point out that this is
09:33something to happen in manufacturing as
09:34well give it to me I love when people
09:36talk about software as drawn from
09:38hardware analogies as my favorite type
09:39of metaphor okay so I've got so Titor
09:42didn't win by making shitty cars faster
09:44they won by making higher quality cars
09:47faster and having shorter time to market
09:49the lean manufacturing method which by
09:50the way also spawned Lean Startup
09:51thinking and everything else connected
09:53and DevOps pulls very strongly from from
09:55lean methodologies so you guys are
09:57probably the only people to have
09:59actually done a large-scale study of
10:00organizations adopting DevOps what is
10:03your research and what did you find
10:05sure so the research really is the
10:08largest investigation of DevOps
10:10practices around the world we have over
10:1123,000 data points all industries give
10:14me like a sampling like what are the
10:16range of it so I've got entertainment
10:18I've got finance I have healthcare and
10:20pharma I have technology government
10:23government education is in every
10:26vertical and then when you tell you
10:27around the world so we're primarily in
10:29North America uh-huh we're in amia we
10:31have India we have a small sample in
10:34Africa right just to quickly break down
10:36like the survey methodology questions
10:38that people have in the ethnographic
10:40world the way we would approach it is
10:41that you can never trust what people say
10:43they do you have to you have to watch
10:45what they do however it is absolutely
10:47true and especially in a more scalable
10:48sense that there are really smart
10:50surveys I gave you a ton of useful
10:51data yes and part 2 of the book covers
10:54this in almost excruciating detail we
10:57like knowing methodologies yes well and
10:59it's interesting because Jaz talked
11:01about in his overview of agile and how
11:03it changes so quickly and we don't have
11:05a really good definition what that does
11:07difficult to measure right and so what
11:09we do is we've defined core constructs
11:12core capabilities so that we can then
11:15measure them we go back to core ideas
11:18around things like automation process
11:20measurement lean principles and then
11:24I'll get that pilot set of data and I'll
11:26run preliminary statistics to test for
11:28discriminant validity convergent
11:30validity composite reliability make sure
11:33that it's not testing what it's not
11:35supposed to test it is testing what it
11:37is supposed to test everyone is reading
11:39it consistently the same way that I
11:41think it's testing I even run checks to
11:43make sure that I'm not inadvertently
11:45inserting bias or collecting bias just
11:48because I'm getting all of my data from
11:49surveys sounds pretty damn robust so
11:51tell me then what were the big findings
11:55that's a huge question but give me the
11:56hit list well okay so let's start with
11:58one thing that jazz already talked about
11:59speed and stability go together this is
12:02where he was talking about there not
12:03being necessarily a false dichotomy and
12:05that's one of your findings that you can
12:07actually accomplish both yeah and it's
12:08worth talking about how we measured
12:09those things as well so we measure speed
12:12or tempo as we call it in the book or
12:13sometimes people call it three per as
12:15well which is a nice full circle
12:16manufacturing idea like the
12:18semiconductor like circuit throughput
12:21I love Hardware now just for a told you
12:22a lot of it comes from leaner so lead
12:24time obviously one the classic lean
12:26manufacturing measures we use how long
12:28does it take we look at the lead time
12:29from checking in to version control to
12:31release into production so that part of
12:33the value stream because that's more
12:34focused on the DevOps end of things and
12:36it's highly predictable
12:37the other one is release frequency so
12:39how often do you do it and then we've
12:40got to stability metrics and one of them
12:42is time to restore so in the event that
12:44you have some kind of outage or some
12:46degradation in performance in production
12:48how long does it take you to restore
12:50service for a long time we focused on
12:52not letting things break and I think one
12:54of the changes paradigm shifts we've
12:56seen in the industry particularly in
12:57DevOps is moving away from that we
12:59accept that failure is inevitable
13:01because we're building complex systems
13:02so not how do we prevent failure but
13:04when failure inevitably occurs how
13:06quickly can we detect and fix it
13:08mtbf right mean time between failures if
13:10you only go down once a year but you're
13:12down for three days and it's on Black
13:14Friday but if you're down very small low
13:17blast very very small blast radius and
13:19you can come back almost immediately and
13:21summers almost don't notice that's fine
13:24the other piece around civility has
13:26changed fail rate when you push a change
13:27into production what percentage of the
13:29time do you have to fix it because
13:30something went wrong by the way what
13:31does that tell you if you have a change
13:33valve so in the lean kind of discipline
13:35this is called percent complete and
13:36accurate and it's a measure of a quality
13:38of your process so in a high quality
13:39process when I do something for Nicole
13:42Nicole can use it rather than sending it
13:45back to me and say hey there's a problem
13:46with this and in this particular case
13:48what essentially the time and I deploy
13:50something to production is there a
13:51problem because I didn't test adequately
13:53my testing environment wasn't in
13:55production like enough those are the
13:57measures for finding this but the big
13:58finding is that you can have speed and
14:00stability together through DevOps
14:03is that what I'm hearing high performers
14:05get it all low performers kind of suck
14:08it all of it medium performers hang out
14:09in the middle I'm not seeing trade-offs
14:11four years in a row so anyone who's
14:13thinking oh I can be more stable if I
14:15slow down I don't see it it actually
14:17breaks a very commonly held kind of
14:19urban legend around how people believe
14:21these things operate so tell me I do any
14:23other sort of findings like that because
14:25that's very counterintuitive okay so
14:26this one's kind of fun one is that this
14:29ability to develop and deliver software
14:31with speed and stability drives
14:32organizational performance now here's
14:35the thing I was about to say that's a
14:36very obvious thing to say
14:37so it seems obvious right developing and
14:40delivering software with speed
14:42instability drives things like
14:43profitability productivity market share
14:45okay except if we go back to Harvard
14:48Business Review 2003 there's a paper
14:50titled IT doesn't matter we have decades
14:54of research I want to say at least 30 or
14:5740 years of research showing the
14:59technology does not drive organizational
15:03it doesn't drive ROI and we're now
15:06starting to find other studies and other
15:08research that backs this up Erik
15:10Brynjolfsson of MIT JD that's out of
15:13Boston University 2017 did you say James
15:16Besson yeah oh I used to edit him - yeah
15:19it's fantastic here's why it's different
15:21because before right in like the 80s and
15:25the 90s we did this thing we're like
15:27you'd buy the tech and you'd plug it in
15:29and you'd walk away it was on Prem sale
15:31model where you like deliver and leave
15:32as opposed likes offers of service and
15:34other way things well and people
15:36complain if you try to upgrade too often
15:38nobody the keys that everyone else can
15:40also buy the thing and plug it in and
15:43walk away how is that driving value or
15:46differentiation for a company if I just
15:49buy a laptop to help me do something
15:51faster everyone else can buy a laptop to
15:54do the same thing faster that doesn't
15:56help me deliver value to my customers or
15:59to the market it's a point of parody not
16:02a point of distinction right and you're
16:04saying that point of distinction comes
16:05from how you tie together that
16:07technology process and culture through
16:08DevOps right and then it can provide a
16:11competitive advantage to your business
16:12if you're buying something that everyone
16:14else also has access to then it's no
16:16longer a differentiator but if you have
16:18an in-house capability and those people
16:20are finding ways to drive your business
16:21I mean this is the classic Amazon model
16:23they're running hundreds of experiments
16:25in production at any one time to improve
16:27the products and that's not something
16:29that anyone else can copy
16:30that's why Amazon keeps winning so what
16:32people are doing is copying the
16:33capability instead and that's what we're
16:35talking about how do you build that
16:36capability the most fascinating thing to
16:38me about all this is honestly not the
16:40technology per se but the organizational
16:42change part of it and the organizations
16:43themselves some of all the people you
16:45studied is there an ideal organizational
16:47makeup that is ideal for DevOps or is it
16:50one of these magical formulas that has
16:52this ability to turn a big company into
16:54a start-up and a small company into
16:56because that's actually the real
16:57question from what I've seen there might
16:59be two ideals the nice happy answer is
17:02the ideal organization is the one that
17:04wants to change that's I mean given this
17:06huge N equals 23,000 data set is it not
17:09tied to a particular profile of a size
17:12they're both shaking their head just for
17:13the nice and high performers among large
17:17companies I see high performers in small
17:19companies I see low performers in small
17:21companies I see low performers and
17:22highly regulated companies I see low
17:24performers in not regulated companies so
17:27tell me the answer you're not supposed
17:28to say so that answer is it tends to be
17:32companies that are like oh and
17:36their two profiles either one they're
17:38like way behind and oh and they
17:41have some kind of funds or they are like
17:45this lovely wonderful bastion of like
17:49they're these really innovative
17:50high-performing companies but they still
17:53realize there are a handful of like two
17:55or three companies ahead of them and
17:56they don't want to be number two they
17:58are gonna be number one those are so the
17:59ideal I mean just like anthropomorphize
18:01it a little bit it's like the 35 to 40
18:03year old who suddenly discovers you
18:05might be pre-diabetic so you better do
18:08something about it now before it's too
18:09late but it's not too late because
18:11you're not so old where you're about to
18:13reach sort of the end of a possibility
18:15to change that runway and then there's
18:17this person who's sort of kind of
18:19already like in the game running them
18:21the race and there might be two or three
18:23but they want to be like number one
18:24nothing to extend your metaphor the
18:26companies that do well are the companies
18:27that never got diabetic in the first
18:29place because they always just a
18:31healthfully like they were already
18:32glucose monitoring they had continuous
18:34glucose monitors on which is like DevOps
18:36actually they were always athletes right
18:38you know diets are terrible because you
18:40know at some point you have to stop the
18:41diet and it has a sudden start and stop
18:43as opposed to a way of life is weird
18:45right exactly so if you just always eat
18:46healthily and never eat too much or very
18:49rarely eat see much and do a bit of
18:51exercise every day you never get to this
18:53data oh my god no I cannot yeah so like
18:57my loving professor Ness nurturer Nicole
19:02also has one more profile that like I
19:05love and I worry about them like mother
19:08hen and it's the companies that I talked
19:10to and they come to me and they're
19:12struggling and I haven't decided if they
19:15want to change but they're like so we
19:17need to do this transformation and we're
19:19gonna do the transformation and it's
19:20either because they want to open they
19:22they've been told that they need to and
19:23then they will insert this thing where
19:25they say but I'm not a technology
19:27company I'm like but we just had this 20
19:31minute conversation about how you're
19:33leveraging technology to drive value to
19:35customers or to drive this massive
19:38process that you do yeah and then they
19:40say but I'm not a technology company I
19:42could almost see why they had that in
19:44their head because they were a natural
19:45resources company but there was another
19:46one where they were a finance company I
19:49mean an extension of software eats the
19:51world is really every company is a
19:52technology company it's fascinating to
19:55me that that third type exists but it is
19:58see world moving in two and I worry
20:01about them also at least for me
20:02personally you know I lived through this
20:05like mass extinction of several firms
20:07and I don't want it to happen again and
20:09I worry about so many companies that
20:11keep insisting they're not technology
20:12companies and I'm like oh honey child
20:14your tech company you know one of the
20:16gaps in our data is actually China and I
20:18think big China is a really interesting
20:20example because they didn't go through
20:22the whole you know IT doesn't matter
20:23phase they're jumping straight from no
20:25technology to Alibaba and 10 cent right
20:28I think US companies should be scared
20:31because yeah the moment $0.10 a nanny
20:33Barbara are already moving into other
20:35developing markets and they're going to
20:38be incredibly competitive because it's
20:39just built into their DNA so the other
20:40fascinating thing to me is that you
20:42essentially were able to measure
20:43performance of software and clearly
20:45productivity is there any more insights
20:47on the productivity side yes yes I want
20:49to go jumping around and like waving his
20:53hands so tell us the reason the
20:55manufacturing metaphor breaks down is
20:57because in manufacturing you have
20:58inventory yes we do not have inventory
21:01in the same way in software in a factory
21:04like the first thing your lien
21:05consultant is going to do walking into
21:06the factories points are the piles of
21:08thing everywhere but I think if you walk
21:10into an office where there's developers
21:12where's the inventory by the way that's
21:14what makes talking about this to
21:15executives so difficult they can't see
21:17the process well it's a hard question to
21:19answer because is the inventory the code
21:22that's being written and people actually
21:24have done that and said well listen
21:26lines of code or an accounting measure
21:27and we're going to capture that as you
21:30know I mean it seems like an invitation
21:32to write crappy unnecessarily long code
21:34that's exactly what it's like the olden
21:36days of getting paid for a book by how
21:37long it is and it's like actually really
21:38boring when you can actually write it
21:40like one writer the layman right you
21:41know you thinking of Charles Dickens in
21:43general you know you prefer people to
21:45write short programs because they're
21:46easier to maintain and so forth but
21:48lines of codes have all these drawbacks
21:50we can't use them as a measure of
21:51productivity so if you can't measure
21:53lines of code what can't you measure
21:54because I really want an answer like how
21:55do you measure productivity so velocity
21:57is the other classic example agile
21:59there's this concept of velocity which
22:01is the number of story points a team
22:05manages to complete in an iteration so
22:08before the start of an iteration in many
22:11sigelei scrum based processes you've got
22:14all this works to you like we need to
22:15build these five features how long will
22:17this feature take and the developers
22:18fight over it and they're like oh it's
22:20five points and then this one's gonna
22:21take three points it'll take two points
22:23and and so you have a list of all these
22:25features you don't get through all of
22:26them but at the end of the iteration the
22:28customer signs off will accepting this
22:29one this one's find this one's fine this
22:31one's a hot mess go back and do it again
22:33whatever the number of points you
22:34complete in the iteration is the
22:36velocity so it's like the speed at which
22:37you're able to deliver those features so
22:39a lot of people treat it like that but
22:41actually that's not really what it's
22:42about it's a relative measure of effort
22:44and it's for capacity planning purposes
22:46so you basically for the next iteration
22:48will only commit to completing the same
22:50velocity that we finished last item sets
22:51relative and it's team dependent and so
22:53what a lot of people do is say they
22:55start comparing velocities across teams
22:56then what happens is a lot of work you
22:59need to collaborate between teams but
23:01hey if I'm gonna help you with your
23:02story that means I'm not gonna get my
23:05story points and you're gonna get your
23:07story points right people can game it as
23:09well you should never use story points
23:11so lines of code doesn't work velocity
23:13doesn't work what works so this is why
23:15we like see things in particular one
23:18thing that it's a global measure and
23:19secondly that it's not just one thing it
23:22mixes two things together which might
23:24normally be in tension and so this is
23:26why we went for our measure of
23:29performance so measuring lead time and
23:32release frequency and then time to
23:35restore and change fail rate lead time
23:38is really interesting because lead time
23:39is on the way to production right so all
23:42the teams have to collaborate it's not
23:43something where you know I can go really
23:45fast in my velocity but nothing ever
23:47gets delivered to the customer that
23:48doesn't count in lead time care of that
23:50problem of the incentive alignment
23:52around the the competitive dynamic also
23:54exactly outcome it's not an output
23:57there's a guy called Jeff Patton he's a
24:00really smart thinker in their kind of
24:01lean agile space he says minimize output
24:05maximize outcomes which i think is
24:08simple but brilliant it's so simple
24:10because it just shifts that words to
24:11impact and even we don't get all the way
24:13there because we're not yet measuring
24:14did the features deliver the expected
24:16value to the organization or the
24:17customers well we do get there because
24:19we focus on speed and stability which
24:25come to the organization profitability
24:27productivity market share but the second
24:29half of this which I am also hearing is
24:31did it meet your expectations did it
24:34perform to the level that you wanted it
24:37to did it match what you asked for or
24:40even if it wasn't something you
24:41specified that you desired or needed
24:43that seems like a slightly open question
24:45so we did actually measure that we
24:47looked at nonprofit organizations and
24:49these are exactly the questions we
24:50measured we asked people did the
24:52software meet a quorum or the exactly
24:55effectiveness efficiency customer
24:56satisfaction delivery mission goals
24:59best thing that you do at nonprofits
25:01because that is a larger movement on
25:02profit measurement space that try to
25:03measure impact but we captured it
25:05everywhere even profit seeking firms
25:08still have these goals in fact as we
25:11know from research companies that don't
25:12have a mission other than making money
25:14do less well than the ones that do right
25:15but I think again what the data shows is
25:18the companies that do well on the
25:20performance measures we talked about
25:21outperform their low performing peers by
25:23a factor of two a hypothesis is what
25:26we're doing when we create these high
25:28performing organizations in terms of
25:29speed and stability is we're creating
25:31feedback loops what it allows us to do
25:34is build a thin slice a prototype of a
25:36feature get feedback through some UX
25:39mechanism whether that's showing people
25:41with a prototype and getting their
25:42feedback whether it's running a B tests
25:44or multivariate tests in production it's
25:46what creates these feedback loops that
25:47allow you to shift direction very fast I
25:49mean that is the heart of Lean Startup
25:51it's the heart of anything you're
25:54putting out into the world is you have
25:55to kind of bring it full circle it is a
25:57secret of success to Amazon as you cited
25:59earlier I would distill it to just that
26:01I think I heard Jeff Bezos say the best
26:03line it was at the internet Association
26:04dinner in DC last year where you come
26:07announcement about an innovation he's
26:08like to him an innovation is something
26:09that people actually use right and
26:11that's what I love about the feedback
26:12loop thing is it actually reinforces
26:13that mindset that's what innovation is
26:16right so to sum up the way you can frame
26:18this is DevOps is that technological
26:20capability that underpins your ability
26:22to practice Lean Startup
26:24and all these very rapid iterative
26:26processes so I have a couple of
26:28questions then so one is you know going
26:29back to this original taxonomy question
26:32and you guys described that there isn't
26:33necessarily an ideal organizational type
26:35which by the way should be encouraging I
26:38agree I think it's sue
26:39encouraging and more importantly
26:40democratizing that anybody can become a
26:43hit player we were doing this in the
26:45federal government I love that but one
26:47of my questions is when we had Adrian
26:49Cockcroft on his podcast a couple years
26:50ago talking about micro services and the
26:52thing that I thought was so liberating
26:54about what he was describing the Netflix
26:55story was that it was a way for teams to
26:59essentially become little mini product
27:02management units and essentially self
27:04organize because the infrastructure by
27:07being broken down into these micro
27:09pieces versus say a monolithic kind of
27:13uniform architecture I would think that
27:15being a you know organization that's
27:18containerized its code in that way that
27:19has this micro services architecture
27:21would be more suited to DevOps or is
27:25that a wrong belief I'm just gonna
27:27understand again that taxonomy think of
27:29how these pieces all fit together so we
27:31actually studied this as a whole
27:32sectional architecture in the book where
27:33we look exactly this question
27:34architecture has been studied for a long
27:36time and people talk about architectural
27:38characteristics there's the a time the
27:40architectural trade-off model that
27:41Carnegie Mellon developed there's some
27:43additional things we have to care about
27:45testability and deployability
27:47can my team test its stuff without
27:50having to rely on this very complex
27:52integrated environment can my team
27:54deploy its code to production without
27:56these very complex orchestrated
27:57deployments basically can we do things
27:59without dependencies that is one of the
28:01biggest predictors in our cohort of IT
28:04performance is the ability of teams to
28:06get stuff done on their own without
28:07dependencies on other teams whether
28:09that's testing or whether it's deploying
28:11or it's planning even just communicating
28:14yeah can you get things done without
28:16having to do like mass communication and
28:18checking and permissions question I love
28:20love love asking on this podcast is we
28:22always revisit the 1937 Coast paper
28:24about the theory of the firm and this
28:26idea that transaction costs are more
28:28efficient and this is like the ultimate
28:29model for reducing friction those
28:32transaction costs communication
28:34coordination costs all of it that's what
28:35like all the technical and process stuff
28:37is about that I mean done rynason once
28:39came to one of my talks on tins delivery
28:40at the end he said so continuous
28:42delivery that's just about reducing
28:43transaction costs right and I'm like an
28:46economist view of you know up your long
28:48white used my entire body of work to
28:51it's so much Conway's law right it's
28:54what remind me what kind of way so
28:55organizations which design systems are
28:57constrained to produce designs which are
28:59copies of the communication structures
29:01of these organizations all right it's
29:02that idea basically that your software
29:04code looks like the shape of the
29:05organization itself right and how we
29:06communicate right right so which you
29:09know just just summarized if you have to
29:11be communicating and coordinating with
29:13all of these other different man and
29:15controls looks like waterfall a more
29:17decentralized model looks like
29:18independent teams right so the data
29:20shows that one thing that I would say is
29:21a lot of people jump on the micro versus
29:24containerization bandwagon there's one
29:26thing that is very important bear in
29:27mind implementing those technologies
29:29does not give you those outcomes we
29:31talked about we actually looked at
29:32people doing mainframe stuff you can
29:34achieve these results with mainframes
29:36equally you can use that you know Kiva
29:39Nettie's and you know docker and micro
29:42services and not achieve these outcomes
29:43we see no statistical correlation with
29:46performance whether you're on a
29:47mainframe or a Greenfield or a
29:49brownfield system if you're building
29:51something brand-new or if you're working
29:52on existing build and one thing I wanted
29:55to bring up that we didn't before as I
29:57said you know day one is short day two
29:58is long and I talked about things that
30:00live on the internet and live on the web
30:02yeah this is still a really really smart
30:05approach for packaged software and I
30:07know people who were working in and
30:09running packaged software companies that
30:12use this methodology because it allows
30:14them to still work in small fast
30:17approaches and all they do is they is
30:18they push to a small package pre
30:21production database and then when it's
30:23time to push that code onto some media
30:26they do that okay so what I love hearing
30:29about this is that it's actually not
30:30necessarily tied again to the
30:32architecture or the type of company
30:33where there's opportunity for everybody
30:35but there is this mindset of like an
30:37organization that is ready it's like a
30:39readiness level for a company I hear
30:41that all the time I don't know if I'd
30:43say there's any such thing as readiness
30:45right like there's always an opportunity
30:46to get better there's always an
30:48opportunity to transform the other thing
30:51that really like drives me crazy and
30:54makes my head explode
30:55this whole maturity model thing okay are
30:58you ready to start transforming well
31:00like you can just not transform and then
31:03maybe fail right right
31:04thirty models they're really popular in
31:07industry right now but I really can't
31:09stress enough that they're not really an
31:10appropriate way to think about a
31:12technology transformation I was thinking
31:14of readiness and a kind of like NASA
31:15technology readiness levels or TRL's
31:17which is something we think about a lot
31:18for very early stage things but you're
31:21describing maturity of an organization
31:23and it sounds like there's some kind of
31:24a framework for assessing the maturity
31:26of an organization and you're saying
31:28that doesn't work but first of all what
31:29is that framework and why doesn't it
31:31work well so so many people think that
31:33they want a snapshot of they're like
31:36DevOps or their technology
31:37transformation and spit back a number
31:40right and then you will have one number
31:42to compare yourself against everything
31:44the challenge though is that a maturity
31:47model usually is leveraged to help you
31:50think about arriving somewhere and then
31:53here's the problem once you've arrived
31:55what happens oh we're done you're done
31:58and then the resources are gone and by
32:00resources I don't just mean money I mean
32:02time I mean attention we see year over
32:06year over year the best most innovative
32:08companies continue to push so what
32:11happens when you've arrived I'm using my
32:13finger quality stop pushing you stop
32:14pushing what happens when executives or
32:16leaders or whomever decide that you no
32:19longer need resources of any type I have
32:22to push back again though doesn't this
32:24help because it is helpful to give
32:26executives in particular particularly
32:28those that are not native coming from
32:30the seeds of the engineering
32:31organization some kind of metric to put
32:34your head around where are we where we
32:35at so you can use a capability model you
32:39can think about the capabilities that
32:41are necessary to drive your ability to
32:43develop and deliver software speed and
32:45stability and another limitation is that
32:47they're often kind of a lockstep or a
32:49linear formula right no right it's like
32:51a stepwise ABCDE one two three four and
32:55in fact the very nature of anything era
32:56t'v is it's very nonlinear and circular
32:58feedback loops are circled right or
33:01maturity models just don't allow that
33:03now another thing that's really really
33:05nice is that capability models allow us
33:08to think about capabilities in terms of
33:10these outcomes capabilities drive impact
33:14maturity models are just this thing
33:17you have this level-1 level-2 level-3
33:19little bit more affirmative
33:21and then finally maturity models just
33:24sort of take this snapshot of the world
33:27and describe it how fast is technology
33:31and business changing if we create a
33:33maturity model now let's wait let's say
33:36four years that maturity model is old
33:38and dead and dusty and gone do new
33:41technologies change the way you think
33:43about this because I've been thinking a
33:44lot about how product management for
33:45certain types of technologies changes
33:47with the technology itself and that
33:48machine learning and deep learning might
33:50be a different beast and I'm just
33:52wondering if you guys have any thoughts
33:52on it yeah I mean me and I Farley wrote
33:54the continuous delivery book back in
33:562010 and since then you know there's
33:58docker and communities and large-scale
34:01dosh and the cloud and all these things
34:02that you had no idea what happened
34:03people sometimes ask me you know isn't
34:06it time you wrote in your addition at
34:07the work I mean yeah we could probably
34:08rewrite it does it change any of the
34:11fundamental principles no do these new
34:13tools allow you to achieve those
34:15principles in new ways yes so I think
34:18this is how I always come back to any
34:20problem is go back to the first
34:21principles the first principles I mean
34:24they will change over the course of
34:25centuries and we've got modern
34:27management vs. and kind of scientific
34:28management but they don't change over
34:31the course of like a couple of years the
34:33principles are still the same
34:34technologies give you new ways to do
34:36them and that's what's interesting about
34:37them equally things can go backwards a
34:40great example of this is one of the
34:41cases we talk about in the book is
34:43working off a shared trunk or master in
34:46version control not going on these
34:47long-lived feature branches and the
34:49reason for that is actually because of
34:53you know developers love going off into
34:54a corner putting headphones on the head
34:56and yes coding something for like days
34:58and then they try and integrate it into
35:00trunk you know and that's a total
35:01nightmare and not just for them more
35:03critically for everyone else who then
35:04has to merge their coding so ever
35:07so that's he usually painful git is one
35:09of these examples of a tool that makes
35:10it very easy some people like I can use
35:12feature branches so I think again it's
35:14nonlinear in the way that you describe
35:15gives you new ways to do things
35:17are they good and bad it depends but the
35:19thing that strikes me about what you
35:20guys have been talking about it as a
35:21theme in this podcast that seems to lend
35:24itself well to the world of machine
35:25learning and deep learning where that
35:27technology might be different is it sort
35:29of lends itself to a probabilistic way
35:31and that things are not necessarily
35:34yeah and that this is not a beginning
35:36and an end and that you can actually
35:38live very comfortably in an environment
35:40where things are by nature complex and
35:42that complexity is not necessarily
35:43something to avoid so in that sense I do
35:46think there might be something kind of
35:47neat about ml and deep learning and AI
35:49for that matter because it is very much
35:51lending itself to that sort of mindset
35:53yeah in our research we talked about
35:55working in small batches there's a great
35:57video by Brett Victor called inventing
36:00on principle where he talks about how
36:02important it is to the creative process
36:04to be able to see what you're doing and
36:06he has this great demo of this game he's
36:08building where he can change the code
36:10and the game changes its behavior
36:11instantly when you're doing things like
36:13no and the whole thing with machine
36:15learning is how can we get the shortest
36:17possible feedback from changing their
36:19input parameters to seeing the effect so
36:21that the machine can learn and that the
36:23moment you have very long feedback loops
36:25the ml becomes much much harder because
36:27you don't know which of the input
36:29changes caused the change in output the
36:31machine is supposed to be learning from
36:32so the same thing is true of
36:33organizational change and process and
36:35product development as well by the way
36:37which is working in small batches so
36:39that you can actually reason about cause
36:41and effect you know I changed this thing
36:43it had this effect again that requires
36:45short feedback loops that require small
36:47batches that's when the key Heyford as
36:48we talked about in the book and that's
36:49what devops enables so we've been this
36:51hallway style conversation around all
36:52these themes of DevOps measuring it why
36:55it matters and what it means for
36:56organizations but practically speaking
36:58if a company and you guys are basically
37:01arguing at any company not necessarily a
37:03quote company that thinks it's a tech
37:05company and necessarily a company that
37:07has like this amazing modern
37:09infrastructure stack it could be a
37:10company that's all working off
37:11mainframes what should people actually
37:12do to get started and how do they know
37:14where they are so what you need to do is
37:16take a look at your capabilities
37:18understand what's holding you back right
37:20try to figure out what your constraints
37:22are but the thing that I love about much
37:24of this is you can start somewhere and
37:28culture is such a core important piece
37:31we've seen across so many industries
37:34culture is truly transformative and in
37:36fact we measure it in our work and we
37:38can show that culture has a predictive
37:40effects on organizational outcomes and
37:42technology capabilities we use a model
37:44from a guy called Ron westrom he was a
37:47social scientist studying safety
37:49outcomes in fact in safety critical
37:51industries like healthcare and aviation
37:53he greatest a typology where he
37:55organizes organizations based on with
37:58their pathological bureaucratic or
38:00generative especially agreed topology I
38:02want to apply that to people I date I
38:03know right I'm trying to
38:10anthropomorphize all these
38:11organizational things into people but
38:13anyway yes instead of thirty five love
38:15languages three relationship types so
38:18pathological organizations are
38:19characterized by a low cooperation
38:21between different departments and up and
38:23down the organizational hierarchy how do
38:25we deal with people who bring us bad
38:26knees to be ignore them or do we shoot
38:28people who bring us bad knees how do we
38:30lower the responsibilities are they
38:31defined tightly so that when something
38:33goes wrong we knows whose fault it is so
38:35we can punish them or do we share risks
38:37because we know we're all in this
38:40exactly how do we do with bridging
38:42between different departments and
38:43crucially how do we do with failure as
38:45we discussed earlier in any complex
38:47system including organizational systems
38:49failure is inevitable so failure should
38:52be treated as a learning opportunity not
38:54whose fault was it but why did that
38:55person not have the information they
38:57needed the tools they needed how can we
38:59make sure that when someone does
39:01something it doesn't lead to
39:02catastrophic outcomes but instead it
39:04leads to contain small blast radiuses
39:06not an outage on Black Friday right
39:08exactly and then also how do we deal
39:11with novelty is novelty crushed or is it
39:13implemented or does it lead to problems
39:14one of the pieces of research that kind
39:16of confirms what we were talking about
39:18with some research that was done by
39:19Google they were trying to find what
39:21makes the greatest Google team you know
39:23is it for stem for graduates and no
39:25developer and fire all the managers is a
39:27data scientist and an ojs programmer and
39:30a yeah manager right one product manager
39:32paired with one system engineer with one
39:34yep and what they found was the the
39:36number one ingredient was psychological
39:39safety does the team feel safe to take
39:43risks and this ties together failure and
39:45novelty if people don't feel that when
39:48things go wrong they're going to be
39:50supported they're not going to take
39:52risks and then you're not going to get
39:53any novelty because nobody by definition
39:55seyton risks to say we see that one of
39:58the biggest things you can do is create
40:00teams where it's safe to go wrong and
40:02make mistakes and where people will
40:04treat that as a learning experience this
40:05is a principle that applies again not
40:08just in product development you know the
40:09Lean Startup fail early fail often but
40:11also in the way we deal with problems an
40:13operational level as well and how we
40:15interact with our team when these things
40:16happen so just to kind of summarize that
40:18you have pathological this is a power
40:20oriented thing where you know the people
40:22are scared the messenger is gonna be
40:23shot then you have this bureaucratic
40:25kind of rule oriented world where the
40:28messengers aren't hurt and then you have
40:30this sort of generative and like again I
40:32really wish I could apply this at people
40:34but we're talking about organisations
40:35here for culture which is more
40:38performance oriented and I just want to
40:39add one thing about this you know
40:41working the federal government you would
40:42imagine that to be a very bureaucratic
40:43organization word actually and actually
40:46what was surprising to me was that yes
40:47there's lots of rules the rules aren't
40:49necessarily bad that's how we can
40:51operate at scale is by having rules but
40:53what I found was there was a lot of
40:54people who are mission oriented and I
40:55think that's a nice alternative way to
40:57think about generative organizations is
40:59to think about mission orientation the
41:01rules are there but if it's important to
41:03the mission break the rules and we
41:05measure this at the team level right
41:07because you can be in the government and
41:09there were pockets that were very
41:10generative right you could be in a
41:12start-up yeah and and you can see
41:14startups that were very bureaucratic
41:18logical right and the CEO where it's not
41:22charismatic inspirational vision but to
41:24the expense of actually being heard and
41:26the messenger is shot at cetera and we
41:28have several companies around the world
41:30now that are measuring their culture on
41:32a quarterly cadence and basis because we
41:34show in the book how to measure it
41:35westerns typology was the table itself
41:38and so we turned that into a scientific
41:40psychometric way to measure it now this
41:42makes sense why I'm putting these
41:43anthropomorphic analogies because in
41:46this sense organizations are like people
41:48they're made of people teams or organic
41:50entities and I love that you said that
41:51the unit of analysis is a team because
41:53it means you can actually do something
41:55you can start there and then you can
41:56like see if it actually spreads or
41:57doesn't spread bridges doesn't bridge
41:59etc and what I also love about this
42:01framework is it also moves away from
42:04this cult of failure mindset that I
42:05think people tend to have where it's
42:07like failing for the sake of failing and
42:09actually want to avoid failure right and
42:11the whole point of failing is to
42:12actually learn something and then be
42:14better and take risks so you can
42:16implement and very certain risks so
42:19what's your final I mean there's a lot
42:20of really great things here but like
42:23what's your final sort of parting
42:25takeaway for listeners or people who
42:26might want to get started or think about
42:28how they are doing so I think you know
42:31we're in a world where technology
42:32matters anyone can do this stuff but you
42:34have to get the technology part of it
42:36right that means investing in your
42:38engineering capabilities in your process
42:41in your culture in your architecture we
42:43dealt with a lot of things here that
42:45people think are intangible and we're
42:46here to tell you they're not intangible
42:47you can measure them they will impact
42:49the performance of your organization so
42:51take a scientific approach to improving
42:53your organization and you will read the
42:55dividends when you guys talk about you
42:57know anyone can do this that teams can
42:58do this but what role in the
43:00organization is usually most empowered
43:01to be the owner of where to get started
43:04is it like the VP of engineering is it
43:06the CTO the CIO I was gonna say don't
43:09minimize the role of and the importance
43:12of leadership DevOps sort of started as
43:14a grassroots movement but right now
43:16we're seeing roles like VP and CTO being
43:20really impactful in part because they
43:22can set the vision for an organization
43:24but also in part because they have
43:26resources that they can dedicate to this
43:28we see a lot of CEOs and CTOs and CIOs
43:30in our business we have like a briefing
43:32Center we hear with top of mind for them
43:34everyone thinks they're transformational
43:36so like what actually makes a visionary
43:38type of leader who has that not just the
43:40purse strings and the decision-making
43:41power but the actual characteristics
43:43that are right for this right and that's
43:46such a great question and so we actually
43:48dug into that in our research and we
43:49find that there are five characteristics
43:50that end up being predictive of driving
43:53change and really amplifying all of the
43:56other capabilities that we found and
43:58these five characteristics are vision
44:00intellectual stimulation inspirational
44:03communication supportive leadership and
44:05personal recognition and so what we end
44:08up recommending to organizations is
44:10absolutely invest in the technology also
44:13invest in leadership in your people
44:15because that can really help drive your
44:18transformation home well Nicole
44:19Jezz thank you for joining the a 6nz
44:23book just out is accelerate building and
44:26scaling high-performing technology
44:27organizations thank you so much you guys
44:29thanks for having us thank you
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
