00:00I would like to introduce Jared Frieden
00:02my partner and his esteemed panel who he
00:04will introduce to talk about technology
00:06thank you thank you Jeff okay well I am
00:14super lucky to have a very esteemed
00:17group of guests with me here today the
00:19everyone on this panel is is a tactical
00:22founder of a really successful company
00:25everyone here has built like a really
00:26cool company in a really amazing
00:28technology organization we've changed
00:33the name of the event for today it was
00:35originally called a CTO advice and on
00:37the advice of Lillian thank you so much
00:39we have changed it to tactical founder
00:41advice which i think is a better
00:42description and I'd also like to thank
00:46everyone on the startup school forum who
00:48wrote in with questions for the panel we
00:51posted a few weeks ago and solicited
00:53questions we got over 150 responses so
00:54thank you everyone who wrote in with
00:57those questions we're gonna do our best
00:58to cover as many of them as we can and
01:00then at the end we'll open it up to the
01:02in-person audience or some questions as
01:03well okay so let's get started could we
01:09start by having everyone introduce
01:10themselves and tell us about your
01:13company and about your technology like
01:19what's your tech stack what's some of
01:21the interesting technology that you
01:23build how does your technology
01:25organization look like today oh you
01:29monster hello everyone my name is Ralph
01:32I'm CTO and co-founder of plain grid
01:34clean grid were 350 people based out of
01:37Mission San Francisco we write beautiful
01:40easy-to-use software for the 17 trillion
01:42dollar construction industry so what
01:45that looks like to you the analogy often
01:47uses we're like github for construction
01:49construction has blueprints blueprints
01:51change rapidly version controls
01:53extremely important if you have changes
01:55that means there's issues that are
01:56happening issues need to be tracked and
01:58then we build collaboration tools the
02:00top of that too as well as a lot of
02:01other tools for the construction
02:02industry that go into some deep jargon
02:05our stack we're you know based on AWS we
02:09used to be based on a variety of other
02:10things we had to move everything into
02:12over time on our back end mostly Python
02:14on the back end that we've got some go
02:15and other things and then one of our
02:17challenges is we actually write native
02:19for every platform so we've got iOS with
02:22objective-c Android all Java and Kotlin
02:25the web reacts and then windows we have
02:27a full Windows app as well which has
02:29done that hi everyone I'm Calvin French
02:35Owen and CTO and co-founder of segment
02:38segment is a single API to collect
02:41organize and adapt all of your customer
02:45data into hundreds of downstream tools
02:47you might be using whether that's an
02:49analytics tool like Google Analytics or
02:51Mixpanel maybe a customer success tool
02:54like gain site maybe an email tool like
02:57customer i/o segments helps you collect
02:59data from where it lives and get it
03:02where it needs to be in terms of the
03:05company overall we're a little over 300
03:06people right now we have our
03:08headquarters in San Francisco but we
03:10also have offices in Vancouver New York
03:12and Dublin and the engineering product
03:15and design team which is kind of how we
03:17build product here at segment is a
03:19little over 80 or 90 right now in terms
03:24of our tech stack we're also built
03:25entirely atop of AWS in terms of how we
03:29run our infrastructure we run on top of
03:32ECS Amazon's container service and
03:34pretty much all of our different
03:35services our containerized today we're
03:37running about 300 different micro
03:39services which are kind of piping
03:41together various kafka topics and
03:43reading and writing transforming this
03:45data getting it to where it needs to go
03:47and our back-end is primarily written in
03:49go good morning everyone so my name is
03:54diana who i was the founder and CTO for
03:58SEO reality and we're building the
04:00backend for augmented reality and i say
04:02i was because my company just got
04:04acquired by Niantic the makers of
04:06pokemon go so what we were building an
04:09Escher and now we actually continue
04:10doing it in Niantic is building this
04:14back-end technology for augmented
04:17reality to enable developers to build a
04:20our experiences as easy as possible so
04:23we handle all the complexity with the
04:26rhythms with all the interaction with
04:28the backend all the hard parts of
04:30getting things to render so that
04:32developers could build easy a our apps
04:34in minutes so that's what we do and we
04:38provide a lot of advanced they are
04:40features like multiplayer persistence
04:44so that is easy for developers are just
04:46developing and let's say on unity and it
04:49just works so in terms of tech stack has
04:53been a bit of an adventure back at Asher
04:55we were we had a service that was hosted
04:58in AWS but that didn't matter too much
05:01because we had everything in docker
05:02containers now a Niantic is I think is
05:05heavy user of Google Cloud so move
05:07everything to Google Cloud but we have a
05:10lot of the bulk of the code is actually
05:12for us native means more C++ literally
05:15because we is a lot more efficient for
05:18us to write the code and a lot of the
05:19algorithms once and then cross compile
05:22it across all the architectures for
05:24Android or iOS and even the back end so
05:27that we run some of the CVL rails we
05:29could prototype it in the phone and then
05:31we could easily move them into the
05:33server because our service actually also
05:35written in C++ hi everyone I'm Lillian I
05:41am CEO oh not CTO of second measure so
05:47second measure we're about 50 people we
05:50got started back in 2015 and we're based
05:52in San Mateo what we do is we analyze
05:55credit card data so basically we take
05:58billions of credit card transactions and
06:00try to build a daily view into the
06:03public and private company performance
06:04we analyze these billions of
06:07transactions automatically every day and
06:10just them clean them enrich normalize
06:12them and then service that in a
06:15front-end application for our clients
06:18which include VC firms hedge funds and
06:21big brands like blue apron or Spotify to
06:24check trends do any sort of competitive
06:28intelligence or customer intelligence so
06:31we actually don't have a CTO so that's a
06:34fun fact about us and that's why I'm
06:38and myself and my co-founder actually
06:40both technicals that's been really
06:44pretty fortunate we don't have to deal
06:46with as many of the challenges around
06:48starting with non-technical founders so
06:53our I mentioned that our team is about
06:5550 people primarily technical so our
06:59technical organization is about 30
07:00people and that's actually split evenly
07:02between data scientists and engineers
07:04and that's probably something that makes
07:07us unique is that our core product is
07:10actually the data itself so our data
07:12scientists and engineers have to
07:14collaborate super closely on a day to
07:17day basis and probably one of the things
07:20that has technically been interesting is
07:23a lot of our users want to explore and
07:27dive deep into our data and building a
07:30front-end that allows that flexibility
07:33of exploration while still creating a
07:36good user experience basically requires
07:38us to figure out how to rapidly query
07:41data and run really complex queries on
07:45yes about tech stock we're also
07:48primarily on AWS so for our pipeline we
07:51leverage services like lamda and spark
07:54and then for our front-end we react
07:56based and leveraged column columnar data
07:59stores on the backend to service queries
08:02so I think redshift awesome guys that
08:06was super cool it's so cool the
08:08diversity of different kinds of
08:10technology organizations and products
08:13that are here today okay so most of the
08:17company is in start-up school are really
08:19early so I want to take everyone here
08:21back to their v1 the very first version
08:25before you had actually shipped anything
08:27tell us the story of how you built your
08:30v1 who built it how did you built it how
08:34long did it take you to build it and
08:36what things went wrong in the process I
08:40guess I'll start this is a great story
08:43I've actually got my CEO and co-founder
08:45over there watching me right now and um
08:48so that's company hi Tracey yeah hey
08:50Tracey how's it going
08:51so that's that's where we started Tracy
08:53told me about her idea of putting
08:54blueprints on an iPad this was in 2011
08:56right at the launch of the iPad so it
08:58was very early for the technology and I
09:01said yeah no problem I could put
09:02blueprints on my PI it's like a PDF
09:04that's no problem at all
09:05so I was attempting to impress her
09:07really quickly by putting blueprints on
09:09an iPad and it turns out there was
09:10actually some hard core challenges with
09:12putting these giant images on a really
09:14low computer availability on the iPad
09:18there's no graphics chip at the time so
09:20basically these are 20,000 pixel images
09:23and they would just crash or overrun the
09:25video memory and they're in PDF which is
09:27a kind of painful format to deal with so
09:29the first prototype sucks um is really
09:31slow and that actually told me that
09:33there's something real meaty here
09:34there's something that's actually a
09:35challenge and it took me about a month
09:37to write the second prototype based off
09:40some my background actually I used to
09:42work at Pixar so I had some graphics
09:44experience there didn't use any
09:46proprietary stuff but use some
09:48off-the-shelf computer graphics
09:49knowledge to write the first blueprint
09:52viewer that ever ran on mobile so that
09:54was our first prototype back then and I
09:56think the best thing I remember back on
09:58it is um the only thing we really
10:00focused on the first prototype was what
10:02was technically impossible there was no
10:04way you could load blueprints onto an
10:05iPad at the time and that was our
10:07prototype which meant that all the data
10:09got there by side loading which meant
10:11there was no web interface and the
10:12little crappy web interface we had had a
10:14delete button right next to a publish
10:16button that was not a pixel apart and
10:18there was no confirmation on the delete
10:20button so we actually as a team we
10:24managed everything on the back end we
10:26kind of like behind the curtains loaded
10:28all the documents for our first 30 or 40
10:30customers but they all they saw was the
10:32the pot the prototype that was fast and
10:34did what they thought it would they
10:35never saw the man behind the curtain so
10:38to speak so that's all the story of our
10:39first prototype cool um I'll share a
10:44little bit of a story of our v1 and then
10:46maybe our v1 next I think for those of
10:50you has seen the startup school talk
10:51where my co-founder Peter talks about
10:53finding product market fit he goes into
10:55this story and way more detail than I
10:57could today but I can share a little bit
10:59of our journey so when we first started
11:04oh I see 2011 class which sounds like a
11:07dinosaur now in terms of YC years yeah
11:11but at the time we were actually
11:13building something completely different
11:14from segments today we were building
11:17this classroom lecture tool which would
11:19help professors get feedback about what
11:22their students were thinking in the
11:24middle of class we were students at the
11:26time we figured hey like this is seems
11:28like a great ideas professors will
11:30sometimes go on for five minutes and
11:32they'll lose the entire class and the
11:34class gets confused and just waste
11:35everyone's time let's solve that problem
11:36we built it out over the course of the
11:39summer and we deployed it back in Boston
11:42in the fall and the whole thing just
11:44kind of crashed and burned
11:45like students would go to Facebook
11:47YouTube Wikipedia like in retrospect all
11:51the things that you would assume college
11:52students would do if they have a laptop
11:53in front of them in a lecture but we
11:56didn't quite see it that way
11:57so we started looking around for a new
11:59idea because we've just raised a seed
12:01round and we say okay what are the
12:05problems that we have with this college
12:06lecture tool and one of them was that we
12:09couldn't answer questions about how
12:11people were using our tool we can tell
12:14you how a college students at Harvard
12:16we're using the tool differently than
12:18students at MIT or we couldn't tell you
12:20how anthropology classes used it
12:22differently than biology classes and so
12:25we switched again to something which
12:26also we don't do today but effectively
12:29it was a competitor to Mixpanel or
12:32amplitude is this tool that would allow
12:34you to cluster your users by different
12:36rules and break them up into segments
12:39which is where the name came from and
12:41from there you could then reach out to
12:43them or figure out what you're most
12:44engaged users were doing and so we built
12:47out that system for about 15 months the
12:51whole time we were doing revs on the
12:52backend infrastructure we were building
12:54it out and we're trying to get users and
12:56no one wanted our tool and so we arrived
13:00fifteen months later we actually moved
13:02back to the Bay Area after living in
13:04Boston for a year we go and talk to PG
13:07and we tell them about kind of our whole
13:10saga that's been running for the past
13:13and you listen to our story and he
13:15listens to everything that we've done
13:16and when we're finished he just sort of
13:20pauses I think it was maybe 400 feet out
13:22there walking around the roundabout he
13:24says wow so you just burned half a
13:27million dollars and you're still at
13:29square one they're just like yes he said
13:34well on the plus side you still have
13:35some runway left so try again like want
13:37something new and at the time we had
13:40this internal tool that we had been
13:41using as kind of a growth hack to get
13:43ourselves more users and the whole idea
13:45was that you would use the same API to
13:47send us data as you would send to
13:49Mixpanel KISSmetrics Google Analytics
13:51all these other tools and people
13:53actually liked that single API and they
13:55were contributing to this open source
13:57library and github and starring it and
13:59using it and so my co-founder Ian said
14:01hey why don't we turn this into a
14:03why don't we launch this and see what
14:05happens and my co-founder Peter said oh
14:07that's the worst idea I've ever heard it
14:09will never work he's CEO now but he's
14:12seen the light and so I he said okay
14:16I've got to figure out a way that we can
14:17test this idea incredibly quickly so we
14:20cleaned up the library and we launched
14:21it on Hacker News and to our surprise it
14:23actually got about a thousand stars that
14:25first day I'm github and so our real v1
14:28was that first week taking that library
14:31and we basically just had a landing page
14:33up which said hey if you're interested
14:35in a hosted version of this leave your
14:37email and about 500 people left their
14:39email and we built out that v1 over
14:42about a week where just everyone was in
14:44Mad hackathon mode and building out the
14:47product which we launched a week later
14:49and so today I think there's actually no
14:52parts of that product which still live
14:54on today but it was enough to get us
14:57kind of the seed of product market fit
14:59and then start getting us a user base so
15:03one week is the short one week one week
15:06everybody hear that one week ok that's
15:10kind of amazing one week sir story is
15:13definitely more than one week so me and
15:17my co-founder have been thinking of
15:18building different things and one of the
15:21technology trends that we really saw was
15:24it comes from a robotics background with
15:27a lot of experience with slam slam is
15:29this album called simultaneous
15:31localization and mapping and it's one of
15:32the core algorithms that run today in AR
15:34which helps tell where your camera
15:38position is while you also build this
15:40map of the world so what happened is
15:42there's this category of algorithms that
15:44have been used mostly for robotics now
15:47AR had been kind of tried and many times
15:52but never really being picked up there's
15:54AR with markers and that never really
15:57took off and then we thought about why
15:59do we bring these algorithms and run
16:01them on the phones and we got to the
16:03point where we thought we could do it
16:05because the computation at that point
16:07with phones for example for iPhone 7 has
16:11the same competition actually as a
16:13MacBook Pro from 2013 so that's you
16:16think about that that's kind of amazing
16:17in terms of all the trends with Moore's
16:19law and some of the ones with power
16:22efficiency and compute so we decided to
16:23go do it so we were at that time I was
16:26actually still working and I was leading
16:29a data science team back then and say
16:31okay I'll do it because I was thinking
16:32of doing some transition and I was
16:34working part time with another engineer
16:36and this one of the engineers that that
16:39we kind of got excited to work with us
16:40so we struggle a lot to try to get a
16:44version working the first was very duct
16:46tape and it wasn't very encouraging it
16:48was only working at five frames per
16:50second which is terrible because if you
16:53want a good AR experience the whole
16:55thing is that you need it to render
16:56nicely and be at least 30 frames per
16:58second so that was very discouraging in
17:03a sense and then because it was all kind
17:04of put together this and then we started
17:06kind of really removing all the research
17:08code out and then it started working and
17:12seeing a lot of promise and it was
17:13running then at at least 30 frames per
17:15second and note that when we were
17:18working on this was at least about a
17:20year and a half before air kid had
17:22launched and then we were excited about
17:25this so we were both very more from a
17:28technology side so we don't know product
17:30market fit way to go with this so we had
17:34to find the market so the exercise we
17:36did is actually we went in an interview
17:38people that we thought would use it and
17:41we found a really good for it with
17:42gaming so we decided to focus on that
17:46YC application and the thing that really
17:48took us and this took off and this is
17:50actually I guess a good story too is the
17:53first day of YC Apple announced air kit
17:56which is basically what we had built and
17:58that was sort of a moment of truth for
18:00us it's sort of this really moment where
18:03you are looking at yourself and it is
18:05really go flight-or-fight so we decided
18:10why not let's just take on Apple so we
18:13decided to just double down and we had
18:15this idea that the start wasn't just on
18:18the device which we had all our demos
18:20and Technology was just working on the
18:22phone we didn't have anything on the
18:23backend yet so back in YC we got it
18:26working with a back-end and also with
18:28Android and basically accelerating the
18:30roadmap that we thought it was about a
18:31year shrink it and got it done in three
18:33months and that was the thing that once
18:37we put it out there and let people sign
18:40up if they were interested we go got
18:42about a thousand developers signed up in
18:44about a week so they find okay I think
18:47we found something and then after that
18:50we got a lot of interest and among those
18:52Niantic was one and the story is we got
18:56so how long was it from the time that
18:59you started working on the like original
19:02slam algorithms until you had it like a
19:04working product in users hands so we
19:07were not really working full time it was
19:10probably about a year of part time and
19:15even by the time was part time wasn't
19:18really working so when full time I took
19:21another six months of just me and
19:22another engineer to do it and then
19:25during the summer YC we hired two more
19:28engineers and that's when we were able
19:30to really accelerate and get it working
19:33well cool so for us our v1 probably took
19:40assuming full-time two or three months
19:43to build and I actually would say that
19:46most of that time was just figuring out
19:47what our product was going to be we knew
19:52that we were working on is gonna be
19:53really interesting initially we were
19:55focused on investors basically trying to
19:58get them an information edge on sort of
20:00any investment decisions that they're
20:02making whether that be hedge funds focus
20:05on public markets or VCS focus on
20:07private markets we ran through a few
20:11ideas we're like hey do these investors
20:14want predictions we they just want to
20:16know how like this company public
20:18company's quarterly sales are gonna hit
20:21and the short answer is yes but but no
20:25that doesn't really seem like a really
20:28sort of sustainable business model so
20:31then we shift it a little bit like okay
20:33maybe these investors really really want
20:35to go deep and they just want to cut
20:38data any way they want so we put
20:41something together really quickly put
20:43that in front of some of our friends
20:44your investors and found out very
20:47quickly through some white user testing
20:49that that was way too overwhelming our
20:52users wanted some guidance so what
20:54ultimately landed on is we needed to
20:56build our own self-service platform
20:58where we can apply our opinions around
21:02how our customers should be viewing this
21:05data and getting quick access to
21:07insights so once we figure that part out
21:10I was actually pretty quick so my
21:13co-founder focused on building the
21:16analytics product piece so the the front
21:20end application and I'd say you know and
21:23together we kind of focus together we
21:25built sort of the data pipeline and sort
21:27of the transformations on the data but I
21:30think some important choices that we
21:32made early on is and maybe some that you
21:36guys are facing right now is what
21:38technology do you want to use like what
21:40do you want to build this in ultimately
21:42we decided to build our first
21:43application in groovy using the Grails
21:46framework which is very not shiny and
21:49not that many people are doing that but
21:51that was actually the code that we had
21:54the most experience in so my co-founder
21:57Mike had built large production scale
21:59system servicing hundreds of thousands
22:02of concurrent users and pretty much
22:05the bones of the application and kind of
22:08like some of these other stories here
22:10like none of us it sounds like are
22:11running rv1 anymore so it's more
22:16important in the early days to be able
22:18to iterate quickly and usually that
22:21comes from using the technology that you
22:23know the best building things in a
22:26modular way such that once you actually
22:28find that product market fit and know
22:31what your audience needs then you can
22:34focus on that area and actually select
22:35tech technologies that are best fit for
22:38that problem awesome
22:44okay so I'm gonna go next to the very
22:47top voted question from the startup
22:48school forum which is about the
22:51trade-off between engineering best
22:53practices like good test coverage and
22:56security and scalability and he done and
23:00see versus writing code as quickly as
23:04possible and shipping something so can
23:08everyone talk about how you made that
23:12trade-off for your v1 and then how it's
23:14evolved since then and sort of like the
23:16timetable of its evolution as your
23:19company grew and those things became
23:20presumably more important and just to
23:22mix it up let's try flipping the order
23:24and we'll start with Lilly in this time
23:28so speed is paramount nobody's gonna pay
23:32you for having excellent test coverage
23:34so in the early days the it's it's you
23:39definitely want sort of the bit of like
23:41what you need right so it was that's the
23:43risk to your business around sort of
23:45security and not having things working
23:47and obviously you want your whatever you
23:50build to run every day you don't want to
23:53be fixing it breaking all the time and
23:56not actually building but really it is a
23:59balance and I do believe that initially
24:03it's more important to have that speed
24:07of development and incorporate
24:09constantly testing and incorporating the
24:12learnings into your product than it is
24:14to have like the most robust
24:18most scalable product great you're
24:23trying to find something that people
24:25will pay you money for ourselves a real
24:27problem and that takes time and a lot of
24:29iteration that has definitely evolved
24:32since we got started you know once you
24:35find product market fit a lot of these
24:37problems change you know your system
24:39gets more complex your team grows you
24:42have a lot more people contributing code
24:44a lot more ways for your systems to
24:45break a lot more users who are relying
24:48on your product so for us the way that
24:51manifested itself is yeah we started
24:54writing unit tests to verify each
24:56engineer would be verifying that you
24:58know their code works then we introduced
25:00CIN CD to make sure that that could work
25:03with the whole system our director of
25:07engineering who leads our engineering
25:08organization started developing and
25:11formalizing our best practices around
25:14like code reviews testing etc in
25:20defining those processes and then
25:22finally building some more controls
25:24directly into our system to make it
25:26harder to break things and specifically
25:30like what was the rough timeline for
25:32those things like are we talking about
25:33like two months after launch two years
25:37so I'd say most of that stuff I was just
25:43talking about happened probably the last
25:46year to a year and a half so and that's
25:49how many years after about a year and a
25:52half after launch and the main reasons
25:56for that again were the growth and the
25:58complexity of our systems and you know
26:00our techno are technical staff tripled
26:03in the last year so just that alone
26:06introduces a lot more need for process
26:11in our case it was always about trying
26:16to get to MVP that was reasonably
26:19walking wet well and all that was
26:22definitely duct tape code there is
26:23nothing that I'd be proud of it's just
26:26Frankie's I had put together and barely
26:28working as much as possible so we were
26:33kind of private launch in a sense when
26:35we that the private beta and got
26:37customers who kind of sign up and we
26:41were actually had a beta a private beta
26:43that we had some number of game
26:45developers working with us and that was
26:47also the duck tech code
26:48the only time we started actually having
26:50all the best practices is actually once
26:51we joined I antic because now we have to
26:54integrate into the games which pokomoko
26:57has hundreds of millions of users so
26:59we're bearing ourselves to get a lot of
27:02the we pretty much we wrote the whole
27:04codebase none of what we had before is
27:06there it's all rewritten through pair of
27:09do that that expect that number of users
27:12cool I think for us at segments there
27:15are kind of three distinct phases of our
27:17development I'd say the first one was
27:20when we were in full hackathon mode
27:22right where we definitely didn't have
27:26we're I'd say there was an 80% chance
27:29that we were just going to throw it out
27:30two weeks later because it wasn't going
27:32to stick and then we were going to move
27:34on to the next thing I think because we
27:37had been burned so much by building out
27:39all of this infrastructure and investing
27:40a lot of these ways to provision your
27:42infrastructure and write tests and spin
27:45up different new services over the past
27:47year and a half and it really I gotten
27:49we had no users it has created this like
27:53pretty aligning homing instinct for us
27:56where we just said no matter what the
27:58only thing that matters is getting users
27:59at this point in the game so that's
28:02where where we started and I think that
28:05lasted us probably for about the first
28:07nine or ten months where we were just
28:09focused on rapid iteration getting more
28:11users making sure that we didn't run out
28:13of money and then the whole thing
28:16wouldn't be been mattered I think from
28:19there the the first phase that then
28:21shifted was when we brought on our first
28:23engineer TJ Wright for those of you who
28:26know are involved with the node
28:28TJ is one of the best engineers I think
28:30I've ever worked with I think 90% of
28:33node shops run on some part of his code
28:35and he basically came in two segments
28:39and he looked around at well this mess
28:42that we created the product that we
28:46said well there's no way for me to work
28:48in this like I have a horrible time
28:50onboarding I think it took him a week to
28:52get his entire laptop set up and so we
28:56kind of moved from this point where the
28:57entire development team shares the same
28:58tube of toothpaste to now starting to
29:01have more and more engineers involved
29:03with the project and when that happened
29:05it really stepped up our game in terms
29:08and just reproducibility when it came to
29:12running the builds and running the stack
29:14because suddenly we had to expand
29:16outside of just the four of us and then
29:20I think so maybe that period lasted
29:23another two or three years probably in
29:26the past two three or four years from
29:29now or from this point in time we've
29:33gone through another shift where we've
29:35started thinking a lot more about
29:36end-to-end testing security kind of best
29:39practices around handling all of our
29:42customers data and the real reason for
29:44that is that now we actually have a
29:46pretty significant amount both like
29:48revenue and customers and reputation to
29:51lose from day one we had no users so it
29:54didn't matter if the whole thing went
29:56down for a day or hours or whatever was
29:58just the calculus didn't make sense
30:01there but today we have thousands of
30:03customers who are relying on segments to
30:05publish their data to not lose it to
30:07treat it and handle it securely and so
30:09for us the investment makes way more
30:11sense and and that that last period like
30:16roughly what scale did you have to reach
30:18before you started thinking seriously
30:23about those things yeah it's a good
30:24question I think when we started having
30:27enterprise customers who are paying
30:29north of six figures per year that's
30:32when it started to shift and we started
30:34thinking oh we really have to take care
30:37and how we're handling this data because
30:39these customers are really depending on
30:40this to power their business so I think
30:44my story's a little different than the
30:46other panelists this might be
30:47interesting I would say all three of
30:49those different facets I think I heard
30:51security I heard scalability and I heard
30:53engineering best practices we really
30:54have to treat them independently at
30:56plain grid klinger's used to build all
31:00most most hospitals most jails most
31:03heavy civil roadwork were used on all
31:04the tech campuses all of the different
31:07government buildings that go up so
31:08obviously security from day one I mean
31:11that's that's key to us so we had no
31:13choice but to always be super conscious
31:14of security and scalability for that
31:17matter because our customers the only
31:19way we're useful is if we're used to
31:22build the project like we're the
31:24replacement for blueprints which means
31:26in construction if we're not working
31:27properly like no one builds and not
31:30building for a dam construction can
31:32seriously impact the bottom line of
31:34these businesses so for us the first two
31:36security and scalability we're always
31:38key scalability is a challenging one
31:40because it's I don't know I'm not sure
31:44if there's a way to do it without it
31:44being kind of reactionary you either
31:46over engineer everything and then you
31:48maybe never get a product to market or
31:50you eventually have to deal with scaling
31:52issues and for us we tried to architect
31:55it in such a way that we would last
31:56through the foreseeable future the
31:58foreseeable future would come and some
32:01customer would find the first kind of
32:02flaw in the scalability and we're used I
32:04mean our to give you an idea of the
32:06scale we've probably about we've
32:09customers with projects that are like
32:11500,000 blueprints with a ton of changes
32:13and maybe over 5 million annotations on
32:17one project and we're work we work
32:19offline as well so this is all
32:20downloaded onto a cache on the device we
32:23can actually our physical size on an
32:24iPad can take up like a hundred
32:26gigabytes for certain projects so we
32:28have like all kinds of scaling issues
32:29we've had to approach and we always try
32:31to balance it between engineering like a
32:33year ahead of time but not maybe three
32:35years ahead of time which is a trade-off
32:36so you know find that trade-off yourself
32:39I remember when we were in - I see we
32:41had all these stories of companies that
32:42had built the most ironclad architecture
32:45and just never had a product to market
32:47so that was on our minds while we were
32:49building that for engineering best
32:51practices are v1 some of that code still
32:55exists in the product today so some of
32:57the code I wrote for v1 still there I
32:59would say it's probably our measure of
33:00technical debt as an engineering
33:02organization so that's humbling but at
33:05the same time like I've seen some
33:07developers and you know we're fully uh
33:10you know we do everything cool now see
33:13we run integration tests we've got a UI
33:15testing team so this is feedback I'm
33:17giving from the past but you know if
33:19you're a experienced developer it's
33:22pretty hard to write spaghetti Kraft
33:24code you know it's actually challenging
33:26most experienced developers and
33:28experienced some people are just really
33:30good from the get-go it's normally
33:31something through time and writing a lot
33:32of production products that you just
33:34generally learned the tricks of not
33:36writing spaghetti code pretty quickly
33:38and you know some of that code runs
33:41without heavy testing the other thing to
33:43mention is to properly scale test our
33:45product it doesn't I mean it would take
33:47days during the integration test to
33:49download and upload all the data so we
33:50really have a lot of functional testing
33:52to help us double-check but the thing
33:54I've noticed with UI and unit tests is
33:56we're big fan of unit tests for a big
33:57fan of tester and development I've
33:59definitely seen developers write the
34:00most spaghetti test frameworks and the
34:03most spaghetti tests where they're just
34:04testing their mocks and they're doing
34:06this in the middle of a production
34:07environment you know like a production
34:08need for release and they spent so much
34:10time writing these unit tests that
34:12actually were not testing anything
34:14rather than getting the product out the
34:16door so engineering best practices are
34:19great there's a trade-off between
34:20writing a lot of tests and writing good
34:21code from the get-go
34:24so speaking of test-driven development a
34:28lot of the questions were about the
34:30various engineering methodologies agile
34:34development lean startup test-driven
34:36development extreme programming how do
34:39you guys think about those do you adhere
34:41to any particular methodology in your
34:45company I would describe us as agile ish
34:49so we do not fully adopt any of those
34:52methodologies basically we find what
34:57works well for our team and aren't super
34:59dogmatic about you know is this agile or
35:02is this not you know we run sprints we
35:05have daily stand-ups so we have some
35:07elements incorporated in our development
35:09but I think kind of with anything you
35:12know a lot of these problems you're
35:15going to be solving a lot of problems
35:16for organization and you have to kind of
35:18find what works best for you and so I
35:23think it makes sense to take bits and
35:26that work for you and adapt it to your
35:28organization and then I guess it's just
35:31the one other thing I'd mentioned on
35:33this is as you grow a big a big
35:37challenge is just constantly evolving
35:39how you work right so it's very
35:41different to work on a small team before
35:43where everyone kind of knows everything
35:45and everyone everything is then
35:47everyone's heads versus a team of 30 or
35:5050 or for some of us you know hundreds
35:53that is very different so you kind of
35:56constantly need to be assessing is this
35:59style of working or this methodology is
36:02this right for this stage of the company
36:05yeah I think the point of evolving the
36:08process is something we've been doing a
36:10lot so back when we were that's chair we
36:13were only with four engineers and me
36:15building the product so back then it was
36:18very messy we're just trying to move as
36:20fast as possible really so zero
36:22documentation everything was kind of
36:24whiteboards and everyone kind of could
36:25handle everything in their heads and and
36:28build it out and in terms of we didn't
36:31really do Sprint's because they think
36:33the some engineers are even quite liked
36:34it as much and there were just too many
36:36big tasks and we just trusted different
36:39individuals to go and tackle kind of one
36:41area and then we would integrate it but
36:43now things are very different as we
36:46hired I think team is triple ready in
36:48the past eight months for the AR
36:52platform product when you have a bigger
36:55team you definitely need a lot more
36:56process to be able to be efficient and
36:58communicate because you don't want to
36:59duplicate and not everyone knows
37:01everything and the system grows in a lot
37:03of complexity so we went from zero
37:05documentation to a lot more when from we
37:07used to have a little bit CI but now is
37:10very much of a CI that actually builds
37:12to all the architectural flavors to
37:15linux mac OS android x86 Android arm iOS
37:20etc and actually runs all the tests in
37:23all of them and actually catches a lot
37:24of bugs and we tests we have coverage
37:27and all those things but we also have to
37:29train and get the team to be ok with
37:32having more process so now we do not
37:34quite like a sprint but we do weekly
37:36planning for the week and then we
37:39reconvene and I think the teams are kind
37:42of breaking break it down into sub teams
37:44that work and people kind of float in
37:47and out of their fan focus on areas we
37:49split in three main focuses one is sort
37:51of the back end folder work with the
37:53back end I lower the work on the client
37:55to make it cross-platform the unity API
37:58is and the other the big one is bringing
37:59a lot of the computer vision algorithms
38:02to production so the funny thing is that
38:04everyone in the team has ended up being
38:05and trained up to be a computer vision
38:07out engineer because that's sort of the
38:09core we have CV algorithms on the server
38:11let's see the algorithms in the client
38:12and the core algorithms that run so
38:15there's been this how it kind of shaped
38:18up to be but we don't really have a per
38:21se a process where we do right now
38:24Niantic that's okay our so now we have
38:26all cares as well per quarter so now
38:28we're starting to plan longer longer and
38:30have ideas of before you still have like
38:32a rough idea where things would go but
38:34now this this and this and this yeah for
38:38us we don't have any set process that
38:41teams have to follow at segments kind of
38:44individual engineering teams sort of
38:45like what Spotify does are able to self
38:47organize they're able to run however
38:50they want they want to do Sprint's at
38:51school they want to use JIRA they can do
38:53that if they want to use asana whatever
38:55it is they're allowed to run however
38:57they'd like to run the one thing that
39:00we've introduced over the past two years
39:02similarly the okay our model for those
39:05of you are familiar and this was a model
39:07that was developed at Google it's a idea
39:10of O's or objectives and then key
39:12results so you have your one objective
39:14where you're trying to go and then key
39:16results that are supposed to be
39:17objective measures of how you get there
39:19such if you do every K R then it adds up
39:23to the full objective and so those are
39:24something that we do on a company-wide
39:26basis every single topic team in the
39:29company gets together and puts together
39:32they say one week before each quarter
39:35and then for that three-month period
39:37they're just executing on that plan and
39:41some teams grade those on a weekly basis
39:44and check in and say how am i doing
39:45against these goals some teams grade
39:47them on a monthly basis it kind of
39:50doesn't matter but at the end of the
39:52every team is saying hey here's how well
39:56I did against my stated goals separately
39:59for the teams that I work with and
40:00particular we end up running kind of
40:02just a weekly meeting where at the
40:04beginning of the week we end up planning
40:07out what we want to get done and then we
40:08have kind of a daily stand up I honestly
40:11don't care too much about the content
40:13for those meetings so long as we're
40:14always discussing the most important
40:16problems we're pretty big believers that
40:19the tools and process should be there to
40:21serve us not the other way around
40:25my experience at plain grid echoes all
40:29the other panelists it's agile ish
40:31self-organizing every team can choose
40:33the tools that they want maybe some
40:37things that'd be helpful to you from the
40:38beginning some things we've always found
40:40that's been very huge useful in every
40:41engineering team are again the daily
40:43stand-ups you've got to communicate on a
40:45daily basis as engineers sometimes it
40:47can be a little difficult to want to
40:49communicate every day and not just
40:51program when you wake up but that 15
40:53minutes and you can do it remotely as
40:55well over slack or something has always
40:57been really helpful in just kind of
40:58unblocking people time boxing I mean I
41:01think that a lot of these management
41:02practices all roll up into some abstract
41:04philosophies and sprints or timebox
41:07methods where you're just not going to
41:08work on something infinitely that's very
41:10helpful to kind of keep people's
41:11realities in check because often
41:13estimates can be off and you know
41:14sometimes what we thought was going to
41:16take us a week can take us a month
41:17myself included so time boxing is a good
41:20way to keep keep categories of that I'd
41:23also say some other light weight
41:24management techniques you can probably
41:25employ right now and your team is
41:27probably one to ones weekly one to ones
41:28just make sure if you're talking to
41:30people in a group and you're having
41:31stand-ups make sure you have some time
41:33to actually connect with people
41:34individually and get to learn more about
41:36their wants their needs their emotions
41:37and their careers that's great guys okay
41:42so now I want to change topics to the
41:46right way to work with non-technical
41:48co-founders and I think the topic that
41:51came up most commonly in the startup
41:53school forum was the one that ralph
41:55alluded to which is how to deal with the
41:57deadlines and timeframes particularly
41:59with non-technical co-founders and I
42:02think particularly in the in the early
42:05have thoughts on that any any orders get
42:08I'll volunteer for this since my
42:10technical non-technical co-founder is
42:12staring at me over there um a few a few
42:15ideas here if you've got someone that's
42:17really non-technical and I'm not saying
42:19my co-founders really non-technical I'm
42:21sure she can use your computer but you
42:23know if it's really non-technical this
42:25is an amazing testing opportunity for
42:27you I mean I remember I would write this
42:29software and I'd be so happy about it
42:31and I'd hand it to her and as soon as
42:33she touched it it would break I mean I
42:35don't know what she shook the iPad she
42:37rotated it three times but she had a way
42:39to break the stuff I wrote and that was
42:42amazing for testing in the beginning so
42:43that's one way to engage your non
42:45technical co-founder you eventually have
42:47to learn how to start pouting your
42:48deadlines that's really hard to do I
42:51never got very good at this I'm always
42:52optimistic even to this day I'm like oh
42:54yeah that's gonna take me a week so I
42:56never really got good at this I'm just
42:57aware that I'm optimistic about it and
42:59then I'm always kind of off by week and
43:01she's aware of that as well
43:03so eventually I've talked to other
43:04engineering leaders they keep two books
43:06they keep like a separate set of books I
43:08talked to their their technical
43:09co-founders non-technical co-founders
43:11about say okay here's the deadlines here
43:13and then you have another set of books
43:14that's actually when you think you're
43:16gonna get the job done I think one thing
43:18that can be really helpful is having a
43:19process some very lightweight process of
43:21like here's where we're like testing and
43:24then here's release because often some
43:25people might not realize that iOS has a
43:27release cycle that's not controlled by
43:29the developers it's controlled by Apple
43:31so you got to pad that into your plan
43:32you have to plan a little bit so I found
43:34that a little bit of project management
43:35goes a long way when you're dealing with
43:37a non-technical co-founder and even
43:40better if they're good at project
43:41management that's a great place to
43:42employ them as well so all of my
43:46co-founders are technical so we didn't
43:48really run into this problem in the
43:49early days kind of everyone's like oh
43:50yeah I know what's happening here
43:52if it slips we know exactly why it
43:55wasn't as big of an issue I found more
43:57recently with deadlines kind of the
43:59trick that I've started using is in
44:01one-on-ones actually asking each person
44:04what they think will happen over the
44:05next three or four weeks and when I find
44:08asking one to one is that one people
44:11don't anchor against each other's
44:12answers so you get a very unbiased view
44:15and it like forces each person to think
44:18your term future what's going to happen
44:19and second each person is much more wary
44:22of the parts that they didn't work on
44:24and so you get kind of a sense of like
44:27where there might be trouble spots and
44:29typically the end results is sort of
44:32like a wisdom of the crowds where it's
44:34kind of the average of all three or four
44:35answers and so I've started using that
44:38as a way to get a better sense of
44:40deadlines where each person has their
44:42prediction or mental model of what's
44:43going to happen and then you kind of
44:45average them all out to where it should
44:46be um well I guess in my case my
44:51co-founder didn't work in engineering or
44:55build products or shaped products mostly
44:58just in research so we've been doing
45:00mostly a PhD so in that sense it's kind
45:03a little bit you know technical in a
45:05sense that not having experience of
45:07building and shipping products it's
45:08mostly just kind of the algorithm side
45:10so there was a bit of that work to kind
45:13understand why certain things take long
45:15and why they need to be built a certain
45:18way but at some point it was more he
45:22just letting me handle all of that and
45:24he got engaged mostly on the external
45:27communications business trying to find
45:29partners and all that so that's what
45:31happened sometimes that could be the
45:33split where as the technical founder you
45:35end up owning the product and a lot of
45:38development and having clear
45:40communications and at least try to set
45:42expectations that with deadlines that's
45:45the hard part is always I think this is
45:46always a trade-off between engineering
45:49and product and business right the
45:51trade-off on when to get things and be
45:53able to communicate that clearly so my
45:59co-founder is technical so I don't have
46:00much to offer here not much to add
46:04either on the deadlines and time
46:05estimates to do your best guess do some
46:08padding I'd say one thing kind of on the
46:12non-technical front though that I do
46:14think is worth mentioning is so you know
46:19today a lot of you are probably building
46:21your products and in writing code
46:23especially if you're one of the
46:25technical co-founders at least in my
46:28experience that changes
46:30so I pretty much stopped contributing to
46:32our code base about a year into the
46:35company and it's for us kind of as
46:38leaders well I can't speak for all of us
46:40but at least for me there's a pretty big
46:43shift that happens once you start
46:45gaining traction where it's less about
46:47building a product for a market and
46:50finding that fit and it shifts more into
46:52turn like building an organization and a
46:54company and a system like this system of
46:57humans who are going to build something
46:59long-lasting and great without a lot of
47:02your involvement so I guess what I'm
47:04trying to say is you know at some point
47:07you may end up doing a lot more
47:10things so get ready for that yeah that's
47:15great ok so a lot of the companies in
47:19start-up school are at the phase where
47:22they are trying to figure out the right
47:26way to structure their early engineering
47:28team if they want to hire people locally
47:31if they want to hire people remotely if
47:32they want to hire only employees who
47:35work full-time or if they want to hand
47:37off pieces of the product to contractors
47:39or or to to third-party development
47:44firms can you guys talk about how you
47:47would think about that if you were
47:50now advice that you give them I think
47:52this is going to be the last question
47:53then we're going to open it up to the
47:55audience I can start so we did not start
48:01with contractors we started by building
48:03like hiring full-time employees I think
48:06the main guidance I would give here is
48:10you are building a product in a company
48:13and you're developing these core
48:15competencies I'd say for the most part
48:18like if you end up contracting that out
48:21you're actually not just contracting out
48:24through that technology expertise but
48:26you're learning a lot in these early
48:27days of what's going to work for your
48:30clients wouldn't like what's the what
48:31the product actually is and so moving
48:35that sort of having that external to
48:36your company I think is a really big
48:38loss in the early days just in terms of
48:39all that knowledge I could see an
48:42argument if you're just like
48:43getting off the ground and you're doing
48:45contracting kind of as a way to evaluate
48:48potential new hires I could see that
48:50definitely being a path but again that
48:53sort of with the intention of building
48:56your team on outsourcing front I think
48:59it's a similar thing of you really need
49:00to look at like what is your core
49:02competency that's what you're investing
49:04in what's your IP you don't want to
49:07outsource that at some point if you do
49:09you're pretty much just a marketing
49:12company and so I'd say if you are if
49:16different business models might have
49:18different requirements so for some of
49:21you it may make sense to outsource a
49:23large part of what you're building but
49:25for us we have experimented a little bit
49:30I'd say our approach has really been to
49:32make sure those projects are very well
49:34defined and not on the critical path so
49:37that we can experiment it is a different
49:40type of project to manage sort of
49:42outsource Talent and that way you can
49:44sort of learn see if that works for your
49:46for your business and if it does great
49:48if it doesn't it's it's fine was there
49:52also a question about remote local
49:55versus remote employees I think that
49:57depends on what type of culture you want
49:58to build so for us we was really
50:01important for us to have our team
50:02together so we hired a local team and
50:07then sort of in the last year so we have
50:10actually started tapping into remote
50:13engineers there's definitely like you
50:16know a little bit of a culture shift
50:17with that we're still predominantly
50:19local but there are a lot of advantages
50:21including being able to tap into other
50:23talent pools and it has actually turned
50:26out to be a really good forcing function
50:28for documentation right explaining your
50:31code explaining your decisions which is
50:34good for a number of reasons including
50:37just like as you grow communication is
50:39one of the harder problems you have to
50:41solve so I think for us because so much
50:50of the core that technology is what our
50:52company is we didn't quite outsource any
50:56of that we did contract people
50:57as more of an interview process so we
51:00would work with and contact someone for
51:02a month before we gave them an offer so
51:04that's what we did but that we did
51:06outsource more things that we really
51:09thought they were not essential to her
51:12company for example building 3d models
51:14some of the design some of the actually
51:18writing technical documentation at some
51:20of it website things that were we could
51:23do it if we had the time I wanted to
51:26we knew we could do it stuff like I
51:27could build a website but our time was
51:29better spent on that core text so those
51:31sort of things we did outsource and in
51:35terms of remote versus local because
51:37everything was so complex with what we
51:39build we chose the culture to be more
51:41all on-site and to this day we still are
51:44building the team locally but I think
51:46you could handle a lot of stories of
51:48companies that do remote but you have to
51:50kind of start remote first cool yeah I
51:53can share a little bit from our story so
51:57in the very early days we basically only
52:00hired full-time folks we did an
52:02experiment with contractors kind of the
52:04only case where we'd outsource is when
52:06eight of us could take some piece of
52:08infrastructure that we're building or
52:09like some new product feature and we
52:11could build on top of that I think that
52:14was still probably the right move
52:15especially in the early days it feels
52:16like a start-up is really fragile and
52:18you want a bunch of people who were just
52:19all pointed in the same direction and
52:21really kind of in it for the long haul
52:24on the local versus remote piece we
52:28actually started hiring pretty much only
52:29remote people we're really involved with
52:34a lot of different open-source
52:35communities on github
52:37particularly there were some node in the
52:39very early days some JavaScript ones
52:41component etc and so these were people
52:46who we had been working with and
52:48collaborating with for years at this
52:49point who then when we were ready to
52:51start hiring we just said hey which you
52:53like to work more closely together I
52:56think it came with its own set of
52:57trade-offs on the plus side we had
52:59access to these people who were insanely
53:01talented not in the San Francisco Bay
53:04Area but who were used to working with
53:07us already and who we very clear
53:11like built an insane amount of value for
53:14I think the trickier part for us to get
53:16right was around the communication piece
53:18and all kind of being pointed in the
53:19same direction from a product
53:20perspective so while we grew remote from
53:24about the first 10 or 15 hires we then
53:27only grew locally through about the next
53:3070 and then only recently started
53:33opening remote backup and I think now
53:36it's because we actually have enough
53:38bandwidth to create a really good remote
53:40culture where we're putting an emphasis
53:42on that documentation that communication
53:44piece and we're actually really able to
53:46hire folks who have been working
53:47remotely for years as well some really
53:51good advice given on this panel so far I
53:52think the trade offs are anything for
53:54you to keep in mind the most I was
53:56thinking back towards contractors and
53:57I'll tell you a quick story of a
53:58contractor our first two hires were
54:02actually as I was thinking about it
54:03contractors one of them helped us
54:05organize our bills and our like kind of
54:08office stuff and the other one helped us
54:09develop some technology someone I worked
54:11with when I used to work at Johns
54:13Hopkins and even though he was a
54:16contractor at first he actually you know
54:18we needed money to pay him because he
54:20so we paid him in equity and he joined
54:23us as our first engineer eventually and
54:24helped us build some really core
54:26technology for our company and really
54:28was a great addition to playing grid so
54:30I think that even though I wouldn't
54:32suggest hiring a contractor immediately
54:34we just did what was right for us at our
54:36company size and what we needed right
54:38then was like heli I need this kind of
54:39right some stuff for me and he'll take
54:40equity as payment wonderful because I
54:42don't got anything else to pay man so I
54:45mean I think just be open hearted with
54:47what you think is needed for your
54:48business right then no there's a lot of
54:50trade-offs with contractors the other
54:52thing to note is this was my friend
54:53we've had contractors have hired as well
54:55there's this ignorant feeling I can't
54:57even now I feel it when you hire a
54:58contractor that's like oh great I don't
55:00want to manage this person and that is
55:03management of contractors is significant
55:06time for you as a founder that is not a
55:07free hire it sometimes is worse actually
55:10it's more management time for a
55:11contractor than a full-time employee so
55:14keep that in mind as well as the remote
55:17thing this is also an interesting topic
55:18when we started our company we tried to
55:20aim to have people in San Francisco but
55:23yet I realize actually as Kofi
55:25we split off for like months at a time
55:26we'd be like oh you know I'm gonna go
55:27back to Chicago for a month and then you
55:29know one of my co-founders to work out
55:30of Chicago for a month or two and that
55:32was totally cool as well we've gone back
55:34and forth on this over our eight years
55:37over and over again between allowing
55:40remote not allowing remote allowing
55:41remote when it's a friend or a really
55:43good you know best in class I see that
55:46can really develop a lot of things now
55:48we're completely open as a company it
55:49doesn't matter we hire managers remote
55:51we hire local you know designers
55:52products everyone doesn't really matter
55:54we're just looking for the best talent
55:55we can get so I think the best advice
55:57you've gotten already is just do what's
56:00right for you at the size of your
56:01company and know that there's trade-offs
56:03with all these decisions but there's no
56:04easy answer for any of these questions
56:06great okay I think we've got time for
56:09one or two questions from the audience
56:11questions questions one over there you
56:26okay I'm just going to repeat the
56:28question so everyone can can hear it um
56:30if you were going to start another
56:32company now what would you do to
56:34accelerate the development process any
56:36takers yeah sure I mean I think we spent
56:40too much time building technology in my
56:41on my side to try to really get that
56:44product market fit as as soon as
56:47and it's a challenge in terms of the
56:49areas that you pick because if you're in
56:50a kind of frontier tech space with like
56:52AR VR or this summer biotech those lead
56:55time could be really long and think
56:57about what kind of company you want to
56:59build I'd say get in front of users all
57:05the time find your user base it's really
57:09it's actually a lot harder to build a
57:11product in this vacuum where you're just
57:14like in your head maybe your ID ating
57:16with your co-founder and you're like
57:17what if we do this what if we do this
57:19these all sound great no this idea
57:21sounds bad but it's like way easier to
57:23just put it in front of somebody and
57:25have them say this is great or this
57:27sucks that feedback loop is invaluable
57:30yeah echo the sentiment on users 100%
57:34get out in front of them start showing
57:36things to them start
57:38make sure your digging deep to make sure
57:40that you're really solving their problem
57:41I think if I were to start again today
57:44actually I'd build two or three versions
57:47which I just intended to throw away
57:48maybe this is biased on probably prior
57:51experience but I think the fact that you
57:53can get reps and a very safe manner
57:56where then when you're really ready to
57:58start you can just kind of go nuts
58:00building out the v1 I think is really
58:02powerful you know I love mobile
58:06I love Swift I love Kotlin but I don't
58:08think I would choose to have to write
58:10for different platforms if I was going
58:12to write again I probably would have
58:13pick one of the cross-platform
58:15development libraries I don't know which
58:16is best they all you'd have to tell me
58:18if anyone has like after this come your
58:19strong opinions at which one of this is
58:21best they all seem to have trade-offs
58:22probably the best advice I could give
58:23though as I'd stopped coding a little
58:25bit earlier and I would start hiring a
58:27little bit earlier because I think
58:29there's a lot of me that wanted to
58:30passionately write those features and if
58:32I instead been passionately hiring other
58:34developers we would have you know we
58:36would have built faster I feel
58:38interesting okay one more question over
58:42there and your driving features given by
59:03the users what would be the
59:05communication style get them in front of
59:07the users bringing them to the meetings
59:08get them over the phone well then give
59:11the questions you know kind of set it up
59:13a little bit where you're trying to bait
59:14the users into giving the feedback that
59:16you're looking for or you won't get the
59:18feedback you're looking for and maybe
59:19there's some adjustment there too so get
59:21them in front of users as much as you
59:22can that's the most powerful tool you
59:23have my opinion okay thank you all so
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
