Building an Engineering Team by Ammon Bartram and Harj Taggar

Y Combinator2018-09-26

YC#Y Combinator#Startup School#Harj Taggar#Ammon Bartram

65K views|5 years ago

💫 Short Summary

Hajin Amin from Triplebyte shares insights on building successful engineering teams, discussing challenges, strategies, and the importance of effective hiring practices. Utilizing personal networks, hiring marketplaces, and structured interviews are key. Focusing on skills, avoiding biases, and providing a positive candidate experience are emphasized. Startups should prioritize productivity and ownership in early hires. The significance of humility in interviewing skills, feedback, and mock interviews is highlighted. Transparency in compensation, company culture, and mentorship are crucial for attracting and retaining top talent. Accurate assessments are key to successful hires, with a focus on technical skills and role-playing for improvement.

✨ Highlights

📊 Transcript

✦

Insights on building a successful engineering team from Hajin Amin of Triplebyte.

00:17

Triplebyte focuses on helping companies hire engineers efficiently by emphasizing challenges and strategies involved.

Importance of evaluating technical skills and the process of making offers and closing deals is discussed.

Warning about the time-consuming and difficult nature of hiring, emphasizing the need for careful preparation and prioritization.

The significance of effective hiring practices in scaling and competing in the market is stressed in the video segment.

✦

Hiring Process in Startups

03:39

Hiring in startups involves repetitive tasks like messaging, interviews, and meetings, often resulting in rejections.

The hiring process is compared to creating a funnel with sourcing, screening, and closing stages.

Personal networks are recommended for hiring in startups, allowing for assessing both skill fit and personal compatibility.

Hiring the right person early on is crucial for startup success, as a wrong hire can be fatal.

✦

Utilizing personal networks for hiring effectively.

05:51

Create a list of all good engineers known, regardless of their availability.

Reach out to each person on the list and make the ask, even if it seems unlikely.

Persistence is key in convincing potential candidates to join.

Tap into personal networks of team members through team events as the company scales for successful hires.

✦

Utilize shared spreadsheets, LinkedIn, and Facebook to identify and describe roles and candidates effectively.

08:21

Encourage team members to spend time finding potential fits on social media platforms and follow up personally with candidates.

Consider offering referral bonuses to motivate team participation in the hiring process.

Explore hiring marketplaces for quick and efficient hiring processes, similar to dating sites for connecting companies and engineers.

Be proactive in reaching out to candidates due to high demand for engineering talent, and utilize marketplaces to expedite the hiring process and reduce founder time spent on recruiting.

✦

Tips for Efficient and Cost-Effective Hiring Process.

11:07

Costs of hiring can range from 15-20% of the first year salary.

Aircast can provide better candidates through hiring marketplaces.

LinkedIn and GitHub are recommended sources for finding engineers, but response rates are dropping due to high volume messaging.

Personalized and targeted messages are more effective for early-stage startups, requiring time for research and personalization.

✦

Tips for Reaching Potential Candidates

14:51

Personalize messages by referencing candidate profiles and keeping them concise.

Use email for higher response rates rather than messages.

Utilize job boards like Stack Overflow Jobs and AngelList to find engineers, but expect underqualified applicants.

Attract quality candidates by making job listings unique, interesting, and highlighting personal stories, company mission, and technical challenges.

✦

Effectiveness of in-person meetups for recruiting debated.

16:46

Concerns about low attendance and lack of genuine interest in general.

Technical meetups may be more successful in finding qualified candidates.

Hosting meetups at the office can help build a network of potential hires.

Timing of hiring a technical recruiter should be after the first engineer is onboard.

✦

Importance of Hiring Your First Employee and Recruiter Delegation.

18:47

Hiring your first employee before delegating to a recruiter is crucial for understanding the hiring process and company needs.

Different types of recruiters include contract recruiters, in-house recruiters, and agency recruiters.

Recommendations include starting with a contract recruiter focused on sourcing and filling your calendar with promising calls.

✦

Importance of experience and data analysis in screening technical talent for companies.

21:54

Triplebyte offers a unique perspective in evaluating candidate performance across different companies.

Standard tech company hiring process starts with resume screening and includes multiple steps.

Speaker provides insights and advice based on extensive interviewing experience and data analysis.

✦

Overview of the hiring process for engineers.

24:23

The hiring process includes resume review, recruiter phone call, technical phone screen, onsite interview, and decision meeting.

Companies extend offers to only 2-8% of engineers who apply, with drop-off rates at different stages.

Around 95% of hired candidates succeed, but 5% are fired within months.

Top programmers receive more job offers, highlighting the challenge of interview inconsistency and the need for a repeatable process.

✦

Importance of inter-rater reliability in interviews.

27:29

Low agreement levels among interviewers on candidate evaluations, similar to online movie review agreement.

Only 20% of engineers willing to participate in trial employment periods.

Most engineers prefer traditional interviews due to time constraints.

Recommendation to offer trial employment as an option alongside standard interviews to attract top talent.

✦

Tips for Reducing Noise in Job Interviews

29:32

Defining important skills for the company is crucial to avoid biased questioning and hiring decisions.

Structuring interviews around specific questions regarding the type of programmers needed, their motivation, technical skills, and expertise can improve the hiring process.

Establishing clear criteria for evaluation helps companies select candidates more effectively.

✦

Importance of structured interviews in the hiring process.

33:21

Structured interviews are more predictive and should be favored over freeform interviews.

Asking every candidate the same set of questions helps reduce bias and noise in the evaluation process.

Providing interviewers with defined criteria for evaluation helps mitigate biases related to race or gender.

Unified decision-making is essential to ensure fair evaluation and selection of candidates.

✦

Best practices for conducting interviews in larger companies.

35:06

Having one person or group making final decisions reduces confusion and ensures consistency.

Use better interview questions to minimize noise and avoid questions that require a leap of insight.

Opt for questions with a gradual ramp of steps leading to a solution.

Multi-step problems can help assess candidates' problem-solving abilities, and providing hints can guide candidates to demonstrate their skills effectively.

✦

Tips for conducting technical interviews.

38:10

Focus on basic CS concepts like lists and hash tables rather than specialized knowledge.

Budget three times the solving time for a problem when creating interview questions.

Effective interview questions are often easier than expected, and asking multiple questions can provide a more consistent assessment of a candidate's abilities.

Provide candidates with a solution to a problem and assess their ability to implement it in code to evaluate their skills accurately.

✦

Importance of not being biased by a candidate's credentials, especially from well-known companies like Google, in evaluating programming skills.

40:32

Interviewers often influenced by strong credentials, leading to potential false positives in hiring decisions.

Focus recommended on assessing actual programming ability during interviews to find skilled individuals who may lack traditional credentials.

Consider the false negative rate in interviews to avoid missing out on talented candidates who could positively contribute to the team.

Potential consequences discussed of false positives in hiring decisions.

✦

Importance of False Negative Rates in Interview Processes

42:51

Not all candidates who fail interviews are unfit for employment.

Focus on candidates' maximum skills rather than average or minimum skills during assessments.

Creating a positive candidate experience increases the likelihood of candidates accepting job offers and performing well during interviews.

Reducing candidate stress during interviews by allowing them to work in a familiar environment.

✦

Importance of designing a structured interview process based on necessary skills.

46:10

Avoid hazing practices during interviews and focus on providing a positive experience for candidates.

Set criteria for interviewers, use good interview questions, and calibrate around the maximum skill each candidate offers.

Tips are applicable to companies of all sizes, with specific considerations for smaller startups in the seed stage.

✦

Key skills for the first hires in a startup should focus on productivity and ownership.

48:33

Accepting potentially lower code quality for quick results is common and recommended during the early stages of a company.

Being less aggressive in screening out candidates early on is suggested, especially for small startups struggling to find applicants.

Designing questions to assess the most important skills can aid in better screening all applicants.

✦

Importance of humility and awareness in interviewing skills.

51:08

Speaker recommends a mock interview exercise with purposely bad answers to emphasize self-awareness and feedback.

Role-playing good and bad answers can enhance objectivity in assessing candidates.

Exercise helps interviewers acknowledge limitations and be more humble in evaluations.

Debrief session feedback offers valuable insights for improvement.

✦

Importance of honest and rigorous feedback in interviews.

52:43

Use of reverse role-playing to identify weaknesses.

Significance of speed in the hiring process for startups.

Training interviewers on bedside manner and technical questions.

Addressing common complaints of interviewers not being familiar with technical questions.

✦

Tips for discussing company culture during interviews.

55:10

Emphasize diversity and how it is incorporated into hiring practices.

Use unique terms to describe your culture instead of common ones like 'open, transparent, collaborative'.

Discuss the trade-offs of your culture to attract the right candidates.

Involve your team and investors in the hiring process to increase candidate interest and closing rate.

Offer full and transparent compensation details to candidates for clarity.

✦

Importance of Transparent Salary and Equity Offers in Hiring Process.

59:26

Full details of salary and equity offers should be provided to candidates to prevent a negative candidate experience and improve acceptance rates.

Founders need to comprehend stock options to effectively address candidate inquiries.

Startups can compete with tech giants like Google and Facebook for engineers by highlighting learning opportunities, real decision-making responsibilities, and growth potential.

Despite tech giants offering higher compensation, startups can still attract and retain talent by providing engaging learning experiences and growth opportunities.

✦

Importance of mentorship in startup vs. larger company environments.

01:01:01

Big companies have checks and balances to prevent new hires from causing significant damage.

Emphasis on career progression and rapid advancement in the tech industry.

Working at a startup offers unique and varied experiences compared to safe technology companies.

Evaluate the level of mentorship available before choosing between a startup or a larger company.

✦

Importance of learning from experienced engineers for valuable insights and best practices.

01:02:31

Assess technical skills for hiring by evaluating past work and portfolios.

Background checks are discussed but not prioritized in hiring process.

Advocate for background blind hiring based on direct skills assessment.

Challenges non-technical founders face in evaluating technical skills and the advantage of having a technical co-founder.

✦

Importance of role-playing in job interviews for honest feedback and performance improvement.

01:08:05

Making intentional mistakes during mock interviews allows for constructive criticism without social pressures.

This method enables genuine evaluation and helps individuals identify areas for growth.

Experienced engineers can benefit from role-playing to enhance their interview skills.

Role-playing is a humbling experience that promotes self-improvement and prepares candidates for real-world interviews.

✦

Discrepancy in reported success rates of hires.

01:10:24

Only 10% of hires considered top performers.

Surveys show 5% of hires get fired, 30% are average employees.

Emphasis on accurate assessments in the hiring process.

00:00as a slides of loading there there is no

00:02topic that should occupy your minds more

00:05as you build your company then bring it

00:09on the team that's going to make your

00:11company successful as you move forward

00:14Hajin amin from triple byte YC alumnus

00:17are going to talk about building in

00:21today's day and age perhaps the key

00:24portion of that team the engineering

00:27team so please welcome harsh you're

00:30starting yeah harsh alright thanks for

00:38having us everyone

00:39so I'm Hodge I'm one of the cofounders

00:41of trip I along with Arman previously I

00:44used to be a partner at Y Combinator and

00:46part of the inspiration of sighing

00:48triple buy it was noticing how after

00:50graduating y Combinator

00:51and raising the first investment around

00:52everyone's number one problem was hiring

00:54and specifically hiring engineers

00:56because the hardest hiring challenge so

00:59three working on to a bite which is a

01:01hiring marketplace that's used by

01:02engineers to find new places to work

01:05Armen and I have gathered lots of data

01:07on what works well when it comes to

01:09hiring engineers

01:10I've personally focused a lot on

01:12spending time with companies helping

01:14them think through their strategies for

01:15finding engineers and obviously getting

01:17the most out of triple buy it and Arman

01:19spent a lot of time thinking through the

01:21details of how do you evaluate an

01:23engineer how to evaluate the engineering

01:25skill and answer the question of are

01:27they a good engineer or not so we're

01:29going to share that and sort of have a

01:30divide and conquer strategy going on

01:32here the four main topics we're going to

01:34cover are where to look for engineers

01:36and when you should start thinking about

01:38using recruiters and that's what I'm

01:40going to start with then we'll talk

01:42about evaluating technical skills which

01:44are Mons gonna cover and then I'll

01:47finish up by talking through the process

01:48of making offers and closing which is

01:51getting people to actually join your

01:52team but before we start on any of that

01:55I want to just issue a warning and make

01:58sure you're well prepared for the fact

01:59that hiring really well and truly sucks

02:01it's like incredibly painful process

02:04because for many reasons which I'll

02:06describe in detail the first is it takes

02:09a lot of time like it takes a lot of

02:11time just to convince someone good to

02:13eat

02:13and have a conversation with you and as

02:16a founder as you know time is a very

02:18scarce resource there's bugs to fix

02:20there's sales customers to close various

02:24things going on and hiring will never

02:27feel like it's your top most urgent

02:29priority and it's very easy to

02:31procrastinate and push it back but if

02:33you do that for too long you won't scale

02:35and you won't grow your startup and

02:37someone else will come along do that and

02:39take the market to hiring involves a lot

02:42of repetitive work so actually as Tyler

02:44was giving his presentation I was

02:46talking to Jeff about this back there

02:48there's a lot of similarities between

02:49sales and hiring and actually there's a

02:51lot of similarities between sales and

02:53hiring and fundraising and buy lots of

02:56things that you do as a start-up founder

02:57large part of it is effectively selling

02:59all of the time and selling as Tyler

03:02pointed out involves a lot of repetitive

03:04work this so hiring will involve lots of

03:07messaging and revolve taking lots of

03:09coffee meetings lots of phone calls lots

03:11of interviews and most of those will

03:14result in the dead end and be a complete

03:15waste of your time but you have to keep

03:17going and finally you will get your

03:20heart broken you will inevitably end up

03:23getting rejected by people you really

03:25wanted to hire who would have been the

03:27perfect fit to help you hit your growth

03:29goals but it turns out they were never

03:31really that serious about leaving their

03:33comfortable job at a big company to join

03:36your exciting but risky startup so be

03:39prepared for all of this and as you're

03:42thinking through building a hiring

03:43process I encourage you to think about

03:46this as a funnel that you're creating

03:48that has three parts to it the top of

03:50the funnel is sourcing and that's

03:51finding people who could be good fit the

03:54second is screening that's answering the

03:56question of do you want to hire this

03:58person or not and the final part of the

04:00funnel is closing making the offer and

04:02getting the offer accepted I'm going to

04:05start by talking through some strategies

04:07for building the top of your hiring

04:08funnel these are the five places I'd

04:12recommend that you look for making

04:14engineering hiring for making engineer

04:15hires I'll talk through the pros and

04:19cons of each of these they are personal

04:20networks hiring marketplaces LinkedIn

04:23github job boards and meetups I'll talk

04:26about how you can

04:27the most out of each of these and this

04:29list is ranked or it's sorted in order

04:31of where I think you should start out

04:33focusing most of your attention and

04:35energy down to where you focus the least

04:38so we start with personal networks in my

04:42opinion personal networks are the best

04:43place to hire especially when you're

04:45early and making your first few hires by

04:47far and the reason is anytime you're

04:50deciding if you want to hire someone

04:51you're essentially asking yourself two

04:53questions one does this person have the

04:55skills that you that you need for them

04:57to do the job and two can you personally

05:00work effectively with this person when

05:03you're a big company you can mostly

05:06focus on answering just the first

05:08question because they're your large

05:09enough there's enough people there's

05:10enough teams that it's likely that

05:12someone one team somewhere will be able

05:15to work effectively with anyone but when

05:17you're small that's not true whether you

05:20can work effectively with someone or not

05:21is a big determinant of your success and

05:23if you hire the wrong person early on

05:25that could literally be fatal so when

05:28you hire someone that you've worked with

05:29previously or you hire someone that's

05:31worked with the person you trust you do

05:33risk the chances that you won't be able

05:35to work well together which is a big

05:37thing to consider early on so that

05:39probably sounds like somewhat obvious

05:41advice and yet I'm surprised by how

05:44often founders still don't really use

05:47their personal networks effectively when

05:48they're hiring and I think there's two

05:50reasons for this the first is they don't

05:53use a process to exhaustively search

05:55through everyone that could potentially

05:57hire and the second is they don't

05:59actually make the ask that's usually

06:02comes down to being afraid of being

06:04rejected by your friends it's somewhat

06:06easier actually to be rejected by a

06:08complete stranger than it is to ask your

06:10best friend to come join you and they

06:11say I'm not sure the idea is that great

06:14there's there's also like you can also

06:16worry about what happens to the

06:18friendship of the startup doesn't work

06:19out like it's the sort of more that goes

06:22into it when you're talking someone you

06:24know personally that when you're talking

06:25to someone you don't but the truth is

06:28you just have to sort of suck it up and

06:30do it

06:30but you won your startup to be

06:31successful hiring from people you know

06:34it's a tremendously valuable resource

06:36and you just have to make that ask so

06:38I'd recommend you follow a strict

06:40process here

06:41start with just making a list of every

06:44good engineer you know whether you think

06:45they're available or not like that's

06:47actually irrelevant doesn't matter if

06:49they just sold their company for a

06:50billion dollars put them on the list

06:52then ping each and every one person on

06:55that list to meet up and commit to

06:57asking them if they would join if you

06:59have a crazy you think it is however

07:00unlikely commit to making the ask if

07:03they say no they're hesitant so

07:06deviously a little B and say will you at

07:07least come by the office and see what

07:09we're working on and if the office is

07:10your apartment that's totally fine to

07:12you but just keeps pushing until you've

07:14at least shown them something that

07:16you've done keep working on convincing

07:18them if it doesn't work out if they say

07:21no and you get a definitive no thence

07:23then ask them if they were in your

07:25position who would they try and hire

07:27make a list and go out and repeat this

07:30exact same process with them and this

07:32process just never ends like I know

07:34public company startup founders know

07:36long startups pod like probably company

07:37founders who kids still do this on a

07:39daily basis like this is just like a key

07:41thing you have to embed in yourself as a

07:43start-up founder as your company does

07:46scale and grow and the tea and you start

07:48putting two team together you want to

07:50start tapping into the personal networks

07:51of your team and the way I'd recommend

07:54doing this is team events where people

07:56brainstorm potential hires and this is

07:58commonly referred to as a sourcing party

08:01the way I had to recommend going about

08:03this is get everyone together send out a

08:07shared spreadsheet and describe the role

08:11that you're hiring for so if it's an

08:12engineering role sort of described in

08:13detail like who you're looking for who

08:15are examples candidates what are skills

08:18and qualities you're you'd be excited

08:20about and then literally have everyone

08:21spend 30 to 45 minutes going through

08:24their LinkedIn or their Facebook or

08:26whatever right there and then thinking

08:28of everyone that could be a fit and

08:29putting them into the spreadsheet

08:31once that's done a trilobite I'll

08:34actually then personally follow up with

08:35anyone on that list who seems like a

08:37good candidate or not and we've made

08:38several really great hires doing this it

08:40works works really really well and you

08:42can kind of make it like a fun thing to

08:44write so we'll do it at the end of the

08:45week just before our Friday all-hands

08:48food and drink and you can also offer

08:50referral bonuses to your team to

08:52incentivize them to do this so

08:55really make sure that you use the great

08:57you're sticking to an exhaustive process

08:58you're making the ask of people you know

09:00and then as you scale tap into the

09:03personal networks of your team once you

09:07sure that you've exhausted your personal

09:10network for leads the next place I'd

09:11start looking would be hiring

09:12marketplaces hiring marketplaces are

09:16actually relatively new they've become

09:18more popular over the last few years as

09:20has become harder to hire engineers by

09:22using traditional methods like reaching

09:24out on LinkedIn or github and I'm gonna

09:26talk more about that next the way I

09:28think about hiring marketplaces is

09:30actually work a lot like dating sites

09:32the idea is there's engineers who create

09:35profiles companies that create profiles

09:37and both are advertising their best

09:39selves you message each other and you

09:42figure out if it's worth meet up in

09:43person and you know if it all works out

09:46you make a hire the dynamics the

09:49marketplaces those such that the demand

09:52for good engineering talent far exceeds

09:53supply and so typically it's the

09:56companies that are being a lot more

09:58proactive in terms of reaching out first

09:59to the candidates the candidates are

10:01getting multiple of inquiries and the

10:04candidates or the engineers are the ones

10:05that are choosing who they want to speak

10:07to and who they don't a big benefit of

10:10using a marketplace especially in the

10:12early stages is that they can help you

10:15hire very quickly because everyone most

10:17candidates are on the marketplaces are

10:19actively looking for a place to move

10:21right now it's very quick to get on a

10:23phone call with them and start pitching

10:24and if you run a good closing process

10:26you can significantly reduce the amount

10:29of time you'll spend as a founder on

10:31hiring which is obviously great the

10:33downsides though are that they tend to

10:35be quite competitive so engineers are

10:38being reached out to do by multiple

10:40companies at the same time so you'll

10:42have to be very effective at convince

10:44them to join if you want to make hires

10:45and the second is that they can be

10:47expensive so most marketplaces will work

10:50on a fee per hire basis which can be 15

10:52to 20 percent of the first year salary

10:54that's cheaper than a recruiting agency

10:57but still a significant cost if you're

10:58in early-stage startup I'm obviously

11:01biased Aircast requires a hiring

11:03marketplace but I'd say like the three

11:05main ones that come up in conversation

11:07at least when we're pitching customers

11:08we

11:09be trilobite higher than Vetri you

11:12should try they're all free to use to

11:14get started so you're welcome to try I'd

11:16say provide the way we differentiate

11:18ourselves is essentially by having

11:20better candidates and we measure that by

11:22what percentage of candidates that

11:25companies interview through to provide

11:27do they make an offer to you and that

11:28tends to be twice the rate that they

11:30make through other sources and just as a

11:32general note hiring as a funnel you're

11:36optimizing your funnel so you should pay

11:37attention to what percentage of

11:39candidates are making it through each

11:40step in your funnel when you're early

11:42you won't have that many candidates so

11:44you can't be that scientific about it

11:45but start sort of capturing that data

11:48and just building that into the habits

11:50you have when you're thinking about

11:51hiring the third source that I recommend

11:55you go to and Mehcad is LinkedIn and

11:57github and these are effectively the

12:00biggest online directory of engineers in

12:02the world and most of hiring that's done

12:05at big companies is through teams of

12:07technical recruiters reaching out to

12:09engineers on LinkedIn or github finding

12:12the ones that fit sort of certain

12:13keyword criteria and sudden cold

12:15messages the reason they and they go for

12:18a very high volume approach here so

12:20technical Kreutzer may well sound over

12:22100 messages a day just in the hope that

12:24they get a few replies and a dynamic

12:26that occurred especially over the last

12:28few years is there's more technical

12:30recruiters there's more messages going

12:32out on these platforms so response rates

12:34are dropping for everyone which means

12:36for early-stage startups in particular

12:37it's going to require a lot of your time

12:39sending a lot of messages in order to

12:42get a few interested candidates so

12:45making this work for you requires in my

12:48opinion not playing to a high volume

12:51approach like a big company recruiting

12:52team word and instead spending the time

12:55actually researching the reading the

12:57details of profiles or reading through

12:59someone's Linkedin looking through their

13:01github looking at the details of the

13:03work that they've done and sending a

13:04smaller number of personalized targeted

13:07messages and emphasizing when you send

13:10the message why would someone be a good

13:11fit for your company specifically and

13:13give them clear evidence that you've

13:15read their profile and you're interested

13:17in them as an individual as opposed to

13:19sending a spam message

13:21but that doesn't mean the message has to

13:23be super long I still advise keeping it

13:25short and concise just the key fact is

13:27there's proof that you've read their

13:29profile final note here send emails

13:33instead of messages so if you sign up

13:36for LinkedIn recruiter light which is

13:38about $120 a month that will give you

13:40access to connect to fire it's a chrome

13:42plug-in that makes it really easy to

13:44pull out email addresses from anyone's

13:46LinkedIn profile and you'll consistently

13:48see much higher response rates through

13:50emails and you want messages so

13:51definitely definitely go for that okay

13:54the fourth place I try looking for

13:56engineers would be job postings or job

13:59boards these the two main ones for

14:02startup so stack overflow

14:04jobs and AngelList I haven't included I

14:08can use jobs on there because it's only

14:09available to YC companies but how can

14:13use jobs is somewhat unique and just it

14:14has a particularly high quality of

14:16engineer and I rank it second on this

14:19list actually if it were sort of

14:20stand-alone source of Engineers job

14:23board in general do suffer from a

14:25quantity of a quality problem so what's

14:27good about them is you don't have to

14:29spend a huge amount of time posting to

14:31one of them the downside though is that

14:33the time suck can come in later because

14:34most of the applicants you'll get will

14:36be vastly under qualified and you'll get

14:39a lot of applications from people who

14:40aren't even really software engineers

14:42and it will take a lot of your time

14:43reading through all of these resumes and

14:45applications to find the one or two good

14:47applicants so to maximize your return

14:51and maximize the number of good

14:52applicants you do get I'd recommend

14:54focusing on making your job listings

14:56unique and interesting and bear in mind

14:59that the majority of job descriptions on

15:00the internet are written by someone in a

15:03recruiting or marketing department

15:04that's using corporate boilerplate

15:06language that isn't going to especially

15:09using an appeal to an engineering

15:10audience right so as a start-up founder

15:12you can serve experiment to try and

15:15bring through a bit of your personality

15:17in the job listing so one thing you

15:19might try is writing the first person

15:22about the personal story for why you

15:24started the company why you're excited

15:26about the mission and making it seem

15:28like you get that sort of excitement and

15:30passion across other things you could

15:32try is there anything unique about the

15:34culture is there anything

15:35specific about the technical challenges

15:38or product challenges you're facing so

15:40if you put in more details that again

15:42will stand out because big companies

15:43tend to be very generic and vague when

15:46they're talking about what you actually

15:47get to work on they're the final source

15:50I talk about is actually just like

15:51physically in person meet ups so I don't

15:55think that these are actually going to

15:56be very effective and they're sort of a

15:58long shot like I just the numbers don't

16:00really work out like in-person meetups

16:02don't have that large of a number of

16:05people in attendance and truth is most

16:07of the time people are there for free

16:08food and drink more than trying to

16:10actively find somewhere to work so it's

16:12unlikely that you're gonna find both a

16:14really qualified candidate who's

16:16actively looking to move who's excited

16:18about your company and you also

16:20personally need to be very effective at

16:22talking to strangers and convincing them

16:24of things it all for this to work at all

16:26right so I've accrued it on there

16:28because I do know startup founders who

16:30have had success through meetups but

16:32they are few and far between if you do

16:35try this approach I'd recommend focusing

16:37on technical meetups and by that I mean

16:39the local closure programming group

16:42where people come together and meet up

16:44and bring laptops and work on things is

16:46more likely a better source of engineers

16:48than going to Dreamforce the final thing

16:52you could try here is hosting meetups at

16:54your own your own office right so you

16:57could also combine this with the

16:58personal network hiring and use as an

17:00excuse to have friends come by the

17:02office or have their friends who are

17:03engineers come by the office at a

17:05Tribble bite for example one of our

17:07engineers is a fanatical Emacs user and

17:09he hosts the Bay Area Emacs meets up at

17:11our office and it's sort of a it's a it

17:14hasn't worked for us for hiring but it

17:15is a good way to just meet and build a

17:17network of good engineers that could

17:19come invaluable at some point in the

17:20future so it's worth the definite worth

17:23considering okay now I'm going to talk

17:26about when you should think about using

17:27recruiters truth is there's there's not

17:29really any hard rule on when you should

17:31hire a technical recruiter I've seen

17:34companies of less than ten people hire

17:36one I've seen companies wait until

17:37they're 50 people on plus it's sort of I

17:41treat these are my opinions or when you

17:42should and I treat these as a rule of

17:44thumb right so first I think you

17:47absolutely should wait until you've at

17:48least

17:49you're first engineer before you

17:50consider bring you on a recruiter in any

17:52capacity the reason for that is actually

17:55I think general startup advice applies

17:57here so one is just in general when

18:00you're hiring it's good for you to do

18:03the job yourself for a bit so you feel

18:05the pain and you understand it and you

18:07sort of understand the details of what

18:08makes someone good at that role at your

18:10startup in particular before you go and

18:12hire someone you'll be better able to

18:14assess them so if you feel the pain of

18:15recruiting yourself I think you have a

18:17better shot of hiring the right

18:18recruiter for you and the second the

18:23second reason I say is just it's it's

18:25like sales like it's actually good for

18:27you to go out and try and pitch and

18:28convince people to join your startup so

18:30you can just understand yourself like

18:32what message works and what doesn't

18:34because as a startup founder you are

18:35always selling and you never know when

18:37you might bump into someone who could be

18:39a really great hire and if you've

18:41already sort of practiced the pitch for

18:42convincing engineers to join you and

18:44you'll have those ready to go so I'd

18:46recommend making sure you always make

18:47your first hire before you try and

18:49delegate this to a technical recruiter

18:51second I'd expect to have a good hiring

18:54cadence so somewhere around hiring an

18:55engineer a month for the next six months

18:57or so and before you start bringing on a

18:59recruiter otherwise it's likely they'll

19:01run out of work to do a quickly and then

19:04finally as a rule firm if you're

19:06spending more than 50% of your time

19:08sourcing that's like all the things I

19:10mentioned before and doing initial phone

19:12calls and screens and getting people to

19:14come and meet you in person over 50

19:16percent is probably about the time where

19:18you want to start thinking about

19:19bringing on help because 50% is about

19:22the amount of time you want to be

19:23spending on hiring so recruiters

19:27themselves come in roughly three types

19:28you have contract recruiters who you pay

19:30by the hour and they can do anything

19:33from just : messaging on LinkedIn all

19:35the way to doing initial phone screens

19:37this in-house recruiters it's just

19:39hiring a full time technical recruiter

19:40and that works as a member of your team

19:43and the final would be agency and

19:45agencies are essentially teams of

19:49salespeople that are paying lots of

19:51engineers on LinkedIn or wherever they

19:52can

19:53and then sending their resumes out to as

19:54many companies as they as possible and

19:56they tend to charge 25 to 30 percent or

19:59first year salary if you hire an

20:02engineers

20:02maybe they sent you they do tend to be

20:04fairly high touch so they're quite

20:06involved in trying to give you

20:07information that will help you close an

20:09engineer but they're also sending that

20:10information to multiple companies at the

20:12same time so my recommendations here

20:15would be when you get to the point where

20:17you feel it's time to bring on help or

20:19bring on a recruiter start with a

20:21contract recruiter and have them focus

20:24on just the sourcing piece of it so have

20:26them just focus on reaching out to

20:28engineers on LinkedIn and github and

20:29their main deliverable for you should be

20:32filling your calendar with calls with

20:34promising candidates then you're doing

20:36the pitching and practicing or yielding

20:38the pitching and convincing when you get

20:40to a point where that becomes too much

20:41work for just you then I'd consider

20:44hiring a full-time in-house recruiter

20:45and training them to do the pitch cool

20:48so now they're both sourcing and doing

20:50the initial calls and sort of setting up

20:52the on-site interview process for you so

20:57to sum up most of startup hiring plan if

20:59I were just getting going and building

21:00an engineering team would be start by

21:02making sure you've exhausted your

21:03personal Network spend lots of time

21:05taking people out for lunch coffee

21:08making the ask to you experiment with

21:11the hiring marketplaces your mileage

21:14will vary on these depending on how

21:16effectively you can pitch your company

21:17but even if you don't make hires from

21:20them you'll get valuable experience

21:22pitching real engineers and real

21:23candidates and learning what resonates

21:25about your company to that audience

21:28third spend some amount of time doing

21:30personalized and targeted outreach to

21:32engineers on LinkedIn and github making

21:34those messages really personalized and

21:35finally treat job boards and meetups

21:38into meeting people in person as a

21:40background process you run where you're

21:43not expecting to make any highs from

21:45them but you're sort of building a

21:46general pipeline that could be valuable

21:48in the future cool that's the first part

21:53of this almonds are now going to talk

21:54about how to screen and evaluate a

21:56technical school of engineers then I'll

21:58jump back in a wrap-up with making

21:59offers and closing

22:03awesome thank you hard so I'm Aiman I'm

22:08hard as co-founder of a triple byte and

22:10before this I started Socialcam with

22:13Michael Seibel and I was an early

22:15employee at twitch and I'm gonna talk

22:17about just the screening step so how to

22:20identify skilled engineers at your

22:23company so the first question here is

22:26just you know why you should believe me

22:28about any of this and one answer is that

22:32I've done a lot of interviews so since

22:35starting trilobite I've interviewed over

22:37a thousand engineers personally I but I

22:41think a better answer is that you know

22:44trilobite has a pretty special vantage

22:47point we're able to you know see how the

22:51views do see how candidates rather do

22:53you know in the interviews at multiple

22:55different companies are we can see how

22:57the same candidate performs in multiple

22:59companies and that gives us a data set

23:02that I think no one else has and you

23:06know that data is is you know where the

23:09advice I'm gonna give today comes from

23:13before I jump in I want to just go over

23:16the the basic hiring process that most

23:19companies use and this is actually

23:21pretty standardized so probably 95% of

23:24tech companies use these basic steps to

23:26screen campaigns so first is a resume

23:29screen someone applies to a company they

23:32send in a resume any recruiter looks

23:34over the resume and decides if this

23:35looks like someone who's a basically

23:37good fit then a recruiter call so this

23:41is a usually a 30-minute phone call with

23:43the recruiter I just ask about the

23:45candidate background judge culture fit

23:47and make sure they're interested in the

23:49company then a technical phone screen

23:53and so this would be between 30 minutes

23:55and an hour with an engineer usually

23:59solving a single programming problem so

24:01this is sometimes something like

24:02fizzbuzz or a little bit harder often

24:05this point done over over a synchronized

24:07text pad of some sort then this is

24:11optional sometimes I take home project

24:13so a substantial project

24:16the candidate completes on their own

24:17time and then you know sends in to the

24:20company to be evaluated then finally the

24:23on-site interview so it can they comes

24:26into the office and does between three

24:29and six one-hour sessions with engineers

24:33at the company I you know covering

24:35covering individual problems and then

24:38finally a decision meeting so you so

24:39usually the next day after the candidate

24:41has gone home everyone who interviewed

24:43the candidate and the hiring manager

24:44gets together in a room and talks about

24:47sort of their perceptions and make you

24:50know they they as a group make a hire no

24:51hire decision some stats on this

24:56companies make offers to between two and

25:00eight percent of all of the engineers

25:02who apply but interestingly exactly

25:05where in the process that drop-off

25:07happens is is very different between

25:09companies so we so we see companies

25:11where you know seven five percent of

25:14people who apply get screened out you

25:16know at the first step on a culture fit

25:18call and then we see companies where

25:20almost all applicants make it through to

25:22the final interview and that's where all

25:23this reading happens and about ninety

25:25five percent of people who are hired

25:27work out that is about five percent of

25:29technical hires result in someone being

25:30being fired within a few months and then

25:34from the candidates side what we see is

25:37a distribution of success in interviews

25:40so the the the top you know few

25:42percentage points of programmers by

25:44skill receive job offers after most of

25:47their interviews but then the bulk of

25:49programmers are somewhere in the middle

25:50where they receive you know the you know

25:53they do interviews and they receive

25:54offers after somewhere between you know

25:55fifteen to thirty percent of the

25:56interviewers they do and an interesting

25:59point is that no one passes all their

26:02interviews so there are not magical

26:04engineers who receive offers after every

26:05interview they do and yeah this gets at

26:10what I think is the the the major

26:14challenge when designing any of you

26:15process and this is inconsistency so

26:19this is noise in the interview process

26:22it's kind of the idea you know is your

26:25interview fundamentally repeatable and

26:28meaningful

26:30you know if you had to if you could

26:32somehow you know reading to view your

26:34co-workers your employees or everyone

26:37you who passed review in the last year

26:38if you could be any of those people how

26:41many of them would pass again and it's a

26:45pretty scary question but kind of

26:49interesting thing is that we've able to

26:50add some data on this that's what I did

26:53was I calculated a stat called the

26:57inter-rater reliability between all of

27:00the interviewers at all of the companies

27:02on trilobite and so what that means is

27:05this this is the statistical measure of

27:06the extent to which different

27:09interviewers tend to agree about which

27:12candidates are best and it's on a scale

27:16of zero to one where zero would be no

27:20agreement or the amount of agreement you

27:22would expect to see you know in random

27:23data from chance alone and one would be

27:26perfect agreement and what I found was

27:29an agreement of just over 0.1 so so the

27:35first point is that that's obviously

27:36much closer to to no agreement than it

27:38is to perfect agreement but for some

27:40context than that I calculated the same

27:43stat on a data set of online movie

27:47reviews and what I got was a agreement

27:52of very similar but it's actually

27:55slightly higher so it ends up that you

27:59know interviewers agree about which

28:02engineers are the best at about the same

28:04rate that Netflix viewers agree about

28:07which movies are best and that's yeah

28:11that's scary I interviews are

28:14fundamentally noisy and they are they

28:16are more noisy than the data shows them

28:17to be more noisy than most hiring

28:20managers sort of want to believe so I

28:23know the first the first question here

28:25is sort of why do invested all that

28:26infinitives are noisy why do them at all

28:27why can't we just use something like

28:29trial periods

28:31atashi engineers and yeah I think this

28:35is actually a great idea it is almost

28:38certainly much more you know if you can

28:40work with someone for a week you can

28:41almost certainly get a much

28:43sense of if they're a good employee then

28:45and then you could in in a you know if

28:46we are interview with them the problem

28:50is that most engineers don't actually

28:53want to do trial periods so we did some

28:55research on their sexual bite and it

28:57ends up that only 20% of engineers are

28:59willing to to do trial periods and

29:02there's actually some there's some

29:04adverse selection there so the you know

29:06most of the best programmers are in the

29:0880% who would prefer to do a standard

29:11technical interview because it takes

29:12less their time and is faster so I think

29:15that trial employment is an excellent

29:18option and you can offer that as an

29:19option but you know if you don't want to

29:22scare away most good programmers you do

29:23have to still run a standard traditional

29:25interview process so we're gonna talk

29:29about sort of today is you know specific

29:32pieces of advice that you can use to try

29:34to reduce the noise in a traditional

29:36interview and I'm gonna go over seven

29:38points so point one the first way to

29:42reduce the noise in an interview is to

29:45decide what skills matter for your

29:49company there are a lot of different

29:51ways that a programmer can be skilled so

29:55you know someone for example can be very

29:58productive or they can be you slow

30:00careful write great tests and and make

30:02sure they don't commit bugs you know

30:04someone can be strong in math and

30:07computer science or they could be you

30:09know deep knowledge about the internals

30:11of the Linux kernel and and you know

30:13scheduling and real-time operating

30:14systems or something and if you don't

30:17decide as a founder which of those

30:19skills matter for your company then your

30:22interviewers are going to decide that

30:23for you so they're going to come up with

30:26questions that they asked you in the

30:27interview process and they will fail

30:30people who answer poorly on those

30:31questions whether or not those are areas

30:34that that that should matter for your

30:36company and this is actually a major

30:39source of noise in interviews every you

30:43know engineer has this bias where they

30:44think the things that they know the best

30:45are the most important skills one could

30:47know an absent specific direction from

30:50above about what to look for they will

30:52fail people for areas that are not

30:53important for your company

30:55so my

30:56first piece of advice here is that you

30:59should go through and ask yourself these

31:02questions before you start hiring us the

31:05first is I need to clarify these are

31:06these are the axes along which we see

31:08most disagreement between interviewers

31:10the first question is you know do you

31:13need sort of fast iterative programmers

31:15or someone who's careful and rigorous do

31:18you want to look hire someone who's

31:19motivated by solving hard tech problems

31:21or someone who is motivated by building

31:24you know beautiful products for users I

31:26important that someone comes in with

31:29skill in a particular technology pick

31:31your language say or can you hide you

31:34want hire a smart person and let them

31:35learn your tech stack on the job is

31:38academic computer science or algorithm

31:41ability something which is important for

31:43you or is that an irrelevant skill and

31:45then is there any other sort of specific

31:48expertise that you need in people you're

31:52hiring it's actually so you know it's

31:54fine to answer both some of these

31:56questions you don't have to specify only

31:58a single profile of person you're hiring

31:59the important thing is to decide what

32:03matters even if that's multiple profiles

32:04um that sets you up to design the rest

32:07of your process and make sure that

32:08you're not you know failing people for

32:09being bad in relevant areas okay point

32:12number two second way you can reduce

32:14noise in interviews is to use structured

32:17interviews so to define this a free-form

32:22unstructured interview is an interview

32:23where a new viewer gets in a room with a

32:26candidate and they ask questions and

32:28they follow their intuition they they

32:31you know based on the answer the critic

32:32a that gives they ask follow-ups and

32:33they try to get a sense what the

32:35candidate feels like if that person can

32:36be a good fit at the company and at the

32:38end to make a global yes/no I don't know

32:40how a decision based on that Daddy

32:42interview and that contrasts with a

32:44structured interview where the

32:45interviewer comes into the room with a

32:47question which they're going to ask and

32:49a defined criteria they're trying to

32:52evaluate and the very interesting point

32:55here is that everyone thinks that that

32:57freeform interviewers feel better almost

33:00all interviewers prefer freeform

33:02interviews and they think they are more

33:04accurate in many candidates if you ask

33:06them actually say they prefer to be

33:08entered in that way but the

33:10things that this is completely opposite

33:11to all of the available data on this

33:13structure knickers are simply more

33:15predictive they are better at predicting

33:17success on the job and there's no excuse

33:20not to use them we should all be using

33:21structure interviews and so in the purse

33:24exams what that means um the first thing

33:26is that you have to hold the process

33:27constant so the goal of an interview is

33:31essentially to evaluate variants and

33:33candidates and you know if you do not

33:35hold the rest of your process constant

33:37between candidates you're introducing

33:39noise so there's simply no excuse for

33:41not asking every candidate for the same

33:43job the exact same set of questions I

33:46think the reason this is not more common

33:49is that the interviewers themselves tend

33:51to find this boring and you know I can

33:53say it suck it up like hey you like it

33:54you have to do that so second point on

33:58structure interview is that you want to

34:00give your interviewers defined criteria

34:03to evaluate so rather than putting in

34:05the room and saying decide if this K

34:07this person to be a good fit for the

34:08company say we care about coding

34:10productivity and you know knowledge of

34:13back-end web systems and so your goal is

34:15to ask this question the interview and

34:17grade the candidate on coding pregnat

34:19ivities and knowledge of back-end web

34:21and there's actually some great research

34:23on this it ends up that a lot of the

34:26sort of worst biases that can come up in

34:29interviews are made worse when the

34:31interviewer is trying to make a global

34:33decision so if something's making a

34:35global - this person feel like a good

34:36employee decision that's where things

34:38like you know do they look like someone

34:40got you in the past or you know their

34:42race their gender that's what those

34:44things come into play and interviewers

34:45are much better at ignoring those

34:47criteria ignoring those attributes when

34:49they're given a defined criteria to

34:51evaluate and then final point is that

34:55you want a unified decision-making on

34:57this and this applies posted to larger

34:59companies but you want to make sure that

35:00one person or one group of people is

35:02involved of all the final decisions so

35:06rather than viewing interviewers as

35:07people making decisions view them as you

35:09know people taking notes grading

35:11candidates on criteria and that all

35:13those notes and criteria go to a

35:14centralized person or a centralized

35:16group who makes the final decision and

35:18the idea is just that again the goal is

35:20consistency and it's far more easy to be

35:21consistent if one person

35:23making all the decisions point three for

35:25how to reduce noise is that it's just

35:27used better interview questions and so I

35:29have some tips on that

35:31the first is that you want to avoid

35:33questions that require a leap of insight

35:37from the candidate and rather you are

35:40questions where there is a gradual sort

35:42of ramp of steps that they can follow

35:43that lead to a solution and this is the

35:47general idea the rule of thumb I like is

35:49that you can ask yourself can this

35:51question be given away so if it's a

35:53question that has a sort of a single

35:55fact in the solution which a candidate

35:58could know in advance and perhaps be

36:00communicate them by a friend or by

36:01anything they read in Glassdoor then

36:03that means that it's probably a bad

36:05question and the example of that is

36:07there's a plastic question the question

36:09is you know imagine you're staying at

36:11the bottom of a flight of stairs and

36:13every step you can either take a single

36:15step up one stair or a double step up to

36:16stairs you know how many unique paths

36:18are there at the top of the flight of

36:19stairs and you know it ends up the

36:22solution of that is the Fibonacci

36:23sequence

36:23kind of strangely if someone knows that

36:26obviously it's you know that's the

36:28solution if they don't know it they may

36:30well struggle then think about it and go

36:31down some some rabbit hole and so that's

36:34exactly what is the solution example of

36:38a good question is you know can you

36:40please implement the game Connect four

36:42for me so there you know it's a series

36:44of steps each one relatively

36:46straightforward but that leads to a

36:48solution and and there's nothing F

36:50summons friend could tell them in ten

36:51minutes which would give them no

36:53massively unfair advantage you know in a

36:55game Connect four so another idea here

36:58this is the kind of related but you want

37:00multi-step problems and there's tend to

37:04lead to problems I don't have insight

37:06but also candidates will often get stuck

37:09in an interview even good candidates and

37:11if your problem has multiple steps to it

37:13you can you can get them a hint you can

37:16help them out in one portion and it

37:18still have enough left for them to go on

37:20to do well and be themselves and

37:21demonstrate skill whereas if your

37:23problem is all in one one one one one

37:25sort of nugget of difficulty if they

37:26can't solve that you have to help them

37:27at that point they basically failed or

37:29done poorly on that section you want to

37:33avoid specialized knowledge

37:36so this is you know if your goal is to

37:38assess general program ability you

37:40probably want to ask questions that

37:41evolve you know things like list and

37:43hash tables and strings rather than

37:46questions that evolve you know tries or

37:48prefix trees you know this list of court

37:51you know unless you've decided that you

37:53really care about algorithm

37:54you know structure knowledge and your

37:55goal is to make sure that me knows about

37:56tries in that case it's totally fine to

37:57ask about them but if you're if you're

37:59measuring something else if you if a try

38:01the portion of the problem you know some

38:03portion of your your people your your

38:05candidates will be familiar with us

38:07others well in that will introduce noise

38:08so in general I think it's good to stick

38:10with the sort of classic most basic CS

38:13concepts another route you want you run

38:16a budget about three times the amount of

38:18time it takes you to solve a problem for

38:20the candidate so if you come up with a

38:22question and you solve it yourself it

38:23takes you ten minutes that's probably a

38:26good question for a 30 minute interview

38:28session and the reason here is just that

38:31it's it's far easier to be the

38:33interviewer that is to be the candidate

38:35it's far easier to ask the question and

38:36we tend to downplay how hard the

38:40questions actually are and we can

38:42actually research on this we went

38:43through all the questions we asked we

38:45asked a terabyte and we looked which are

38:47most correlated with candidates going on

38:49to succeed and it ends up that the most

38:51effective questions tend to be much

38:54easier than what you know our intuition

38:56going in was predicting so it ends up

38:58that most interviewers think that you

39:01know the optimal question is quite a bit

39:03harder than the optimal question

39:04actually is you wanna make sure that you

39:07ask

39:08four or more questions in an interview

39:10they hear is that you know each

39:13individual question carries a certain

39:14amount of noise as the person seems

39:15questioned before is it you know did

39:18they have whatever they lucky answering

39:19it if you ask more questions you'll get

39:21a a more consistent signal out and then

39:26finally just one one tip type of

39:29question we like a lot a trilobite is a

39:30question where we we give the candidate

39:32a problem and then rather than wanting

39:34them to devise a solution we tell them a

39:37solution to the problem and then their

39:38goal is to take that idea and implement

39:41that in the code and so we give them an

39:44algorithm and we see if they can

39:44implement that algorithm rather than

39:46require them to devise an algorithm

39:49a fourth way to reduce noise is to

39:54ignore credentials during your interview

39:57so by credential I mean things like I

40:01you know did someone study at a

40:03well-known school or has someone worked

40:04at a well-known company and I'm not

40:08claiming that credentials are meeting

40:10are meaningless credentials are

40:11important right so the productivity case

40:13the group of people who you know our ex

40:16new colors are indeed better engineers

40:18the group of people who have not worked

40:20at Google I'm so it's totally you know

40:22legitimate to take that fact into

40:25account when when when deciding about

40:27the hire or someone however it's not

40:30relevant to their actual programming

40:32skill so my advice here is to make sure

40:35that you're not biased by the fact that

40:37someone comes from Google when you are

40:39narrowly evaluating the programming

40:40skill so I recommend you hide that

40:43credential from the interviewers and the

40:45reason is we found that interviewers are

40:47actually pretty biased by this if they

40:49know someone has you know what were your

40:52strong credentials they're more likely

40:53to interpret the result of the coding

40:55screen as in a positive fashion or this

40:57person you know yeah they didn't know

41:00that answer but I'm sure they you know

41:01I'm sure there was a temporary slip-up

41:02and so how do pencils from your

41:04interviewers let them assess program

41:06ability and then when making the final

41:08decision in the decision meeting

41:09consider credentials and also the

41:11performance in the interview and yeah I

41:13this will help you find the programmers

41:15who are skilled and who lack traditional

41:17credentials and those are the

41:19undervalued people in the market and as

41:21a start-up if you can you know get if

41:23you're good at finding those people that

41:25is a big advantage okay point five is

41:28that you want to think about the false

41:30negative rate in your interview so a

41:33false negative is when someone fails

41:37your interview who could have gone on to

41:39do the job well and the opposite of

41:40false positive is when you have someone

41:42past if you hire someone who then goes

41:44on to do poorly and probably be fired

41:46and both false positives false negatives

41:49are very costly so if you hire a bad

41:52person if the fire of them that's

41:53terrible it'll hums the morale for the

41:54team and also also very expensive its

41:55actual money big problem but you know if

41:59you're a start-up and you're really

42:00hungry you know you're held back but not

42:01having

42:02engineers if you pass up a person who

42:04could have joined your team and couldn't

42:05productive that's also very expensive

42:07and I'm making a pretty subtle point

42:10here but I think there's a bit of a

42:11cognitive bias where false positives are

42:16very we're very aware of them

42:17we're very like we hire a bad person we

42:19feel that pain for a month or a month

42:21plus best-case and we've seen comes into

42:25a bite they're generally they generally

42:26are have too much faith that the folks

42:29who fail their process must have been

42:30bad engineers and couldn't have done the

42:31job and that's empirically not the case

42:34right if you watch you know as I said

42:36earlier no engineer will pass all

42:38interviews right so a significant

42:40portion of people who fail interviews do

42:41go on to be employed very productively

42:43at other companies and so I just

42:45recommend that that folks designing any

42:47of you processes think about the false

42:49negative rate and try to give that some

42:51way in in in their calculations of

42:53setting studying hiring bars um one of

42:57my goals would apply to actually just

42:58get to a point where you can actually

42:59measure this rate because cuz no one

43:01knows what it is no one knows what the

43:02false negative rate on the interview is

43:03because to measure that you have to just

43:04hire people randomly and see how they

43:05perform that's very expensive

43:07my goal is to get to where we can do

43:09that let's see okay point six is that

43:15you want to genuinely calibrate on the

43:18maximum skill that each candidate brings

43:20rather than their average skill or their

43:22minimum skill so someone comes in an

43:24interview they're very strong in one

43:26area or the weaker and others what

43:28matters the most is what they were

43:30strongest in everyone can look stupid

43:33right everybody you know if you ask me

43:35the right question I will definitely

43:36look very stupid that's true for

43:37everyone out there and so the problem is

43:39that someone might go through an

43:40interview and do very well on some

43:42things that would be you know useful for

43:43the company but look stupid on one

43:45question and you know if that

43:47interviewer gives a blocking know that

43:49that person was stupid that's

43:50interesting noise in the process perhaps

43:52now again if someone does poorly in a

43:54crack in the area that is important for

43:56the job totally failed them but be open

43:58to the fact that everyone does look

44:00stupid sometimes and don't fail someone

44:02just because they look stupid on one

44:03portion of the interview okay a final

44:05last point here is that you want to

44:08think about the candidate experience

44:09when designing an interview you know you

44:12want to you know make sure that every

44:14candidate who goes through

44:15I likes you likes your company and this

44:20is true for a few reasons one is just

44:21you know if they enjoy the process they

44:23have a higher probability of accepting

44:24an offer you make and so this will help

44:26in the closing step but a interesting

44:28point is that it will also actually make

44:30your screening itself more accurate

44:32because stress has a big impact on

44:35performance high percentage of

44:37candidates get very stressed in

44:38interviews and underperform their actual

44:40peak ability and so some tips here to

44:43help you do stress include just letting

44:46everyone bring in their lap and work in

44:47their own environment

44:48you know what throw and throw but their

44:49own language their own tools they will

44:50be they will be able you're much more

44:51productive much less stressed and then

44:54coaching your interviewers in just some

44:57soft skills so being friendly

45:00providing breaks for the candidates and

45:03you know when they are doing poorly on a

45:05section training them and how to

45:07intercede anyway which isn't too

45:08stressful insulting to the candidate

45:10rule of thumb here that comes from the

45:13the old Jolan software blog is that you

45:16you you want every candidate no matter

45:18how they do to finish your interview

45:20wishing that they had you know wanting

45:22join your company you know even people

45:23who fail your any very poorly you want

45:25them to end your interview wanting to

45:28join you being signs about the

45:29opportunity and the final point I you

45:32want to avoid hazing so this is this is

45:34rare but results in some of the worst

45:36horror stories of interviews this is

45:37where you know I mean who takes on the

45:38role of you know a ritual of acceptance

45:42into a group you know think if this if

45:44this happens it's terrible and as a

45:46hiring manager you can just stay totally

45:47away from that okay those are my points

45:52um I want to emphasize that I'm not

45:56saying that you should lower the bar for

45:58who you hire I think if you follow this

46:01advice you will get the more accurate

46:03signal you can then set your barbecue

46:05higher RF you want on that signal i but

46:09you'll still be making at that point

46:10better hires so just to summarize here i

46:13recommend that the first step you follow

46:16is to decide what skills matter for your

46:18organization and make sure that you're

46:20screening on things that you actually

46:21care about and then design a structured

46:24interview around those skills so come up

46:27with ways to assess each skill

46:29and stretch the criteria for the

46:31interviewers so they are you know less

46:32likely to be biased by by outside

46:35factors then you want to use good

46:38interview questions you know multiple

46:40parts in the lives of insight you want

46:43to hide credentials from the technical

46:44interest because that interest is noise

46:46in the process you want to think about

46:48the false negative as well as the false

46:49to positive cost in your interviewers

46:51and you want to genuinely calibrate

46:53around the maximum skill that each

46:55candidate brings well doing all that you

46:58want to try to provide a positive

46:59experience for the candidate so those

47:03are those are my my tips so far I think

47:06this all applies to both big and small

47:08companies so I'm gonna go over a few

47:10points here that I think are specific to

47:13let's say Series A and smaller companies

47:16so one one point here is winning what if

47:18your what if you're so small you don't

47:20have the scale to standardize your

47:21process so your seed stage start off

47:24your hiring your interviewing your first

47:25few people you obviously can't run an

47:28extremely standardized process because

47:29the first two candidates have seen those

47:31questions that's a totally real point I

47:35think it still is worth trying to run

47:37instruction interviews so you know you

47:41won't have you'll still you know your

47:42play-doh pie you simply Google Doc with

47:44some tips tips written down but it still

47:46is helpful to think about what skills

47:47matter and try to design questions that

47:50assess those skills that will still

47:53reduce you know the bad bias in the

47:54interview let's say that you're in

47:57trouble sourcing so you're already

47:59staged start up and your number one

48:00problem is we know how do you get enough

48:03qualified applicants for your company so

48:05in that scenario it's fairly obvious

48:07that a false negative costs more so if

48:09you're struggling to get people to apply

48:10to your company screening out someone

48:11who's bad stress all right so here

48:13someone who could have been good is a

48:14very expensive more expensive than for a

48:16large company but the flipside is that

48:18if you're a small company hiring a bad

48:20person is also way more cost you know

48:22from the way more expensive so it's not

48:25cream if the ratio of those two costs

48:26changes so I think you still have to

48:28care a lot about both I think what you

48:31can do is be less aggressive about

48:33screen folks out early so if you're an

48:36early stage companies struggling to

48:37source candidates I recommend you are

48:41less aggressive in speeding out

48:43you're on site you know pass more folks

48:45through and accept a lower final

48:48interview you know success rate in

48:50exchange for you know better screening

48:52all the applicants let's say that you're

48:56a small start-up in use don't know what

48:57skills matter so you're not sure if you

48:59if you want hire someone who's you know

49:01very CCS focus or someone who's very you

49:03know web focused and there's no crisp

49:07answer here so we have plenty of

49:08examples of you know billion-dollar

49:10companies that have taken various routes

49:13here my personal advice from from

49:17trilobite and from Socialcam and from

49:19twitch when it was small is that yeah I

49:22believe strongly that that that that the

49:23two most important skills for the first

49:25few hires are productivity and ownership

49:27so being able to basically take a

49:29project figure out has to be built and

49:31just make that happen and I recommend

49:33applies for that at the expense

49:35potentially of code quality so I think

49:38the first few hires at a company you

49:40should accept that perhaps the writing

49:41crappy code but you know by god they're

49:43writing it quickly and they're getting

49:44stuff done and I yeah just say that was

49:49the case during the early stages of you

49:51know all through comes I've been

49:52involved with and I think that's normal

49:53and probably good so let's say that

49:56you're hiring for an area where you do

49:58yourself don't have technical expertise

50:00so you're you're a web developer and

50:02you're you need to hire an iOS engineer

50:04what do you do so what enters you can

50:06you sure you can use a stick neutral

50:07byte I will help you do that

50:08I you can call them friends did you any

50:11before you if you have them but I think

50:14a good point here is that a trick that

50:16I've used in the past is to just ask and

50:18asking this to explain things so your

50:20web developer you ask the iOS engineer

50:22question they give an answer

50:23you have no idea if it's a good answer

50:24or a bad answer what you can do is just

50:26ask them to explain you say oh that's

50:28interesting can you explain why that why

50:29that's a good idea and for the most part

50:32if someone truly is a skilled iOS

50:33engineer they should be capable of

50:35explaining their answer in terms out of

50:37web screen flow much get to understand

50:38it's not always the case sometimes

50:41someone come along who's you know good

50:43at assumption but the bad communicator

50:44and that that might fail them but this

50:45is a pretty good trick to hire outside

50:47your area of expertise

50:50okay that's basically I want to end act

50:54with an ask for you guys so this is this

50:56is a we've been developing an exercise

50:59for interviewers that we use when

51:00training new interviewers patrol bite

51:03and I want to see if I get you guys to

51:04try this and I sort of email me and let

51:07me know how it goes because I'm curious

51:08but this works more broadly and the idea

51:13is that and you know interviewers tend

51:18to sniffily underestimate how noisy

51:20interviews are they they overestimate

51:22their own ability to distinguish to

51:24engineer some bad engineers and part of

51:27becoming a better interviewer is to is

51:29to sort of become a little more humble

51:31and more aware of your limitations and

51:34so this exercise is designed to sort of

51:36highlight that and so what I want you to

51:39do is to do a mock interview with one of

51:43your co-workers so have one of your

51:45co-workers interview you where they're

51:47role-playing you know they're asking

51:48questions they're the interviewer and

51:49you're role-playing a candidate you're

51:50giving answers and I want you to tell

51:52them in advance that you're going to be

51:54giving bad answers to some of the

51:56questions you're gonna be roleplay I

51:57can't think who makes mistakes and then

52:00during the interview they ask questions

52:01I want you to half the time go ahead and

52:04give a bad answer

52:04roleplay a poor answer to the question

52:06and the other half the time giving your

52:08best shot and actually try to answer the

52:11best possible answer you know to that

52:12question and then after the interview is

52:14done do a debrief session where you know

52:18your coworker goes through and sort of

52:19goes over all the mistakes you made and

52:21gives you feedback and what's great here

52:24is that they don't know what answers

52:25that you gave were intentionally bad and

52:27what were you trying your best and if

52:29they MIT look if they don't highlight

52:31the mistake you made where you were

52:33trying to make a mistake they're gonna

52:34look a little bit bad so they're fully

52:36incentivized to be completely honest and

52:38rigorous when putting out what the flaws

52:39they saw in the interview and though you

52:42know the criticism they give will apply

52:43both of the points where you're

52:45intentionally making mistakes but also

52:47to your attempts to give your best

52:48possible answers this is being really

52:50interesting we've been doing this a lot

52:51a trilobite and it's great highlighting

52:55how interviewers disagree so if you do

52:58this with your critic or worker and you

53:00do it a few times reverse roles you know

53:02the result

53:03is that you

53:04all well you'll highlight your area so

53:06you're both weak but you'll also

53:07highlight a bunch of areas where you

53:09have legitimately strong disagreements

53:12about what constitutes the best answer

53:13to it to it to a technical question

53:15there will almost certainly be more

53:17disagree more than you're expecting

53:18going in yeah I found this super

53:21insightful I'd love you guys could try

53:22this and I email me let me know how it

53:25goes awesome that's a thing for me

53:27hardies gonna talk about closing and

53:29then we'll both answer questions okay

53:36I'm gonna wrap up here by going through

53:37best practices and point to optimize

53:40your chances of people actually

53:41accepting the offers you make them okay

53:43I have five main pieces of advice here

53:45the first is just understand as a

53:48start-up in particular speed is a huge

53:50hiring advantage that you have over

53:52bigger companies so through two of why

53:54we work with startups and larger

53:56companies and we ourselves are surprised

53:58by how often a big company can take

54:00weeks just to deliver the actual final

54:01offer details and when someone's getting

54:04to the end of their job search process

54:05that's when there's most keen to just

54:08make a decision and move on and know

54:09what they're doing so as a start-up if

54:11you're quick at every step in your

54:13hiring process from the moment you have

54:14the first contact through to when you

54:16make the offer you increase your chances

54:18of closing a candidate so just be fast

54:21responsive at every step

54:22second Almond already mentioned like

54:25train your interviewers to have a good

54:28bedside manner with interviewees and so

54:30we ask candidates or engineers on

54:32trigger by what were some of the the

54:35reasons you had bad experiences when you

54:36went and interviewed in person at

54:38companies and all of the reply centered

54:41around their experience with the

54:42interviewers and the top two complaints

54:44were interviewers not being familiar

54:46with the technical questions they were

54:48asking and second the interviewer being

54:50determined to show how smart they were

54:52specifically smarter than the

54:53interviewee so if someone has a bad

54:56experience interviewing with you they

54:58are highly unlikely to accept your offer

55:00three be prepared to talk in some detail

55:04about your company culture because

55:07almost every candidate is going to ask

55:08you that question what I like to work

55:10here what's your company a culture like

55:11and I think we've seen a particular

55:14uptick in this question

55:16over the past year anything frankly a

55:18lot of stuff around

55:19uber and just a lot of the focus on the

55:22culture in the technology industry in

55:23Silicon Valley is making people

55:25especially individuals think more about

55:28the kind of culture they want to work in

55:29so two specific bits of advice here I

55:32definitely be prepared to answer

55:34questions about how you think about

55:35diversity if your teams already diverse

55:37or not I think you should have some

55:39thought around how you plan on

55:41incorporating that into your hiring

55:42practices going forward and second when

55:46you're describing your culture

55:48be aware that almost every company like

55:51a significant majority essentially use

55:52three adjectives I open transparent and

55:54collaborative and we've seen this on

55:56trim why because we see them carrying

55:57their profiles right and so that can be

56:00an effective strategy because those all

56:02three of those things are good

56:04aspirational qualities right like no one

56:06would not want to work at an open

56:08transparent collaborative place and but

56:10it's not very good for differentiating

56:12yourself right so one strategy you could

56:15take when you're talking about your

56:16culture is take more risk and the

56:19example the risk might be talk about the

56:21trade-offs in your culture so if you are

56:23if you believe yourself to be an open

56:26culture talk about the trade-off there

56:28right like openness means like being

56:31honest about things and being honest

56:33about things can be difficult and

56:34uncomfortable for people so if you're

56:35willing to kind of go in that direction

56:37you may a lien ate some candidates but

56:40you may also really increase your

56:42closing rate on the candidates are most

56:44excited about you forth get your team

56:47and investors involved in the process so

56:50once you made an offer make sure your

56:52team reaches out afterwards offers to

56:54meet up with the candidate again answer

56:56more questions and generally be involved

56:58in their decision making process and do

57:01the same with investors and in

57:02particular pick any investors you have

57:04who would be particularly good at

57:05closing that candidate so if they're on

57:08the fence about whether to leave a big

57:09company for your startup do you have an

57:11investor that successfully left a big

57:13company and joined a successful startup

57:15and finally present full and transparent

57:20offers with all the details the

57:22candidate needs to understand how much

57:24they're being compensated

57:25this sounds against probably somewhat

57:27obvious but I've been surprised by

57:29seeing

57:30companies engage in this sort of

57:32hypothetical battle where instead of

57:34just telling in candidate here's how

57:36much salary and equity you get so posing

57:39questions like how excited are you about

57:41us if we made you an offer do you think

57:43you join and this always creates a bad

57:46awkward candidate experience it's also

57:48unfair because the truth is no matter

57:50how powerful you powerful your mission

57:52is compensation is a big factor in where

57:54someone's going to work and so asking

57:56them to commit before giving them

57:58details it just puts them in an

58:00uncomfortable position and decreases the

58:02chances that they would accept your

58:04offer to join so when you do make the

58:06offer provide full details this slide

58:09has I take the minimum of what you

58:11should put on an offer letter salary is

58:13fairly self-explanatory with equity make

58:17sure that when you present the details

58:19of an equity offer so the number of

58:21stock options exercise price etc that

58:24you're also checking into a Sikandar how

58:26comfortable are they with this right if

58:28they're if they're a senior engineer and

58:29they've worked at 6 or 7 different

58:31startups they probably understand this

58:33you don't have to go into too much

58:34detail if they've spent most of their

58:37life working at Intel they may not be

58:40familiar with how startup equity works

58:41right so ask them how comfortable they

58:43are don't overwhelm them when you

58:45present the offer details if they're not

58:47just send them follow-up resources and

58:49that they can read on their own time and

58:51also this sounds somewhat obvious but

58:54make sure you yourself as founders

58:55really understand how stock options work

58:57because people will ask you questions

58:58about it and you actually don't end up

59:01spending much time on this because you

59:03yourself have restricted stock and

59:05fundraising doesn't usually include

59:07options but you should understand even

59:09down to level all I'd say tax

59:10implications of different types of

59:11options a final thing I'm doing close on

59:16here is a common thing that comes up

59:18with startups is how do you compete for

59:19engineers against the tech giants like

59:21Google and Facebook in particular right

59:23so Google and Facebook obviously can

59:26offer much larger liquid compensation

59:28packages than a start-up can and frankly

59:31like the public tech company stocks have

59:33been doing great over the last few years

59:34so it's becoming more challenging to

59:36compete against them still we routinely

59:39see startups do it and win out against

59:41much larger companies offering more

59:43money

59:43think there are four pictures they make

59:45that have been quite compelling first is

59:48emphasized learning the the case I'd

59:51make here is that you learn the most or

59:54you learn the fastest when you're given

59:55real decision-making it's a

59:56responsibility which means if you make a

59:59mistake like bad happens and

01:00:01there's very very little chance

01:00:03practically no chance that that's going

01:00:05to happen to a big company right like

01:00:07big companies have checks and balances

01:00:08in place to make sure that no one can

01:00:10create too much damage in particular for

01:00:13new hires right so if you tell someone

01:00:15that hey like we just don't have that

01:00:17luxury we're growing quickly we've got

01:00:19to get everything shipped yesterday

01:00:20you're gonna get thrown straight into

01:00:22making a real decision making authority

01:00:24and if you screw up that's going to be

01:00:26bad for the company and that's the way

01:00:27that you learn and build real experience

01:00:29so emphasize that too I talked about

01:00:33career progression and one bottom two

01:00:36macro comment I'd make is if you look at

01:00:38executives at public technology

01:00:40companies they tend to be you're often

01:00:43can be much younger than you see at the

01:00:45counterparts that like public deactive

01:00:48or like a public utilities company right

01:00:50and the the reason for that is you can

01:00:53grow up and be like chief product

01:00:55officer Facebook at 38 or however old

01:00:57the chief product officer is because you

01:00:59join the startup when it was really

01:01:01early and you grew along with it right

01:01:03and the tech industry is really the only

01:01:05place that happens like it's more of a

01:01:07meritocracy you don't have to pay your

01:01:0920 years dues at a company before they

01:01:12consider you for a senior role you can

01:01:15rise up as quickly as a start-up grows

01:01:16so emphasize that third talk about

01:01:20opportunity cost and in particular the

01:01:23way I'd frame this or talk about this

01:01:24would be at any given moment in time

01:01:27there's always going to be a basket of

01:01:29big safe technology companies to go work

01:01:32at the names might change sort of every

01:01:34five years or so but the experience of

01:01:36working at one is largely fungible

01:01:38whereas the experience of working at a

01:01:40start-up varies wildly right so I would

01:01:43emphasize the fact that your startup

01:01:45right now the team you have the

01:01:46opportunity is a unique one that's a

01:01:48uniquely good fit for that candidate and

01:01:50that they won't get again and if it

01:01:52doesn't work out they can always go back

01:01:54to generic big company

01:01:57and final point this one specific to

01:01:59hiring more junior candidates talk about

01:02:02mentorship so one thing that we see is

01:02:04especially for new grads people started

01:02:05off college is that they're worried they

01:02:07won't get the right level of mentorship

01:02:09they won't learn how to best practices

01:02:11work if they join a start-up and maybe

01:02:13they should go work at Google for a

01:02:14couple of years get that under their

01:02:15belt and then join a startup I mean

01:02:18first decide if that's actually true or

01:02:20not right like if you're if you yourself

01:02:21have only junior engineers on your team

01:02:23and frankly you just own one to mentor

01:02:25or get someone up to speed and then

01:02:27don't make that case and maybe don't

01:02:29hire someone like that and but if you

01:02:31have experienced engineers on your team

01:02:33emphasize that and emphasize the fact

01:02:35that they will get to learn alongside

01:02:36experienced engineers who have worked at

01:02:38places and learn best practices etc etc

01:02:41so that's everything I think we're gonna

01:02:45open up for questions I will ask the

01:02:48questions about the topics we've covered

01:02:49you wanna come up like background things

01:03:16are things that rule out the candidate

01:03:22background background checks frankly

01:03:25most startups I know don't do them

01:03:26they're especially easy to do now though

01:03:28because you can integrate Checker in

01:03:30with your payroll service so sure but

01:03:32that's not something I would like focus

01:03:34on I focus more on like if someone can

01:03:36be a good fit or not yeah

01:03:40[Music]

01:03:54that's a hard question so the majority

01:03:57of computer science engineers are

01:03:58probably not gonna be a to a pic Euler

01:04:00good job of doing the work you want that

01:04:05said this sounds like a case where you

01:04:06could probably just evaluate past work

01:04:08this person has done so you could

01:04:10probably ask this person to take to give

01:04:11you a portfolio or other examples of

01:04:13work they've done and if that is at a

01:04:14level and of a type that you're happy

01:04:16with hiring them would probably work

01:04:31yeah probably so the thing we focus on

01:04:35is background blind hiring so we focus

01:04:37very narrowly on just direct skills

01:04:38assessment rather than looking at other

01:04:40things that that correlate in our

01:04:41predictive so you know personality type

01:04:43is real people vary in their personality

01:04:45so that stuff matters for performing

01:04:47jobs I think it's more research on like

01:04:49the big five rather than big five

01:04:51percent of traits rather than - Briggs

01:04:53all relevant but I would try to keep it

01:04:56pretty separate from technical

01:04:57assessment so it's assess technical

01:04:58attributes yeah and then you know

01:05:01separately try to assess you know

01:05:02culture for it friendliness soft skills

01:05:03and then combine that all and they try

01:05:05to make a global decision yep sorry back

01:05:09in the sunglasses

01:05:15[Music]

01:05:31that's the extreme version and I said

01:05:33you probably can't so you okay so the

01:05:36question is if you're a non-technical

01:05:39founder how can you assess an applicant

01:05:42technical skills and I think the answer

01:05:44in fortune is that you probably can't so

01:05:47you can fall back in the third device I

01:05:48try to look at past work but I think

01:05:51this is why it's very helpful to have a

01:05:53technical co-founder is because you are

01:05:55going to be at a fundamentalist

01:05:56advantage there yeah I okay so the

01:06:11question is how do you find a technical

01:06:12co-founder and you figure out they're

01:06:14good or not I think sort of Arlen's

01:06:17advice basically applies here they the

01:06:19most thing is the even if you use a

01:06:20friend or some that you know who is an

01:06:22engineer to talk to them and figure out

01:06:24they're good engineer or not otherwise

01:06:26there's there's not really any other

01:06:27options yep yeah honey I'm not precise

01:06:33with my pointing yes but I think this is

01:06:54largely focused I'm sorry moving the

01:06:56question the question the question is

01:06:59other ways to work with the universities

01:07:00and colleges to generate like a pipeline

01:07:03of talent that you could hire from that

01:07:04accurate I'd say yes there certainly are

01:07:09on you know they exist like career fairs

01:07:11and sponsored events at colleges et

01:07:13cetera et cetera but I think those most

01:07:15there's probably work better for larger

01:07:16companies that can fit themselves around

01:07:18the college graduation deadlines and

01:07:21that schedule for start-up is kind of

01:07:23tough because usually you want someone

01:07:25that's available right now to hire as

01:07:26opposed to waiting for them to graduate

01:07:29so I probably wouldn't recommend

01:07:30investing too much time in that as a

01:07:32start-up and just to clarify big

01:07:34companies at this point make offers a

01:07:36full year in advance with large signing

01:07:38bonuses you know you deliver it a year

01:07:40in advance so it's very hard for

01:07:41startups to compete with college hire

01:07:49[Music]

01:07:59yeah the question is to clarify unlike

01:08:03my last point about role playing a bad

01:08:05interview so the key point here is that

01:08:06you're gonna roleplay need to do in the

01:08:08cat you know the co-worker who's playing

01:08:10the candidate who's getting the answers

01:08:11they're going to intentionally make some

01:08:14mistakes and what that does is it frees

01:08:17up their coworker to be critical so

01:08:19normally if you say if you do a mock

01:08:21interview between two co-workers the

01:08:22problem is that you know in the end

01:08:24everyone's gonna pull their punches

01:08:25they're not gonna honestly give their

01:08:28opinion of the performance that that

01:08:29their co-worker gave it just doesn't

01:08:31quite work due to all kinds of social

01:08:32pressure if you tell them I'm gonna be

01:08:35intentionally making some bad I'm saying

01:08:37I'm gonna tell you things and mistakes

01:08:39that frees them up where they're taking

01:08:41they're then free to point out the flaws

01:08:43in your interview but only free to they

01:08:45have to because if you say I'm making

01:08:47mistakes and you ask them what's your

01:08:49opinion of not even the results if they

01:08:51don't point out a mistake you made that

01:08:52means you got something past them right

01:08:54so it flips the social pressure and

01:08:56creates pressure where there then

01:08:56incentivize to be critical which is

01:08:59extremely useful because you know as

01:09:01experienced engineers like weird like me

01:09:03it's been a while your senior engineer

01:09:05on a team after small company it's been

01:09:07a while since you've had someone really

01:09:09brutally critique how good you are at

01:09:12answering your own questions and that

01:09:14you know what will happen is that they

01:09:16will point out legitimate flaws you'll

01:09:18be trying to give your best answer and

01:09:19they will point out things you said that

01:09:21are wrong or things that could be better

01:09:22and that it's very humbling it's very

01:09:26humbling and I recommend that they do

01:09:28that's I think this is exercise you can

01:09:29give yourself that experience of being

01:09:31humble and having flaws and even your

01:09:33own performance pointed out in the

01:09:40middle yes got it

01:09:43you talked about a 95% plus success rate

01:09:48on one of the slides and I have given

01:09:53the the fact that the interviewers tend

01:09:57not to have a very high rate of

01:09:59reliability it seems time to believe

01:10:01that there's a 95% and also doesn't

01:10:04match my experience that 95% of the

01:10:07hires successful so when you think is

01:10:12the real success rate where you know

01:10:16you're after hire something's still

01:10:17there and they're pop before yep that's

01:10:21gonna be so this comes from some so that

01:10:24number I 5 comes from some surveys you

01:10:25get upon companies okay sorry so the

01:10:29question was I my slides I said that

01:10:32that there is that tires are successful

01:10:34ninety-five percent of time but that

01:10:36doesn't mesh with you know the

01:10:39experience of being a hiring manager

01:10:40height people many hire are not not

01:10:43great so the question is what's the

01:10:44actual rate at which you know candidates

01:10:46end up being top performers and yeah

01:10:49questions totally right so that that

01:10:51number comes from some surveys we did

01:10:53where we asked companies of all their

01:10:55hires what percentage got fired and we

01:10:57actually asked web stands our top

01:10:58performers and that five times the fire

01:11:01rate so five percent actually get fired

01:11:03you know an additional 30% are people

01:11:06who stick on but are not particularly

01:11:07great employees and then about 10

01:11:10percent are in the bucket that cut that

01:11:11companies said were there their best

01:11:13hires and top performers thank you

01:11:18[Applause]

01:11:28you

🎥 Related Videos

a16z Podcast | Things Come Together -- Truths about Tech in Africa

a16z Podcast | The Infrastructure of Total Health

The Robot Lawyer Resistance with Joshua Browder of DoNotPay

a16z Podcast | Bots and Beyond

Design Sprints as a Tool for Organizational Change

a16z Podcast | Valuing Today's Fast-Growing Software Companies

🔥 Recently Summarized Examples

Former Priest REVEALS Jesus' MYSTICAL Lost Years & His Connection to BUDDHA! | Fr. Seán ÓLaoire

Kim Kardashian's Plastic Surgery Reversal: Is She Trying to Rewind Time?

How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]

BITCOIN EMERGENCY: NEXT PRICE TARGETS REVEALED!! Bitcoin News Today & Ethereum Price Prediction!

Uncovering Ancient Atlantean Ruins: Exploring Evolutionary Pathways and Psychic Phenomenon

Samsung Technician Knives TV To Void Warranty

View original video