AI killed the tech interview. Now what?
kanenarraway.com258 points by ghuntley 2 days ago
258 points by ghuntley 2 days ago
The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone call with a member of the team. You never failed to know within a few minutes whether the person could do the job, and be a good coworker. This process worked so well, it created the best team, most productive team I've worked on in 20+ years in the industry, despite that company's other dysfunctions.
The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"
I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)?
Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source.
Indeed, the other problem is what you mention: it doesn't scale to high throughput.
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
In all pair programming interviews I have run (which I will admit have been only a few) I would fail myself as an interviewer if I was not able to guide the interviewee away from a dead end within 15 minutes.
If the candidate wasn't able to understand the hints I was giving them, or just kept driving forward, then they would fail.
Exactly! Who calls "researching how to build X, but then letting my pair-programming partner fall down a rabbit hole so I can feel superior" "pair programming".
Way too many career and remuneration focused techbros. Often driven by bullshit management and stack ranking style promotion/firing pathways.
> I would fail myself as an interviewer
You are not most interviewers, alas.
That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
Hiring doesn't scale, period; deal with it.
Exactly. People go like “the ideal way to interview is <whatever they themselves are the best at>”. Pair programming interviews suck and don’t scale, just like every other alternate way of hiring.
IMO interviewing is the biggest bottleneck and if interviewing was decoupled from hiring then it wouldn't be a problem. But this requires a Guild-like organization to manage interviewing/vetting and for companies to use said Guild for hiring. The companies could then do a single team culture meeting (if they wanted) before hiring.
I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.
Triplebyte is the company.
You still interview with the end companies, but technical interviews aren’t given.
I personally think outsourcing hiring is the worst idea ever, and completely lose confidence in companies that do.
They’re outsourcing technical interviews, not the same thing as hiring.
(Update: technically I would guess they're only outsourcing the first few rounds. Presumably their customers will conduct further interviews.)
What if the best candidates have already been filtered out by then?
Create a scalable, practical interview process where the result is reliably the best candidate, and you'll take over the world.
At this point in my career I no longer even believe that "best candidate" is a meaningful description except perhaps for highly specialized roles. We're all stumbling blindly in the dark.
Exactly. Its almost like optimising for finding your best possible match for marriage. You don't go over a billion prospects, you choose from the ones locally available to you, as they come.
it isn't about scale. It is about core principle of the tech hiring - all the companies hire only the best. Not only it is impossible to scale, it is plain impossible. Even if all the companies hired only "above the average" it would still be a pretty tall order :)
There is no one-size-fits-all for software developers. Different companies are looking to hire people at different experience/pay levels, with different skill sets, etc. Just as with dating, most companies are also going to have some understanding of their own "attractiveness", and not try to date out of their league.
FAANG companies offering industry leading compensation packages and prestige are in a position to be able to hold out for the best (even if their interviewing practices may fail to achieve that), but most companies are just looking for someone that checks the boxes and seems like a good fit.
Employers trying to hire for fit and culture has resulted in the most inhumane and counterproductive processes I've witnessed. If that's what you really want to do, let the person work for at least a month in the team.
Provisional hires (which often exist in theory but they're pretty much a formality in general) don't work for the most part. Lot of overhead for the company. And in many cases you're asking for the candidate to quit a job and possibly relocate on the possibility they'll get a new position assuming that they click in a short time interval.
Who said anything about relocation? That has to be a tiny percentage of hires.
Relocation used to be pretty common for professional jobs. Don't know about today when there's more remote work. And maybe companies aren't as willing to pay for in general.
So, accept the overhead as the cost of hiring the right people?
There's even more overhead on the people being provisionally hired.
Yes, sometimes things just don't work out. But, if someone quits a job and maybe relocates, that's a big personal cost. It's just the way things work in some limited contexts (e.g. professional sports) but it's not and shouldn't be the norm.
I suppose you can give a huge sign-on bonus with no claw-back provision, but that's never going to happen in most cases.