I’ve been hiring in tech for almost a decade, and helped coach lots of team-members to be better at hiring. Ignoring for a moment how we actually found our candidate and how our hiring pipeline is designed, here’s how to get the most out of an interview.

How to ask good questions

There are quite a few interview formats, like interactive coding, or reviewing a take home assignment, but here I’m just talking about a vanilla questions-only interview. I’ll start by explaining how to choose questions to ask, I’ll talk about how to figure out what level to bring a candidate in at (if it’s relevant for your organization), then I’ll finish by giving an example interview structure.

Good questions are disqualifying

Don’t ask questions that someone you’d want to hire would get right, ask questions that someone you wouldn’t want to hire would get wrong.

This was the advice that a tech recruiter friend of mine gave me near the start of my career and it’s what I’ve found most useful, but also the most at odds with how people tend to approach interviewing.

Ask questions that disqualify, not that qualify.

Imagine you are their new teammate and you’ve found out they don’t know the answer to this question. Given their job title and level, if this is not a red flag then this is not a good question.

Here’s that thought experiment highlighting a bad question:

Oh they don’t know good patterns for working with de-normalised data in NoSQL databases? Maybe they’ve mostly worked with SQL databases in the past and are used to normalization.

Here’s that thought experiement highlighting a good question:

Oh they don’t know when to use a composite index and they’re a senior backend developer? Who hired this person?

Another way to think about this is we are asking questions where a good candidate has to get all of them right, not some of them.

Finding truly disqualifying questions that are non-trivial is very difficult in practice, but it’s a good ideal to aim for. The closer you get to it, the more information each question will give you. Speaking of non-trivial…

Good questions aren’t too easy

Possibly this sounds obvious, but I’ve seen many junior interviewers make this mistake, so to be clear:

Don’t ask questions that a bad candidate could bluff through

These are questions for which any obvious answer would be sufficient. They tend to be some variant of:

Do you have any experience with X?

The above only works as an introduction to a line of disqualifying questions. If you’re not going to ask a follow-up question that requires them to demonstrate their knowledge of X at the appropriate level, then don’t ask the question in the first place.

The first section here is about lowering the question bar if it’s too high. This section is about raising the question bar if it’s not too low.

There is a frontier of ability that a good candidate should be at or beyond, and you are trying to cover the breadth of that frontier.

Examples of good and bad questions

Here are a few more examples (very heavily dependent upon what job and what level you are hiring for - some of these ❌ examples would absolutely disqualify a Staff engineer):

❌ What does idempotent mean in the context of HTTP requests? (Specific buzzword knowledge which they could understand without knowing the specific word.)

✅ Explain to me the difference between a PUT request and a POST request in HTTP (even a junior web-app developer should have an answer to this.)

❌ What is the algorithmic complexity of radix sort? (Random Googleable fact with no on-the-job advantage to having this knowledge in your head, biases towards those with CS degrees.)

✅ What is the performance of testing if an element is a member of a set compared to a list?

❌ Tell me about your experience with NoSQL databases (fluffy question, could be answered with just buzzwords.)

✅ What are the different types of NoSQL database you are familiar with, and when would you use them? (Requires you to understand NoSQL as a tool to solve a problem, not just know buzzwords.)

Reminder that all of these examples are heavily dependent upon what job you are hiring for, and at what level. If you had a colleague that didn’t know the answer to these ❌ examples and you were surprised, then they are ✅ questions for your interview.

Advantages of this approach

To understand the advantages of the disqualifying approach you have to understand the alternative, which is the “qualifying” or “vibes” approach. In this approach you ask deeper questions which it would be great if they knew the answer to, but not the end of the world if they don’t, or simple questions that don’t give you any information; then you take the overall percentage of correct answers, mix it with the general vibes of the candidate, and get an overall score and impression.

This is biased towards candidates who:

• By luck of past jobs, have similar experience to yours and have depth in similar areas.
• By luck of birth, are familiar to you in their method of communicating.

Qualifying questions are often full of gray areas of how “right” a candidate is. Disqualifying questions I have found much more black and white - either a candidate knows what a POST request is compared to a PUT request, or they don’t. Very little room for bias.

Don’t be afraid to end the interview early

Your time is precious. The advantage of the disqualification approach is that you can end an interview early without worrying that you didn’t give the candidate a fair shake. (If this isn’t the case then your questions are wrong!)

Something I do but don’t recommend is…

If a candidate has only failed one or two disqualifying questions, I tend to shift into a sort of “Hail Mary” mode where I ask myself “what could they get right at this point that I’d still want to hire them”. This is more to settle my conscience than anything else - I have never, to my memory, had a candidate salvage an interview by answering a Hail Mary question, but at least I feel better about rejecting people, particularly if I end the interview 30 minutes early.

Don’t leave any room for doubt

Don’t let candidates get away with ambiguity. If you are not 100% sure they answered the question correctly, then drill in with follow-up questions.

This is a key element of disqualifying questions - they are easy so you have to be certain the candidate got them right. It also increases your confidence in the candidate, which is the goal in the first place.

A composite index? It’s an index on multiple fields and I’d use it when I need to query multiple fields.

With the answer above I’d worry that they don’t necessarily know how field order is relevant, so I’d drill into that with a follow-up.

You might worry that the failure is on your side in understanding their answer, but this interview is about the candidate’s ability to communicate their knowledge as well as the knowledge itself, so this is still a failing on their side. Drill in and give them a chance to communicate the knowledge if they have it.

This isn’t just about knowledge

This notion of disqualifying questions is not just the candidate’s knowledge or experience. It can apply to problem solving ability, communication skills, or any other attribute you are looking for.

Asking yourself “would I be shocked if a teammate could not answer this” is a good way to figure out if you are asking the right questions, regardless of the shape of the questions themselves. Be honest with yourself about where the baseline really is for the role you are hiring, and ask questions that cover the breadth of this baseline.

How to level-set a candidate

This section is just if you are hiring for multiple levels of a job simultaneously, and you want to figure out what level to bring a candidate in at.

I thought I’d failed your interview and was really surprised when I got the job offer

Paraphrasing but this has been a frequent quote at work socials after a few drinks with my past teams. Turns out most people come out of my interviews thinking they’d failed. This is because they didn’t realize they’d passed half way through the interview and I was using the remaining time to figure out what level they were.

Larger companies often hire for multiple levels of the same role at the same time, so an interview is also used to figure out what the level of the candidate is. For this you can just use the same approach, but increase the difficulty of the disqualifying questions step by step - which is to say, disqualify for a higher level role. I’ll often ask a chain of related questions until the candidate fails, like:

What is an index in the context of a database and when would you use one?

Why not automatically index every field?

How does a primary index manifest vs. a secondary index in most databases? What about highly scalable NoSQL databases, like DynamoDB?

Another technique is to ask questions that are more open-ended, and see how much the candidate can fill in the gaps. Make sure you are clear on what the disqualifying baseline is for the question though.

As for the feedback I got from successful candidates coming away feeling like they’d failed - maybe there’s a lesson in there for me about reassuring candidates more during an interview 🤷

Interview structure template

I tend to set interviews at 50 minutes, and I structure them like this:

Part 0: Small talk, and interview outline (5 min)

This is where you make the candidate feel comfortable, and outline the structure of the interview. I start with a bit of small talk, then I explain this exact structure to them. I find this helps them relax and know what to expect.

Part 1: Introductions (5 min)

Here I will introduce myself to them and invite any other interviewers to do the same, before passing it to the candidate to introduce themselves and their past experience.

Make sure to let the candidate know you are expecting them to keep their introduction within a time limit, say two minutes. This is to make sure we have time for the meat of the interview, to see if they can communicate their experience succinctly, and to see if they can follow instructions.

Part 2: Attempt to disqualify (30 min)

Here we ask the “disqualifying but not too simple” questions we talked about earlier. This should be the bulk of the interview.

If I am part of an interview pipeline, with many interviewers before or after myself, I will make sure that our questions are co-ordinated, so we are not asking the same questions.

Part 2*: Level-set (whatever remaining of the 30 min)

As mentioned above, if you are hiring for multiple levels of the same role, once you have failed to disqualify a candidate for the base level (and hence would hire them) you can ratchet up the difficulty of the questions to figure out what level they are at.

Part 3: Candidate asks questions (10 min)

Good candidates ask good questions. This is a chance for them to ask you about the role, the team, the company, or anything else they are curious about. This is also a chance for you to sell the role to them.

If a junior candidate doesn’t have too many questions I don’t see it as much of a red flag, but a senior candidate should absolutely be grilling you about how your team works, and what the role would look like if they decided to take it.

Please do not keep this section as a rushed afterthought after the time for the interview has already run out. Making enough time for this shows respect to the candidate and gives you a chance to sell the role to them. After all, you should be interviewing people good enough that landing them needs a bit of selling.

Summary and final thoughts

TODO