February 4, 2006

Good Interview Questions

Posted in Interviewing, Recruiting at 7:28 am by mj

I admit it, I suck at interviewing.

My interviews usually fall into a pattern:

  1. read candidate’s resume
  2. based on experience level of candidate, make up a couple of coding/design questions
  3. meet candidate
  4. completely ignore pre-made coding/design questions
  5. ask candidate to use the whiteboard to explain the workings of one or two of her previous projects, including architectural diagram, class hierarchies/design patterns, and data model
  6. shout “oh shit!” when I realize I’m already over my allotted time

Frankly, I’m happy to play the part of the guy who asks about the candidate’s past experience, and ensures both that she’s not lying, and that she understands enough about what was going on (even in parts she didn’t write) that she’d be a solid teammate. In an interview sequence consisting of 4-6 interviewers, that role needs to be filled anyway.

Moreover, I’m not a believer that any sequence of technical interview questions is going to tell you a whole lot about the candidate. A lot of people can answer basic technical questions. That leaves either tough coding questions, or design questions. The problem with the latter is you need a well-defined problem statement, or else your judgement of whether the candidate can produce good designs is going to be mostly based on whether she’s operating on your wavelength at that particular moment.

My team used to interview as a team, which I actually prefer. After the acquisition, we’d invite managers from other teams to interview with us. This exposed me to some good technical questions. But, again, even the best questions seemed mainly aimed at showing how smart/experienced the interviewer was, and said little about how the candidate would perform in the real-world. (I’m thinking of one challenging question in particular, which this one really smart guy from another team always asked.)

But that’s not why I ignore my pre-made questions. I tend to ignore those because I chicken out in the interview, and start thinking one of the following:

  • the interview question is too easy
  • the interview question is too hard for this candidate
  • the interview question isn’t clear
  • the interview question takes too long to set up, and I’m a lazy bum

In my Friday interview, I had two questions prepared. They were:

Coding question: Implement the classes necessary to iterate over a CSV file, and access fields either by their position or by their name, given that the first line of the CSV file contains all field names.

I like this question. Since all of the parsing rules and edge cases aren’t defined (and there are many CSV exceptions), the candidate should be savvy enough to ask about them before starting her code. I can then follow-up with questions about how she’d handle the new requirement of supporting tab-separated files, or supporting CSV files produced by a legacy application with slightly different parsing rules, or supporting files without field names on the first row, etc.

The one time I asked this question (a year ago), the candidate totally blew it. And we never heard from this person again. Ever. I felt bad.

What’s confusing for me is: is this question too hard (it can be implemented in a hundred lines, way fewer in an interview using pseudo code and empty methods for the unimportant stuff), or is it too easy? Yesterday, during the interview, I chickened out because I thought the guy might take offense at having been asked a question that one might ask an intern.

The second question was a bit more germaine to real-world requirements:

Design question: How would you design a framework for batch jobs to run? Each job must be monitorable in terms of its current state and duration, and be able to tie into a notification system when it fails to complete successfully after some given time. Each job must also be able to easily save its current state, so that it can start where it left off in the event of a software or hardware crash or restart.

This also lends itself to some follow-up questions, such as distributing workload across multiple servers, or having redundant servers such that if one dies another will automatically start running the jobs, etc. Design trade-offs could be assessed, as could platform decisions (does it run in a resin JVM or not?), and industry knowledge (is there already an open source framework that you’d recommend to solve this problem?). It’s also not a totally easy problem to solve, and nobody would be expected to get it 100% correct in an interview.

But it doesn’t assess whether they’d be good at solving our particular kinds of problems, for which we opened the job req in the first place.

And here’s an important point: I prefer interviews (regardless of which side I’m on) to be more conversations than quizzes. I want to talk about what problems my team has, and what kinds of experience the candidate has which she can lend to my team. With that in mind, I always feel downright uncomfortable drilling the candidate like that. Maybe I’m just crazy.

But, at least I know I suck at interviewing.

Advertisements

1 Comment »

  1. Nits said,

    I liked this blog… frankly, interviewing is a tough job and I always wonder if I am doing enough justice with the guy on the other side of the table.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: