Almost every senior hire ends with a conversation that felt like it told the board something. In many cases it told them very little, and left them more certain anyway.

The experiment that should worry every hiring panel

In 2013, Dana, Dawes and Peterson published a set of studies in Judgment and Decision Making titled "Belief in the unstructured interview: The persistence of an illusion." In the first, 76 students were asked to predict a fellow student's semester grades, some after interviewing that student for 20 minutes and some from background information alone. The interviews made the predictions worse. The detail that stays with you is that in some conditions the person being interviewed was secretly answering at random, and the interviewers still formed confident impressions and felt they had learned something useful. The authors' recommendation was blunt: those making screening decisions should not use unstructured interviews.

These were students predicting grades, not a board hiring a chief financial officer, and the study shows a general weakness in human judgment rather than proving the same effect size for senior roles. But it fits what the wider selection research keeps finding.

What the larger evidence says

Sackett and colleagues revisited decades of selection research in the Journal of Applied Psychology. Their revised estimates put the predictive validity of structured interviews at 0.42 and unstructured interviews at 0.19, a little under half. The older, widely cited estimates from Schmidt and Hunter had them at 0.51 and 0.38. Whichever set you use, structure is what separates a useful interview from a pleasant conversation.

Why it feels like evidence

An interview produces something that looks like data: stories, tone, rapport, a gut sense of the person. Dana and his colleagues point to two forces. Relevant facts get diluted by irrelevant impressions, and interviewers build a confident story out of whatever they hear. Fluent, warm, well-prepared candidates produce the most satisfying stories, so an interview can end up rewarding the skill of interviewing. That is a risk worth naming at the start of any senior search.

What to do instead

Keep the interviews, but change what they are for. Decide in advance what the role requires, ask every finalist the same behavioral questions, and score the answers against those criteria before the panel compares notes. Put a validated, job-relevant assessment next to the interview, not in place of it, and test the stories against people who were there. The aim isn't to remove judgment. It is to give judgment better evidence than the feeling in the room. It is the approach behind our methodology.