Banner message can go here.

AI Sourcing: Promise, Bias, and the Human-in-the-Loop

This episode explores how AI sourcing can uncover passive candidates and expand talent pools beyond keyword searches. It also digs into the risks of automated bias, ranking gravity, and why transparent, auditable human oversight is essential.

Powered by the OAD Team

The Science of Leading is brought to you by the OAD team—pioneers in predictive hiring, 
role-fit psychology, and high-performance team building. As hiring evolves, smart companies are rethinking the questions they ask—and OAD equips them with the behavioral insights to do it right.

With decades of experience in decoding what makes people thrive at work, OAD empowers leaders to ask sharper questions, avoid mis-hires, and build teams that grow with purpose.

Book a free demo at OAD.ai and discover how our behavioral insights can help you hire smarter, strengthen your teams, and scale with confidence.

You’ll learn:

No Risk. No Credit Card. Just Answers.

TRANSCRIPT: 

[Claire Monroe]
Welcome to the show—Edwin, if a recruiter types “account executive, SaaS, Chicago” into a search bar, they might get 200 profiles. But if an AI sourcing tool looks for adjacent signals instead—pipeline ownership, vertical experience, even patterns in career moves—it suddenly surfaces people no one explicitly searched for. The quiet candidates.

[Edwin Carrington]
Yes, and that idea—quiet candidates—is really the center of this. The strongest prospects are often not actively looking. They’re doing good work somewhere else. And historically, they were invisible unless their profile matched the exact language a recruiter happened to search for.

[Claire Monroe]
Right, and language is messy. One person writes “customer expansion,” another writes “account growth,” and a traditional search treats those as completely different—even though they’re basically the same capability.

[Edwin Carrington]
Exactly. So now you’re not just automating search—you’re defining competence. And that’s where this gets more serious. A well-tuned system expands your understanding of talent. A poorly tuned one just automates a narrow definition of it.

[Claire Monroe]
So when vendors talk about “faster sourcing,” you’re basically saying… speed isn’t the interesting part?

[Edwin Carrington]
Speed matters, but it’s not the differentiator. Consistency is. A human recruiter on Monday morning and that same recruiter late Friday are not evaluating candidates the same way. An AI system applies the same logic every time. That can be useful—but also dangerous.

[Claire Monroe]
Because “same logic every time” sounds fair… until you realize the logic itself might be flawed.

[Edwin Carrington]
Precisely. Consistency is not the same as accuracy. If the model overvalues certain employers, certain career paths, or uninterrupted trajectories, it will apply that preference perfectly—at scale. It will be wrong very efficiently.

[Claire Monroe]
And it looks clean while doing it. That’s the unsettling part. You get a dashboard, neat scores—82, 61—and suddenly people treat those numbers like they were discovered, not designed.

[Edwin Carrington]
That distinction matters. These systems don’t discover truth. They reflect decisions. What data was included, what success was defined as, what patterns were rewarded. If past hiring favored linear careers, the model may quietly penalize anything that deviates from that.

[Claire Monroe]
So AI sourcing is helpful when it finds patterns humans miss—but risky when it just repackages old habits as data?

[Edwin Carrington]
That’s close, but I’d sharpen it. It’s not just old preferences—it’s accumulated organizational habits. Things that were never formally decided, just repeated. The model treats those as signals, even if they were never intentional.

[Claire Monroe]
And a lot of those habits come down to… comfort. Same background, same industry, same type of profile. It feels safe.

[Edwin Carrington]
Yes, and the system can reinforce that without anyone noticing. If every shortlist keeps surfacing the same kind of candidate, it can look like quality—when it’s actually repetition.

[Claire Monroe]
There’s also the outreach side. “Personalized messaging” sounds great—until it turns into very polished, very irrelevant emails.

[Edwin Carrington]
Exactly. Good personalization reflects actual relevance. Something like, “I noticed you’ve led post-merger integrations”—that’s meaningful. Referencing a university or a generic title is just noise. Bad sourcing doesn’t improve connection. It just scales interruption.

[Claire Monroe]
So if someone wants the upside without turning hiring into a black box… where do they start?

[Edwin Carrington]
Start small. One use case. For example, passive candidate discovery for a difficult role. And define success clearly—better response rates, less manual sourcing time, broader candidate pools without lowering quality.

[Claire Monroe]
That constraint matters. Because otherwise teams roll this into everything at once, and when something goes wrong, nobody knows where it started.

[Edwin Carrington]
Exactly. Narrow scope allows for real evaluation. And anything that influences decisions—especially shortlists—needs human review. Not symbolic review. Actual scrutiny.

[Claire Monroe]
“Not symbolic review” is important. Because sometimes the human is technically there—but they’re just approving whatever the system suggests.

[Edwin Carrington]
And that’s not oversight. That’s delegation of judgment. A shortlist determines who gets seen, and in hiring, being unseen is often the deciding factor.

[Claire Monroe]
Let’s make this practical. If I’m evaluating a tool through something like the OAD lens—what do I need to understand upfront?

[Edwin Carrington]
Three things. First, transparency—what data is being used. Second, exclusions—what’s missing. And third, auditability—can you track outcomes over time and see if certain groups are consistently ranked lower.

[Claire Monroe]
Because bias doesn’t always show up as rejection. Sometimes it’s quieter. A candidate just keeps landing lower in the list, so no one ever reaches them.

[Edwin Carrington]
Exactly. Bias often shows up as ranking patterns, not outright exclusion. That’s why you need to compare input and output. Who entered the pool, who rose to the top, who received outreach, who moved forward.

[Claire Monroe]
And then there’s data quality. If profiles are uneven, outdated, or overly polished, the system might just reward whoever sounds the best on paper.

[Edwin Carrington]
Correct. Poor data doesn’t improve through processing. It becomes consistently misleading. So the real question isn’t whether the AI is sophisticated—it’s whether the signal it’s using is reliable.

[Claire Monroe]
Let’s test a scenario. A team uses AI sourcing for engineering managers. The tool finds strong passive candidates quickly. Recruiters spend less time searching and more time talking to people. That’s the ideal.

[Edwin Carrington]
That’s the productive version. The system handles breadth and initial ranking, and the recruiter focuses on relationships and fit. In that case, AI removes repetitive work and increases meaningful interaction.

[Claire Monroe]
But the failure version is… the same speed, just aimed in the wrong direction. The shortlist leans too heavily on certain companies or titles, and everyone moves faster toward the wrong candidates.

[Edwin Carrington]
Exactly. Faster toward the wrong candidates. That’s the risk. Velocity feels like progress because activity increases, but if no one questions the assumptions behind the ranking, speed becomes acceleration without direction.

[Claire Monroe]
So the real question isn’t whether AI can move faster. It’s whether a team can stay deliberate when speed is no longer a constraint.

[Edwin Carrington]
Yes. And the organizations that handle this well will make a conscious choice to keep judgment slower than the software. If you’re looking to operationalize that kind of structured, evidence-based hiring, you can test OAD’s tools—like behavioral assessments—for free at o-a-d-dot-a-i. It’s a practical way to introduce structure without losing oversight.

[Claire Monroe]
Which is probably the balance everyone’s actually trying to get to—speed where it helps, and thinking where it matters.