Questions to ask before you buy an AI hiring tool

An AI hiring tool should help your team judge candidates faster and keep the judgment with a person. The questions below separate a helper from something that quietly decides for you, and you can ask most of them in the first demo.

Candidates are already adapting to screening software. In Greenhouse's 2026 Future of Hiring Candidate Report, nearly two-thirds of job seekers (62%) said they had tried tactics to get past automated screening, and 60% had changed how they write an application so it wouldn't be mistaken for AI-generated. A tool that mostly measures who is best at beating the tool won't tell you much about who can do the job.

What decision is this tool helping with?

Write the decision down before you watch a demo: which step the tool touches, what evidence it handles, and who still decides. A tool that summarizes interview notes after an interviewer has recorded evidence does something very different from one that ranks applicants before a person reads them.

Ask the vendor to describe that decision in one sentence, without adjectives. Then ask for a specific example of a hire the tool helped and a candidate it got wrong.

What evidence does it look at?

Every score should trace back to something you would accept as proof that a person can do the work. Ask what signals go in, what they are meant to predict, and what the tool does with a candidate who looks unusual but is strong.

AI can help format notes or flag missing evidence. The interviewer still decides whether the evidence is valid, sufficient, and relevant, and a polished summary can still leave out context or repeat a biased note.

Who can disagree with the output?

“Human in the loop” means nothing if the reviewer has no time to read anything. Ask who reviews the output, how long they get, whether they can see the source evidence, and what happens when they disagree with it. A review step that takes eight seconds per candidate is a rubber stamp.

You also want the reviewer to see what the tool used and what it ignored, and to keep the original notes long enough to check the decision later.

How does it treat people who don't fit the pattern?

Candidates differ in speech, movement, timing, and communication style, and an automated assessment can read those differences as weakness. The EEOC notes that federal anti-discrimination laws apply to AI used in hiring, and that a practice which looks neutral can still produce unfair results for some groups.

Ask how the tool performs for candidates with disabilities, accents, or assistive technology, and how someone requests an accommodation. Then test it yourself: run a few candidates whose outcomes you already know through the tool and see whether it agrees with the people who interviewed them.

What will candidates be told?

People should know when software is part of an assessment and who to ask about it. Ask what candidate-facing language the vendor provides, whether you can edit it, and what a candidate is told when a tool contributes to a rejection.

What happens when the tool gets it wrong?

Ask what the vendor does when a summary is obviously wrong, two tools disagree, an interviewer cannot explain a score, or a candidate challenges an assessment. You want a named owner on your side, a way to pause the tool, and a record you can review afterward.

Then ask how you hear about model changes and security incidents, and how much notice you get before either lands on your candidates.

What stays with you after you sign?

Accountability for the hiring decision stays with your team, whatever the contract says. Confirm what data the tool receives, where it goes, how long it is kept, and what you get back when you leave.

NIST's AI Risk Management Framework is voluntary guidance, and it makes a useful checklist for this part: name an owner, map where the tool touches candidates, measure how it performs over time, and decide in advance when you would switch it off.

FAQs

Should AI screen résumés?

It can help organize applications when the criteria are job-related, written down, and reviewed by a person. Treat any score as a draft that needs testing, oversight, and a way to correct errors.

Do we have to tell candidates we use AI?

In most cases you should, in plain language, and your counsel should confirm what your location requires. Candidates should know when software materially affects an assessment and how to ask a question or request an accommodation.

Is NIST's AI framework required?

No. NIST describes the AI Risk Management Framework as voluntary guidance. It structures the work, and your legal obligations come from employment law and your own policies.

What is the fastest way to test a tool?

Run it on candidates you already decided on. If it agrees with your interviewers for the wrong reasons, or disagrees for reasons nobody can explain, you have your answer.

The practical takeaway

Buy a tool that makes your evidence clearer and keeps a person accountable for the decision. Ask what it measures, who can disagree, how it treats people who are different, and what happens when it gets something wrong.

How Crossbridge keeps people at the center of hiring

Our talent team personally interviews every candidate before you see a profile, so a person who understands your role makes every recommendation. You meet 3 to 6 vetted candidates per search, you pay nothing until you hire, and every placement comes with a 60-day replacement guarantee. That care is part of why 95% of our placements are still in the role at 12 months.

Get candidates for your open role or see how Recruiting as a Service works.