AI in Hiring and Employment: Can an Algorithm Discriminate Against Job Applicants?
Quick Answer: Yes. An AI system can potentially produce discriminatory outcomes in recruitment, hiring, promotion or other employment decisions. In the United States, the fact that an algorithm made or assisted the decision does not automatically remove the employer from the scope of existing employment-discrimination laws. Federal law can prohibit discriminatory employment practices involving race, colour, religion, sex, national origin, age, disability and genetic information, subject to the applicable statutory requirements.
Imagine applying for a job.
You submit your CV.
You never speak to a human recruiter.
An AI system analyses your qualifications, employment history, writing style and online information.
It assigns you a score of 62.
Another applicant receives a score of 91.
You are rejected.
You ask why.
The employer says:
“The algorithm determined that you were not a strong match.”
But there is a problem.
The algorithm systematically scores applicants from one demographic group lower than similarly qualified applicants from another group.
Who is responsible?
The software developer?
The employer?
The HR department?
Nobody?
Employment law does not generally disappear simply because an employer introduces artificial intelligence into its decision-making process.
The Equal Employment Opportunity Commission has specifically recognised that AI can be used in recruiting, screening and hiring and that existing federal employment-discrimination laws continue to apply. :contentReference[oaicite:1]{index=1}
The problem is that AI can make discriminatory decision-making less visible.
A human recruiter may consciously or unconsciously discriminate.
An algorithm can produce a discriminatory outcome without anyone explicitly instructing it to discriminate.
That creates a difficult legal question:
When an algorithm discriminates, is the discrimination caused by the machine, the data, the employer or the humans who designed the system?
The answer may involve all of them, but their legal responsibilities are not necessarily identical.
Legal disclaimer: This article provides general educational information and is not legal advice. Employment-discrimination law varies by jurisdiction and depends on the specific facts, employer coverage and applicable statutes.
Key Takeaways
- AI can potentially discriminate against job applicants.
- Employers generally cannot avoid employment-discrimination laws merely by using an algorithm.
- AI can create both intentional discrimination and discriminatory effects.
- Resume-screening systems can reproduce historical patterns contained in training data.
- AI interview tools can create disability and accessibility concerns.
- Facial, voice and behavioural analysis can create additional risks.
- Disparate-impact principles can be relevant where a seemingly neutral system disproportionately disadvantages protected groups.
- Employers should understand how AI tools are used rather than treating them as black boxes.
- Vendor contracts do not necessarily eliminate an employer's legal responsibilities toward applicants.
- New York City has specific requirements for certain automated employment decision tools.
- Human review can reduce risk but does not automatically cure a discriminatory automated process.
- AI hiring systems should be tested, documented and monitored for adverse outcomes.
What Is AI Hiring?
Quick Answer: AI hiring refers broadly to the use of artificial intelligence, machine learning, statistical modelling or related automated systems in recruitment and employment decisions.
AI can be used to:
- Search CVs.
- Rank applicants.
- Recommend candidates.
- Analyse applications.
- Schedule interviews.
- Evaluate recorded interviews.
- Assess written responses.
- Predict candidate suitability.
- Identify supposedly high-potential candidates.
The technology can therefore enter the recruitment process long before a human interview.
Can AI Hiring Systems Discriminate?
Quick Answer: Yes.
An algorithm does not automatically become neutral merely because it uses mathematics.
AI systems learn patterns from data.
If the underlying data contains historical bias, the system can potentially reproduce or amplify that bias.
For example, suppose an employer historically hired mostly men for a particular technical role.
An AI system trained on historical hiring decisions may learn that certain characteristics associated with previous successful applicants predict “success”.
If those characteristics indirectly correlate with sex, the system may produce discriminatory results even without receiving an explicit instruction to reject women.
What Is Algorithmic Bias?
Quick Answer: Algorithmic bias occurs when an automated system systematically produces outputs that unfairly disadvantage particular individuals or groups.
Bias can enter an AI system through:
- Training data.
- Labels.
- Feature selection.
- Model design.
- Measurement methods.
- Human decisions.
- Deployment conditions.
Not every statistical difference is unlawful discrimination.
The legal analysis requires consideration of the applicable law and facts.
What Is Disparate Impact?
Quick Answer: Disparate impact generally concerns a facially neutral employment policy or practice that disproportionately disadvantages members of a protected group and cannot be justified under the applicable legal standard.
This concept is particularly relevant to algorithmic hiring.
An employer may never tell its AI system:
“Reject women.”
Instead, the system may use a neutral-looking scoring methodology.
Yet the outcome could disproportionately exclude women.
The absence of an explicit discriminatory instruction does not automatically end the legal analysis.
The EEOC explains that federal employment-discrimination laws can prohibit neutral employment practices that have a disproportionately negative effect on protected groups when the relevant legal requirements are satisfied. :contentReference[oaicite:2]{index=2}
What Is Disparate Treatment?
Quick Answer: Disparate treatment generally refers to intentional discrimination in employment decisions because of a protected characteristic.
In an AI context, intentional discrimination could arise if an employer deliberately designs or configures a system to favour or exclude applicants based on a protected characteristic.
For example, an employer might deliberately instruct a system to rank applicants from a particular demographic group differently.
That would raise very different issues from an algorithm that unintentionally produces a disparate impact.
Does Title VII Apply to AI Hiring?
Quick Answer: Yes, where Title VII applies, AI-assisted employment decisions remain subject to its requirements.
Title VII prohibits covered employers from discriminating in employment on specified protected grounds.
The EEOC states that federal employment-discrimination law applies to applicants and employees and covers areas including recruitment, hiring, promotion, pay and termination. :contentReference[oaicite:3]{index=3}
AI does not create an exception merely because the employer delegates part of the decision to software.
Does the ADA Apply to AI Hiring?
Quick Answer: Yes. The Americans with Disabilities Act can apply when AI systems are used in recruitment or employment decisions.
The ADA prohibits covered entities from discriminating against qualified individuals on the basis of disability in job application procedures, hiring, advancement, discharge, compensation, training and other employment terms. :contentReference[oaicite:4]{index=4}
This creates important risks for automated assessments.
How Can AI Hiring Discriminate Against Disabled Applicants?
Quick Answer: An AI assessment may disadvantage applicants with disabilities if it measures characteristics that are unrelated to actual job performance or fails to accommodate disability-related limitations.
Consider an automated video-interview system.
The system analyses:
- Speech.
- Facial movement.
- Eye contact.
- Response timing.
- Body movement.
An applicant with a disability may communicate or move differently from the patterns the system expects.
The system could therefore produce a lower score even though the applicant is fully capable of performing the job.
The EEOC and Department of Justice have specifically warned that algorithmic employment tools can disadvantage applicants with disabilities and highlighted the need for reasonable-accommodation processes. :contentReference[oaicite:5]{index=5}
Can an AI Interview Discriminate?
Quick Answer: Potentially.
AI-powered interview systems may evaluate:
- Language.
- Speech patterns.
- Facial expressions.
- Response speed.
- Word choice.
- Behavioural patterns.
Some of these characteristics may correlate with protected characteristics or disability-related conditions.
The critical question is whether the assessment is genuinely job-related and legally permissible.
Can Facial Recognition Be Used in Hiring?
Quick Answer: Facial-recognition or facial-analysis technology can be used for legitimate purposes in some contexts, but employers should carefully assess discrimination, privacy, accessibility and accuracy risks before using it in employment decisions.
A facial-analysis system may perform differently across demographic groups.
If the resulting error rate affects hiring decisions, the employer may face significant legal and compliance concerns.
Can AI Analyse an Applicant's Personality?
Quick Answer: AI systems can attempt to infer personality or behavioural characteristics from applications, interviews or other data, but employers should carefully evaluate whether the method is valid, job-related, reliable and legally compliant.
There is an important difference between:
“The applicant possesses the technical skill required for the job.”
and:
“The algorithm believes the applicant has the personality of a successful employee.”
The second claim can involve significantly more uncertainty.
Can AI Reject a Job Applicant Automatically?
Quick Answer: Automated rejection is legally possible in some contexts, but employers must still comply with applicable employment-discrimination and other laws.
The greater the role of the automated system in the final decision, the greater the importance of understanding:
- What data the system uses.
- How it scores applicants.
- Whether it has been validated.
- Whether adverse outcomes are monitored.
- Whether accommodation procedures exist.
What Is an Automated Employment Decision Tool?
Quick Answer: An automated employment decision tool, commonly abbreviated as AEDT, is a category of technology used to substantially assist or replace discretionary decision-making in employment decisions.
New York City's Local Law 144 defines an automated employment decision tool in terms of computational processes involving machine learning, statistical modelling, data analytics or artificial intelligence that produce outputs such as scores, classifications or recommendations used to substantially assist or replace discretionary decision-making. :contentReference[oaicite:6]{index=6}
What Is NYC Local Law 144?
Quick Answer: New York City's Local Law 144 regulates certain automated employment decision tools used by employers and employment agencies.
Among other requirements, covered users of an AEDT must ensure that the tool has undergone a bias audit within the required period, make information concerning the audit publicly available and provide required notices to candidates or employees. :contentReference[oaicite:7]{index=7}
The city's current worker-rights information confirms that employers and employment agencies using covered AEDTs must ensure that a bias audit has been conducted and provide required notices. :contentReference[oaicite:8]{index=8}
What Is a Bias Audit?
Quick Answer: A bias audit is an assessment designed to evaluate whether an automated employment decision tool produces materially different outcomes across specified demographic groups.
The purpose is not simply to ask:
“Does the software work?”
It is also to ask:
“Does the software produce problematic disparities?”
A technically accurate algorithm can still create legal concerns if its use produces discriminatory outcomes.
Does Every AI Recruitment Tool Fall Under NYC Local Law 144?
Quick Answer: No. The law is directed at covered automated employment decision tools that substantially assist or replace discretionary employment decision-making.
The statutory definition excludes tools that do not automate, support, substantially assist or replace discretionary decision-making and do not materially impact individuals. :contentReference[oaicite:9]{index=9}
This means that not every piece of HR software is automatically an AEDT.
What Notices Must Employers Give Under NYC AEDT Rules?
Quick Answer: Covered employers and employment agencies must provide specified notices concerning the use of an AEDT and related information.
NYC's current guidance states that candidates and employees must receive notice that the tool will be used and information concerning the opportunity to request a reasonable accommodation. The city also requires specified information concerning the tool's data sources and retention policies. :contentReference[oaicite:10]{index=10}
Can Employers Blame the AI Vendor?
Quick Answer: An employer should not assume that purchasing a third-party AI system automatically transfers its legal responsibilities to the vendor.
Imagine an employer says:
“The vendor assured us that the system was unbiased.”
That statement may be relevant to the contractual relationship.
But the employer still needs to consider its own legal obligations toward applicants and employees.
Vendor due diligence should therefore form part of AI employment governance.
What Should Employers Ask AI Hiring Vendors?
Quick Answer: Employers should ask vendors for sufficient information to understand how the system works, what data it uses and how performance and bias are evaluated.
Questions can include:
- What data does the system process?
- What characteristics does it evaluate?
- What validation has been performed?
- Has adverse-impact testing been conducted?
- What demographic groups were included?
- What are the known limitations?
- How are errors handled?
- Can applicants request accommodation?
- Can the employer audit outcomes?
- How is applicant data retained?
What If an AI System Uses Proxy Variables?
Quick Answer: A variable does not need to explicitly identify a protected characteristic to create discrimination concerns if it acts as a proxy that systematically correlates with one.
For example, an algorithm might not receive an applicant's race.
But it might use variables that correlate strongly with race.
The absence of the explicit field does not necessarily eliminate discrimination risk.
Can Zip Code Create AI Hiring Bias?
Quick Answer: Potentially.
Geographic information can correlate with demographic characteristics.
If an algorithm uses location in a way that produces discriminatory outcomes, the employer may need to examine whether the variable is genuinely job-related and legally defensible.
This is one example of why simply removing protected characteristics from a dataset does not necessarily guarantee fairness.
What Is Historical Bias in AI Hiring?
Quick Answer: Historical bias occurs when past human decisions or social conditions reflected in training data influence an algorithm's future decisions.
Suppose an organisation historically hired mostly candidates from a narrow group.
An AI system trained on those outcomes may learn that the historical pattern represents “success”.
The algorithm can therefore reproduce the past instead of objectively identifying the best candidate.
Can More Data Solve AI Hiring Bias?
Quick Answer: Not necessarily.
More data does not automatically mean better or fairer data.
A large dataset can contain large amounts of historical discrimination.
The relevant questions include:
- Where did the data come from?
- Was it representative?
- Were historical decisions themselves biased?
- Were labels accurate?
- Were protected groups adequately represented?
Should Humans Review AI Hiring Decisions?
Quick Answer: Human oversight can provide an important safeguard, but simply placing a human at the end of an automated process does not automatically eliminate discrimination.
A recruiter may simply accept the algorithm's recommendation without meaningful review.
This is sometimes described as automation bias.
The employer should therefore determine whether human review is genuinely substantive.
What Is Automation Bias?
Quick Answer: Automation bias refers to the tendency of humans to place excessive trust in automated recommendations or outputs.
A recruiter may think:
“The software scored this applicant 25/100, so there must be a reason.”
The recruiter may never investigate the underlying score.
The result is effectively automated decision-making despite nominal human involvement.
Can AI Hiring Systems Be More Objective Than Humans?
Quick Answer: Potentially, but objectivity should not be assumed merely because a system is automated.
AI can reduce some forms of human subjectivity.
But it can also introduce new forms of statistical or technological bias.
The correct question is therefore:
“Has the system been shown to make accurate and legally appropriate decisions for this particular employment purpose?”
AI Hiring and Reasonable Accommodation
Quick Answer: Employers using AI in hiring should have processes for applicants who require reasonable accommodation because of disability or other legally protected circumstances.
The EEOC and DOJ have warned that algorithmic tools can screen out people with disabilities even when those individuals could perform the job with reasonable accommodation. :contentReference[oaicite:11]{index=11}
This means an AI recruitment process should not be designed as though every applicant must interact with technology in exactly the same way.
AI Hiring and Age Discrimination
Quick Answer: AI recruitment systems can create age-discrimination concerns where an automated practice disproportionately disadvantages older applicants or otherwise violates applicable age-discrimination law.
Under U.S. federal law, the Age Discrimination in Employment Act generally protects individuals aged 40 and older.
The EEOC expressly identifies age as one of the protected characteristics covered by federal employment-discrimination laws. :contentReference[oaicite:12]{index=12}
AI Hiring and Gender Discrimination
Quick Answer: AI systems can potentially create sex-discrimination concerns if their design, data or outcomes disadvantage applicants based on sex or related protected characteristics.
The legal question is not simply whether the algorithm contains a field labelled “sex”.
The entire decision-making system may need to be evaluated.
AI Hiring and Race Discrimination
Quick Answer: Employers cannot assume that removing race from an algorithm automatically eliminates race-discrimination risk.
Proxy variables and historical patterns can reproduce demographic disparities indirectly.
Where federal employment-discrimination laws apply, employers must consider whether their AI-assisted practices produce unlawful discriminatory outcomes. :contentReference[oaicite:13]{index=13}
AI Hiring Compliance Framework
| Risk Area | Recommended Control |
|---|---|
| Historical bias | Training-data review |
| Disparate impact | Outcome testing |
| Disability discrimination | Accommodation process |
| Opaque vendor model | Vendor due diligence |
| Automation bias | Meaningful human review |
| Proxy discrimination | Feature analysis |
| Candidate privacy | Data-governance controls |
| Regulatory compliance | Jurisdictional assessment |
| Incorrect scoring | Validation and monitoring |
What Should an Employer Do Before Deploying AI Hiring Software?
Quick Answer: Employers should conduct legal, technical and operational due diligence before using AI in material employment decisions.
- Identify the purpose of the tool.
- Identify the data being processed.
- Determine which employment decisions it influences.
- Assess whether discrimination laws apply.
- Test for adverse outcomes.
- Review disability-accommodation requirements.
- Evaluate vendor claims.
- Document validation.
- Establish human-review procedures.
- Monitor performance after deployment.
What Should a Job Applicant Do If AI Rejected Them?
Quick Answer: An applicant who believes an AI system contributed to discriminatory treatment should preserve relevant communications, identify the employer and tool involved, request appropriate information where available and consider seeking legal advice.
Useful evidence can include:
- Job advertisement.
- Application materials.
- Recruiter communications.
- Assessment instructions.
- Interview records.
- Rejection communication.
- Accommodation requests.
- Any notice concerning automated decision-making.
Can an Applicant Sue an Employer for AI Discrimination?
Quick Answer: Potentially, if the facts satisfy the requirements of an applicable employment-discrimination law.
The applicant generally needs more than simply showing that AI was used.
The relevant legal elements depend on the particular claim and jurisdiction.
Possible issues include:
- Protected characteristic.
- Adverse employment action.
- Discriminatory treatment or impact.
- Causation.
- Employer coverage.
- Applicable procedural requirements.
Can an Applicant Sue the AI Company Instead?
Quick Answer: Potential claims against an AI or HR-technology vendor depend on the facts, contractual relationships, applicable employment law and other causes of action.
The employer is often the central actor in an employment decision because it controls whether and how the hiring recommendation is used.
But technology vendors can also face contractual, privacy, consumer-protection or other legal issues depending on their conduct.
AI Hiring Due-Diligence Checklist
| Question | Employer Action |
|---|---|
| What does the AI evaluate? | Document the model inputs |
| What decision does it influence? | Map the decision workflow |
| Could protected groups be disadvantaged? | Conduct impact testing |
| Can disabled applicants participate equally? | Provide accommodation mechanisms |
| Can the employer explain the decision? | Maintain appropriate records |
| What does the vendor guarantee? | Review contractual terms |
| Is human review meaningful? | Define review procedures |
| Does local law apply? | Conduct jurisdictional assessment |
Frequently Asked Questions
Can AI discriminate in hiring?
Yes. AI systems can potentially produce discriminatory outcomes through biased data, model design, proxy variables, inappropriate features or deployment practices.
Is AI hiring legal?
AI hiring can be lawful, but employers must comply with applicable employment-discrimination, privacy, accessibility and other laws.
Does Title VII apply to AI hiring?
Yes, where Title VII applies, employers remain subject to its employment-discrimination requirements even when AI is used in recruitment or hiring.
Does the ADA apply to AI hiring?
Yes. The ADA can apply to AI-assisted job application and hiring procedures and can require reasonable accommodation for qualified applicants with disabilities.
What is algorithmic discrimination?
Algorithmic discrimination generally refers to discriminatory outcomes or treatment resulting from an automated decision-making system.
What is AI bias in recruitment?
AI recruitment bias occurs when an automated recruitment system systematically disadvantages certain candidates or groups in a manner that may raise legal or fairness concerns.
Can an employer blame an AI vendor for discrimination?
An employer should not assume that using a third-party vendor eliminates its own employment-law responsibilities.
What is an automated employment decision tool?
An AEDT is a category of automated technology used to substantially assist or replace discretionary employment decision-making. NYC Local Law 144 specifically regulates certain AEDTs.
What is NYC Local Law 144?
It is a New York City law governing certain automated employment decision tools and requiring measures including bias audits, public disclosure of audit information and specified notices. :contentReference[oaicite:14]{index=14}
What is a bias audit?
A bias audit evaluates an automated employment decision tool for potentially discriminatory differences in outcomes across relevant demographic groups.
Can AI resume screening be discriminatory?
Potentially. Resume-screening algorithms can reproduce historical patterns or rely on features that indirectly correlate with protected characteristics.
Can facial recognition discriminate in hiring?
Potentially. Facial-analysis systems may produce different error rates or performance across demographic groups, creating legal and compliance concerns when used in employment decisions.
Can AI interview software discriminate against disabled applicants?
Yes. An assessment can disadvantage applicants with disabilities if it measures characteristics unrelated to the essential requirements of the job or fails to provide reasonable accommodation.
Does removing race from an AI system prevent discrimination?
No. Other variables can operate as proxies for protected characteristics.
Should employers allow humans to override AI decisions?
Meaningful human review can provide an important safeguard, but merely having a human approve an algorithmic recommendation does not automatically eliminate discrimination.
Can a rejected applicant challenge an AI hiring decision?
Potentially. The available remedies depend on the jurisdiction, employer coverage, facts and applicable employment laws.
How should businesses audit AI hiring systems?
Businesses should evaluate the system's purpose, inputs, validation, error rates, demographic outcomes, accommodation procedures, vendor controls and applicable legal requirements.
Conclusion
Artificial intelligence promises to make hiring faster.
It can process thousands of applications.
It can identify patterns.
It can rank candidates.
It can automate repetitive recruitment tasks.
But efficiency is not the same thing as fairness.
An algorithm can reproduce the assumptions contained in the data used to build it.
It can also introduce new forms of discrimination that are difficult for employers and applicants to see.
That is why the central question should not be:
“Is AI making the hiring decision?”
The better question is:
“Is the employment decision lawful, job-related, reliable and appropriately monitored regardless of whether a human or an algorithm performs part of the process?”
Existing U.S. employment-discrimination law already provides an important foundation.
The EEOC states that federal law prohibits employment discrimination based on protected characteristics and that neutral policies producing disproportionate negative effects can create legal issues when the applicable requirements are satisfied. :contentReference[oaicite:15]{index=15}
The disability context is particularly important.
AI systems that analyse speech, facial movements, response times or behavioural characteristics can potentially disadvantage applicants with disabilities.
The EEOC and DOJ have specifically warned employers about these risks and emphasised reasonable-accommodation considerations. :contentReference[oaicite:16]{index=16}
Local regulation is also becoming more important.
New York City's AEDT framework requires covered employers and employment agencies to undertake bias audits and provide specified notices before using covered automated employment decision tools. :contentReference[oaicite:17]{index=17}
This demonstrates a broader regulatory movement:
AI employment systems are increasingly being treated as governance issues rather than merely software purchases.
Employers should therefore maintain documentation concerning:
- What the system does.
- What data it uses.
- How it was validated.
- What risks were identified.
- How applicants can obtain accommodation.
- How adverse outcomes are monitored.
- How human review operates.
Employers should also remember that an AI vendor's promise that its software is “bias-free” is not a substitute for independent legal and operational due diligence.
Ultimately, an applicant does not become less deserving of legal protection simply because a computer made the decision.
The central legal principle is simple: an algorithm may make an employment decision, but the use of technology does not automatically excuse an unlawful employment decision.
Legal Disclaimer
This article is provided for general educational and informational purposes only. It is not employment, discrimination, technology, regulatory or legal advice and does not create an attorney-client relationship. Employment laws vary by jurisdiction. Employers and applicants should obtain advice from qualified counsel regarding specific circumstances.