Definition
Algorithmic Bias refers to systematic and repeatable errors in a computer system that create unfair outcomes, typically by privileging one group of people over another. These biases can emerge from the design of the algorithm itself or, more commonly, from the data used to train the system.
Why It Matters
Algorithms are not “neutral”; they inherit the biases of their training data and their creators. If left unchecked, algorithmic bias can automate and scale discrimination in hiring, policing, and lending, creating “digital redlining” that is invisible but devastating.
Core Concepts
- Data-Driven Bias: If historical data contains human prejudices (e.g., gender bias in hiring or racial bias in lending), an AI trained on that data will learn and amplify those prejudices as “patterns” to be optimized.
# Example of encoded bias via proxy variables
def approve_loan(applicant_data):
# Zip code can act as a proxy for race/socioeconomics
if applicant_data['zip_code'] in high_risk_zones:
return False
return True- Redlining (Implicit Bias): Even when protected categories (like race) are removed, an AI can “reconstruct” them using proxy variables (like zip codes or shopping habits), leading to the same biased outcomes.
- Fairness vs. Accuracy Trade-off: Enforcing mathematical definitions of fairness (e.g., equal false-positive rates across groups) often results in a slight decrease in overall prediction accuracy or profit.
- Inscrutability: Deep learning models are often “black boxes,” making it difficult to explain why a biased decision was made or to provide a “right to an explanation” for affected individuals.