Collider Bias: How Data Filtering Creates False Correlations
A collider is a variable C influenced by two other variables A and B through causal paths: A → C ← B. Conditioning on C—whether through filtering or stratification—creates a spurious correlation between A and B.
Real-world examples:
- Age → Usage → Quality (surviving items appear higher quality).
- IQ → College Admission → Parental Wealth (among students, wealthier ones seem less intelligent).
- Talent → Success → Work Ethic (successful talented individuals appear lazier).
In data science, this leads to misleading models: gradient boosting on filtered data inherits the bias.
Simulating with Uniform Data
We generate 1,000 independent points for A and B from uniform(0,100). The collider flag is set if either A > 50 or B > 50.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from statsmodels.nonparametric.smoothers_lowess import lowess
NUMBER_OF_VALUES = 1000
A_RANGE = (0, 100)
B_RANGE = (0, 100)
A_THRESHOLD = 50
B_THRESHOLD = 50
a = np.random.uniform(*A_RANGE, NUMBER_OF_VALUES)
b = np.random.uniform(*B_RANGE, NUMBER_OF_VALUES)
data = pd.DataFrame({'a': a, 'b': b})
data['flag'] = (data['a'] > A_THRESHOLD) | (data['b'] > B_THRESHOLD)
print(f"Correlation between A and B: {data['a'].corr(data['b']):.2f}")
print(f"Correlation where flag is True: {data[data['flag']]['a'].corr(data[data['flag']]['b']):.2f}")Results:
- Full dataset: corr ≈ 0.00
- Filtered subset: corr ≈ -0.34
The scatterplot visualization shows how removing non-surviving points (blue) shifts the trend downward.
sns.scatterplot(data=data, x='a', y='b', hue='flag', palette={True: 'red', False: 'blue'}, alpha=0.5)
sns.regplot(data=data, x='a', y='b', scatter=False, color='black')
plt.show()After filtering, the lmplot shows a negative slope: the upper-right quadrant becomes empty, disrupting balance.
Realistic Scenario: Normal Distribution
Now we use np.random.normal: A ~ N(50,5), B ~ N(30,5). The collider is probabilistic: normalize values, compute selection probability as max(a_norm, b_norm)*1.5, clip to [0,1], then assign flag based on random draw.
a = np.random.normal(loc=50, scale=5, size=NUMBER_OF_VALUES)
b = np.random.normal(loc=30, scale=5, size=NUMBER_OF_VALUES)
normal_data = pd.DataFrame({'a': a, 'b': b})
a_norm = (normal_data['a'] - normal_data['a'].min()) / (normal_data['a'].max() - normal_data['a'].min() + 1e-6)
b_norm = (normal_data['b'] - normal_data['b'].min()) / (normal_data['b'].max() - normal_data['b'].min() + 1e-6)
selection_prob = np.maximum(a_norm, b_norm) * 1.5
selection_prob = np.clip(selection_prob, 0, 1)
normal_data['flag'] = np.random.random(len(normal_data)) < selection_prob
print(f"Correlation in full dataset: {normal_data['a'].corr(normal_data['b']):.2f}")
print(f"Correlation when flag is True: {normal_data[normal_data['flag']]['a'].corr(normal_data[normal_data['flag']]['b']):.2f}")Results:
- Full dataset: corr ≈ 0.03
- Filtered subset: corr ≈ -0.09
The effect is weaker but still significant in A/B tests, field studies, or machine learning pipelines.
How to Avoid Collider Bias in Analysis
- Draw DAGs: Use directed acyclic graphs to identify colliders before analysis.
- Avoid outcome-based filtering: Analyze the full population or apply IPCW (inverse probability censoring weighting).
- Check correlations pre- and post-stratification: Compare trends across groups.
- Use instrumental variables: For causal inference when bias is suspected.
- Leverage Bayesian networks: Explicitly model dependencies and conditional relationships.
Key Takeaways
- Collider bias occurs when conditioning on a common cause, creating artificial negative correlations.
- Simulation with uniform data yields corr ≈ -0.34; normal data gives ≈ -0.09—effect scales with data structure.
- Related to survivorship bias, Berkson’s paradox, and selection bias.
- Always sketch a DAG before filtering data.
- In ML: filtered data leads to biased models, resulting in under- or overestimation of feature importance.
— Editorial Team
No comments yet.