As a black statistician, I felt the need to look into this plot going around Twitter. As far as I can tell, it's being used to push the narrative that black people's poor relationship choices are causing bad outcomes for black children.
At first, I thought the analysis looked sloppy based on just the plot but I decided to dig in and realized the methodology was much better than I thought and it actually addressed my major concerns BUT the narratives on Twitter don't seem to reflect what's in the actual report.
1. CAUSATION. The actual analysis WARNS against making CAUSAL claims and admits that a "number of factors not measured...may confound the associations between family structure and child outcomes".
In other words, the report itself admits that they probably didn't do a good job controlling for all the possible causal factors because there are so many and it's so complex.
So bottom line: We shouldn't be saying that family structure or race CAUSE these outcomes because SCIENTIFICALLY and STATISTICALLY the analysis doesn't support that.
2. INCOME. The second thing which I was wondering about is this: two parents can have two incomes and one parent might only have one income. So this could be an issue right? What do they say about that?
Basically, they say income and having two parents in the home are highly coupled variables to the point where the individual effects of each variable would be hard to separate in an analysis. EXACTLY!
In my opinion, when you have a situation like this where you can't really tell if it's income or the two parents, the FAIR thing to do is to repeat the analysis looking at things based on income levels and see if that gives a more convincing story.
3. RACE. At first, I was going to criticize this analysis for not considering things on a multiplicative scale. What do I mean? Well, I noticed that 36/18 = 2 and 53/28 is almost 2. So it looks there might be no racial difference here on the multiplicative scale.
Assuming causality (which is a HUGE assumption), the idea would be that having two parents in the home MIGHT double the chances of graduating regardless of race.
Also, 24/14=1.71 and 18/8=2.4. Somewhat close again. At this point, I was starting to think race might not even be a factor here. I almost wrote a thread criticizing the analysis for not considering this and for not using a real statistical model like logistic regression but...
Look at that! They did do a logistic regression! And now we can see the numbers are very similar for both races. I bet if they added confidence intervals, we would see an overlap suggesting no evidence of race being a factor. So the racial interpretation seems inappropriate.
The logistic regression is much less sexy than the plot but it's probably the more accurate story. Bottom line: the analysis doesn't support the conclusion that race is a factor.
So basically, as far as I can tell, they did a pretty statistically defensible analysis but for some reason the framing on Twitter is misleading and introduces a racially-tinged, causal interpretation that's not supported by the data analysis.
Here's the report if you want to check it out for yourself: ifstudies.org
Loading suggestions...