GirlChat #726377

Start A New Topic!  Submit SRF  Thread Index  Date Index  

no significance tests means results worthless

Posted by EthanEdwards on Friday, August 24 2018 at 11:23:31AM
In reply to not necessarily posted by Baldur on Thursday, August 23 2018 at 05:01:53AM

The simplest fundamental problem with the study is "... no tests of statistical significance for that rank ordering" of abuse combinations.

In social science, you have to use tests of statistical significance before you draw any conclusions. That is not a subtle methodological critique scientists would disagree about. It's an indisputable problem.

Mike Bailey (a scientist who Dissident respects) took a quick look at this and said he agreed with my analysis.

The "post hoc" fallacy notes that people are very good at coming up with explanations after the fact -- if you take two groups and tell them science shows one result or its opposite, both will come up with good-sounding explanations. I was wrong to suggest that the fact that I couldn't make sense out of two particular cases was what made the study unreliable. But when you take one methodological approach, such as Puzzled did, that "if you add abuse form Z to the combination XY and results are better (measured by average correlation coefficient) then Z is good for someone" then there is a great deal to explain besides what he draws attention to. But failure of testing for statistical significance nullifies the study and makes all of this just speculation.

Is it possible that the results would be significant if you did the tests, and they just didn't get around to doing them? I did a quick-and-dirty T test on the correlation coefficients in my base post between two conditions Puzzled compared and got a significance value of 0.666. In a study of this kind, with lots of coefficients, you would want a value below 0.01 and preferably 0.001 to make any inferences.

We can ask about the significance of individual correlations, as there are significance tests for those. As the authors note, one interesting result in Table 1 in the study is a notable correlation between both kinds of physical neglect and both kinds of sexual abuse. One mundane explanation for that is that if the kid is being left alone a lot (their definition is "lack of food, lack of shelter, inadequate hygiene, etc."), the parents would not be around to notice someone else doing sexual things with the child and stop it.

One thing we could have fun with on GirlChat is taking studies that we think are relevant to the pro- and anti-legalization positions and analyze them in detail.





Follow ups:

Post a response :

Nickname Password
E-mail (optional)
Subject







Link URL (optional)
Link Title (optional)

Add your sigpic?