The problem with intellectual curiosity is that it leads you to worry about things that you did not previously know existed. This would not be a problem were it not for the fact that you usually have no control over these problems. It’s better to know about issues over which you have some control.
Today, my wife sent me a link to a warning on Facebook that many teas contained unacceptably high levels of pesticides. It went on to say that the pesticides mentioned could predispose you to cancer. It then gave you options: 1) drink the teas and take your chances, 2) avoid a list of teas (mostly commonly available teas sold in grocery stores in bags) 3) switch to white tea which has lover levels of the chemicals 4) buy from a list of organic loose leaf teas that had acceptable levels.
I did not know there was a problem, but I have manageable choices.
In a second article there was a critical analysis of a statistical tool for measuring the value of meta-analysis in studying “themes” of neurological and psychological problems.
This was technical, and I used to read studies using meta-analysis when I was working, and because I like technical/statistical studies (a definite mark of nerdiness) and because I used to wonder just how useful meta-analysis was I found it interesting.
Here’s the scenario. Say you want to know whether there is a relationship between having an overly protective mother and the development of habit-tic. (Habit tics are those peculiar physical movements such as repeatedly squinting one’s eyes when nervous, or rubbing your nose frequently. In neither of these is there a neurological problem. It is just an acquired habit that somehow relieves tension).
You go to the literature and can’t find a study with – say – more than 5 individuals studied. With studies like this the number of individuals studied increases the statistical “power” of the study. Another way of saying that any finding is more likely to be valid.
So, a meta-analysis might take 20 studies with anywhere from 1 to 5 subjects giving more statistical probability with the caveat that each study needs to be examined for how well the study was done and possible bias on the part of the researcher and other compounding effects.
Typically, each study is scored and studies with higher ratings count more.
In “thematic” studies you might look at all studies done on subjects with habit tic looking at anything else as a contributing factor.
Well the “funnel” test is a statistical tool to look for selection of only the studies that you like. Ideally the plot should look like an inverted funnel with smaller studies – with more statistical error – being spread across the bottom and larger studies at the top of the funnel. An asymmetric plot is a sign of selection bias.
An asymmetric plot suggesting publication bias
Well, it isn’t at all clear that there is any validity to doing “thematic” meta-analysis in the first place. What does it mean that you have twenty studies of subjects with habit tics which studied everything from how often they went scuba diving to the ratio of the length of a subject’s ears to the distance between their eyes? Maybe, it’s just an excuse to get published. Maybe buried in all of that data is a study that, while small, had a startling finding that needed a larger study.
Now, I feel that I need to warn people about the validity of these studies. Not really. I’m OK with the fact that I am an observer and not a participant now, but at an earlier age I might have worried about this.
Before knowing about it I would have been blissfully ignorant.
But, maybe I’m not as sanguine about this as I think. After all, am I not writing about it?
And worrying about pesticides in my tea was off my radar until I got the warning. And, because I’m naturally suspicious I have to go research the warning to find out whether the author works for one of the more expensive loose-leaf tea suppliers. And, on and on.