Jeez, Now I Have to Worry About Thematic Meta-analysis

The problem with intellectual curiosity is that it leads you to worry about things that you did not previously know existed.  This would not be a problem were it not for the fact that you usually have no control over these problems.  It’s better to know about issues over which you have some control.

Today, my wife sent me a link to a warning on Facebook that many teas contained unacceptably high levels of pesticides.  It went on to say that the pesticides mentioned could predispose you to cancer.  It then gave you options: 1) drink the teas and take your chances, 2) avoid a list of teas (mostly commonly available teas sold in grocery stores in bags) 3) switch to white tea which has lover levels of the chemicals 4) buy from a list of organic loose leaf teas that had acceptable levels.

I did not know there was a problem, but I have manageable choices.

In a second article there was a critical analysis of a statistical tool for measuring the value of meta-analysis in studying “themes” of neurological and psychological problems.

This was technical, and I used to read studies using meta-analysis when I was working, and because I like technical/statistical studies (a definite mark of nerdiness) and because I used to wonder just how useful meta-analysis was I found it interesting.

Here’s the scenario.  Say you want to know whether there is a relationship between having an overly protective mother and the development of habit-tic. (Habit tics are those peculiar physical movements such as repeatedly squinting one’s eyes when nervous, or rubbing your nose frequently.  In neither of these is there a neurological problem.  It is just an acquired habit that somehow relieves tension).

You go to the literature and can’t find a study with – say – more than 5 individuals studied.  With studies like this the number of individuals studied increases the statistical “power” of the study.  Another way of saying that any finding is more likely to be valid.

So, a meta-analysis might take 20 studies with anywhere from 1 to 5 subjects giving more statistical probability with the caveat that each study needs to be examined for how well the study was done and possible bias on the part of the researcher and other compounding effects.

Typically, each study is scored and studies with higher ratings count more. 

In “thematic” studies you might look at all studies done on subjects with habit tic looking at anything else as a contributing factor.

Well the “funnel” test is a statistical tool to look for selection of only the studies that you like.  Ideally the plot should look like an inverted funnel with smaller studies – with more statistical error – being spread across the bottom and larger studies at the top of the funnel.  An asymmetric plot is a sign of selection bias.

An asymmetric plot suggesting publication bias

Well, it isn’t at all clear that there is any validity to doing “thematic” meta-analysis in the first place.  What does it mean that you have twenty studies of subjects with habit tics which studied everything from how often they went scuba diving to the ratio of the length of a subject’s ears to the distance between their eyes? Maybe, it’s just an excuse to get published.  Maybe buried in all of that data is a study that, while small, had a startling finding that needed a larger study.

Now, I feel that I need to warn people about the validity of these studies.  Not really.  I’m OK with the fact that I am an observer and not a participant now, but at an earlier age I might have worried about this.

Before knowing about it I would have been blissfully ignorant.

But, maybe I’m not as sanguine about this as I think.  After all, am I not writing about it?

And worrying about pesticides in my tea was off my radar until I got the warning.  And, because I’m naturally suspicious I have to go research the warning to find out whether the author works for one of the more expensive loose-leaf tea suppliers.  And, on and on.

Views: 135

Comment by JMac1949 Today on March 25, 2017 at 1:46pm

Hope for the best...

Comment by Rodney Roe on March 25, 2017 at 2:21pm

I thought tea had to be grown in the mountains of Nepal or something, but there is a tea plantation near Charleston, SC  I ordered some of their tea.  I ordered in bulk and it got dry before I finished it.  I drink a lot of hot tea, year round.

Comment by Rodney Roe on March 25, 2017 at 2:34pm

kosh, sorry.  When I deleted the other (double) post I deleted your comment.  You wrote, "I learned the other day that "organic" doesn't mean without pesticides. Great."  Organic doesn't mean much of anything anymore.  It has been watered down by statute.  I don't have a clue what GMO free means.  Especially since there are laws to protect those who market GMO plants from having to reveal that they have been genetically modified.

The Code of Hammurabi was written, according the preface, to "protect the weak from the strong" and then went on to protect the rich and powerful.  Nothing has changed over the last 3500 years.

JMac, that song is good for about every situation :-)

We had a brief movement here toward locally grown and marketed, organic, GMO free agriculture.  Actually, there are some pretty big farmers here who try to grow and market in the spirit of that movement, but they are not strictly organic.  The movement died mainly because the organizer was a sort of pie-eyed guy who went by "Feather" and he turned people off.  And he was not a local.

Comment by koshersalaami on March 25, 2017 at 3:44pm


Comment by Drew-Silla on March 25, 2017 at 8:17pm

I understand your trepidation, Mr. Roe. Before writing my award-winning Open Salon post, What Not to Put in Your Vagina, I spent weeks sorting through dozens of thematically dreary anti-meta-analyses pertaining to my thesis (most of them involving mentally challenged individuals with various sorts of dairy products) before realizing I could greatly facilitate the process by simply doing a Google Search employing the words "What Not to Put in Your Vagina." A word of caution, though; if you, too, should choose to go that route, be absolutely sure not to accidentally make it a Google Image Search. 

Comment by Rodney Roe on March 26, 2017 at 2:10pm

I'll keep that in mind, Drew-silla.  

You may not remember a comedian from the past who built a career around doing a sort of extended monologue that went something like, "You can call me, Ray, or you can call me Jay, or you can call me Ray Jay, or you can call me Ray Jay J., or you can call me Ray.Jay J.,jr, or you can call me, Ray Jay Johnson, or you can call me Ray Jay Johnson, junior,.but you doesn't have to call me Mr. Johnson.".

Comment by J.P. Hart on March 26, 2017 at 2:32pm

ain't none of us gettin' out of here alive

Jim Morrison

Thunder Bay, 1969

Comment by koshersalaami on March 26, 2017 at 2:45pm

Morrison certainly didn't

Comment by Drew-Silla on March 26, 2017 at 3:18pm

Live Fast

Die Young

Leave a good-looking corpse -

George Santayana, Madrid 1907

Jim Morrison managed two out of three, so good for him.

I'm not sure if I'm comfortable calling you Ray Jay, Mr. Roe. Would Rodney be alright?

Comment by Rodney Roe on March 26, 2017 at 6:38pm

I'm cool with Rodney.


You need to be a member of Our Salon to add comments!


How not to wake up

Posted by koshersalaami on January 23, 2019 at 5:21am 1 Comment

Setting Sail

Posted by Doc Vega on January 22, 2019 at 9:07pm 2 Comments

Cults in Our Midst

Posted by John Manchester on January 22, 2019 at 11:59am 6 Comments

Trump as role model

Posted by Dicky Neely on January 22, 2019 at 10:35am 1 Comment

The Unheard Listen

Posted by Robert B. James on January 22, 2019 at 9:09am 6 Comments

A Question About Tulsi Gabbard

Posted by Ron Powell on January 21, 2019 at 7:16pm 39 Comments

MLK; The dream is still alive

Posted by Dicky Neely on January 21, 2019 at 3:24pm 0 Comments

© 2019   Created by lorianne.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service