My Pet Peeves: Inflated Statistics

Standard DeviationI’m a science geek.  It comes with getting a PhD.  But I also happen to have received a minor in Psychometrics during my graduate studies because I love statistics.  I know, some of you working with stats will think I’m absolutely bonkers, but I can’t help it.  I could honestly spend hours a day sitting and doing stats, and before my daughter, I did.  It’s why I’m as honest as the data will allow in my posts and while people may disagree with my interpretations, I’ve yet to be challenged on the data at hand (outside of individuals trying to use anecdotal data to contradict research).  Numbers are numbers.  The way we interpret them is varied, and it’s what makes statistics so fascinating, but at the end of the day, you have certain effects and they may be big, small, non-existent, negative, or a million other things, but they are what they are.  Contrary to popular belief, you cannot fake results.  You can omit certain important qualifiers—as they used to do when studying co-sleeping by not examining smoking, drinking, or other risk factors—or take a biased sample, but unless you make up your data, the numbers you get are the numbers that are there.  These numbers will vary from study to study and the more convergence you have between studies, the more confident you can be that the results you’ve got are real.  This is why systematic reviews and meta-analyses are so important – they not only amalgamate responses but are also able to judge the studies qualitatively as it’s true that some studies are better than others.

So it pisses me off something fierce when I see inaccurate statistics about something I believe in.  And oddly I’m not talking about stats that go against what I believe because you can always fight those if you know your data.  I’m talking about people who share the same belief but use false statistics to make a point.  Let me give you a couple examples…

Co-Sleeping Statistics

In November 2011, Dr. Dan Flanders posted the following on the Analytical Armadillos site:

Some stats to put this into perspective: 150,000 children die each year in the US as a consequence of motor vehicle accidents. Approximately 2500-5000 SIDS deaths occur per year in the US (many of which are not attributable to bedsharing). One could argue that banning children from automobiles would have a far greater beneficial impact than recommending against bedsharing. But we are generally comfortable with the risks of driving. On the other hand, we are led feel far less comfortable or far more guilty sleeping with our babies despite the risk being orders of magnitude lower.

I called him out on the numbers because 150,000 seemed a tad high, especially given that I’d recently done a piece and examined car deaths myself.  He admitted it was a typo and supposed to be 15,000 and that included children up to 18, but even that was still high in my mind.  Using the site he sent me to[1], I got 4,373 for children 0 to 18 and 437 for children 0-2.  That’s far less than the SIDS number he reported, which comes from the national SIDS site and shall be taken as accurate.

Circumcision Statistics

A few weeks ago I was sent to an older post by Dr. Momma, who I normally adore.  It was on circumcision statistics, again something I have become rather well-acquainted with given my recent 4-part piece.  Here are the statistics listed in the piece:

Out of 100 Circumcised boys:
75 will not readily breastfeed post-op
55 will have adverse reactions from the surgery
35 will have post-op hemorrhaging to one degree or another
31 will develop meatal ulcers
10 will need to have the circumcision surgery repeated to fix prior surgical problems/error

8 will suffer infection at the surgical site
3 will develop post-operative phimosis
2 will have a more serious complication (seizure, heart attack, stroke, loss of penis, death)
1 will require additional immediate surgery and sutures to stop hemorrhage
1 will develop fibrosis
1 will develop phimosis
1 will be treated with antibiotics for a UTI (urinary tract infection)
1 will be treated with antibiotics for surgical site infection
Of those who do receive pain medication for the surgery (about 4% of those boys undergoing circumcision in the U.S.) some will have adverse reactions to the pain medication injected.

Here’s the problem, while Dr. Momma gives us a list of books for which she credits these statistics and claims they are an average of all studies done over a 20-year period up to 2009, that’s just not true.  I don’t doubt some of the books she used for the data did suggest such a thing (in fact, it’s quite likely), but somewhere along the line, someone’s lying or seriously misinformed.  There are no doubt complications, and sometimes very serious complications, to circumcision which is why I’m not a supporter, but they aren’t this.  For example, the idea that 35% will have post-op hemorrhaging is not based on any average, and I can find only one study as cited in George Kaplan’s 1983 review, and even that review had a range of 0.1% to 35%[2].  Furthermore, the most recent systematic review was done in 2010 (far more recent) and found an average incidence of bleeding of 1%[3].  In this recent comprehensive review, the range of all complications for neonatal circumcision was 0 to 16% with a median rate of 1.5%.  If 1.5 out 100 will experience any complication, which includes all of the above (except breastfeeding), how can those numbers hold?  If even the highest of studies finds a 16% rate, how can 35% experience excessive bleeding?

Now you may wonder what happened to that 35% study which was in Kaplan’s review but not the most recent.  Good question.  Remember I mentioned that systematic reviews and meta-analyses analyze the quality of papers?  Well, ones that don’t fit the bill, don’t get included and this means case studies or case series or studies in which not enough information is provided to get an accurate assessment of the methods, the measures, and the outcomes.  The other issue is time.  Kaplan’s review was done in 1983 and the studies that found a 0.1% to 35% complication rate were done in the 1950s and 1960s.  With so much changing in the medical field, including the use of surgeons to perform circumcision, many of these studies are considered outdated and of little use to understanding the current picture.

Now there are arguments and one study that suggests the number of circumcision-related deaths reported is too low[4] and generally I think they’re right.  However, they suggest that 117 infants in the US die each year and I think that number is also a little disingenuous.  Why?  Because to get that they do the same kind of statistical faux-pas as the anti-co-sleepers do by ignoring mediating factors.  For example, on one site[5], they provide cases of infants who react to anesthetic or had heart surgery.  Yes, you can say that perhaps they would not have died if they hadn’t gone in for a circumcision, but in the case of anesthesia, if the child had an adverse reaction, he would have had it at any point in his life.  If he had to have a tooth pulled or surgery for something else, the same result would have occurred and no one would say the dentist was wrong.  This is why they cannot classify this as a circumcision-related death.  But even if we accept 117 as factually true, it would equal a death rate of 0.0078% based on WHO data for the number of circumcisions in the US.  Kinda far from 2%, even if it is included with a few other serious complications, wouldn’t you say?

Some of you may wonder what my problem is here.  Doesn’t it sound like I’m against co-sleeping and for circumcision?  Those of you who read this regularly know that neither is even close to the truth, so please any newcomers, take my word for it.  But if I’m for co-sleeping and against circumcision, shouldn’t I be happy people are sharing these somewhat skewed statistics?  Won’t that help to move more people?

I don’t think so.  In fact, the reason I get so pissed off is that I think this type of misinformation has the opposite consequence.  Take the co-sleeping example, if people actually go do the analyses, suddenly they find what I did, that in fact in the same age group, the number of infants killed in car accidents is a faction of those killed by SIDS.  Furthermore, most people already think too many kids die in car accidents so to see a five-fold increase for SIDS, well, it looks like those anti-co-sleepers have something going for them, doesn’t it?  Forget the fact that the SIDS numbers don’t have accurate assessments of co-sleeping versus cot death, or even safe co-sleeping versus unsafe co-sleeping.  All people see is approximately 2500 infants die each year from SIDS and people are telling me that co-sleeping is a big culprit.  By providing inaccurate information, the point has been flipped on its head.

Now what about circumcision?  Again, you get the same problem.  Suddenly the rates of complications are much lower and people who are on the fence don’t see it as that big a deal.  One and a half percent?  That’s small.  Less than 1/1000th of a percent risk of death?  That’s tiny, especially if that’s the lenient estimate!  And as pointed out in most articles, the risks can be made much smaller by taking care to make sure a qualified surgeon is used and that the infant is healthy.  By presenting misleading evidence it detracts from the real statistics and the real problems associated with circumcision.  The American Academy of Family Physicians says the death rate for circumcision is 1 in 500,000.  We’ll assume that’s an underestimate, but it’s the number the people on the fence will look to, so why not fight to say that that’s still far too many for an elective and frankly useless procedure?  Why not focus on the fact that 1.5% is STILL A LOT even if it’s not the inflated numbers you’d rather see?  The most recent estimate is that 33-50% of US boys are circumcised each year, and there are approximately 2 million boys born each year so we get 640,000 to 1 million circumcisions.  With a 1.5% complication rate, that’s between 9,750 and 15,000 boys who will suffer a medical complication.  Why isn’t that enough?  Not to mention that you also have one of the biggest arguments (that I’m pleased Dr. Momma did bring up) in that all of these infants have lost a part of themselves without their consent.

Look, I’m not trying to knock people sharing information they believe is true, but it’s not like we’re promoting ideas that are readily accepted, and so we need to check and double check that data before we start sharing information.  When there are city-funded campaigns against co-sleeping and doctors saying we need to institute routine circumcision, we have to be very, very careful about the data we post.  The last thing we want is to turn people off because they think we’re the ones being disingenuous with our message.   Don’t we have a big enough hill to climb without sabotaging our climb up?



[2] Kaplan GW.  Complications of circumcision.  Urologic Clinics of North America 1983; 10: 543-9.

[3] Weiss HA, Larke N, Halperin D, Schenker I. Complications of circumcision in male neonates, infants and children: A systematic review. BMC Urology 2010; 10: doi:10.1186/1471-2490-10-2.

[4] Bollinger D. Lost boys: An estimate of U.S. circumcision-related infant deaths. THYMOS: Journal of Boyhood Studies 2010; 4: 78-90.

[5] http://www.circumstitions.com/death.html

—–

The following link will take you to your local Amazon:

Comments

  1. JJoleen says

    Brilliant post, brilliant point. I used to hate stats but now i finally understand them i am starting to love them i.hate to see them falsely used too though.

  2. Jessica says

    Have you chanced upon this dreadful example? http://www.naturalnews.com/033858_unvaccinated_children_health.html
    Its so important to be able to recognise flaws in the data collection method. In this case a voluntary questionaire is used to collect information from non-vaxing families from many countries, and this information is compared to official health data on German children. No conclusions whatsoever can be drawn from such a comparison, yet the author concludes that unvaccinated children have half the rates of every imaginable disease compared to vaccinated children. Either this author has no science training, or is deliberately misleading his readers.
    I lament that schools spend so long teaching trigonometry, calculus, long division etc and very little time teaching statistics, and specifically the interpretation and critique of statistics. This is such an important life skill. We are presented with stats every day by news media, doctors, schools, yet the majority do not have the skills to avoid being tricked by bad or deliberately misleading reporting.

  3. says

    My personal pet peeve with statistics is bad grouping. Like the options are 1) very positive 2) positive 3) neither positive nor negative 4) negative 5) very negative 6) undecided. Now, I see how it would make sense to report both 1 and 2 responses as ‘positive’ for a result category. But then studies do things like report 1, 2, 3, and 6 as ‘for’ and only 4 and 5 as ‘against’. Which is very misleading and leads to greatly inflated statistics. On a recent health questionair for insurance purposes (so you *know* it’s going into some statistical database somewhere) there was a question about tobacco use which had the options ‘smoke everyday’, ‘smoke some days’, ‘have smoked in the past but do not currently smoke’ or ‘have never smoked’. Ok, not a bad list of options, but compair that to the illicit drug use question: ‘daily’, ‘some days’, ‘rarely or never’. So someone who uses drugs (illicitly) once or twice a month is being grouped into the same category as someone who has never done an illicit drug. Can you imagine the statistical analysis for someone with an agenda? ’100% of respondants admitted to at least rare drug use’.

    • says

      That’s another big problem. Typically good researchers know how to handle their options (including using analyses appropriate for categorical data), but survey researchers tend to truly screw the pooch on this one!

  4. Jess says

    I agree completely! I have 5 children, 2 girls, 3 boys, and i am also a supporter of co-sleeping and against circumcision!

  5. gina bottello says

    Great article, extremely valid points you make! And I totally agree, it’s irksome when people undermine the validity of our opinions with ridiculous seeming (and false) statistics, when there exist perfectly valid ones to support them. I think there is one small mistake, unless I am incorrect in my understanding – 117 deaths is .000078% of 150,000,000. I think you meant to write .0078% (117 out of 1.5 million circumcisions per year is .0078%).

Leave a Reply

Your email address will not be published. Required fields are marked *