Though acupuncture is often used to treat seasonal allergies, there has been limited scientific evidence that it actually works. A new study finds that it does — kind of, maybe, but not really.

The study, led by Dr. Benno Brinkhaus of the Charité-University Medical Center in Berlin, examined the outcomes of 422 patients with seasonal allergic rhinitis (pollen allergies) over an eight-week period. They were randomly assigned to different conditions to evaluate whether acupuncture helps alleviate symptoms compared to the drug cetrizine and fake acupuncture.

NEWS: Now Cough … For Three Weeks

Though the study has been widely reported as showing that acupuncture likely works, there are important red flags suggesting that the results may not be all they are cracked up to be.

The Placebo Effect

First, there is strong evidence that much of the improvement in allergy symptoms may simply have been caused by the placebo effect. When a patient is given a treatment and told that it will help them, often it will — even if there’s no active ingredient. A person’s expectation that they will feel better often actually makes them feel better subjectively.

One problem with trying to test acupuncture’s effectiveness apart from the placebo effect lies with the treatment itself. If doctors are testing a drug, they can make pills that look and taste identical, both with and without active ingredients. Because the patient doesn’t know if they are taking a placebo or not, the difference in outcomes can be attributed to the medicine. But in the case of acupuncture, it’s very difficult to fool someone into thinking they’re not being poked with needles.

Statistical Versus Clinical Significance

There’s another, more serious issue. The researchers noted, “We found that acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use after eight weeks of treatment compared with sham acupuncture and with alone, but the clinical significance of the findings remains uncertain.”

To see why is this a problem it’s important to understand the difference between statistical and clinical (or practical) significance; they are very different things. Let’s say that in a clinical trial Cold remedy A has been shown statistically to work better on some measure than Cold remedy B. Does that mean that Cold remedy A is better than Cold remedy B?

News: Coughing Robot Spews ‘Flu Germs’

Not necessarily, because outside of the laboratory and in the real world, people look for practical, not just theoretical or statistical significance. Maybe the average person sneezes 20 times per hour using Cold remedy A but only 18 times per hour using Cold remedy B. So what? If Cold remedy A shortens the duration of a cold by half a day that might be a good reason to choose it — unless it costs twice as much as its competitor, or has significantly more side effects, or any number of other reasons.

Or, to use an example outside of the medical field, let’s say that research is done on the price of food in a certain metropolitan area, and, on average, Supermarket A tends to be about 2 percent cheaper than Supermarket B after prices are compared and calculated. Does that mean that Supermarket A is better or worth shopping at more often? Not necessarily. For example that 2 percent difference is only a statistical average across many products that not every household uses, and may not necessarily reflect the products you regularly buy. Even if it does, the savings may not be worth the effort to save $1 on a $50 purchase. This is especially true if Supermarket A is out of the way: there’s little point in spending $3 in gasoline to drive across town to save $1.50 on groceries — not to mention the extra time and hassle.

Flu, Whooping Cough And Vomiting Virus: What’s Going On?

Indeed, the acupuncture researchers acknowledge that “no effect of active treatment on individual symptom severity could be shown.” In other words though the improvement was statistically significant across the study groups, any given person’s sneezing and itchy eyes were just as bad whether they used acupuncture or not. There may be compelling evidence that acupuncture can treat allergies, but this study is not it.

The study was published in the February 2013 journal Annals of Internal Medicine.

Image: iStockphoto