Prevagen Memory Study Falls Short

UPDATE: In February 2019, an appellate court ruled that the FTC and New York can proceed with their lawsuit against Quincy Bioscience. For additional updates on the case, click here. Our original post follows.

In 2011, four years after launching Prevagen, Wisconsin-based Quincy Bioscience embarked on a study to prove that the active ingredient in the supplement — apoaequorin — improves memory. It did not yield the results Quincy was hoping for. In fact, the Madison Memory Study failed to show a statistically significant improvement in the treatment group over the placebo group — scientist speak for Prevagen wasn’t any better than a placebo at improving memory.

But rather than pack it in and start afresh with a new study, like a high school biology student whose hypothesis has been proven wrong but who still needs an A to pass the class, Quincy concocted new, less reliable ways to look at the completed study. It instructed its team of researchers to scour the data in hopes of finding something — anything — that might support its hypothesis that Prevagen improves memory. This resulted in more than 30 “post hoc” analyses, which in turn yielded two overlapping subgroups whose members displayed cognitive test results that Quincy apparently deemed favorable to its aims. These would serve as the basis for claims that Prevagen “improves memory.”

It’s the kind of substantiation for a health-related claim that would rarely hold up in court. Yet that’s exactly what it did last September when a district court dismissed the FTC and New York attorney general’s case against Quincy on the grounds that the post hoc analyses of the two subgroups, while so far removed from the initial hypothesis of the study as to render them misleading, provided sufficient support for Prevagen’s memory-improvement claims.

This week, TINA.org joined with AARP and its charitable arm, AARP Foundation, as well as the National Consumers League and a group of advertising law academics in filing a legal brief in support of reversing the dismissal. In the brief, TINA.org and its partners argue that the district court erred in its interpretation of post hoc analyses as competent and reliable scientific evidence — the legal standard for substantiation when it comes to making health claims.

The brief states:

In direct contradiction to the Court’s reasoning, post hoc analyses are not part of prospective, double-blind, clinical trials; they are separate retrospective analyses of trial data performed after the study has concluded to try to find patterns that were not primary objectives of the study.

The brief goes on to explain the dangers in that:

Once the study results are sliced and diced in multiple overlapping ways (in a practice known as data dredging or P-hacking, wherein P is a measurement of statistical probability), the researchers have decreased their sample sizes and simultaneously increased the chances of getting a false positive.

Indeed, in the two subgroups that Quincy claims are proof that Prevagen actually improves memory, only one computerized assessment with purported positive results — a task that measured “visual learning” — was shared across both groups. The brief refers to this as “a classic case of elevating likely false positives to the status of a proven claim.”

The FTC and attorney general’s lawsuit was sparked by a 2015 TINA.org investigation into the deceptive marketing of Prevagen and subsequent complaint to the federal agency. At the time, the market for dietary supplements and products aimed specifically at brain health was valued at $2.3 billion, as more than one in three people over the age of 70 suffer from a form of memory loss. These are the people that Quincy targets with its unsubstantiated memory-improvement claims for Prevagen, sales of which totaled $165 million as of mid-2015. It is a fiction that must be stopped. The brief concludes:

If a federal district court was misled by Quincy’s marketing and proffered substantiation, what hope is there for the millions of aging Americans concerned about memory loss and cognitive decline to accurately differentiate scientific facts from Quincy’s fiction?

The FTC and New York attorney general are appealing the decision, arguing the case was improperly dismissed.

Find more about TINA.org’s investigation into Prevagen here.

UPDATE 11/20/20: A federal district court has approved a class-action settlement over claims that Quincy Bioscience falsely advertises that Prevagen can improve memory. TINA.org had objected to the proposed settlement on the grounds that the terms were unfair to consumers, who are only eligible to receive a maximum refund of $12 unless they kept receipts. The FTC and state of New York’s lawsuit against Quincy is pending. For updates on that case, click here.