How we read a study

You don't need a science degree to tell a strong study from a weak one. A few questions, asked before you get excited, go a long way. These are the ones we ask about every study PD Brief covers, and you can use them on anything you read.

1. How many people were in it?

Size is the most useful thing a non-specialist can check. A study of 20 people can suggest an idea, but it can't prove one. Small studies are unstable, and by chance alone a few of them will turn up a striking result that disappears when someone tries again with 200 people.

As a rough guide for treatment trials, under 50 people is preliminary. Fifty to 200 deserves attention. Several hundred, followed over years, is the kind of evidence that changes practice.

2. Was there a comparison group, chosen at random?

If everyone in a study received the treatment, and they improved, that improvement could be the treatment. It could equally be the natural fluctuation of the disease, the attention that comes with being in a study, or the placebo effect.

Parkinson's has an unusually strong placebo response. Expecting a benefit can trigger real dopamine release and produce measurable improvement in movement. Any Parkinson's trial without a placebo group should be read with that firmly in mind.

The strongest design is the randomised controlled trial: assign people to treatment or placebo purely by chance, so the groups are otherwise comparable.

3. Did anyone know who was getting what?

If participants know they're on the real drug, they tend to report feeling better. If the doctor scoring their symptoms knows, they tend to score more generously. Nobody has to be dishonest for this to happen. It's just how human judgement works.

Double-blind means neither side knows. It is the standard for good reason.

4. Was it in people, or in mice?

A lot of exciting Parkinson's coverage is about work in mice, rats or cells in a dish. That research matters, because it's where ideas come from, but most of those ideas fail once they're tried in people. Plenty of treatments have cured Parkinson's-like conditions in rodents. None of them has cured Parkinson's.

When you see a hopeful headline, check who or what was actually studied. That alone often explains it.

5. Did it measure what actually matters?

Some studies measure how a person feels and functions. Others measure a number in a scan or a blood sample. The second kind is faster and cheaper, but a drug can move a laboratory measurement without helping anyone.

Ask what changed: a person's ability to walk, dress and sleep, or a marker that is merely believed to correspond to those things.

6. Did it hit the target it named in advance?

Researchers declare a primary endpoint before a trial starts: the one main question it will answer. This exists to prevent a trial that failed from being rewritten around whichever measurement happened to look good.

If a report emphasises a benefit in a subgroup ("worked well in women under 60") while the main result was negative, that is a hypothesis for a future study, not a finding.

7. Is the effect big enough to feel?

Statistically significant does not mean large. With enough participants, a difference far too small to notice will still clear the significance threshold.

So ask the second question: how big was it, and would a person living with the disease be able to tell? The two questions are independent, and the second is often the more important one.

8. Who paid for it, and who ran it?

Funding does not invalidate research, and most drug trials are paid for by the companies that make the drug. Knowing who paid still tells you something. Independently funded studies of a treatment tend to report smaller effects than company-funded ones.

Good papers state their funding and conflicts of interest openly. The information is usually near the end.

9. Has anyone repeated it?

A single study is a data point. Science advances when independent groups reproduce a finding. The first report of anything, however striking, is provisional by definition.

That's why PD Brief keeps an archive instead of only showing the newest issue. Watching a claim get confirmed, narrowed down or overturned over several years teaches you more than any single paper.