Last month, we drew on the insight of a group of experts to look at ways to assess the credibility of scientific studies, or health articles citing scientific studies and describing their results (with varying degrees of accuracy). This month, we’re going to look at the different types of scientific studies, as well as their benefits and drawbacks. But let’s start by taking a step back and look at what all these studies have in common.

All scientific studies begin with an observation and a hypothesis, or educated guess on why it is so. Then, scientists will design some kind of experiment, or a method to get the information they need which will theoretically prove or disprove their hypothesis. Next, they run their experiment or collect their information, record and analyze their data, and then release their results.

Researchers design studies in a variety of ways. This is by no means a comprehensive list, but includes some terms you’re likely to come across—conveniently organized in alphabetical order. Knowing what the different types of studies are, and what the key terms mean, may come in handy the next time you’re analyzing the latest health claim and drawing your own conclusion. So let’s get to it.

Case-control studies

A case-control study finds the side effects of a drug or substance. For example, a case-control study might evaluate the relationship between, say, caffeine consumption and heart disease. One way to do this is to create a group of people with heart disease and a control group of people who have not. Then, researchers will control for factors such as age, economic status, etc. so that they’re comparing people who share those attributed. The two groups will then describe their caffeine intake, and those answers will be compared. (Unfortunately, the possibility exists that people who have suffered from heart disease would be more likely to remember their coffee drinking. This is called a recall bias.)

Clinical Trials

Clinical trials are used in medical research, as well as drug development, to test different types of drugs, medical devices, or other health interventions. They are the next step after smaller pilot studies, and are typically double-blind trials (defined below.)

Controlled experiments

These are experiments in which scientists change one specific factor, which is called an independent variable. They’ll keep everything else constant, and just change that one specific thing. Why? Because if you start a new diet and a new workout plan and start taking a new supplement, and have amazing results, you won’t know which of those things is specifically responsible for the result. That might be okay for you, but it wouldn’t necessarily be useful for anyone else. Controlled experiments are generally considered more reliable than observational/epidemiological studies, but sometimes present an ethical dilemma. For example, researchers would probably not ask pregnant women to drink a glass of wine a day to help them determine whether or not this would lead to fetal alcohol syndrome.

Crossover Studies

Ever wonder whether the experimental group in a study is just genetically superior to the control group? Crossover studies somewhat control for this by having the groups switch. After the trial, and a short break, the control group becomes the experimental group, and the experimental group is given the placebo. This allows researchers to analyze differences in individuals, rather than just groups.

Double-Blind Trials

Double-blind studies mean that both the patients and the clinicians are “blind,”—neither of them know which group is receiving the substance they are testing, and which group is receiving a placebo. (Most double-blind studies are “randomized,” so the groups are chosen randomly.) Not all types of studies can be blinded. For example, it’s be pretty difficult for patients and clinicians not to know what diet or exercise program they’re on. However, in many studies—such as those where the placebo effect can be very powerful—absolutely require blinding.

Epidemiological Studies

Also known as observational studies, epidemiological studies find associations in populations between, for example, lifestyle factors (or genetic factors) and the incidence of specific diseases. One example would be looking at the relationship between obesity and heart disease. The biggest benefit to these studies is that they are often long-term, and usually observe a very large number of people. The biggest drawback is that the research isn’t necessarily conclusive; it’s often correlative and not causal. For example, say a specific group of people who drink alcohol (or eat whole grains) happen to live longer. This doesn’t necessarily mean the alcohol, or whole grains, specifically lead to longevity.

In vitro studies

These are studies done in test tubes and petri dishes. They’re definitely a good starting point (antibiotics were discovered on a petri dish), but not quite as conclusive as studies taking place in an actual living being. The next step is in vivo research, in which experiments involve either people or living animals. Although animal studies sometimes accurately predict effects on humans, studies on humans are obviously more conclusive.

Meta-analyses

A meta-analysis takes all the studies in a specific area (as found in a systematic review of the available literature), analyzes those that fit within parameters they select (such as double-blind studies lasting more than 6 months which took place within the last 5 years) and comes up with a conclusion based on those results. Meta-analyses draw on data derived from systematic reviews.

Peer review

When an article has been peer-reviewed, that means that editors of an academic journal send it out to other scholars in the field (the author’s peers) who will take a look and provide their comments on whether the study is relevant, done properly, and so forth. While some good science may not make it into a peer-reviewed journal, and some poor science will inevitably sneak its way in, whether a study has been published in a peer-reviewed journal is another factor to look at in assessing its credibility.

Some Resources

Let’s say you read last month’s article, and this one, and are really itching to get your hands on the latest study you read... but you don’t want to dish out $54.99 for it and don’t have access to the university library, or even a friend with access to the university library... nor do you have a blog which you could write about the study in (in which case you could probably get a free press copy). Here are a couple of options:

PLOS: http://www.plos.org/
This nonprofit publisher and advocacy organization makes every article they publish available for free.

SciVerse: Sciencedirect.com
This site allows guests to access all citations and abstracts for free, along with a limited amount of full-text content. Articles can also be downloaded at a fee, but it’s typically a bit discounted.

In addition, conspiracy theorists amongst us may wish to check out the following three sites.

HealthNewsReview: http://www.healthnewsreview.org/
This site critically evaluates health care journalism and advertising, improving public dialogue on health care and helping people like you or me analyze health care information on their own.

PharmedOut: http://pharmedout.org/
This Georgetown University Medical Center project, led by a team of academic decisions, educates readers on how pharmaceutical companies influence prescribing, and helps foster access to unbiased information about drugs.

ProPublica’s Dollars for Docs: http://projects.propublica.org/docdollars/.
In case that last link made you feel a little bit paranoid, this site allows you to look up your own doctor to see if he or she is receiving payments from drug companies in exchange for promoting the drugs.