When the Key to Good Genetics Research Isn't in the Genes

In the last couple of weeks, two new papers have had genetics enthusiasts buzzing: one a study that turned up 95 gene variants linked to cholesterol levels, and the other a similarly designed study of personality traits that turned up no genes at all. There must be a reason the findings came out so differently. But it probably isn't that cholesterol is influenced by genes and personality is not. Both run strongly in families. Cholesterol and personality are subject to environmental influences, yes—but previous work, such as studies of twins and adoptees, has suggested that genes set the predispositions and the environment sculpts them.

A better explanation for the differing findings may be something less obvious: either that cholesterol is influenced by a different type of genes than personality is, or that it's much easier to measure. (Both could be true.) If the problem turns out to be primarily the latter issue—cholesterol is easy to measure, personality isn't—it will have all kinds of important implications for how science gets done, not just in genetics but also in any number of medical fields in which diagnosis and description are somewhat subjective arts.

The cholesterol study, published in Nature, has a lot to recommend it. Two thirds of the genes the scientists point to have never been linked to lipid levels before. The researchers have gone to great pains to make sure their results are real. "The number of false positives on this list should be zero," says study coauthor Sekar Kathiresan, director of preventive cardiology at Massachusetts General Hospital and an associate member of the Broad Institute of MIT and Harvard. The scientists looked for the genes in many populations of different ethnicities (most studies of this kind aren't so comprehensive). They also began to explain what some of their results might mean, exploring in fine resolution what one of the newly identified genes does, biochemically speaking, in mice.

The paper is an important volley in the ongoing debate about how much scientists can learn from genome-wide association studies (GWAS), of which it is one. In the past, some GWAS efforts have come up disappointingly empty, as the personality study did. GWAS, which examine the DNA for small mutations that vary from person to person, are necessarily limited compared with studies that sequence the entire genome. They have the statistical power to find genetic variants that are common in the population being studied. Most of these variants nudge people's health risks up or down just a tad. Even if rare, powerful variants are more important in determining a person's health, a GWAS will overlook them. GWAS may also miss genes that cause disease because they're structured in peculiar ways (for instance, there are too many identical copies of them in the genome, as if they've been accidentally Xeroxed).

The bigger its sample size, however, the more a GWAS can find, and the Nature paper has an enormous sample size—it looks at 100,000 people, five times as many as have been studied before. It may have found as many as half of all the genes that influence cholesterol levels. (Kathiresen thinks there are about 200 such genes.) Its findings are being received in some quarters as "a true testament to the success of GWAS" and a validation of the method's usefulness. The study suggests that scientists shouldn't necessarily abandon GWAS techniques, even if the initial results are underwhelming; instead, they should apply the techniques to bigger groups of people.

The personality study is a GWAS, too. It has a much smaller sample size: 5,117 subjects. But would simply increasing the number of subjects have helped? The authors clearly expected that their sample size would yield something: as they write, it has "90% power to detect variants that explain only 1% of the trait variance."

So why did the study draw a blank? The authors blame a phenomenon called "missing heritability," which amounts to arguing that the genetic variants that determine personality can't be found using GWAS—they're either too rare or they're structured in such a way that the technique can't help but overlook them.

Two popular neuroscience bloggers, however, raised a different possibility last week that ought to be troubling for anyone who studies genetics and psychology. In a masterful dissection of the personality study, the pseudonymous blogger known as The Neurocritic noted that the scientists might be "dealing with a flawed set of personality constructs to begin with." Jonah Lehrer, one of the most interesting writers on neuroscience around, picked up that argument and ran with it:

There has been a longstanding debate among psychologists about the proper way to measure and define human personality. On the one hand, there are plenty of researchers and clinicians who endorse tests like the Myers-Briggs Type Indicator (MBTI), which seeks to categorize people based on a series of supposedly innate personality dichotomies. (You've probably taken this test, and been given a summary in capital letters that describes your tendencies towards extraversion, intuition, judgment, etc.) On the other hand, there's a camp of scientists which argues that these vague categories are mostly meaningless, and that asking people a few dozen multiple choice questions is a terrible way to summarize the soul ... This might be why the [personality] study came up empty: We're trying to find the genes for personality constructs that don't exist. It's not that people don't have personalities, or that these personalities can't be measured—it's that we aren't the same person in every situation, which is what all these "tests" implicitly assume.

Cholesterol levels aren't "the same in every situation" either. They fluctuate naturally according to the time of year and the phase of the menstrual cycle, and, of course, they can be modified with changes in lifestyle and medication. But at least scientists agree on how to measure them precisely. The personality study, by contrast, looked at four traits—"novelty seeking," "harm avoidance," "reward dependence," and "persistence"—that may or may not reflect real, heritable, constant dimensions of character. There's certainly no blood test for them. They're determined by a questionnaire, a methodology on the softer side of the scientific spectrum.

The point here is that it's very hard to link a gene to a condition if you're not exactly sure how to define that condition in the first place. In scientific parlance, knowing a person's genotype may be rather beside the point unless you also have a good handle on his phenotype.

Consider what this means for people studying the genetics of mental illness. Some psychiatric diagnoses are poorly defined with blurry boundaries; they overlap; they change with new editions of the DSM. If they're made based on "self-reported" symptoms, they can also change depending on how a patient perceives himself—and this can be influenced by all sorts of factors, including, fascinatingly, the fact that the person knows something about what is in his own genes. No wonder, then, that the search for the genetics of personality and mental illness has run into so many frustrating dead ends, or that pharma is apparently cooling on the idea of developing new psychiatric drugs based on genetic research. The cholesterol paper shows that with an extraordinary amount of care and attention to detail, the promise of genomics can be realized. The personality paper is a reminder that for that to happen, the rest of medicine needs to be equally precise.