New Mind-Reading Technology Re-Creates Digital Images of What You're Thinking

Mind-reading machines may sound like they belong firmly in the realms of science fiction, but they may be closer to reality than you think. Researchers have developed a technique that can produce a digital image of what a person is picturing in their mind, simply by reading the electrical activity of their brain.

As part of a new study, accepted for publication in the journal eNeuro, a team from the University of Toronto Scarborough (UT) used a noninvasive test known as an electroencephalogram—which is usually used in the diagnosis of brain tumors, epilepsy and other conditions—to record this electrical activity in participants who were presented with visual stimuli on a computer screen.

The researchers hooked up the volunteers—13 healthy adults between the ages of 18 and 27—to the EEG via electrodes placed on their heads. They then showed the participants the faces of 140 individuals, one-by-one, on the computer screen, while recording their brain activity.

Finally, on a computer installed with special machine-learning algorithms, the team re-created images of the faces that the participants were looking at using the data recorded by the EEG. Essentially, the algorithm searched for brain signals related to the mental image that the volunteers were holding in their mind and reproduced it as a digital image. (Machine learning refers to computer systems that can adapt their programming according to the data that is fed to them, without being explicitly reprogrammed.)

Related: Scientists captured a single thought traveling across a human brain from start to finish

The work draws on the brain's natural ability to create what is known as a mental percept, or a mental impression, of whatever we are seeing. "We were able to capture this percept using EEG to get a direct illustration of what's happening in the brain during this process," Dan Nemrodov, a postdoctoral fellow in the Department of Psychology, who developed the technique, said in a statement.

While the reconstructed digital images are not perfect copies of the originals—they are somewhat lower in resolution—the detail in them is impressive, the researchers say.

"What's really exciting is that we're not reconstructing squares and triangles but actual images of a person's face, and that involves a lot of fine-grained visual detail," added Adrian Nestor, assistant professor of psychology at UT.

This study is not the first time that researchers have digitally reconstructed mental images. But the new technique offers some distinct advantages over existing methods.

Previously, neuroscientists have used functional magnetic resonance imaging (fMRI), which measures brain activity by detecting changes in blood flow rather than electrical activity. EEG, however, is more practical, because it's more readily available, portable and cheaper.

EEG is also more sensitive to tiny changes in brain activity, capturing data at the scale of milliseconds. By contrast, fMRI can only record activity at the scale of seconds.

But Nemrodov notes that many researchers doubted the potential of EEG for this kind of image reconstruction, given that the technique produces a lot of noise—or meaningless data—in the measurements it takes. It is also not very accurate at locating where in the brain signals are coming from. Despite this, the new study, he says, demonstrates EEG's advantages over fMRI. The results could widen the scope of this technology, opening the door for its use in a variety of applications.

"The fact we can reconstruct what someone experiences visually based on their brain activity opens up a lot of possibilities," said Nestor. "It unveils the subjective content of our mind and it provides a way to access, explore and share the content of our imagination."

According to Nestor, the technique could be as a means of communication for people who are unable to communicate verbally, perhaps due to paralysis. "Not only could it produce a neural-based reconstruction of what a person is perceiving, but also of what they remember and imagine, of what they want to express," Nestor said.

The technique could prove useful for law enforcement when gathering information about potential suspects from eyewitnesses, instead of relying on inaccurate sketches or verbal descriptions.

The next step, the researchers said, is to determine whether the technique would work when test subjects recall a memory from scratch instead of looking at an image.