scientists can now read your thoughts. well, sort of...

New technology allows computers and fMRI machines to decode what you're thinking as a stream of text. It sounds creepy, but it could save lives.

How do you train an artificial intelligence model like ChatGPT? Well, you have it read lots and lots and lots of text. This way it can understand what words are close to what other words, and infer the rules of grammar and spelling. But what happens when you put three people in a functional MRI scanner, read them stories while measuring the activity in parts of their brains associated with speech, then use that activity to train a precursor to ChatGPT? First, you save some money because it’s a much smaller and cheaper to train model. Second, you get an AI that can translate brain function into a continuous stream of text reflecting a person’s thoughts.

Okay, hold on, why would that work? How does the noise of your mind recorded as a sequence of zeroes and ones turn into words? It’s because computers don’t care how you define symbols and their relationships, and AI models take that to the extreme. If you think your computer innately understands ASCII or UTF-8 encoding for text, you are sorely mistaken. We simply have pieces of code which know that 68 65 79 21 in hex means “hey!” in a particular context after applying the ASCII standard, so display that or send it down the wire. In this case, the AI knows that a sequence of brain noise encoded as binary signals equates to a certain block of text.

The difference is that in a standard encoding, it has a key-value map which it consults while the AI model has no such luxury. Instead, the AI tries to use a statistical formula and calculus to find a way to reliably break down and predict what each sequence will say. Once it can do that with accuracy found acceptable by researchers, it can run the same logic for future inputs and predict that fMRI sequence one means “hey there, I am a human!” and fMRI sequence two means “destroy all humans,” or what have you. After that, as the test subjects watched silent movies and thought through what each showed, the machines were able to summarize their thoughts as text.

Okay, so when is the though police coming for you? Not anytime soon, actually. First of all, each set of impulses is unique to each person, so even slight changes would be problematic for a model not trained on your exact brain to decipher with any degree of accuracy. Secondly, fMRI is a pretty intensive and restrictive technique. You would fail miserably if you tried to deploy it in, say, an airport to catch security threats. Plus, the reading is very surface level. Change your train of thought and the reader will veer right off with you into a stream of consciousness rather than focusing on some deep, subconscious inner monologue a lot of people don’t even have.

Why would you want to go through all this trouble? To save people’s lives and improve quality of life and treatment for patients who are “locked in,” unable to move or talk to the outside world, yet fully aware of their surroundings. If you could listen to what the speech cortices of a patient’s brain are saying, you would know who is awake, aware, but may not be able to communicate it. Granted, you wouldn’t know what they wanted to say and have to train a model just for them to actually talk — which has worked in a similar study on augmenting a speech motor cortex — but you could see if the noise is random or seems to be an actual stream of consciousness.

The other nice part about this approach is that it’s not invasive. Earlier efforts to turn crystalized thoughts into speech relied on implanting sensors and probes in patients’ brains. By contrast, this decoder can just watch the blood flow inside the target parts of the mind and return a very good approximation. This means you’re not risking any patients’ already shaky and fragile health with surgical intervention just to figure out whether they might be locked in. Just observe them in an MRI machine for a standard session, or 16 hours worth if you want a non-invasive communication tool. So, rather than a menacing development, this research may be a lifesaver for thousands.

See: Tang, J., LeBel, A., et al. (2023) Semantic reconstruction of continuous language from non-invasive brain recordings. Nat Neurosci 26, 858–866. DOI: 10.1038/s41593-023-01304-9