How The Hell Do We Link Our Brains To Computers? (Part 2 of x)
Emerging Technologies
Read Part 1 here.
Conflict of interest statement - I know some of the people working on these technologies socially. I have no business ties to the sector, but I do want everybody in it to succeed.
It takes a long time to bring new medical technologies to market - I once heard that for an innovation to make it from first paper to mass adoption in hospitals, it takes 17 years. This paper estimates that from concept to FDA approval a drug takes 12 years, and a medical device - 3 to 7 years. This means that there is a sizeable amount of technologies that are already practical, but still haven’t fully made their way from the lab to patients. Let’s look at BCI approaches that are trying to make that leap.
BrainGate is the most pedigreed BCI project out there. It has produced 85 papers (so far) since 1995. It uses a fully invasive implanted microelectrode array (Utah Array) to read data from human brains.
Over the years, it has been used to allow paralyzed people to control a robotic arm, type 39 characters per minute, and even regain control over their limbs, when combined with another implant that electrically stimulates muscles.
That’s incredibly powerful stuff. However, there’s a big hurdle this approach needs to overcome in order to achieve widespread adoption in practical medicine - longevity. Inserting a foreign body into a brain is a violent act from the perspective of the brain. The brain reacts to this damage with glial cells (non-neuron cells in the brain) forming a protective layer around the implant. Over time, the electrodes lose their recording effectiveness - possibly because of this layer, but other culprits are also being investigated, such as infection or electrode degradation.
This is likely not an insurmountable problem. A multitude of approaches is being studied, including changing electrode stiffness, topography, and surface chemistry. Some BrainGate implants work even 1000 days after implantation, though my understanding is that that result is a pretty big outlier.
The implants used by BrainGate are manufactured by a company called Blackrock Neurotech. The company recently raised $10M (Peter Thiel is one of the investors), which is a sign that it is planning to transition from only making devices for researchers to becoming a fully-fledged medical device company.
The Utah array uses 100 silicon electrodes. It records only from the tips of those electrodes, so the amount of neurons it can read from is fairly limited. In 2016, DARPA announced a new program called Neural Engineering System Design, the goal of which was to record from a million neurons, write to 100,000, and interact both ways with at least 1000. It awarded $65M to six teams that use different approaches to try and achieve that goal.
The first one is Paradromics, a company based in Austin, Texas, and led by Dr. Matt Angle. Its approach is conceptually similar to BrainGate - it also uses microelectrodes implanted in the brain. The electrodes themselves are different - instead of silicon needles, they are microwires, which are smaller, so they damage the brain less. Because of it, Paradromics can use more of them. In a 2020 publication, the company described its Argo system, which reads from 65,536 channels. That’s a lot of information! The system produces up to 26Gbps worth of information. That introduces a lot of challenges, including data transmission, processing, and perhaps most importantly - power. To process (or transmit) that much information, you need a fair bit of power, and implanting large batteries into humans is not safe. The current system which records from all of the channels dissipates 20 watts - it gets pretty hot! So, Paradromics is working on analog signal processing and on-chip feature extraction. That means that they are trying to pre-process the data using extremely low-power methods within the brain and reduce the amount of data that needs to be transmitted by a factor of 100. That results in power requirements that are compatible with what we can do with implants. They are calling their system the Neural Input-Output Bus. Here is a mockup of how the first medical device by Paradromics might look:
As you can see, only a part of the device is installed in the brain. The power and telemetry module is too large (and produces too much heat) to be implanted in the brain, so it is implanted in the shoulder and connected to the brain implant by a wire running through the neck. This device, to the best of my knowledge, is currently being tested in animals and is expected to begin human testing in 2023.
The second recipient of a DARPA NESD grant is a Brown University team led by Arto Nurmikko. They call their approach Neurograins. The idea is to have tiny chiplets implanted in the brain and have a patch on the skin with a device that provides power and data transmission wirelessly to the chiplets.
There are several obvious advantages to this approach:
It allows you to record from different parts of the brain.
It doesn’t require a power source being implanted into your body.
Smaller implants might be less damaging to brain tissue.
However, Neurograins is at a very early stage. The only work that’s been published so far describes device performance on the bench and ex vivo (in cells but outside of a living body). And while we know that there is early animal testing ongoing, we have no idea how Neurograins perform in a living body along any of the dimensions we care about (accuracy, longevity, etc.).
The BCI Funkiest Tune Award undoubtedly goes to The Neural Matrix (also partially funded by a NESD grant):
The Neural Matrix is a soft implant. That means that it’s way less likely to damage the brain, and already early results show that it is tolerated well for considerable periods of time. However, since it only records from the surface of the brain, the quality of the signal is worse than that of penetrating electrodes. We don’t know yet whether the data we get from this approach would be good enough to control robotic hands or decode speech.
NESD funded some other teams, but I have been unable to find any publications by the other teams that show promise as far as developing general-purpose BCIs goes. One team published a cool paper on an artificial retina, but it’s not really relevant here.
A couple of years after giving out NESD grants, DARPA launched another program to fund non-invasive and minimally invasive neural interfaces. This one is called Next-Generation Nonsurgical Neurotechnology (N3). Now, as non-invasive and minimally invasive devices are a lot more difficult to make, the approaches funded are at an earlier stage, and include some very… Weird, interesting, and exciting ones! Let’s dive in.
A team led by Battelle is working on a project called BrainSTORMS (Brain System to Transmit Or Receive Magnetoelectric Signals)1. The gist of it is to inject tiny devices called nanotransducers into a human’s bloodstream, guide them to the brain, and then interact with them through a helmet generating and recording magnetic fields. If you think that this technique sounds wild, it’s because it absolutely freaking is, but! Apparently, it works. In a paper published in June 2021, a team reported injecting live mice with the nanotransducers, guiding them to a part of the cortex, and then successfully activating neurons in that part of the cortex magnetically! Wow. I don’t think that they have been able to record from these nanotransducers yet, and they face a major limitation - the nanotransducers work for less than 3 days. But since introducing nanotransducers2 requires merely an injection and not surgery, you can see this being a one-time therapy (or repeated sessions?) type of an approach. Two of the most important people in DARPA’s neurotechnology history (Doug Weber and Justin Sanchez) now work at Battelle, which to me is a pretty big endorsement. I will definitely follow BrainSTORMS with great interest.
Hanging around neuroscientists, you meet a lot of smart people. So it’s refreshing when a team decides to solve a problem by simply being smarter than everybody else. That is my understanding of the approach taken by the CMU team that received an N3 grant. Instead of inventing new Bond-villain-tier technologies like nanotransducers, they have decided that they can understand the dynamics of the brain better than everybody else. They will use known technologies like fNIRS and transcranial stimulation, but because of their superior computational models, they think that they will be able to stimulate/record at higher resolutions than other people using the same technologies. Will it work? I have no earthly idea, just looking at some of the publications by the lead researcher is making my brain hurt.
The Johns Hopkins APL team is also working on extending fNIRS, but they actually plan to push the technology itself forward. Their work aims to make optical recording so precise that you can detect it when neurons contract when they fire. They are calling it Digital Holographic Imaging because they are using multiple light sources and their interference to reconstruct a 3D model of the insides of your brain and detect changes in it. They are planning to move to human trials fairly soon, I will be eagerly awaiting any resulting publications.
A PARC team got funding for an acousto-magnetic approach of writing to the brain. It involves somehow combining ultrasound and magnetic waves to create local electrical currents? I haven’t been able to find good sources of information on any intermediate results.
A Rice University team named their project Moana, presumably so it sounds friendly, but if anyone is up to some real Bond-villain antics, it’s them. They use optical tomography for reading from the brain, but for writing… Are you ready for this? They are using a magneto-genetic approach. They are planning to infect brain cells with a virus that alters their genetic code so that it’s easier to activate them with a magnetic field. Similar techniques (most notably optogenetics) have been used in animals for a while, but this is the first time I’ve heard of somebody planning to use genetics in a live human brain.
The final recipient of an N3 grant is Teledyne. They plan to use micro optically pumped magnetometers to record small changes in the magnetic field caused by brain activity. For writing, they plan to use focused ultrasound. I haven’t been able to find much information about how this project is going either.
This is it for N3-funded projects, but there are a few other approaches that deserve a shoutout. The first one is Stentrode (TM). The stentrode approach involves inserting a sensor into the brain through the blood vessels, as to avoid surgery. It looks kind of like this:
Stentrodes recently got FDA approval to start human trials for patients with severe paralysis. There are already patients with stentrodes implanted who can control computers with their brain, without having had their skull cut open. Pretty cool! You can see them in action in this video:
Another big name in BCIs is Kernel. Kernel takes known technologies, like fNIRS and OP-MEG, and puts a ton of money and talent into engineering cutting-edge devices based on these technologies.
My understanding of their capabilities is that they allow the equivalent of fMRI-quality recording in freely moving humans. This will likely result in awesome new human studies of things like sports, psychedelics, etc. But these approaches are unlikely to be high-definition enough to, for example, control a robotic arm or decode speech.
Finally, there’s Neuralink. I know that some readers of mine are fans of Elon Musk, but there’s no getting around the fact that Neuralink is a complete shit show. Here’s a brief overview of what has happened with Neuralink so far:
Musk announces that he is putting $100M into BCI.
Neuralink pays top-tier scientists to take their research and put a Neuralink brand on it.
Neuralink demos some of the tech that the scientists had invented and Neuralink has improved in animals.
Without publishing any data on the longevity of the implants (or even waiting long enough to know internally what it might be), Musk announces that he wants to do human trials soon. Makes a metric ton of impossible promises.
The scientists say “No, we can’t do human trials unless we establish safety of the implants in animals”.
Neuralink leadership says “How about we do trials in Russia or China where they don’t value human lives”.
Scientists quit in disgust.
By now, none of the big scientists who initially joined Neuralink still work there.
The way things have turned out… Isn’t surprising? I feel like I called it in 2019 after Neuralink’s first public demo.
In software, for example in web apps, you can iterate very quickly. In self-driving cars… You can try to iterate quickly. I don’t think that’s a good approach, but so far Tesla has been allowed to test buggy software on live drivers and pedestrians who hadn’t signed up for the test.
You absolutely cannot do that with invasive brain-computer interfaces. Every device you implant into a living human brain needs to have been tested for safety and longevity in animals for a long time. As far as I can tell Neuralink’s plan was “we aren’t going to do that because Musk is Musk and he doesn’t play by the rules”, which, frankly, is pretty lame. None of the experts in the field I know thought this would work, and it didn’t.
This is it for the emerging technologies in BCIs. I am going to be watching Paradromics, BrainSTORMS, and stentrodes closely for developments in BCIs. I am also excited about Kernel, but not because of fundamental breakthroughs - I am excited by the studies they will enable.
At a later date, I will write about approaches that are so far merely theoretical - which have been suggested as potentially feasible but haven’t been demonstrated to work yet. I will likely publish a post on virtual worlds that feel alive between now and the 3rd part of this series, though.
Scientists are not known for their naming skills.
Try saying that rapidly. Introducing nanotransducers, introducing nanotransducers. I can’t even get two repetitions right.
Aww, I was excited about Neuralink. Should have known...