VISION FOR THE FUTURE CREATES HISTORY. To determine the evolution and manner in which science, technology, and society will unfold requires vision. The ability to imagine what can be and work towards that goal. Without creativity, without passion, and without perseverance, we are lost to roam shiftless and blind like a ship without a sail in the night. - Eric C. Leuthardt
Catching Elephant is a theme by Andy Taylor
Part 2: Market Considerations for Neuroprosthetics.
I went into Brain Computer Interface research with the intention of enabling spinal cord injury patients to move and locked-in patients to communicate. After working on this for 10 years it still hasn’t happened. There has been enormous excitement around the possibilities, smart people are poring into the field, and government agencies are funding the research like never before. But still – it hasn’t happened yet – why?
Most would say that there is still a lot that needs to be done, key technical hurdle to be overcome, issues of reliability, there would be a lot of hand waving and declarative words like “milestones,” “stakeholders,” and other useless technical jargon used. Lets ask this question, if there was an infinite amount of money available and all the research and technology was created next week, would we be able to help people with devastating spinal cord injury. Sadly the answer is still no.
The reason lies in the story of a company called Cyberkinetics. The company raised millions of dollars from venture capitalists, they got FDA approval, and even demonstrated that quadriplegics could use robotic arms. The reason for the failure was over inflated hype around the market. Namely, that once they went through all this, there were only a handful of patients who would actually require the technology. By handful, I mean several thousand patients per year, which is small in market terms (not small in terms of human suffering). There were also some technical and engineering shortcomings, but at the end of the day there were not enough customers that wanted the product and the company regressed into a corporate acquisition footnote.
Before getting cynical on the idea that money is all that matters, lets look at the food truck guys outside the med school where I work. They have these great touch screen billing devices that are portable and wireless. They’re called iPads. When you think about it, iPads were not created for food truck workers. Nor were they created for any other niche business operator. If they were, nobody would invest that much money to make such a slick and highly refined technology. Instead they were created for the largest market possible, everyone. As a result there was a collateral benefit for all these additional uses.
Similarly for neuroprosthetics, we have to be smarter and think more strategically to get our technology to markets that favor the creation of this technology so that they can then collateralize to all the niche clinical needs. This is in no way minimizing the needs of patients with severe motor disability, rather its finding better ways to get the technology to them. That path is not linear. Going back to the analogy of market forces being similar to gravity, rather than fruitlessly trying to push a boulder up a hill why not use a waterfall to drive a waterwheel that pulls it up. Use gravity to work for you, rather than against you. In the same light, capturing market forces that favor BCI applications will necessarily help brain computer interface (BCI) applications towards small market applications. This is in part why my BCI efforts turned towards solutions to stroke. There are 700,000 strokes per year that leave patients with motor impairment (versus 10,000 per year with spinal cord injury). That’s a market.
The stroke market is a pretty decent stream to power the possibility of BCIs getting clinically applied. To really push this technology, however, to every clinical corner require we need something like the Three Gorges Dam that generates twenty three thousand megawatts (minus the ecological disaster). To big river app, the Yangtze of BCI, is a device, like the iPad, that serves everyone. Similar to the Apple we need to create technologies that enhance everyday lives. We need to figure out how to make neural augments and neural complements to our brains that add to the usual grind of our lives. Use these technologies to make us smarter, more creative, have better memories. Once we do this then all the people who really need this stuff actually benefit.
Living in the Giant
If you ask one of my neurons, or a myocyte taken from my arm, it won’t tell you its name is Eric. Similarly, if you ask me what my name is, I won’t say the United States of America. Common sense tell us that the cells that make up our body are all part of a larger whole that makes up a unique and self-aware entity. In my case, it’s Eric Leuthardt. It goes against everyday intuition, however, to think of a nation as a sentient organism that is conscious. A hive consciousness usually harkens to far-out notions like “the Borg,” which as we all know, nothing kills conversation like a reference to Star Trek.
So putting space fantasy aside for a moment, the actual notion is likely more real than our human egos are likely to accept. I was recently reading an article by the evolutionary biologist Mark Pagel. http://edge.org/conversation/infinite-stupidity-edge-conversation-with-mark-pagel. He argues that ideas are essentially the new DNA of the human species. For the last 1.5 billion years the combination of nucleotides (a biologic pattern) which is encoded in DNA (a biologic carrier) has been the cellular software that has shaped the physical development of biological organisms. Over the past two hundred thousand years, however, he posits that ideas (a biologic pattern) carried by human brains (a biologic carrier) are the new information bearing medium that has played the dominant role in our evolutionary development. While I think he is right that ideas held in human brains are the new form of biologic information that shapes development, he misses the mark in its importance. Ideas are not only the “DNA” that are shaping our human development, they are the DNA that underpins a whole new form of life– a “social macro-organism,” or more simply, the Giant.
It’s probably a bit of our own hubris to think that the evolutionary process stopped aggregating living components when it came to us. Ongoing biologic development has favored compiling biologic structures. It started with the formation of amino acids, which eventually became combined into proteins. Proteins then became embedded within plasma membranes to form the first single celled organisms. These differentiated and eventually formed molds, slimes, and early plants. Complex cellular structures eventually became mobile and more differentiated. Creatures were no longer clusters of cells, but had specialized substructures (different tissues, organs, appendages, etc) that did various things to give them some selective advantage to survive – legs to run, gills to breathe, teeth and claws to hunt. Eventually after a lot of trial and error, humans and their idea-carrying brains came on to the scene. My question here is why would nature stop aggregating life forms for higher levels of specialization and survival benefit?
Our government has already tacitly acknowledged that certain groups (i.e. corporations) can be treated as “people” in the eyes of the law. Namely, a corporation (a group of humans) has a legal personhood with rights equal and equivalent to single human. Corporations can exercise human rights against real individuals and the state, and they can themselves be responsible for human rights violations. Wikipedia is quick to note, however “that corporations are not living entities in the way that humans are.” Really? Why not? It’s a cluster of organisms (humans) that consumes resources, is built to grow, and has behavior that is distinct from any of its components with regards to self preservation. That is the same description of any multicellular organism – humans included. One could counter that a corporate entity (i.e. company, nation, or social movement), is not self-aware like a human. There is an absence of sentience like a living breathing creature (no matter how primitive). Returning to the individual homo sapien– is any one of the cells that make up my body “aware” of my thoughts, my feelings, and my intentions? No - just as any one cell or any one part of a body is not equipped to understand my higher order consciousness, so to we as individual humans may not be able to appreciate the sentience of a larger social organism. Who is to say self-awareness hasn’t already happened on the large scale, but we don’t have the capacity to understand or communicate on that level.
The perspective is a little disconcerting because it demotes one’s self-perceived importance in the world. I am in relationship and in someway bound by the giant I live in. Using a medical example to underline this point. I typically take care of myself and try to avoid any physically self-destructive behaviors – smoking, drug abuse, car surfing, juggling sharp objects, etc. That said, if a part of my body threatens my health, such as an appendicitis, or a cancerous growth – I’ll remove it – regardless if it’s a part of me. If the whole is threatened, the part has to go. Corporations and countries certainly act the same way when it comes to people for the purposes of self-preservation. Despite being a citizen, if one of those citizens endangers the greater health of a nation; that individual (or group of individuals) are removed. The US didn’t hesitate to kill Anwar al-Awlaki and another America-born militant with a drone – he was a threat. The point here is that we are not only in relationship with each other, we are in relationship with a larger being that we don’t fully understand and can not communicate with.
Taken together, the Giant may, and often does, have it’s own priorities. Moreover, just as history has shown, the evolution of the nervous systems leads to more complex and sophisticated means for the transit and processing of information. Just as our neurons became more numerous and myelinated with more complex connections, the Giant’s integration of information has been exponentially increasing with the printing press, telephones, and the internet (see Blog: Why is the World Changing So Fast?). Thus, the Giant is also getting smarter and more nimble. So in returning to the early Star Trek reference of the Borg, will we ever become part of a collective consciousness that can think and acts on its own? Well, in truth, I think we have already been assimilated.
Epilepsy, monkeys with rakes, and the many different types of human body parts.
What is the one thing that a patient with epilepsy wants? Is it for the seizures to stop? To be off the medications? The top of the list that I hear time and time again is this – to be able to drive a car. Without a doubt seizures are dangerous and socially debilitating (image going about your day to day life with the thought of abruptly losing consciousness). Due to their epilepsy, these patients are legally prohibited from driving because their seizures put them at risk for getting into an accident. At first glance, that of course sounds inconvenient, but to my patients it often means everything. The question is why? Why is driving so important? I think as we delve into the answer it tells us something about how humans have evolved in the modern era.
To answer the question I would cite some work done by Atsushi Iriki’s lab at the Riken Brain Science Institute in Japan. He did some interesting experiments with monkeys in which he trained them to use rakes to pull a morsel of food toward themselves while recording brain activity with electrodes. The electrodes were in the post-central gyrus, a part of the brain involved with sensory perception as it relates to our bodies. So if someone touches your hand there will be increased neuronal firing in that part of the brain. Similarly, Iriki identified neurons that would start to fire when you touched the monkey’s hands. After they were trained to use the rake when they touched the rake, the same activity appeared. Thus, according to that monkey’s brain, in a very real way that tool became a part of his body.
What does that have to do with epileptics driving cars? From monkeys to humans, the tools we use quite literally become a part of who we are. We perceive them as a part of our body. One of the most fundamental tools in modern society is the automobile. It is our new legs in a mechanized environment. Actually, its probably more than that – in addition to mobility we associate cars with our lifestyles, economic class, and personalities (people who drive Hummers are usually quite different from people who drive Minis) Thus, when a patient with epilepsy looses the ability to use a car, he or she has a lost a part of themselves. Essentially, they have lost a physical expression of their persona and, more importantly, they have lost their ability to navigate a specialized environment where most resources are beyond physical walking distance. Interestingly, a patient whose legs are paralyzed have more independence and freedom in the world at large than a patient with epilepsy. Whereas four hundred years ago in a small medieval village, the exact opposite would be true. From a personal standpoint as a surgeon who treats spinal cord injury and epilepsy, whenever I am able to intervene and recover leg function or cure their seizures, the patient’s psychology of being returned to independence is almost exactly the same.
Please don’t confuse my praise of a car as materialistic. Rather, the point here is to show that the boundary of what we call our bodies (according to our brain’s physiology) is more gray than what we would like to believe. Going into the future new “body extensions” will be cropping up with increasing speed and diversity. How many people today feel handicapped without their smart phones, their ipads, or their laptops? More than materialistic desire, the adoption of ever increasing capability is part of how we as humans are built. There is an innate cortical plasticity to take on new functionalities and incorporate those elements into our cognitive model of “me.” It is this cognitive flexibility that allowed our ancestors to first use tools (such as a rake) to advance our ability to survive and proliferate. Interestingly, with the advent of novel human machine interfaces (iphones, cars, and brain computer interfaces), we will see the emergence of new and more impressive capabilities and the emergence of new disabilities when those capabilities are taken away. Thus, you never get free lunch – even if you get it with a rake.
Living In the Model Our Brain Creates
From a traditional scientific perspective, and from a basic common sense point of view, most of us assume that the world we live in is one which we see, hear, touch, taste, and smell. That the environment we perceive around us is the result of our senses telling us the make up of an objective and separate reality. Certain findings, however, conflict with that very fundamental notion. Recent anatomical studies of cortical connectivity tell us something very different, namely, that the input to our brains from sensory organs is actually quite sparse. Rather than synaptic input from sensory organs, the vast majority of connections are interneurons (i.e. neurons talking to other neurons). For the visual system, only about 5-10% o f synapses are connected to the periphery, whereas the majority of neuronal interactions are reflecting intrinsic or thalamocortical connections. When you run the numbers (as one of my graduate students did), the brain consists of approximately 1012 neurons (1 with twelve zeros after it), each making a synaptic contact 5,000 to 10,000 times. That makes for a total of 1016 synapses – a very large number indeed. Now in thinking about information coming in, the retina has about 1,000,000 fibers going into the brain. Again assuming 5,000 to 10,000 synaptic connections that brings us to about 1010 synapses. Taken together, that means there are a million times more connections in the brain than there are for the actual input and processing of vision. The same holds true for the output side of things. When considering the cortiospinal tract, the pathway of all the motor axons that execute every movement we are capable of, again we see a number of about 106 fibers. Again, much much lower than the total number of connections in the brain.
So what’s the brain doing? It would seem the preponderance of brain activity and communication is not in the moment to moment processing of sensory inputs coming from the outside world, nor is it occupied with external behavior either. So again, what gives? There are a growing number of scientists who think the brain is in the business of predicting the world around it, rather than actively processing the activity as its coming in. To predict upcoming events you have to have a “model” to compare against. Maybe another word for a model, is a pattern of expected experiences or behaviors. So rather than sensing the world around us, our brains create a model of what the world should be and use our senses to tell us whether our model is correct or not. From an evolutionary standpoint, being able to predict events – like when I see a shaking of the bushes there will be a tiger there – helps us survive. Its also more energy efficient. Our brains are not busy rebuilding the world on a moment by moment basis, we have a model of what that world is and we check every so often to make sure its correct. If not, then we’ll make changes to the model.
I think what is fundamentally interesting about this proposition is the important role the “observer” plays in their environment. Because it really isn’t our environment that we perceive, its our model of that environment. So we are all really walking around in the world we created for ourselves. Thus, when we are depressed the world is indeed a darker place, and when we are happy the world is brighter. To push this a little further, I think that we can alter and change that model, and thus we can in a certain sense impact the world we live in. So for the cynic, who thinks nothing good come out of anything – they’re right. But the limitation is not that the world is a shitty place, it’s the barriers in their own mind.
The Nuts and Bolts of a Brain Computer Interface
There are four essential elements to the practical functioning of a brain computer interface platform. 1) Signal acquisition, the BCI system’s recorded brain signal or information input. These can be electrical signals acquired from the scalp (EEG), signals acquired from the surface of the brain (ECoG), or signals from within the brain (single units systems). 2)Signal processing, the conversion of raw information into a useful device command. This is where the math comes in. Think of it as the decoding algorithm. 3)Device output, the overt command or control functions that are administered by the BCI system. Basically, what you want to do with the system. 4)Operating protocol, the manner in which the system is altered and turned on and off, programmed and adjusted. Basically the Windows operating system or Snow Leopard of the BC. Lets hope it works more like a Mac. All of these elements play in concert to manifest the user’s intention to his or her environment.
Neuroprosthetics 102: The Neural Code
Fundamentally, neuroscience strives to understand how complex cortical dynamics represent cognitive perceptions and intentions. Its one of the last mysteries of science – how does the brain work. Analogous to the gravitas and impact that bioinformatics approaches have played in genetics and understanding biologic processes, decoding how the information of human experience is represented in cortex stands to substantially alter our fundamental understanding of brain function and change our approaches to treating neurologic disease. Just as the field of genetics was revolutionized by showing how coded sequences of nucleotides encoded very specific biologic phenomenon (rather than simply general biologic associations with a chromosome), so too is neuroscience moving beyond simple localization of cognitive operations to a region in the brain. Neuroprosthetics is demonstrating that it is possible to decode brain signals associated with a human’s intentions such that an individual can control a device with their thoughts alone. This field has shown that not only is it possible to know where a cognitive operation is occurring (e.g., arm movements are located in motor cortex), but also to know exactly what cognitive intention is occurring by the dynamic changes in brain signals (e.g., these brain signals indicate the person will move their hand up and to the left). With computational power becoming faster and less expensive, this nascent field has shown the capability to decode complex brain signals (to “read someone’s mind”) and is exiting science fiction to enter serious scientific consideration. To achieve the same level of bioinformatic impact that has occurred with human genetics will require a more universal understanding of the physiologic principles that govern human perception and intention (i.e. A Neural Code). In realizing the Neural Code, the brain and human experience stand to become as malleable as life forms are today with genetic modulation. Things that become possible when we fully understand this code include are vastly more efficient person to person communications (I can project my thoughts to your mind), seamless human machine interactions, and treatment of neurologic disease ranging from paralysis to psychiatric disease. Additionally, if we can truly model how information is managed in the brain it also opens to door to create that intelligence artificially. Just as the moon landing, heralded frame shifting technical advances ranging from satellite communications to tang, and currently genomics is revolutionizing biotechnology ranging individualized medicine to better agriculture, understanding the Neural Code stands as science’s next grand project. A project that will both literally and figuratively tap into the human imagination like never before.
Brain Computer Interfaces 101: The Basics
As I work to develop this blog, before I can really effectively project my ideas and thoughts on the field and where its going to the blogosphere, it is probably worth giving a “neuroprosthetics 101” lesson to get everyone up to speed as to what people are talking about when they use the terms Brain Computer Interface, Brain Machine Interface, Neuroprosthetics, and so forth.
The most honest statement is that a lot of these terms are used relatively interchangeable and somewhat loosely. Most fundamentally, a Brain Computer Interface (also called a Brain Machine Interface, or a neuroprosthetic) is some kind of device that directly interacts the brain. Of these types of devices there are two basic flavors of how these devices can interact with brain. If the device records brain signals and then converts those signals to some type of external control this would be considered and “output BCI.” Namely, one uses their thoughts or their intentions to control something external to their brain. Conversely, if some element from the environment is converted to an electronic signal that alters the brain’s physiology to create a perception, then this is called an “input BCI.” We have seen examples of both of these types of BCI in various fiction stories. So if anyone recalls the story Firefox (http://www.youtube.com/watch?v=nXaS2EQrX7Q) that would be an example of an output BCI. The idea here was that the Russians had created a fighter plane that could be controlled by ones thoughts alone. In classic eighties style, it was described as the most “the devastating killing machine ever built,” and it was to be stolen by the “the most daring American fighter pilot around,” Clint Eastwood. The point here is that one’s thoughts were converted to real actions, in the Firefox example, it was to shoot rockets and bullets from a plane. Rather than this being a good story, there are numerous projects being funded by the military (http://www.darpa.mil/budget.html) for the express purpose of using brain signals to either control devices for warfare superiority or for the restoration of the injured soldier. In terms of input BCI, probably the best current example of these in fiction is The Matrix (http://www.youtube.com/watch?v=UM5yepZ21pI). The story captured the imagination of people by creating an idea that humans were enslaved by sentient machines who had constructed a completely virtual world which was implemented by prosthetics that were implanted into the back of the person’s head. Basically everything that a individual experienced was an artifice of brain stimulation. Though a far flung story, we also see early beginnings of artificial perceptions being possible. Several research groups are already creating implants in cortex that can create artificial visual perceptions. Rather than having machines enslave us, they could be quite useful in restoring vision to people who are blind. A nice summary of current advances in the field can be found here (http://biomed.brown.edu/Courses/BI108/BI108_1999_Groups/Vision_Team/Cortical.htm).
So to summarize, the question is which direction is the information going. Is it going from the brain to the outside world (output BCI), or is it going from the outside world to the brain (input BCI)? What the neuroposthetic does is create a new channel by which that information can pass from one to the other.
Fiction Precedes Science
I have had several conversations with people regarding recent movies such as Surrogates and Avatar. Most people treat them as fun and fanciful films about things that are far away and can never be. In truth, the likelihood that we will one day all be a society of though driven robots or that we will inhabit alien bodies is indeed low. There is something more prescient and timely about these movies, however. They speak to the growing cultural imagination and fascination with human minds being accessible. The idea that we can, in some way, plug into our brain and have our thoughts, perceptions, and intentions be implemented without the need of our bodies. The reason this growing interest is important is because societies imagination usually precedes its real world creation. To draw on several historical examples, the notion of a ray gun far preceded the discovery of a laser, and people fantasized about flying to the moon as far back as 400 years (Johannes Kepler’s Somnium,1634). The point here is that science fiction invariably focuses our scientific and technical efforts, because society - as channeled and expressed by the artist - have determined that it is something interesting and worth pursuing. What form in the real world this social imagination for neural interfacing will take will remain to be seen. Certainly, early demonstrations that people can control devices with their thoughts is already in existence (see movies presented in blog). How these early findings will be transformed into a larger social adoption is not yet clear - stay tuned for the next episode!
