Neurocosmology, data gloves, and VR - turning theory into communication devices
“The “Neurocosmological Principle” of epistemology: the initial assumption that all information that is perceived about the external environment is filtered, mediated and/or biased by the nervous system”
My PhD research picks up (at least in published research) where Dave left off when finishing his dissertation in 2000. He developed a reference architecture, a way of thinking about the transfer of information from a computer display to a human’s conscious perception. Most data visualization practitioners are from the computer science field, and reference research on cognition, especially the visual system, in order to design the methods with which to display information. In starting with the understanding that we use our nervous system to sense, transduce, and relay signals of the outside world, including computer screens, to our cerebral cortex in order to extract meaning, Dave reoriented the perspective we use when designing data visualization tools.
His experiments eventually led to a focus on data visualization and the development of the openANTZ software used in my research. But his early research is worth reviewing in order to understand the fundamental principles of neurocosmology: information interface to nervous system to human consciousness and back.
VPL Data Glove
In the early 90’s Dave had finished his MD, was working on his PhD in neuroscience, and spending time with the Department of Neurology and Neurosurgery Department at Loma Linda. Using the VPL Data Glove - which captures precise quantitative measurement of finger and hand movement - he worked with a patient that had been admitted to the ICU with a stroke which left them mostly paralyzed. The DataGlove® was used as a tool to establish a communication channel which could reliably convey the intent of the individual. Analysis of the data captured from the glove allowed the patient to convey information - requesting water, asking for a nurse, etc - by making pre-defined shapes with their hand, which became audio and visual output for the hospital staff.
Stroke patient using the Data Glove to communicate with hospital staff
Dave and his team found that the methodology of neurocosmology, applied to operational methods, could be extended to a wider and wider diversity of human needs. They assembled various off the shelf interface devices to continue customizing the system for new users - bastardized game controllers, new VR systems, iterations of the Data Glove, etc. Using neurocosmology changed their perspective about how to communicate with disabled patients and led to novel approaches. They reframed the challenge of communication to that of conveying signals between the input and output channels (our nervous system conveying sensory information to conscious perception, and our brain directing limbs, facial expressions, etc. to convey meaning) available to disabled patients and the input and output channels of information interfaces.
Ashley’s disability was a result of a birth accident. She is classified as C1 with some brain stem involvement (see pictures); she can move her head and she is vent dependent. Over a series fo explorations of the adaptability of their systems, Dave’s team adapted a series of interfaces which detected bioelectric activity in very specific muscles of her face and used those as independent data channels. The idea was, since Ashley had control of her facial muscles, we would use those as her "fingers" to give her differential control into the computer. While Ken Kashuahara of ABC's America Agenda (see photographs) was here, we showed that Ashley was able to navigate around in a virtual environment using only her facial muscles. This was only the second time she had ever even seen this technology.
I’m really lucky to be working with Dave. He’s spent his career leveraging his knowledge and passion for discovery to serve some of the most vulnerable people in our society. I hope to continue that work with my PhD research.