A PATH TO TRANSLATE THE LANGUAGE OF NONVERBAL CHILDREN
Edward Fennell
Haley Hilt
In 1985 Michael Brown and Joseph Goldstein were awarded the Nobel Prize in Medicine for their discoveries concerning the regulation of cholesterol metabolism. Their work set the foundation for drugs we now know as statins. The history of medicine is replete with examples of rare conditions research having universal application. The systemic problems of children with rare diseases affecting both motor and intellectual pathways are no different.
Too often their disabilities prevent them from being interrogated and they are considered as profoundly learning disabled, as if they have nothing to say. For reasons that appear more discriminatory rather than scientific they are customarily excluded from conventional research (Russell et al). However there is research indicating people with profound disabilities have preserved cognition and vocabulary, (Corderre et al). There is also evidence that autism spectrum disorders may be similar to locked-in syndrome (Pines et al). Thus they may be only one of many categories of people deprived of speech due to developmental factors, accident, illness, mental health or aging.
These children offer unique opportunities for cognitive neuroscience to delve deeply into the intricacies of the relationship between the motor and cognitive domains. (Veldman et al). Research projects focused on their nonverbal communication patterns will have universal impact and will open this door. Research projects that can translate wants and needs into intentional autonomous actions will prove especially valuable giving agency to people with severe disabilities.
Unlocking the nonverbal communication patterns will also prove especially valuable to medical professionals and caregivers who seek information necessary for effective treatments, therapeutic interventions or discourse.
In the normal course of communication we rely not only on words but gestures, body movement, posture, tone of the utterance, physiological condition, social setting, environmental conditions, and social status to determine intent and meaning. These various modalities of nonverbal communication are performed by all of us as a suite and their use is idiosyncratic and influenced by cultural factors (Bellieni 2022).
There is evidence that “assessment of functional (nonverbal) communication is increasingly used in large-scale randomized controlled trials as the primary outcome measure” However, “there is little knowledge about how commonly used measures of functional communication relate to each other”, (Schumacher et al).
As is customary in science, research is specialized: one chooses facial expression, another vocalization, another body movement and so on. One such example is the work of the MIT Media Lab where nonverbal vocalizations are seen as speech. (Narain, Johnson) A supplemental work focused on vocalizations that demonstrate “the need and potential for specialized, naturalistic databases and novel computational methods to enhance translational communication technologies in underserved populations”. (Narain, Johnson). Integration of all nonverbal modalities is necessary to provide the answers caregivers and clinicians seek.
Given the multimodal complexity of non-verbal communication, its idiosyncratic usage, unconscious application and a myriad of social situations in which it is exercised, a unique platform for collecting this data is necessary. It will require a unification of sciences to perform the research. The sciences focused on its discrete modalities (ie: facial expression, vocalizations etc.) will require it to partner with artificial intelligence, machine language and ambient technology. Data analysis will replace the reliance on subjective evaluations. This will enable the assembly of digital profiles of large numbers of nonverbal individuals without the customary restrictions of reductive and exclusion selection criteria usually required of studies focused on singular diagnostic categories. Such systems allow for the full range of genotypes and phenotypes and the idiosyncratic profile of each person , while providing opportunities for research to classify differences in nonverbal communication patterns by genotype and phenotype.
The number one priority of parents caring for nonverbal children is communication. “Most parents of non-verbal children would agree that communication is at the top of the symptoms list they would like addressed.” (ORCA Study). Recent work at the MIT Media Lab (ECHOS) focused on nonverbal children’s vocalizations, These works typify current works. They recognize that nonverbal communication is important and multimodal, yet often rely on subjective classification or are unimodal.
In 2013 an article appeared in the Child Development Research journal which reported the results of field study focused on nonverbal ASD children in the classroom. The study revealed that the natural setting approached with structured and informed methodologies will reveal behavioral patterns that cannot be produced in laboratory settings. One of the naturalistic settings that is rich in data on the nonverbal communication patterns of nonverbal children is of course, the home. It is there that researchers may find parents and caregivers who are already skilled in the idiosyncratic nonverbal communication patterns of their children. However, parents and caregivers cannot expect to provide the rigor of high quality good research, nor could it sustain the costs required of quality research and sufficient sample size.
In 2019 a paper appeared in the Proceedings of the SAI Intelligence System Conference on Applying Ambient Intelligence to Assist People with Profound Intellectual and Multiple Disabilities. The application was named the INSENSION project and it was funded by the European Union Horizon 2020 Initiative. It was a consortium of computer scientists, experts in special education and professionals in care provision. Their purpose was to design and develop an Information and Communication Technology (ICT) platform to support people who are nonverbal and have profound learning and multiple disabilities (PLMD). It acknowledged that “for people with profound intellectual and multiple disabilities (PIMD), interaction with the environment is not so easy. In most cases, they are probably dependent on family and caregivers to carry out even basic functions due to severe physical and sensory impairments.”.
The Insension system is designed to monitor the primary user (child) and their surroundings to detect meaningful behavioral signals. These signals are interpreted as the user intent. The system is designed to assist them in carrying out that intent such as switching a light or communicating that intent to a caregiver. It is also designed to capture idiosyncratic nonverbal language without regard to genotype or phenotype and thus opens new possibilities of comparative research well beyond language.
The Insension Project is an endeavor of the Health Division of the Poznan Supercomputing Networking Center (PSNC), an affiliate of the Institute of Bioorganic Chemistry of the Polish Academy of Sciences. The initial prototype was developed under a grant from the European Union. They have initiated Insension 2.0 and are seeking partners for both technological and application technologies.
Lastly, we want to mention that the gateway to cognition is attention. The platform will allow for extensive observation of nonverbal people with the ability to analyze the objects of their attention, research impossible in short term lab visits. PSNC is aware of the necessity of making such a platform scalable for multiple uses and economically feasible for multiple installations.
We will keep you posted as we meet with them biweekly.
Poppy an Haley