Hearing aid through skin sensory for profound deaf people

Introduction

The World Health Organization (WHO) has adopted a grading system based on audiometric measurements to standardize the way in which severity of hearing loss of a person is reported. According to WHO,¹ a profound deafness is defined as a person who has a hearing threshold of 80 to 95 decibels (dB). A person with complete or total hearing loss has a threshold of 95dB or more. On the other hand, a normal hearing person has a hearing threshold of less than 20dB. Furthermore, they stated that almost 30 million people worldwide have profound or complete hearing loss in both ears. Based on their reported statistics on number of people with moderate or higher levels of hearing loss, 80% are from low income and middle-income countries as opposed to 20% that are from high-income countries.¹

A hearing aid or a cochlear implant is the common solution for induvial that are profoundly or severely deaf.² The purpose of hearing aid is sound amplification; therefore, it is more suitable for mild hearing impairment.^3,4 Besides that, the treatment based on cochlear implant is too costly and involves complex and risky surgery. In addition, hearing-impaired users still encounter substantial practical and social challenges with the use of this aid. At present, sign language^5-7 and speech-reading⁸ are commonly applied in schools for the deaf.

Recent studies show that the human ear and the auditory nerve are not the only channel for gaining audio signals.^10,11 There are several processes such as obtaining sound signals, translating voice signals to electrical signals, and producing frequency resolution can be achieved through the advancement of electronic devices. In other words, this additional tactile stimulus could help the profoundly deaf person wearing the electronic device to perceive sounds.

A study on multi-channel array skin-hearing technology was proposed which involved the stimulation of the skin by the electrical signal based on the sound.⁹ This enabled the profoundly deaf person to receive sound signals through the skin to accomplish the purpose of hearing the sound. The advantages of their technology were that it does not rely on intact hearing, and it was cost-effective. Furthermore, Li, J et al.¹⁰ showed that voice signal can be transferred using the cutaneous sensory nerves and it can also distinguish different speech signals. Their findings demonstrate that skin sensory nerves are a reliable replacement for the auditory nerve in the challenges facing individuals with severe-to-profound hearing loss.

A very recent study¹¹ has shown that these individuals can learn to identify sounds that are algorithmically translated into patterns of vibration on the skin on the wrist. Furthermore, the users of the device could identify different sounds from the vibrations alone, which gradually improved over the course of one month. The analysis on the wearable sensory device unlocks the gateway to auditory stimuli, which could be interpreted through skin.¹¹

The aim of this study was to develop a hearing aid that stimulates “hearing” through skin sensory, which is more affordable and accessible for the profoundly deaf or total hearing loss community. Hence, we embarked into this research as studies on hearing through skin sensory seems to be a promising area for the beneficial of the society of profound hearing people in terms of safety and affordability.

Methods

The first phase of this study was to design a skin sensory stimulator by using a hand phone coin vibrator or micro actuator (MEMS) that allows the sound vibrations to be detected through the user’s skin. Figure 1 briefly demonstrates the stages of designing the human sensory stimulator.

Figure 1. Flow design of human sensory stimulator.

A Sound Transducer uses electrical energy to produce mechanical vibrations to disturb the surrounding air to produce sound regardless of whether the frequency is audible or inaudible. Audio Sound Transducer includes an input sensor that transforms sound into an electrical signal such as a microphone, and an output actuator that transforms the electrical signals back into sound, such as a loudspeaker. A preamplifier converts a weak electrical signal into an output signal, which is powerful enough to be noise tolerant. Absence of this will cause the output signal to be noisy or misleading. Due to this, the preamplifier is frequently located near to the sensor which reduces the effects of noise and other interference. A frequency filter is an electrical circuit that either stops or passes frequency from electrical signal with respect to specific frequency needed for further process. There are two types of filters, passive filter which uses passive components, and active filter that uses active components. Audio spectrum analysers are commonly used by sound engineers for various applications. Measuring the frequency response and analysing distortion characteristics of different types of audio tools are some of these applications. Micro actuators are active devices proficient of producing mechanical motion of solids or fluids. Hand phone coin vibrator is taken from the concept of cell phone vibration motor. We developed a simple single audio signal vibration array, which is based on the LM3915 Dot/Bar Display Driver and an electret microphone.

Sound sensor module gives a simple method to detect sound and it is usually used for the purpose of detecting sound intensity. It uses a microphone as a transducer which supplies the input audio signal to an amplifier, peak detector, and buffer. This module is made of three functional elements, the sensor on the front of the module will perform the pick-up (transducer), then the analogue signal is sent to the amplifier to amplify the signal. Accommodated with the adjustable voltage reference and an accurate ten-step voltage divider is the IC LM3915. The advantage of substituting conservative meters with a light-emitting diode (LED) bar graph is that it provides a quicker response, and a more rugged display with high visibility that provides the ease of interpretation of an analog display. As an initial test device, a transducer vibrating device circuit is designed to study the response of subjects to vibrating motor or actuator to skin sensory. The circuit (Figure 2) utilises a LM358 operational amplifier. An electret microphone preamplifier will pick-up the audible signal from the surrounding and amplifies it through a first stage operational amplifier to a process able signal. The signal will then pass through a second stage amplifier comparator to increase the voltage level high enough to drive the vibrator or actuator. A transistor interface is used to drive the vibrator or actuator in accordance with the sound received from the electret microphone.

Figure 2. Single sound to vibration actuator driver.

In this research, the initial testing model was designed based on the LM3915 audio level indicator with LEDs converted to hand phone coin vibrating devices. The circuit design development and drawing were done with the NI Multisim Electronic computer-aided design (CAD) software. The circuit is then simulated to produce a printed circuit board (PCB). The schematic circuit diagram is shown in Figure 3. The initial design (Figure 3) will be upgraded to an audio spectrum analyser (Figure 4), by using the Arduino processor for a multichannel wearable spectrum vibrator.

Figure 3. Schematic circuit diagram.

Figure 4. Audio spectrum analyser

The spectrum analyser measures and displays the amplitude of a given input signal set verses the full frequency ranges of the instrument. Its main purpose is to measure the strength of the spectrum of both known and unknown signals. This audio spectrum analyser enables us to see the frequencies present in audio or sound and produces the graph of all the frequencies that are present in a sound source at any given time. The Arduino is a microcontroller board that contains everything needed to support the microcontroller to create a led audio spectrum matrix. The board is equipped with sets of digital and analogue input/output (I/O) pins that may be interfaced to various expansion boards (shields) and other circuits.

The initial experiment uses a single overall sound to vibrate the device which will be used to stimulate the skin sensory. The electret microphone will pick up audio or sound signal which will feed to a preamplifier with adjustable gain of sound. This is to test the subject whether they can perceive sound through the vibrating motor or actuator. The single vibrating device then will be upgraded to Audio spectrum vibrators in respond to a visible led matrix. The led matrix would then be replaced with coin vibrator or actuator to give more precise vibrating pattern. The sound sensor can detect the frequency range from 20hz -20khz as this is the normal human hearing range. By using this type of sound sensor, it will enable the profoundly deaf people to “hear” the sound through their skin. The hand phone coin vibrators or micro actuator acts as an audio spectrum vibrator to create a vibrating pattern. This vibrating pattern will then be used to stimulate the human sensory system such as the skin for the profoundly deaf person, so they can use the vibration pattern as information. This repeating vibrating pattern stimulation will create a memory of a particular vibrating pattern for future recognition when similar sound is “heard” through the microphone.

The LED Matrix will have different types of frequency columns and its own vibrating devices with different sound intensity (dB) levels. Each column will indicate the frequency level representation based on how it is programmed. For example, if there is a sound present, the frequency of the led matrix column will be activated to the maximum dB level which was picked up by the electret microphone. By changing the LEDs to vibrating stimulation, the profoundly deaf person can feel and “hear” the sound through his or her skin sensory system.

Results

Due to the recent outbreak of covid-19 pandemic, profoundly deaf individuals from different institutions, schools, or hospitals were not able to participate in our study.

Therefore, we randomly selected 30 people: 20 normal, seven partially deaf and three profoundly deaf (Underlying data) (12), who were mostly friends and relatives that lived within a 10 km radius of the authors residential area. All the 30 participants have given a written consent and they voluntarily agreed to participate in this study. We developed four initial vibrating transducers with single spectrum, which were strapped to a belt. Once the individuals wore the belt on his/her waist, the four transducers were placed at the front, back, left, and right side of the individual’s waist (Figure 5). Audible sound was played (stimulus) and the individual was asked whether he/she felt a stimulation at the point of the transducer and if so, state the direction of the sound source. Based on the experiments, both normal and the partial or profoundly deaf individuals felt the vibrating stimulation on their skin and accurately stated the directions of the sounds. This initial experiment was successful, which is an important direction for the future development of a hearing aid through human sensory skin. Further advancement of a more comprehensive device using the concept of spectrum analyser is now in progress. The sound signals will then be fed to multiple audio filters and audio spectrum analyser to produce sound audio spectrum to drive multiple vibration on motors or actuators of the device.

Figure 5. Belt strapped to subject’s waist

Conclusions

Despite the advancement of various hearing devices that improve the life of profoundly deaf people around the globe, there are still many limitations and drawbacks, which emphasizes the need to further improve such technologies. The study of “hearing through the skin” is very promising and leads to affordable and safe solutions for the profoundly deaf and completely deaf people. As far as we are aware, adopting the concept of audio spectrum analyser for developing hearing aid has not been previously studied. By wearing the device, a profoundly deaf person can perceive sound through their skin. Future studies should investigate and address how various types of sound stimulated through the skin by devices such as the belt used in this study, can be interpretated and identified by the the profoundly deaf individual.

Ethical approval

The Research Ethics committee of Technology Transfer Office, Multimedia University has granted ethics approval for this study. Ethical Approval Number: EA1992021. All participants of this study gave informed written consent to take part in the research.

Data availability

Author contributions

Yasothei Suppiah: Conceptualization, investigation, supervision. M Chandran Maruthan: investigation, methodology, resources. Fazly Salleh Abas: validation, visualization, writing-original draft.