Associate Professor OHNISHI Ayumi of the Graduate School of Engineering wants to make people’s lives better using wearable devices, a goal which she pursues through her research. She’s currently working to develop a device to detect fatigue in its wearer and provide support for the five senses, like vision and hearing. Ohnishi attracted attention in 2022 when she was named one of MIT Technology Review’s Innovators Under 35 Japan. We sat down to hear more about the current state and prospects of her research, as well as the possibilities of wearable devices.
Constantly monitoring people’s condition and providing support
What got you interested in research on wearable devices?
Ohnishi:
After entering the Faculty of Engineering at Kobe University, I toured a few different laboratories in the second half of my third year, but the one that captured my interest was the Tsukamoto Terada Laboratory, led by Professor TSUKAMOTO Masahiko and Professor TERADA Tsutomu, to which I still belong. They’ve got a studio and exercise equipment and measure human movement using things like wearable devices, which I found quite interesting.
I’ve been playing tennis since I was a child, and I also loved physics and math when I was in high school. As I was practicing my tennis techniques and trying to improve at the sport, I found myself thinking about things based on the laws of physics, so I think that was what drew me to this type of field.
As a student, I did research that involved collecting data on the nods of participants at a meeting using sensors attached to their heads and used that to analyze the different components of the content of the meeting. Later, after I entered graduate school, I was interested in the agricultural internet of things, so for the two years I spent in the master’s program, I did research in environmental science. Then, when I entered the doctoral program, I returned to engineering and began to really dig into wearable device research.
What exactly are wearable devices? What are some of their characteristics?
Ohnishi:
Typical wearable devices include things like smart watches, which lots of people wear nowadays. Smart glasses, which are worn like regular glasses and allow users to see digital information and take photos, are also rapidly evolving. Hearable devices, which users wear in or on their ears to receive audio information, are also a type of wearable device. All these devices have become quite natural-looking, which has enhanced their wearability as of late.
As opposed to smartphones and personal computers, these wearable devices are always attached to the wearer, which means that they can provide support and gather information on the wearer over a long period of time. Eventually, we may be able to gather detailed information on how people’s minds and bodies change over the course of the day. By clarifying things we didn’t previously know, this could have the potential to prove useful in medical treatment and other areas.
Researching human fatigue and expansion of our five senses
Tell me what kind of research you’re involved in at the moment.
Ohnishi:
I’m mainly involved in research that recognizes human behavior via sensing using wearable devices. I’m not simply researching their behavior, but rather, I’m involved in research for recognizing and understanding sophisticated information on people.
Something I’ve placed a particular focus on is research on human fatigue and the expansion of our five senses. On top of using wearable devices to measure fatigue itself, we’re also thinking about how to support the senses that change due to said fatigue. This “expansion” of our senses refers to both returning the capabilities of the senses to normal levels when they’ve dropped as well as enhancing those capabilities even beyond their usual levels.
Take vision for example: plenty of people notice changes in their vision due to fatigue. That said, vision involves various functions, including visual acuity, visual field, and light and dark adaptation. We need to explore how to measure these changes as well as how we can provide appropriate support. There are still things we don’t know about how our vision changes during our day-to-day, so we first need to use wearable devices to understand how these functions change in daily life.
Right now, there are glasses in development which allow users to alter the strength of the lenses by applying voltage to them, so we’re also studying how such devices can be appropriately adjusted according to each individual’s condition. We’re also considering how to hasten the adjustment time for light/dark adaptation as well as a system for encouraging proper blink timing. Our aim is to use devices to ascertain both a person’s level of fatigue and their surrounding environment in order to provide optimal support.
What about senses other than vision?
Ohnishi:
Actually, it was hearing that led to my idea to research the five senses under fatigue. I realized that when someone talks to me when I’m tired, I have to ask them to repeat themselves over and over. I’m currently researching how often we hear things but don’t fully comprehend them in our daily lives and even whether these sounds are heard differently, as those have yet to be explored.
Many people nowadays use earphones and hearing aids that can automatically adjust their volume depending on the setting. I think we need devices that not only respond to external influences, but also account for the condition of the wearer, such as their level of fatigue.
Estimating a person’s level of fatigue from various measurements collected by wearable devices is a form of advanced context recognition. Other potential targets of such recognition include a person’s level of interest, concentration, and stress. I’d like to continue my research, including things like sensor development, through repeated trial and error.
Exploring unknown influences brought about by wearable devices
Aren’t there any problems or disadvantages caused by the evolution of wearable devices?
Ohnishi:
The influence of wearable devices on people is an important topic. For instance, even when using a device designed to expand the user’s visual field, we do not yet fully understand how wearing the device itself affects the user. so we’re currently performing research on issues like that as well. Moving forward, I’d also like to create guidelines related to wearing these devices.
Protection of privacy and prevention of misuse are also crucial perspectives. Glasses-shaped devices equipped with camera functionality are becoming difficult to distinguish from regular glasses at first glance. We need a system for acquiring information via these devices, be it image data or otherwise, that doesn’t include personal information from surrounding individuals.
Just like your research on the five senses under fatigue, your research on smart shoes is also gathering attention, isn’t it?
Ohnishi:
One big issue we run into when it comes to wearable devices is providing a power source, so we developed these shoes to overcome this issue. They generate electricity from the impact created when the user’s feet hit the ground, eliminating the need to charge them. Led by Professor Terada in our lab, we’ve conducted joint research with Panasonic and Asics.
Using these shoes allows us to identify the type of surface simply by walking. We haven’t yet reached the stage of real-world deployment, but we believe they could be used for applications such as creating real-time accessibility maps, which has attracted considerable interest.
By wearing these shoes over a longer period, we may even be able to perform various measurements, such as changes in walking style. I’d like to think more about what kinds of needs we can respond to in the future.
Potential to support people’s lives in combination with AI
How will wearable devices develop moving forward?
Ohnishi:
Uses for wearable devices may continue to expand in a similar way to how smartphones have become commonplace over the past ten or so years. It’s entirely possible that we can support people’s lives by combining these devices with rapidly evolving AI.
Of course, combining these devices with AI is a topic we’re exploring in our lab as well. Using AI can make interactions between people and devices smoother, help us understand aspects of the surrounding environment that are difficult to detect with sensors alone and enable more sophisticated forms of support.
When we’re throwing around ideas in our lab, we tend to think about problems we face in our daily lives, so we aim to solve various issues by making use of wearable devices and AI.
What kind of research or activities are you looking to conduct in the future?
Ohnishi:
I’m going to continue to focus on research related to fatigue and expansion of the five senses. In addition to people’s current state, I’d also like to analyze how their condition may change in the future and develop wearable devices that can encourage users to take appropriate action. For instance, it could be possible to create a mechanism in which these devices estimate physical and mental changes that could occur in the near future to suggest a change of pace to the user.
As for the senses, I’m interested in research on combining vision and hearing as well as measuring and supporting the sense of smell. Sensing smells and expanding the sense of smell are difficult topics, but by measuring things that were previously unmeasurable to deepen our understanding of humans, I think we can contribute to the accumulation of knowledge and ultimately, to society as a whole.
Kobe University is rich with interdisciplinary research; I myself am working with researchers in nursing. At the moment, I’m developing wearable devices that can measure swelling. I feel that the range of my research has expanded through this collaboration. The research environment at Kobe University has an air of freedom to it; I’m thankful to be able to tackle not only short-term projects, but also topics with a more future-oriented outlook.
On a more personal note, I’ve been doing farm work on weekends since I was a graduate student, so I have sort of a vague desire to tie my research to agriculture. I’d like to reduce the burden of the more demanding aspects of farm work without losing what is good about agriculture. I hope to be able to think broadly on the topic without limiting myself to just wearable devices.
Resume
In 2014, graduated from the Faculty of Engineering, Kobe University. In 2016, completed the master’s program in environmental science at the Graduate School of Frontier Sciences, University of Tokyo. In 2019, completed the doctoral program at the Graduate School of Engineering, Kobe University and received her doctorate in engineering. In 2019, became project assistant professor, in 2020 became assistant professor and in October 2025, became associate professor at the Graduate School of Engineering, Kobe University.
