:::
:::

R&D Focus

CoolSo, a Wearable Gesture Input Device for VR/AR

Video of CoolSo.

CoolSo is a wearable gesture input device that can be used with VR/AR products to improve user immersive experience. It creates an interactive VR/AR interface by using signals from muscle movement as a signal source for gesture control. This device applies an innovative principle which recognizes subtle movements of the human body by sensing vibrations in the muscles of the forearm caused by friction between the fasciculus and serosa. Utilizing MMG (mechanomyogram) to measure muscle contractions, the device adopts G-sensors or microphones as measurement sensors, avoiding the high costs of current electromyography technology and overcoming performance disruption issues due to perspiration. The technology also resolves problems of sunlight interference suffered by image recognition technology, and is available for one-fifth the cost of existing products.

CoolSo uses MMG technology to measure muscle contractions and enable gesture control.

CoolSo uses MMG technology to measure muscle contractions and enable gesture control.

The main components of CoolSo are the MCU device, sensor components, transmission modules, and power modules. The sensor components are acceleration sensors that are economical and accessible for suppliers. Both the MCU and transmission components that ITRI currently uses are universal and readily-available parts. Unlike the transmission of EMG signals from the nerve action potential to the skin, ITRI’s technology reads the vibrations produced from the friction of muscle fibers and the perimysium during muscle contractions. It is capable of measuring muscle movement characteristics and can use the motion signals of other body parts as a signal processing method to filter differentiation.

A core module under 1,000 mm2 including MCU, power module, and connectors of sensors and the transmission module.

A core module under 1,000 mm2 including MCU, power module, and connectors of sensors and the transmission module.

CoolSo uses MMG to measure muscle contractions and has three main advantages over the EMG technology currently in use:

Micro-structure of muscle.

Micro-structure of muscle.

As video games constitute the largest segment of the VR/AR controller market, the target application for CoolSo is in VR/AR video games, where fast and high-precision gesture technology is imperative for game interaction. ITRI’s CoolSo surpasses existing products in signal stability and is less costly. Technologically speaking, the surface of the skin can significantly impact electrical conduction and cause changes to signal status due to the users’ perspiration. Therefore, the accuracy will be affected if the detection uses dry contact electrode EMG. If sensor system correction is not used in the front terminal, an adaptive identification model must be used in the end terminal. ITRI’s CoolSo also outperforms other wearable counterparts in volume, battery life, transmission power, and recognition accuracy. Meanwhile, it offers a notable price advantage over competitor solutions and sets a new standard in the AR market.

This technology was awarded the Grand Prize and Design Recognition Award at the world’s first OpenStack Application Hackathon in Taipei in 2016. ITRI’s team was then invited to the OpenStack Summit in Austin, Texas to showcase its award-winning muscle signal detector for violinists to over 7,500 cloud computing experts from more than 60 countries.