Researchers make virtual keyboard a reality

Researchers at the Swiss Federal Institute of Technology in Zurich (ETH) want to make virtual reality (VR) usable in everyday applications. “Today, VR is used mainly to consume content. In the case of productivity applications such as in office scenarios, VR still has much potential for development to replace current desktop computers,” says Christian Holz, a professor at ETH Zurich’s Institute for Intelligent Interactive Systems, in a press release.

Holz and his research team have developed a sensory technology called TapID. This novel sensor design embeds several acceleration sensors in a normal rubber wristband. These sensors detect the tiniest vibrations to determine which finger the person is tapping – on a table, wall or even thigh.  Each finger generates a different vibration profile on the wrist.

A custom machine learning pipeline created by the researchers processes the collected data in real time. “In combination with the camera system built into a set of VR glasses, which captures the position of the hands, TapID generates extremely precise input,” ETH explains. The researchers have demonstrated this in several applications, including a virtual keyboard and a virtual piano.

This method not only works reliably with the special rubber wristband but could also transfer to existing fitness wristbands and everyday smartwatches, because they are all equipped with inertia sensors.

The researchers will be presenting their TapID sensory technology at the IEEE VR conference at the end of March. They then plan to continue to improve the technology with more test subjects and develop more applications to integrate TapID.