Düsseldorf When shopping online, you never know what the fabric of the new top will really feel like or how the latest tech gadget will feel in your hand. But soon customers will finally be able to touch the online display.
This is made possible by an inconspicuous device that looks like a tablet. If you move your hand over it, you will feel what is on the screen.
The English start-up Ultraleap has built many small speakers into its “tablet”. The “acoustic radiation force” technology uses ultrasound to create virtually tactile surfaces and objects without touching them.
Numerous researchers and companies are currently working on such so-called “mid-air haptics” technology – haptic impressions in the middle of the air. The “touchable internet” is an important step on the way to the merging of physical and virtual reality into the metaverse. A futuristic idea is tempting: touching the handbag in the online shop, shaking a friend’s hand in a zoom call.
That might fundamentally change online shopping, the gaming world and even driving. Market researcher VPA Research forecasts a market for tactile technologies of 28 billion US dollars for the year 2026.
The device below the screen can make touches felt – with a range of up to 70 centimeters.
(Photo: Ultraleap)
At the furniture company Ikea, customers can already visit a virtual showroom with VR glasses. H&M offers digital fitting rooms with individual customer avatars. Nike, Ralph Lauren and Kaufland are experimenting with virtual reality stores.
Technology consultancy Gartner predicts that 30 percent of global companies will offer products or services in the metaverse by 2026. The US investment bank Morgan Stanley estimates sales in the Metaverse in 2030 for luxury brands alone at 50 billion US dollars.
Mid-Air Haptics – a feeling like right next to a concert speaker
“Haptic feedback” has been around longer and provides much more precise mechanical feedback than humming classic vibration motors. On iPhones with a home button, it makes it feel like you’re pressing a real button – although since the iPhone 6S there has only been a fixed touch sensor. The new type of haptic feedback in the Playstation 5 controller also ensures that it feels as if the gaming device is being tapped firmly on the hand. In both cases, however, objects move in the hand.
The “air technology” does not need any objects or even haptic gloves and therefore feels more realistic. Haptics researcher Claudio Pacchierotti from the French National Research Center explains that mid-air haptics technology is comparable to the feeling right next to a concert speaker: “If the music is very loud, you can feel it on your skin.”
The user does not have to touch the tablet to feel surface structures.
(Photo: Ultraleap)
Ultraleap’s device works in a very similar way: with ultrasound, which goes over the skin in so-called shear waves and stimulates the mechanoreceptors there, which are responsible for the sense of touch. This creates many small “pressure points” on the skin, which transmit the tactile image of an object to the brain. “In the future, technology may make it possible to feel things remotely before buying them,” says Pacchierotti.
Combined with virtual reality glasses, this results in a very convincing illusion. Orestis Georgiou, head of research and development at the start-up Ultraleap explains: Sensors in the headset record the movement of the hands, and a tablet connected to it uses ultrasonic waves to create the haptic impression in the right place.
Immense economic importance
Margot Racat is a researcher in digital marketing at the University of Lyon in France and also studies the impact of feeling on consumption. In one of her recent studies, she found that haptic technologies increase purchase intentions. That makes them extremely interesting for trade – and thus for the global economy. According to a UN study, the 13 largest trading platforms in the world sold goods worth 2,400 billion euros last year.
Haptics researcher Pachierotti confirms: virtual purchases and games are more realistic if you can not only see objects, but also feel them. “Haptics are fundamental to the Metaverse.”
Customers might then feel different materials in a shop with mid-air haptics and VR goggles, put their clothing together individually and pick up a fully configured shoe – before it is even made.
Pacchierotti is working on such applications together with the University of Birmingham (England), the University of Delft (Netherlands) and Ultraleap in the H-Reality project. The team is already in contact with potential dealers to test demonstration applications.
Be especially careful with the sense of touch
But how suitable are these new developments for everyday use? For training purposes, such as in flight simulators, haptic technologies are already mature and helpful, says consumer researcher Margot Racat. In their opinion, however, they have not yet been sufficiently tested for the consumer market.
Especially with the sense of touch you have to be careful how you stimulate it with a technology. It is the most intimate contact possibility of a person, and the sense of touch is quickly overwhelmed. A great many materials might not be felt with mid-air technology, which can lead to confusion.
Racat is talking here in particular regarding the younger generation, who only know some products from the online shop: “Subsequent generations will not have any experience with the originals when they touch them virtually.”
She considers mid-air haptics to be useful in combination with holograms: if a three-dimensional image is projected into the physical world via a smartwatch or augmented reality glasses, it would feel normal to be able to touch it. “But we’re still a long way from that,” says Racat.
Substances and textures can only be felt to a limited extent
There are limits to feeling, says Ultraleap researcher Georgiou. It has not yet been possible to make cashmere, wool or polyester tactile with mid-air haptics technology. “But with haptic technology, we can enable an immersive shopping experience,” says Georgiou. Customers might at least feel the structure of a piece of clothing, i.e. whether a sweater is knitted together like a wave or has a smooth finish.
Andreas Noll from the Ludwig Maximilians University in Munich is also researching haptic technology and confirms: “Rich and complex impressions, such as touching different materials, is not yet possible with mid-air haptics.” useful for simple applications, such as when air bubbles burst in a game or rain falls on the skin.
For Noll, gaming in the metaverse is currently the more important area of application for mid-air haptics: “More and more VR glasses are coming out, more and more audio devices, what’s still missing is touch.” Joysticks and controllers might then fall away. People who like to play video games are usually also open to new technologies that connect them even more closely, says Noll. Here he sees the possibility of a first breakthrough in technology.
If the level of detail of the ultrasonic waves continues to increase, substances might also become tactile. Until Mid-Air is ready, Noll is researching an alternative: haptic gloves that enable greater accuracy. “If you feel a lot of small vibrations, you can display more detailed textures and surfaces.”
Finally touch the works of art in the museum with mid-air haptics
Feeling in the virtual realm enables numerous application possibilities. In museums, visitors might touch digital works of art that they see on screens or as holograms. Self-service terminals such as check-in counters at airports or self-service checkouts might also become clearer and easier to use. Touchable hologram buttons and virtual display models might reduce the error rate.
The developers hope that instead of many buttons, holograms on the car’s dashboard might only show what is currently needed in conjunction with mid-air technology, making operation easier or faster. Ultraleap considers its own technology to be particularly promising in this area.
It may be a long time before mid-air haptics find their place in e-commerce and make the metaverse tangible. But scientists and entrepreneurs also agree: the day will come when we can shake hands, rotate a perfume bottle or stroke a sweater via the internet.
More: That’s behind the Metaverse hype