Facial Recognition And AR: Envisioning Who We Could Be

Republished By Plato

Followers: 0

AR facial recognition technology is prevalent across social media, retail, gaming, dating and even healthcare

Facial recognition software can partner with AR to project these fantastic and aspirational visions on ourselves. It can help us shop, dream, fall in love, feel better, and stay safe, all by looking into a camera, writes Snehaal Dhruv, CEO at Cameraah

We have always known that smartphones and other advanced telecommunications devices would change the world. Some of those changes have been a surprise, and in the case of augmented reality (AR), a more literal changing of the world than anyone expected. By loading an app and holding up our phones, we can see the world transformed into a dreamlike place full of fantastic creatures, marvelous vistas, aspirational products, and more.

AR has turned our smart devices into windows on other worlds, and with facial recognition technology, can even change the way we present ourselves to the ever-more-important online world. If this immersive technology can create the world we want to see, pairing it with facial recognition can create the selves we want to be.

To begin with, let’s establish the difference between AR and biometric face recognition. Biometric face recognition verifies an individual’s identity by scanning and confirming their facial features, and is used in security software by companies such as Apple and Bank of America to protect user accounts.

AR facial recognition creates an ‘emphatic camera’ that can detect the presence of a human face and create a grid on which to snap AR assets such as filters, animations, effects, and more onto the appropriate areas of the face. AR software can also detect changes of expression to trigger different effects determined by the user’s mood or effect.

Smile! Now with optional fangs!

With that explanation in mind, it’s easy to recognise the most common use of AR facial recognition software: the ubiquitous selfie filters that fill our social media feeds with pictures of people with animal features, halos, unicorn horns, alien eyes, and more.

Photo filters are by far the most common use of AR facial recognition, turning everyone’s phone into a fantastic toy, a bottomless dress-up closet with all the costumes and effects anyone could want. Options include everything from subtle changes to cartoonish exaggerations to realistic reworkings of the whole face to disguise one’s identity. Many of the best implementations of AR facial recognition use the technology in this way, to let people augment their photographs.

Snapchat Lens Studio - full body tracking — ***Snapchat Lens Studio even offers full-body tracking***

Though Snapchat first brought AR facial recognition filters to prominence, the most successful application of these selfie tools to-date has been on Facebook’s subsidiary, Instagram.

Social media is a great place to leverage this technology as the various platforms already rely so much on projecting a fantastic version of oneself onto the web. Instagram is so associated with these photo accessories that they are widely called “Insta Filters” no matter which platform is being discussed, the way all photocopies are now called xeroxes, regardless of the company that made the copier. The AR filters create hours of extra engagement on the platforms that include them, with users taking and posting many pictures they would not have bothered to take without the added fantastical layer of AR filters.

Shop without dropping

But AR facial recognition is good for more than just selfies. The ability to project a fantasy directly onto oneself is great for advertising, allowing the user to see themselves as a character in a story. Movies, TV shows, and other mass media works especially have been good at leveraging this type of advertising, creating apps or filters on other apps where users can depict themselves as iconic fictional characters such as Transformers, ninja turtles, werewolves, or Pennywise the dancing clown (specifically to keep me from sleeping at night, I assume…).

By filming or photographing themselves with features from these characters molded accurately onto their own faces, consumers can express their own enthusiasm for coming productions, and spread that excitement through their social circles.

Of course, by projecting more realistic scenarios onto an AR facial recognition app, brands can advertise tangible products rather than works of media. Retail businesses such as Macy’s feature AR apps that allow virtual try-on, letting users see themselves in various clothing items and accessories on their own screen. Proper use of physics engines and modern rendering and lighting tools allows consumers to see how different items realistically hang and catch the light, or how certain fabrics drape on their bodies.

Not only does this software create an aspirational fantasy wherein the user can see themselves in the desired item, it also makes online shopping easier and more reliable, as the user already knows how they’ll look in whatever they buy.

AR technology shows us more than just what we can buy—it shows us who we can be. Virtual makeovers allow users to see how they’d look with a new haircut, a change in beauty routine, or even cosmetic surgery.

Doctors can show patients how they’d look after an elective or reconstructive process while barbers and hairdressers let their customers select their new coiffure before touching a hair on their heads. Cosmetics companies are already using virtual makeovers to both inspire and market to consumers, with brands such as MAC expecting significant upticks in sales once people see how they look using various products.

All this potential advertising means more customer service interactions, for better or worse. But AR facial recognition can make more of these interactions better by allowing customer service personnel to monitor customer reactions and moods through expression and body language.

By tracking eye movement, set of the jaw, and angle of the head, facial recognition software can warn call desk operators if a customer is unhappy or growing agitated, or if they are satisfied, regardless of what they say aloud, and data-driven artificial intelligence (AI) can suggest a course of action for every reaction to help make customer experiences positive and productive. Companies can also use filters to mask their employees’ identities in what could become hostile encounters.

Social dynamics

AR facial recognition is good for more than just one-way self-expression wherein a person sees an augmented photo of themselves to try something on or share later. This new technology can open up whole new vistas of online socialisation.

Video conferencing products such as Zoom and Teams already feature AR filters that can both make users appear to be in exotic locales and change their features, or even their whole appearances. This creates the possibility for online costume parties, structured role-play in a team-building environment, or just a chance to add levity to a family group showing up as an animal—or with features taken right from a family member, such as mother’s hairdo or father’s beard.

Social uses of AR facial recognition filters go well beyond conferencing. Gamers have recently begun implementing AR masks, hats, helmets, and more to enliven their video chats and streams.

Facial recognition and AR - Streamlabs Face Masks — ***Streamlabs Face Masks***

Pro gaming teams can design their own team masks as a uniform and put them online for fans to use, and players who wish to socialise without revealing aspects of their identity to what can often be a toxic environment can disguise themselves with filters. Streamers and other online personalities can use facial recognition to create entire personas to keep their identities secret and their personal lives private while embodying a fun, marketable brand.

If facial recognition is good for games, then the game of love is absolutely included. Online dating has only increased in popularity, and AR apps can create situations where people chat and socialise without seeing one another’s faces.

This new approach to online dating preserves user privacy while still letting potential partners read expressions and physical cues. If done right, it can even add an air of mystery and romance to an online dating scene that may seem too demystified and transactional to some individuals. Given how many romances have begun in anonymous spaces where people use fictional personas instead of direct video chat (such as online games and chatrooms), some people may have an easier time wooing a partner from behind an avatar.

Indispensable features

Gaming, dating, shopping, and socialising are all great ways to spend our time, but healthcare is one area we cannot ignore.

AR facial recognition can help healthcare providers meet remotely with patients and use eye-tracking and similar software to read expressions or check if some aspect of a person’s face has changed, such as detecting puffy eyes, sunken cheeks, swollen lymph nodes, jaundice, pink eye, and more.

Therapists and psychiatrists can use expression reading AR filters to detect if a patient is lying, nervous, triggered, or otherwise in distress. So long as the patient gives consent, such programs would only be visible on the doctor’s side, creating a stress-free but enhanced evaluation environment.

Travel and transport are another indispensable field where AR facial recognition could literally save lives, especially in concert with another major technological advancement: autonomous vehicles.

When riding in driverless cars or trucks, users could wear AR-augmented heads-up displays such as smart glasses and can be informed of hazards, changing traffic, and other conditions on the road, and even feed that data to the AI operating the car. People driving their own cars could set up their facial recognition software to recognise signs of distraction and fatigue, and advise them to take a break or even have the vehicle’s autonomous mode kick in during emergency situations such as nodding off at the wheel.

AR technology empowers us to imagine the world we want through our phones, tablets, smart glasses, and other devices. Facial recognition software can partner with AR to project these fantastic and aspirational visions on ourselves. It can help us shop, dream, fall in love, feel better, and stay safe, all by looking into a camera. Perhaps the only thing it cannot yet do is predict what new uses we will think of for AR.