Facial recognition and AR: Envisioning who we could be

Social media is a great place to leverage this technology as the various platforms already rely so much on projecting a fantastic version of oneself onto the web. Instagram is so associated with these photo accessories that they are widely called “Insta Filters” no matter which platform is being discussed, the way all photocopies are now called xeroxes, regardless of the company that made the copier. The AR filters create hours of extra engagement on the platforms that include them, with users taking and posting many pictures they would not have bothered to take without the added fantastical layer of AR filters. 

Shop without dropping

But AR facial recognition is good for more than just selfies. The ability to project a fantasy directly onto oneself is great for advertising, allowing the user to see themselves as a character in a story. Movies, TV shows, and other mass media works especially have been good at leveraging this type of advertising, creating apps or filters on other apps where users can depict themselves as iconic fictional characters such as Transformers, ninja turtles, werewolves, or Pennywise the dancing clown (specifically to keep me from sleeping at night, I assume…).

By filming or photographing themselves with features from these characters molded accurately onto their own faces, consumers can express their own enthusiasm for coming productions, and spread that excitement through their social circles.

Of course, by projecting more realistic scenarios onto an AR facial recognition app, brands can advertise tangible products rather than works of media. Retail businesses such as Macy’s feature AR apps that allow virtual try-on, letting users see themselves in various clothing items and accessories on their own screen. Proper use of physics engines and modern rendering and lighting tools allows consumers to see how different items realistically hang and catch the light, or how certain fabrics drape on their bodies.

Not only does this software create an aspirational fantasy wherein the user can see themselves in the desired item, it also makes online shopping easier and more reliable, as the user already knows how they’ll look in whatever they buy.

AR technology shows us more than just what we can buy—it shows us who we can be. Virtual makeovers allow users to see how they’d look with a new haircut, a change in beauty routine, or even cosmetic surgery.

Doctors can show patients how they’d look after an elective or reconstructive process while barbers and hairdressers let their customers select their new coiffure before touching a hair on their heads. Cosmetics companies are already using virtual makeovers to both inspire and market to consumers, with brands such as MAC expecting significant upticks in sales once people see how they look using various products. 

All this potential advertising means more customer service interactions, for better or worse. But AR facial recognition can make more of these interactions better by allowing customer service personnel to monitor customer reactions and moods through expression and body language.

By tracking eye movement, set of the jaw, and angle of the head, facial recognition software can warn call desk operators if a customer is unhappy or growing agitated, or if they are satisfied, regardless of what they say aloud, and data-driven artificial intelligence (AI) can suggest a course of action for every reaction to help make customer experiences positive and productive. Companies can also use filters to mask their employees’ identities in what could become hostile encounters.

Social dynamics

AR facial recognition is good for more than just one-way self-expression wherein a person sees an augmented photo of themselves to try something on or share later. This new technology can open up whole new vistas of online socialisation.

Video conferencing products such as Zoom and Teams already feature AR filters that can both make users appear to be in exotic locales and change their features, or even their whole appearances. This creates the possibility for online costume parties, structured role-play in a team-building environment, or just a chance to add levity to a family group showing up as an animal—or with features taken right from a family member, such as mother’s hairdo or father’s beard.

Social uses of AR facial recognition filters go well beyond conferencing. Gamers have recently begun implementing AR masks, hats, helmets, and more to enliven their video chats and streams.

About the author

Snehaal Dhruv is the founder and CEO at Cameraah