Originally published as a four-part series, this is the long-read version consolidating the latest research of the global forward march of facial recognition technologies. From Apple normalizing its acceptance, to disruption in the retail / security / healthcare industries, to the emergence of China as the FaceID innovation laboratory, and the ever-growing new global applications, we stare into your auspicious Internet of Things (IoT) future.
Your Face is The Next Computer Cookie
In January 2018, in a D&D Daily "Live from NY" interview, we debated the impact of iPhoneX on FaceID technologies. This new Apple iPhoneX had just placed the disruptive future of facial recognition technology in our pockets.
Less than a year later, it is surprising to observe the continued advanced deployments of FaceID solutions, especially outside the United States. For 2018, the global facial recognition market is valued at $3.97 billion and is estimated to reach $10.15 billion by 2025 at a CAGR of 14.35% during the forecast period.
Unlocking Smartphones with Your Face is Only the Beginning
Very interesting is the mix of the major organizations investing in face recognition. Out of the top 10, eight are academic entities with most based in China. The only USA university that makes the top 10 is Carnegie Mellon. Microsoft leads the top 10 followed by Samsung which already has 186 patents in facial recognition.
The Retail Watch is Underway
Throughout the series of these articles focus will be on presenting industry examples. Personal commentary will be intentionally minimized to allow each of you to decide for yourself the future of this disruptive technology.
- In 2015, Walmart tested facial recognition in some of its stores. The retailer publicly stated they discontinued the pilot as it did not provide adequate ROI.
- MasterCard released a facial identity mobile app to verify payments using the camera of the consumer smartphone.
- Warby Parker released the 'Find Your Fit' app to help consumers find the right frames for their face. "The feature uses the iPhone X’s new TrueDepth Camera to scan your facial measurements then recommends frames best suited for your face shape. This will (hopefully) save you from trying on 50 different frames in the store."
- Vogue announced a partnership with Apple to create an augmented reality feature exclusively for iPhone X users. "The VogueEffect will be an iMessage extension that uses the front camera to create multidimensional filters.
- Caliburger in California links facial recognition to its loyalty program. The software in ordering kiosks recognizes registered members as they approach, activates the loyalty programs, and makes meal recommendations. Caliburger sees solution as a stepping stone to replacing credit cards with face-based payments.
- In March 2018, the American Civil Liberties Union (ACLU) asked the top 20 USA retailers whether they were using facial recognition technologies (substituting Disney for Amazon because of their limited number of physical stores). Eighteen of the companies refused to answer the question with some citing proprietary or competitive reasons. Ahold Delhaize said no and Lowes stated they use facial recognition to identify shoplifters.
- According to Gizmodo, a select few self-service checkouts at supermarkets in the UK will soon be equipped with cameras and facial recognition software. The checkouts are reportedly going to use tech from British startup Yoti to determine if a shopper is old enough to buy alcohol and cigarettes.
- FaceFirst, a facial recognition software provider stated that they had hundreds, soon to be thousands retail store installations. The solutions have reduced theft by more than 30%. Retailers offer not to press charges, if shoplifters opt into database. "Retail stores with facial recognition systems can spot convicted or admitted shoplifters in about the time it takes to walk two paces into an establishment."
Your Face Is the Next Computer Cookie
In multiple previous posts, I have framed the future of retail as the intersection of technology with immersive customer experiences. Realistically this means having greater personal granularity on the consumer walking into the retail door.
Driving the increase utilization of biometric identification in all consumer industries is partly a "century-long trend toward extreme personalization." Your face is potentially your computer tracking cookie.
"When you surf the internet, the various websites you have previously visited can follow you because their cookie is embedded in your browser. Now, the vision is that, as you walk around town or anywhere else, stores that share data with one another will be able to follow you on foot because your face will be associated with their businesses."
This last video presents some eerie potential privacy scenarios. Possible solutions to this challenge will be discussed at the conclusion of this disruptive technology series.
Face It Security Industry, More Technology Disruption Is Coming
Security has been a primary market for face recognition for some time. Threats of terrorism and violent crime have accelerated its adoption by governments and law enforcement worldwide.
STOP, I Can Digitally See You Now
With a 25% CAGR, the security sector is leading facial recognition deployments worldwide.
- From London, Tokyo, Helsinki, Rome and now to Atlanta in USA, airports worldwide are adopting FaceID solutions. Delta Airlines will soon roll out ' curb to gate' facial recognition for travelers departing out of the International Terminal F in Atlanta, the first biometric terminal in the United States. Smile for the camera at self-service check-in including baggage, through security checkpoints, and boarding your flight. Using the technology is optional and will save travelers up to nine minutes per flight.
- NEC has already provided biometric identification systems, including facial recognition to approximately 50 airports around the world. Shanghai's Hongqiao airport unveiled kiosks for flight and baggage check-in, security clearance, and boarding powered by facial recognition.
- The Federal Bureau of Investigation’s database contains over 30 million mugshots of criminals and ID card images.
- Implemented in 2010 and upgraded in 2016, the New York facial recognition system connects to the databases of ID photos and identify information recorded by the Department of Motor Vehicles. Since implementing the upgrade, the governor claims the software has led to more than 4,000 fraud arrests involving criminals attempting to steal identifies.
- Uber uses driver selfies to verify driver identifies before approving passenger rides.
- Australia is developing its own crime-fighting facial recognition surveillance system that can match citizen's faces in real time. "The technology collects driver’s license photos, passport pictures, visa application images, and other photos from state and federal government sources, and pools them in a database. All that biometric data is then compared to images culled from CCTV footage and other surveillance equipment to identify people"
- "Travelers departing from Dubai will no longer need to pass through any sort of security clearance counter or e-gate, they will simply walk through a virtual aquarium tunnel that will scan their face or iris using hidden cameras." The fish displayed in the tunnel invite you to look around at the over 80 cameras to establish your FaceID.
- This fall, new facial recognition technologies identified three impostors trying to enter the United States at Washington Dulles Airport with fake passports.
Your Face Tells Me You Are Sick
Substantial face recognition opportunities are emerging in the healthcare sector. Through face technology and deep learning, "it is already possible to track a patient's use of medication more accurately, detect genetic diseases such as DiGeorge Syndrome with a success rate of 96.6%, and support pain management procedures."
An example healthcare solution is from AiCure which tackles the problem of patients failing to take their medicine. This is a major USA challenge for approximately 50% of patients, leading to an estimated $100 billion in annual costs from hospital admissions.
"When you visit a doctor, it is not just about the eye-to-eye contact, but signs of several diseases can be read from your face. Face recognition technology uses similar ways to find facial traits that reveal clues to health conditions. Every day we look at our face in the mirror and we can’t easily recognize a change in our appearance. Machine learning helps in finding health issues with the help of face recognition technology. The new technology is also helping doctors to treat rare genetic disorders."
Follow the Major Disruptors Money
The investments being made by Google, Apple, Facebook, Amazon, and Microsoft (GAFAM) are a good indicator on the potential future direction and growth of disruptive technologies such as face recognition.
- Apple acquired Emotient, a startup that uses artificial intelligence (AI) to read people's emotions by analyzing facial expressions.
- Facebook acquired MSQRD, a face-swapping app and FacioMetrics which uses machine learning to let smartphones analyze faces in real time through the frame of a camera.
- Google acquired PittPatt which matches people across photos / videos, Moodstocks which develops machine-learning based image recognition for smartphones, and Viewdle which can automatically tag photos.
- Microsoft has so far filed 56 patents in facial recognition and is Number 1 in the top 10 most active organizations with this technology.
- Amazon acquired Orbeus which uses deep learning to provide scalable image and face recognition solutions for businesses and consumers.
The Face Recognition Infused Platforms
FaceID investments have been surfacing in each of the GAFAM software platforms for some time.
- In 2014, Facebook launched DeepFace which can determine whether two photograph faces belong to the same person with an accuracy of 97.25%.
- In 2015, Google introduced FaceNet, "a new recognition system with unrivaled scores: 100% accuracy in the reference test Labeled Faces in The Wild, and 95% on the YouTube Faces DB."
- In May 2018, Amazon was reported as already actively promoting it's Rekognition cloud-based face recognition service to law enforcement agencies. The solution recognizes as many as 100 people in a single image and can perform face match against databases containing tens of millions of faces.
- Microsoft has accelerated development of facial recognition solutions. Major focus is on self-learning systems that can be used for people recognition in events, live presentations, and movies.
- Unlocking the smartphone with your face normalized face recognition as just another daily use technology. At the iPhoneX release, "people raced to test its (face) limits and see if they could break it...Then the hype died down, and people with the iPhone X simply got used to opening their phone with a glance. It proved that facial recognition tech had finally become sharp enough to be useful. It wasn't just cool; it was convenient. A year later, we're all a lot more used to it." The latest iPads just released in October are also transitioning to facial recognition.
Technology convergence and applications outside traditional security are leading to more powerful FaceID based solutions. The combination of artificial intelligence, machine learning, Internet-of-Things (IoT), and cloud computing, plus ever hungry global innovation hubs are accelerating traditional product cycles.
It's an Amazing FaceID Small World After All
China is now THE world FaceID innovation laboratory. Country is installing an extensive CCTV surveillance network, fining of jaywalkers through video, and is disrupting the retail industry with facial recognition.
These Cool Sunglasses Just FaceID You
- Police officers at the Zhengzhou East high-speed rail station, using facial recognition glasses, identified seven fugitives related to hit-and-run and human trafficking cases, plus spotted 26 cases of identity fraud in February this year.
- China is perfecting a nation-wide video surveillance network. 176 million surveillance cameras were in use at the end of 2017 and 626 million are expected by 2020. In a recent facial recognition experiment in the city of Guiyang (4.3+ million people), a BBC reporter was found within the city in just seven minutes.
- Shanghai, which has one of the world's longest and busiest mass-transit systems is installing voice and facial recognition Alibaba solutions. Voice will speed up ticket processing and facial recognition will verify the identities of the commuters.
- Facial recognition systems are being used in China to identify and fine jaywalkers.
- At separate concerts in China, face recognition was used by the police to catch three wanted fugitives. In one case, a man was identified in a crowd of 50,000 people.
- One popular park in Beijing uses face-scanning toilet paper dispensers in public bathrooms to limit each person to a two-foot length of paper every nine minutes.
China's Digital Retail Transformation
Globally China leads digital transformation experimentation in multiple industries, including retail.
- JD.Com currently has six China stores using facial recognition. "Shoppers merely sign up to a JD.com account, via their smartphone. Once they’ve been verified, they can shop freely — there are no queues, nor cashiers. Payments are deducted from their account, as they leave the store."
- "Lenovo has opened an unmanned convenience store in China, using it as a testbed for trialing facial-recognition, e-payment, Internet of Things (IoT), artificial intelligence (AI), and augmented reality (AR) technologies, with its tablets acting as the staffers of the store."
- The Denmark-based Bestseller Fashion Group worked with Tencent to develop two smart fashion shops in China. "By scanning the contours of their faces, consumers are able to receive fashion advice from automated virtual assistant services. Products that they like can be bought directly through WeChat. Shoppers can also elect to join a bestseller facial recognition club, which allows them to make purchases at the automated stores simply by having their faces scanned."
- In 2017, a KFC location in Hangzhou China allowed customers with registered accounts to pay for their meals using facial scans. "The software also recommends orders based on a customer’s age, mood, and gender." An affiliate of Alibaba launched the service.
Banking on China's Face Recognition
- In China, HSBC Holdings PLC, a British multinational financial company allows customers to transfer up to 50,000 yuan ($7,515) per day to new payees by combining facial recognition with passwords on the bank's mobile app.
- Chinese start-up Cloudwalk Technology is helping banks boost efficiency by upgrading facial recognition with machine learning and artificial intelligence to identify VIP customers and their mood. Consumers do not need debit cards to withdraw money from ATMs.
China's Image Recognition Investments Keep Growing
- SenseTime, valued at $4.5 billion, has established itself as China's largest AI company. The platform boasts "an astounding 1,207 network layers that can simultaneously train up to 2 billion facial images. The company claims that is has already processed 500 million identities for facial recognition purposes.
- Home-sharing start-up Xiaozhu, China’s answer to Airbnb, will use the peak travel season of Lunar New Year to test smart locks that can be opened by scanning tenants’ faces.
- In May, a middle school in east China installed cameras to analyze students' facial expressions to determine if they were paying attention in class.
- Students at Hangzhou Normal University in Hangzhou use face-recognition technology to enter their dormitories. The university is also considering using the technology to keep track of attendance.
- Honour of Kings, one of China’s most popular video games, is testing the use of facial recognition to check the ages of users to limit the playtime of younger players.
- An app in China has successfully reunited missing children with their families within one week using facial recognition.
- Volvo which is now Chinese owned is using advanced face recognition systems to help tired driving stay awake by checking their alertness.
- 7-Eleven in Thailand is rolling out facial recognition in all their 11,000 stores. "Combined with behavior analysis, the tech will be used to identify loyalty members, analyze in-store traffic, monitor product levels, measure customer emotions and suggest products to consumers while they shop."
- The International Finance Center Mall in Seoul South Korea is using facial recognition in its information kiosks. " As a customer approaches, the cameras identify the person’s age and gender in real-time, personalizing interactive advertisements accordingly."
- India, which already has one of the largest biometric databases, announced the government launch of facial authentication for July 2018.
- Russia's Central Bank has been deploying a country-wide program since 2017 designed to collect faces, voices, iris scans and fingerprints.
Staring Into the Auspicious Future of FaceID Technologies
The FaceID emerging applications list continues to grow, even with privacy concerns expressed at multiple levels. The genie on the potential disruption and business opportunities of face recognition solutions is out of the global bottle.
The Face of an Increasingly Digital World
- More than one billion smartphones featuring facial recognition will ship by 2020. Apple and Samsung will lead the pack followed by multiple Chinese manufacturers. 64% of all smartphone shipments in 2020 will include facial recognition versus 5% in 2017.
- USA Army researchers have developed an artificial intelligence and machine learning technique that produces a visible face from a thermal image of a person's face captured in low-light or nighttime conditions. Hiding in the dark is not an option.
- Google was recently awarded a patent for 'Facial Recognition with Social Network Aiding'. The invention envisions incorporating "information from social networks — potentially including Facebook, Twitter, Gmail, online blogs, and more — to help identify who people are."
- Walmart has patented technology that would capture shoppers' facial expressions while in the checkout lines so that it could measure levels of dissatisfaction.
- In the UK, the Imperial College London is working on facial recognition methods to improve online teaching strategies. The University College is analyzing faces to detect differences between healthy people and Parkinson's patients.
- In France, Zodiac Aerotechnics has recently patented a camera able to detect whether a passenger is seated in his assigned seat on an airplane or train.
- In Canada, PiP has built a pet identification app. "When a pet goes missing, the application alerts veterinarians, rescue and animal control agencies, subscribers of the app and users of social networks to find it."
- Facial recognition research is now shifting towards sentiment analysis, recognition based on a person's gestures.
- NEC recently announced plans to use facial recognition to identify and authenticate over 300,000 athletes, volunteers, media personnel, and guests for 2020 Tokyo Olympics.
- The 2018 Buick Enspire concept automobile featured facial recognition capable of unlocking the vehicle as the owner approaches and automatically configuring the interior to suit his or her preferences.
- Online dating apps are moving to facial recognition algorithms which can easily filter out irrelevant profiles and find more tailored search results.
- Singapore is planning to install cameras with facial recognition on all 110,000 lamp posts around the city. The 'Lamppost-as-a-Platform' project will allow the government to “perform crowd analytics” and support anti-terror operations.
- Chinese authorities in Beijing and Shanghai have rolled out new Gait recognition technologies which can identify people just by their walking style and silhouette. An improvement on facial recognition, the software works 50 meters (164 feet) away and cannot be fooled by abnormal body motions.
To Hack or Not Hack Your Face
As with any new technology, once adoption increases, disruptors trying to defeat it are not that far behind.
- University of Toronto researchers recently announced new Artificial Intelligence (AI) technologies that can defeat facial recognition.
- "Experts at MIT and Kyushu University fooled an algorithm into labeling a picture of a 3D-printed turtle as a rifle by altering a single pixel."
- Researchers at Carnegie Mellon University demonstrated how specially designed spectacle frames can fool even state-of-the-art facial recognition software.
- In Russia, Grigory Bakunov invented a solution to escape the eyes permanently watching our movements and confuse face detection devices.
- A Vietnamese company used a mask to hack the FaceID recognition of Apple's iPhoneX. "However, the hack is too complicated to implement for large-scale exploitation."
Privacy is a Major Concern
A recent USA Brookings Institute survey found that 50% of participants are unfavorable to the use of facial recognition software in retail stores to prevent theft. Forty-four percent are unfavorable to using this software in airports to establish identity, 44% are unfavorable to it in stadiums as a way to protect people, and 38% are unfavorable to its use in schools to protect students.
We are at a critical time with face recognition technologies and government regulations are not keeping up with the pace of innovation. The President of Microsoft shares similar concerns.
"Facial recognition technology raises issues that go to the heart of fundamental human rights protections like privacy and freedom of expression. These issues heightened responsibility for tech companies that create these products. In our view, they also call for thoughtful government regulation and for the development of norms around acceptable uses."
China is both a leader and a barometer on the potential challenges of this technology. By 2020, China expects to implement a national social credit system for its 1.4 billion people. The system will calculate a social credit score based on income tax, utility, credit bills, criminal activity, traffic records, shopping habits, and social media posts. It could be used to approve or blacklist citizens for travel insurance, loans, employment, internet access and more.
The Discernible Future of Facial Recognition Technologies
As this face recognition series has illustrated, digital innovation is taking us to places where we might not be fully comfortable. Valid concerns about intrusion and privacy need to be tackled by prudent regulations.
For businesses, hiding the use of FaceID technologies is a recipe for failure. Full disclosure, protecting personal data and more importantly providing opt-in options that deliver high value are the critical prerequisites to acceptance.
For retail, the success formula increasingly combines hyper-personal branding and immersive customer experiences through technology. The digital transformation journey is just getting started. Weighing both risks and opportunities, we will continue to stare into the auspicious potential of new technologies such as face recognition working towards an ever-growing digitally connected future.