Paying for Fried Chicken with the Face: A Journey around the Biometric Landscape in China

A customer orders crispy fried chicken using one of the screens at a Kentucky Fried Chicken (KFC) in Shenzhen. At the digital menu, there are pictures of the products and pricing information: a lip-smacking chicken bucket for 39 yuan (almost 5 euro), two chicken burgers plus a chicken bucket for 69 yuan (8.7 euro). After a couple of clicks, the consumer chooses his meal and pays for it. Most of the clients at this establishment use Wechat or AliPay, the two main Apps for mobile payments in China [Read: Inside a Cashless Society (Almost)]. There is a third way: paying with your face.

China has already become the largest force in mobile payments (11 times the transaction value of the United States) and now it wants to lead the facial recognition market. It is a sweet market. It is expected to grow from $ 4.05 billion (3,5 billion euros) in 2017 to $ 14.95 billion (13,2 billion euros) by 2026, according to Stratistics MRC. In China, the market size of the facial recognition industry exceeded 1 billion yuan (127 million euros) in 2016, and it is estimated to reach 5.1 billion yuan by 2021, according to a market research firm, Qianzhan.

IN THE RESTAURANT, SAY: “CHEEEESE”

On September 1, 2017, eleven days before that iPhone X was announced at Steve Job Theater in Cupertino, California, on the other side of the world, in Hangzhou, a land of lakes and green hills and today a busy city that is home to the Chinese e-commerce giant Alibaba, it was the first time in the world that facial recognition was used for payment in a physical store. It was at a KFC restaurant.

Yum China, which holds the rights to KFC in China, introduced Alibaba’s facial recognition software, called “Smile-to-Pay”. Until then facial recognition had already been used regarding security and personal identification in applications and mobile phones. There are already 300 KFC restaurants across China where you can already pay for fried chicken with your face, according to the company.  Outriders have visited some locations in Shanghai, Xi’an, Jinan and Shenzhen. All of them offered the option of facial recognition – few people used it.

The total process only lasts a few seconds. “Choose your meal, then select the form of payment (facial recognition), look at the camera for a moment and confirm the purchase with your phone number”, a waitress explains in a KFC restaurant in Shenzhen. The system scans the face and matches it with photo ID that has previously stored. She says, “until now customers prefer mobile payments”. “But I have seen some clients using it”, she adds. McDonald’s in China has also integrated biometric payments in some of its stores. “Paying with the face is faster, you take a photo, you type the last four digit of your phone, and it is done”, says a customer of the fast-food burger restaurant in Shenzhen.

Retailers are increasingly scanning faces and privacy around the world. In Japan, NEC Corporation has been conducting trials for cashless payment services with some employees since 2016. In Korea, convenience store chain GS25 has opened a location using facial recognition. In Finland, OP Financial Group has announced a pilot program at the company’s staff restaurant in Helsinki. Mastercard has announced that biometric identification will be made available for all its users by April 2019 and Spain-based bank BBVA and the startup Veridas are testing a facial recognition payment system in its corporate headquarters’ restaurants.

A few meters from Hubei subway station in Shenzhen, convenience store 7-Eleven has a self-serve kiosk with facial recognition to expedite orders. It uses the same procedure by Alipay: to click, to smile and to confirm with the phone number. Last March, 7-Eleven introduced facial recognition to all its 11,000 stores in Thailand.

Contactless. Staffless. Privacyless.

IN AIRPORTS AND TRAIN STATIONS

Biometric technology is becoming common in a new way of security in China. Facial recognition is already used: to locate criminals at train stations and 62 airports, to catch jaywalkers on public roads or to detect marathon cheaters as it happened recently in Shenzhen and Kunming. AFP reports that there is a surveillance tech scenario exploring robotic bird drones, called “Doves”.

At the checkpoint between Hong Kong and Mainland China, “now we use facial recognition to pass the control together with a fingerprint”, Wang, a 30 years old computer scientist who has just come back from Hong Kong to Shanghai, explains. He admits that he has never withdrawn money from ATM neither paid with his face “but for privacy reasons, I would not mind.” “However, I would say it might be slower than a traditional card,” he adds.

MORE DATA, LESS PRIVACY IN THE METRO

In Shopping Park station in Shenzhen subway, hurried travellers go through electronic arcs that include facial recognition, before passing their belongings through the scanner. Two cameras on the sides watch the passengers. Last August, Shenzhen Metro Group along with Tencent –Wechat developer based in Shenzhen— and GRG Banking unveiled their latest Metro ticket solution that will combine facial recognition and credit payment. The passengers will not need to buy a ticket, use their cards or mobile. They will just go through the turnstile in the process of only 0.33 seconds. The facial recognition metro ticket is already working in some stations in Guangzhou, capital of Guangdong province. To use it passengers must register in the Metro App, enter their personal details and upload a photo.

HI-TECH HOSPITALITY

In each hotel in China, there is a screen over the counter equipped with a webcam, which photographs the guests. To check-in in China, it is mandatory to show the passport and to be photographed. Now some hotels are streamlining the check-in process with facial recognition. It is the case of Marriott International Hotels. They have partnered with Alibaba to use their Fliggy digital travel service platform. 

“Chinese guests simply need to read and confirm the customer terms of service, scan their Chinese ID cards, pause for a photo and input contact details into a self-service machine”, a Marriot International spokesperson says. She explains, “the information is transmitted to the local police in real-time, which digitalises the traditional check-in process and guests” and once the identity and booking information is verified, “the intelligent device will dispense the room key cards.” Currently, two Marriott Hotels provide facial recognition check-in services in Hangzhou and Dadonghai Bay. The spokesperson says they will wait to see how customers react before expanding it to other locations.

SMART CITIES ARE WATCHING YOU

“This is what we imagined what future would be like,” explains a Yitu Technology spokesperson, a Chinese AI startup based in Shanghai, which has gained recognition for its Dragonfly Eye System platform. It allows facial scanning to identify a person in a few seconds from a database of at least two billion people.

Yitu representative refers to the AI city model that Yitu created in Fuzhou last June. “It can be described as the first AI city model in the world,” she explains. “Yitu helped the government set up an ‘AI City’ model for the First Digital China Summit providing AI services based on facial recognition technology in a range of scenarios: like the welcoming system at the airport and train stations, the self-check-in system at the hotel, at the summit venue entrance and self-service stores.” Founded in 2012, Yitu has received special recognition for its ability to recognise human faces and vehicles.

FROM ATMs TO HOSPITALS

If citizens can pay money with their face, now they can take out money also. An option called “Withdraw cash by face” is offered at some ATMs in China, along with the options: withdraw cash by Apple Pay, Huawei Pay, Samsung Pay, credit card and others. Yitu has deployed a facial recognition system in over 20,000 ATMs in China.

The “smartization” is coming to every field, even healthcare or what Yitu calls: “intelligent hospitals.” In a country of 1.4 billion, “the medical resources are not enough to meet with the demand and most of the resources are provided in big cities”, the spokeswoman of Yitu explains. She gives an example: in some hospitals, a senior doctor has a daily workload of 200 patients, and for each patient, he has 200 CT images for the scan. “It is 40,000 images to read for a doctor a day.” Yitu technology, she says, “can dramatically decrease the diagnosis and takes less than 5 seconds.” Her latest release is an Intelligent 4D Imaging System called CARE.AITM for cancer screening.

FACES BEHIND FACIAL PAYMENT

Other two Chinese AI start-up in the scene are becoming some of the most powerful in the world: SenseTime and Megvii. Hong Kong-based SenseTime describes itself as “the most valuable artificial intelligence startup in the world”. Its co-founder and CEO, Li Xu, 36, is one of the members of Fortune’s list of 40 Under 40. It has recently raised $600 million from Alibaba Group.

Beijing-based Megvii is a developer of Face++ system. The company was founded in 2012, and now it has over 1,500 employees. Its founder, Yin Qi, was included this year at MIT Technology Review’s annual list of Innovators Under 35. In 2016, when he was 28, it appeared on the ‘30 under 30’ list by Forbes. Alibaba is now the investor in both: it invested in Megvii, and it invested in SenseTime.

FROM ONLINE TO OFFLINE (AND VICE VERSA)

The revolution is here, and it is two-way. While traditional services become smart, online companies go offline. Amazon Go, the new retail store in New York, opened by the online shopping giant, is an example. In China, Hema supermarkets by Alibaba is other.

It’s a Saturday afternoon, and Hema in Xi’an is full of couples with children and single customers who go shopping. Under the symbol of a hippopotamus, the futuristic stores combine technology with a traditional market that offers fresh products. When the consumer downloads Hema App, he can scan the codes of each product and know their characteristics or the potential recipes. Another possibility is to choose the product to cook, for example, a crab, and eat it on the spot. Customers line up at the digital kiosks to pay. One of the payment options, of course, is facial recognition, using Alipay.

Next to Hema superstore in Xi’an, its competitor, JD.com, Inc., one of the largest Chinese e-commerce companies, has installed facial monitors on the street. JD operates over 20 unmanned stores across China, according to the company. The clients can simply pick up whatever they want and walk straight out of the store. Every item in the store contains an RFID tag and customers are identified using facial-recognition technology. They only need to access JD.com App, and it generates a locate QR code.

The Alibaba and JD.com’s strategic movement not only allows them to reach consumers who still prefer to buy in the physical store in China. Its strategy also lets them know consumers taste and anticipate future purchases.

Is the real world stop being real? An anecdote can shed some light on the issue. At the end of November, in Ningbo, near Shanghai, the face of a famous Chinese businesswoman appeared on the screens dedicated to detecting and shame jaywalkers in China. However, at that time, Dong Mingzhu, president of China’s biggest air conditioning maker, was not crossing the red light. She was not crossing the street either. The smart camera just captured her face in an advertisement on the side of a bus that was driving through the intersection.