Are you being scanned? How facial recognition technology follows you, even as you shop

If you shop at Westfield, you’ve probably been scanned and recorded by dozens of hidden cameras built into the centres’ digital advertising billboards.

The semi-camouflaged cameras can determine not only your age and gender but your mood, cueing up tailored advertisements within seconds, thanks to facial detection technology.

Westfield’s Smartscreen network was developed by the French software firm Quividi back in 2015. Their discreet cameras capture blurry images of shoppers and apply statistical analysis to identify audience demographics. And once the billboards have your attention they hit record, sharing your reaction with advertisers. Quividi says their billboards can distinguish shoppers’ gender with 90% precision, five categories of mood from “very happy to very unhappy” and customers’ age within a five-year bracket.

Mood is a particularly valuable insight for advertisers, revealing shoppers’ general sentiment towards a brand and how they feel in particular stores at certain times of the day. Unlike gender and age, mood is harder to determine, sitting at around 80% accuracy.

There are now more than 1,600 billboards installed into 41 Westfield centres across Australia and New Zealand. Scentre Group, Westfield Australia’s parent company, emphasises that all data collected is anonymous and that they are using facial detection, not facial recognition technology (FRT).

This means generic information such as a shopper’s age and gender is collected rather than the technology using photo-matching databases to identify who customers are. A spokesperson would not confirm whether or not Westfield would consider using FRT in the future.

Retail companies are increasingly turning to facial detection and facial recognition software to attract and engage a distracted audience. Quividi’s host of international clients include Telstra, 7-Eleven, Coca-Cola, oOH Media and HSBC bank.

Terry Hartmann, vice president of Cognitec Asia Pacific, the company that develops “market-leading face recognition technologies for customers and government agencies around the world”, says using facial detection commercially is no different to Facebook’s manipulation of users’ online search history for targeted advertising.

“You’re not identifying who that person is, you’re just identifying the characteristics of that person. That’s no different to Facebook popping up ads you might be interested in and social media picking up people based on their clicking habits or the shopping that they’ve done.”

While facial detection could be considered relatively benign, it is a step closer to the more problematic FRT.

Dr Dong Xu is the chair in computer engineering at the University of Sydney. He says that under optimum lighting and using high-quality photo data bases, FRT is more accurate than humans at identifying faces and can now recognise an individual from millions of photographs.

According to Xu, the technology is even more reliable at identifying criminals – and presumably other people – than using fingerprints.

This technology is still in its teething stages within the Australian retail sector, but FRT has significant investment and growth potential. International companies including Target, 7-Eleven, Walmart, Google and Facebook are all experimenting with facial recognition. The global FRT market is worth approximately US$3bn (A$4.1bn) and is expected to grow to US$6bn by 2021. Target and Walmart say they trialed the technology in-store to prevent theft and fraud, while 7-Eleven plan to use it to “identify loyal customers”. Facebook has been using facial recognition since December 2017to help users “manage their identity online” while Google has also been using FRT for some time.

China is the world’s leader in facial recognition, with more than 176m CCTV cameras. FRT is used for street surveillance and policing but also in China’s “cashless” stores. In these stores shoppers can buy products by simply scanning their faces, while in ATMS and hotels, all it takes is a glance to check in and take out money. Alibaba and Guess are now experimenting with a project called FashionAI. The project would fit FRT into changing room mirrors, allowing customers to see themselves in outfits without actually having to put them on.

FRT has a number of beneficial uses. In Canada, it has been used to helpthose who have identified themselves as problem gamblers. In a number of international airports, the technology is used to speed up checking in and security procedures. And in Australia, it has long been employed by border security at the arrival gates and in national policing efforts to detect suspects from criminal watch lists.

But there are major concerns about how to protect the privacy of those whose data is collected. In January a coalition of 85 civil rights groups wrote to Microsoft, Amazon and Google demanding the companies commit not to sell face surveillance technology to governments. When asked, Microsoft’s president, Brad Smith, rejected the idea of an outright ban, saying: “A sweeping ban on all government use clearly goes too far and risks being cruel in its humanitarian effect.” He was referring to the reported use of the technology to detect rare diseases and also reunite missing children in India.

But Microsoft has acknowledged the concerns about FRT and called for greater government regulation of the use of this technology. In a December 2018 Microsoft blogpost, Smith identified three main problems governments needed to address, namely the risk of bias and discrimination, new intrusions into privacy, and the potential for mass surveillance to “encroach on democratic freedoms”.

Australia already falls short when it comes to regulations in this area. For instance, under the current regulatory framework, neither Westfield nor Quividi need shoppers’ consent or knowledge to monitor and record them through facial detection by their Smartscreen billboard cameras. Yet throughout Europe, Quividi complies with strict privacy laws implemented by the General Data Protection Regulation enforced last May, which gave citizens the right to control their personal data and be informed about how their information is used.

Rather than burying information in the fine print of lengthy terms and conditions, the GDPR dictated consent must be clear, provided in an easily accessible form with clear and plain language. The regulation also stated it must be as easy to withdraw consent as it is to give it.

Chair of the Australian Privacy Foundation board, David Vaile says Australia’s privacy laws are an international disgrace. “[The GDPR] gives individuals strong rights. They can enforce them, they can go to court and it requires you have much greater specificity. All of those things are not true of Australian privacy law. There’s no right for an individual to enforce the privacy act at the federal level. It’s the opposite of a strong, capable option for remedying problems.”

There is now a bill attempting to regulate facial recognition in its second parliamentary reading. If passed, the bill would allow identity information to be exchanged between the commonwealth government, state and territory governments and “non-government entities” through the creation of a central hub called “the Capability” and the national driver licence facial recognition solution.

The bill is vague, with no date set for its final vote. While consumer consent is required in the bill, it fails to explain how it would be gathered or what exactly is meant by “non-government entities”.

Activists are concerned the Capability hub will contribute to potential data misuse and theft, with the bill unable to keep up with the ever-changing technology.

“People are having information recorded about them. They don’t get to see it, they don’t get to correct it if it’s inaccurate, and they don’t get to opt out of it. These companies get to accumulate very personal information about people and create a data set that could be rife for theft or misuse,” says David Paris, board member of Digital Rights Watch.

While Australia is worlds away from China when it comes to FRT, targeted advertising along with the infiltration of FRT into everyday life may eventually become the norm.

Says Paris: “If we don’t do something about it, the technology will go down the path of eliminating any place where people aren’t surveilled. Unless we have a massive shift in the way data accumulation is governed and limited and protected, we’re in for a really dystopian way of living where we don’t get to escape those sorts of things.”

The Guardian