Modern life is one big photo shoot. The glassy eyes of closed-circuit TV cameras watch over streets and stores, while smartphone owners continually surveil themselves and others. Tech companies like Google and Amazon have convinced people to invite ever-watching lenses into their homes via smart speakers and internet-connected security cameras.
Now a new breed of chips tuned for artificial intelligence is arriving to help cameras around stores, sidewalks, and homes make sense of what they see. Even relatively cheap devices will be able to know your name, what you’re holding, or that you’ve been loitering for exactly 17.5 minutes. It’s the latest development in the tech industry’s campaign to build out the internet of things, a slogan for linking everyday devices to the internet so they are interactive and gather data.
Smart cameras with built-in AI capabilities can offer new conveniences, like a phone notification that a child just arrived home safely, or that the dog walker really did walk the dog. They will also bring new risks to privacy in public and private spaces. Companies are exploring how smart cameras can be used to gather marketing data, or assist law enforcement, for example.
A slim wand with twin lenses like bulbous eyes unveiled last week by mobile-chip company Qualcomm heralds this new era. The company built the six-inch-long, half-inch-thick gadget to demonstrate how a new chip can help security cameras and other devices better analyze images. Products based on the chip are expected to hit the market before the end of the year. Qualcomm already ships 1 million chips a day for use in internet of things devices, including products from Google’s Nest.
In demos, Qualcomm’s camera performed tricks like recognizing individuals from their faces, and detecting and tracking a package held in a person’s hands. Thanks to the new chip, the device could do this independently, without tapping a server in the cloud for analysis. Qualcomm and others working on similar hardware argue this will help AI-enhanced cameras sprout in more places, because they won’t need to rely on a home’s or business’s bandwidth, and can react more quickly.
Democratizing automated surveillance could also bring downsides.
One Qualcomm partner, Pilot.ai, offers software that can log how long people of different sexes and ages linger in various parts of a store. The startup says it won’t let the software identify individual shoppers, but others are willing to offer that feature. Chinese startup Horizon Robotics has developed its own smart camera chip that it says can be used to detect when a particular shopper walks in the door.
Marshini Chetty, a research scholar at Princeton, says such uses could breach people’s expectations around cameras in public spaces. “Today people understand that this video is not used unless something goes wrong,” she says. “They’re not expecting their data to be used for advertising or to track people as they go about their day.”
Horizon launched a camera based on its technology at a security expo in Las Vegas last week. The company’s chip, like Qualcomm’s, is customized to run the neural network software that powers recent improvements in image recognition. Horizon says that one of its cameras can be loaded with up to 50,000 different faces, and recognize any one almost instantly, with 99.7 percent accuracy.
Founder and CEO Yu Kai says he is working with government agencies, shopping malls, and stores in China that want to find find the faces of suspects, or VIP shoppers when they enter a store. The company raised $100 million last fall from Intel and others, and has plans to open a US office.
Further help for companies interested in ideas like those comes from chip designer ARM. In February the company, which licenses designs to Qualcomm, Samsung, Apple, and others, announced a new design to help devices like security cameras and drones detect people and objects.
Security applications are expected to be a big market for smart cameras. Chinese facial recognition company SenseTime, reckoned by Bloomberg to be the world’s most valuable AI startup at more than $3 billion, has adapted its software to run on Qualcomm’s new chips. Chinese government agencies have been enthusiastic adopters of facial recognition. Earlier this month, police plucked a suspect from the crowd at a pop concert in southeastern China after a facial-recognition system identified him among the 60,000 fans.
Smart cameras can do more than find suspects. Qualcomm executive Seshu Madhavapeddy says customers are talking with the company about using its vision chips in cleaning robots, and smart speakers, for example. Three models in Amazon’s Echo line of home devices, the Show, the Look, and the Spot, already have cameras. Chips like Qualcomm’s could give those devices new capabilities, for example customizing what they say or do to people nearby. A Google patent from 2016 described how cameras in a home could use visual clues like books and musical instruments to learn about a person’s interests, and suggest content they might like. Google declined to comment.
Companies working on smart cameras say they are developing and designing the technology responsibly. Qualcomm says its chip is engineered to secure data stored or transmitted by a device. Kai of Horizon notes that by processing images locally, his cameras offer privacy benefits over more conventional models that send everything to the cloud. Google promotes that as a key feature of its Clips camera launched last year, which contains an image-processing chip designed by the search company.
Many functions envisioned for smart cameras depend on transmitting information they’ve detected, such as faces they’ve identified. Chetty of Princeton worries that smart-home enthusiasts may give up data on themselves, their children, or even passers-by that they didn’t intend to share. Interviews with people who use smart speakers and other smart-home devices show they tend to blindly trust their gadgets won’t betray them, and pay little mind to privacy settings or disclosures, she says.
As Facebook’s recent scandal about misuse of data from a seemingly innocuous quiz app shows, once a person sets their data free, it can be used in unforeseen ways. “People are really bad at seeing connections between things in the moment,” says Chetty. “We’re short term thinkers.”