THE SUPERMARKET AI SURVEILLANCE

Cases

Background:

Nina K., a 34-year-old journalist living near a local supermarket, visited the store one day dressed casually and somewhat unkempt. While shopping, she noticed an employee, James, observing her closely. Feeling uncomfortable, she asked him if something was wrong, but James offered no specific explanation. Despite attempting to maintain some distance, he continued to monitor her throughout her visit.

After a few similar encounters, Nina discussed the experience with her friends, who reported comparable observations. One friend, a former employee, explained that the supermarket had implemented an AI system at the beginning of the year, which appeared to influence staff behavior. Motivated by her profession, Nina investigated this technology and published an article in her newspaper. The piece quickly generated public concern, as the AI’s notifications were perceived as discriminatory, effectively labeling certain customers as potentially criminal based on their appearance.

The AI System and Its Purpose:

The supermarket chain had introduced AI to analyze store surveillance footage for operational efficiency and security. The system was designed to:

– Anticipate and respond to security threats, such as theft or property damage.
-Support inventory management and cleaning operations.
-Improve overall customer experience by reducing waiting times and optimizing store operations.

Louis, the branch’s security executive, emphasized the system’s effectiveness. He encouraged employees to follow AI notifications promptly, noting a significant reduction in incidents since the software’s deployment. At the end of each month, he reviewed the notifications and employee responses, issuing warnings if theft or damage occurred despite alerts.

Technical Functioning:

The AI relies on branch-specific customer data, collected from video analyses, including sex, age, race, socio-economic profile, and behavioral patterns. These records are retained for six hours before being automatically deleted. To strengthen predictive accuracy, the system is integrated with the supermarket’s historical security database, mapping customer characteristics against profiles of previous offenders. Notifications are then sent to employees if a customer matches a pattern suggesting potential criminal behavior.

Importantly, customers were not informed that their data were being analyzed or used in this way. While the AI aims to enhance safety and operational efficiency, the ethical implications of monitoring and profiling without consent are significant.

The Conflict:

The board of directors now faces a pivotal decision regarding the software’s continued use. While it has demonstrable benefits in reducing theft and improving store efficiency, it also poses substantial ethical and legal risks, affecting customers, employees, and the supermarket’s public reputation.

Discussion Points for Consideration

  1. How can businesses balance operational efficiency with customer privacy and ethical obligations? 
  2. Should predictive AI systems be allowed to profile individuals based on historical data and personal characteristics? 
  3. What measures can be implemented to increase transparency and accountability in AI-driven surveillance? 
  4. How might the supermarket address employee compliance while minimizing discriminatory outcomes?

DISCUSSION GUIDE

Controversy and Ethical Concerns

Nina’s reporting brought to light several critical issues:

  • Privacy: Customers were unaware that their personal characteristics were being recorded and analyzed. 
  • Discrimination: The system appeared to make judgments based on appearance and socio-economic indicators, raising concerns about bias. 
  • Accountability: Employees were penalized for failing to respond to AI notifications, potentially creating undue pressure.

 

Possible visiting points for developing the case: 

  1. Passive/Active consent 
  2. Public space/Private space
  3. Responsibility
  4. Efficiency/Safety
  5. Surveillance