PETs and AI: Privacy Washing and the Need for a PETs Evaluation Framework

Since the participants came from diverse backgrounds ranging from different topics in computer science to legal and regulatory work, the seminar began with several introductory talks and two panel discussions to bring everyone up to speed. Then, we brainstormed in small groups about all the aspects that could influence whether the deployment of a PET could be considered privacy washing. We subsequently grouped these aspects into four topics: Functionality and Framing, Infrastructure for PETs, Accountability, and Detection of Fake PETs. We split the group into four subgroups to discuss these aspects further and develop criteria by which to evaluate the deployment of a PET leading to a vast catalogue of factors that influence the efficacy of PET deployments. During the plenary meetings after group discussions, the rapporteurs from each group shared the progress made during the group discussions. Finally, we spent the remaining time to merge the results of the four subgroups into a draft for a position paper. The position paper describes what privacy washing is, who is involved in its deployment, who can be affected by it, and the considerations that help to detect privacy washing in deployed systems.

In an era of AI-driven applications, balancing data utility with user privacy is more important than ever. This talk provides a high level introduction to two key approaches addressing this challenge: federated learning and differential privacy. Federated learning enables collaborative model training without sharing raw data, while differential privacy provides strong guarantees against individual data leakage. This talk discusses the fundamental ideas behind these techniques, their real-world applications, and some challenges that remain.

It’s always tempting to cut things at what seem like logical joints so as to make thinking about the individual component easier. In a sense, that’s what we’ve done with privacy, focusing primarily on various forms of data processing. But the world is rarely as orthogonal as we model it to be. This brief talk situated privacy in a wider institutional framework and suggests that we may use an institutional grammar to evaluate the role and effectiveness of privacy decisions in a broader context.

Purpose limitation is one of the requirements under the GDPR. User data has to be processed for specific, explicit, and legitimate purposes. Purpose formulations specify and describe these purposes. In this talk, I presented four examples that, over time, convinced me that these formulations are a weak link in data protection: and thus become a tool for privacy washing.

Paul Comerford (Principal Technology Adviser at the ICO) discussed the role of the technology and innovation directorate at the ICO. The talk focused on the ICOs work on PETs across multiple domains and its upcoming guidance on anonymisation and pseudonymisation. He also discussed our recent work on AI and PETs.

Anonymisation is the main legal paradigm for sharing data while protecting people’s right to privacy. In spite of decades of research, robust anonymisation (“de-identifying”) of individual-level datasets remains an elusive goal. Numerous re-identification attacks have indeed shown how adversaries can use auxiliary information about individuals to single them out in supposedly anonymous datasets. One solution to the data sharing problem is aggregation, whereby data owners share with third parties the results of a computation across all records, while retaining control over the individual-level data. Aggregation solutions include summary statistics, interactive queries over the data, synthetic data, and machine learning. But aggregation does not, on its own, protect privacy, and evaluating the privacy of these solutions is far from trivial. This talk described the two main approaches for this: (1) designing and evaluating privacy attacks and (2) formal methods based on differential privacy, with their advantages and their challenges, together with my perspective on the field.

Companies and governments are increasingly relying on privacy-preserving techniques to collect and process sensitive data. In this talk, I will discuss our efforts to red team deployed systems and argue that red teaming is essential to protect privacy in practice. I will first shortly describe how traditional de-identification techniques fail in today’s world. I will then show how implementation choices and trade-offs have enabled attacks against real-world systems, from query-based systems to differential privacy mechanisms and synthetic data. I will conclude by discussing how this applies to modern AI systems.

Fairwashing refers to the risk that an unfair black-box model can be explained by a fairer model through post-hoc explanation manipulation. In this talk, I will first discuss how fairwashing attacks can transfer across black-box models, meaning that other black-box models can perform fairwashing without explicitly using their predictions. This generalization and transferability of fairwashing attacks imply that their detection will be difficult in practice. Finally, I will nonetheless review some possible avenues of research on how to limit the potential for fairwashing.

Recent applications of Privacy Enhancing Technologies (PETs) reveal a paradox. PETs aim to alleviate power asymmetries, but can actually entrench the infrastructural power of companies implementing them vis-à-vis other public and private organizations. We investigate whether and how this contradiction manifests with an empirical study of Amazon’s cloud connectivity service called Sidewalk. In 2021, Amazon remotely updated Echo and Ring devices in consumers’ homes, to transform them into Sidewalk “gateways”. Compatible Internet of Things (IoT) devices, called “endpoints”, can connect to an associated “Application Server” in Amazon Web Services (AWS) through these gateways. We find that Sidewalk is not just a connectivity service, but an extension of Amazon’s cloud infrastructure as a software production environment for IoT manufacturers. PETs play a prominent role in this pursuit: we observe a two-faceted PET paradox. First, suppressing some information flows allows Amazon to promise narrow privacy guarantees to owners of Echo and Ring devices when “flipping” them into gateways. Once flipped, these gateways constitute a crowdsourced connectivity infrastructure that covers 90% of the US population and expands their AWS offerings. We show how novel information flows, enabled by Sidewalk connectivity, raise greater surveillance and competition concerns. Second, Amazon governs the implementation of these PETs, requiring manufacturers to adjust their device hardware, operating system and software; cloud use; factory lines; and organizational processes. Together, these changes turn manufacturers’ endpoints into accessories of Amazon’s computational infrastructure; further entrenching Amazon’s infrastructural power. We discuss similarities and differences between previous strategic uses of PETs by Google and Apple to expand their infrastructural offerings to third parties. Accordingly, we argue that power analyses undergirding PET designs should go beyond analyzing information flows. We propose future steps for policy and tech research.

In this session we provided an overview on what multiparty computation (MPC) is and how we can think about its variants. We similarly discussed homomorphic encryption (HE). The goal with this session was to establish the breadth of the areas and provide attendees with a common language to think about the way privacy enhancing technologies (PETs) that employ MPC and HE can vary; allowing us to better evaluate the implications of these technologies for privacy and artificial intelligence. We concluded with an overview of some of what is currently possible, in terms of applications, that employ MPC and HE.

In this talk we revisited previous definitions of privacy engineering, showing that data or trust minimization do not necessarily minimize harms. We then argue that purpose minimization is the design goal that helps in this respect. Purpose-oriented thinking additionally has a benefit that it enables to identify fundamental purposes and harms that derive from the goal of the system and have to be assumed as a risk should the system be deployed. We then discussed some missing definitions that would allow to capture harms associated to function creep.

Data forms the backbone of artificial intelligence (AI). Privacy and data protection laws thus have strong bearing on AI systems. Shielded by the rhetoric of compliance with data protection and privacy regulations, privacy-preserving techniques have enabled the extraction of more and new forms of data. In this talk, I illustrate how the application of privacy-preserving techniques in the development of AI systems–from private set intersection as part of dataset curation to homomorphic encryption and federated learning as part of model computation–can further support surveillance infrastructure under the guise of regulatory permissibility. Finally, I propose technology and policy strategies to evaluate privacy-preserving techniques in light of the protections they actually confer. I conclude by highlighting the role that technologists can play in devising policies that combat surveillance AI technologies.

The panel explored how privacy-enhancing technologies (PETs) and regulatory tools are increasingly used for privacy washing – creating a surface-level appearance of compliance while sidestepping real accountability. Sandboxes and red teaming were called out as processes that can be used for legitimizing privacy-invasive systems without addressing underlying risks. Technologies like differential privacy, synthetic data generation and federated learning were highlighted as particularly vulnerable to misuse, especially when their implementation details are obscured or when their guarantees are undermined through practices like budget resetting or general misconfigurations. A key point raised was that evaluations should prioritize the actual impact on individuals and society, not just technical compliance or claimed adherence to norms.

The conversation also focused on the role and limits of transparency. While transparency was broadly supported as essential for accountability, it was acknowledged that legal barriers like trade secrets and competition law often prevent meaningful oversight. There was an agreement that transparency should go beyond abstract metrics and provide explanations that are intelligible to non-experts. At the same time, concerns were raised that transparency alone can also be co-opted as another form of privacy washing if not paired with enforcement and verification. The discussion underscored the need for enforceable standards, empowered regulators, and a shift away from over-optimizing technical frameworks toward addressing broader systemic and structural issues in privacy governance.

The panel discussed the gap between how privacy-enhancing technologies (PETs) are developed and how real users understand, need, or experience them. There was a recurring argument that users often lack the language, awareness, or mental models to demand privacy – much like people once lacked the concept of clean tap water – yet that doesn’t mean privacy isn’t essential. PETs should be designed to be invisible and default, not something users must consciously engage with. Communication breakdowns between researchers, usability experts, and end-users were identified as major barriers. There was also a push to broaden the definition of “users” to include software engineers and institutional actors, since engineers are often key decision-makers and operate much closer to the tools in practice. The lack of actionable usability research and the assumption of a clearly defined privacy “problem” were cited as weaknesses in the current ecosystem.

Beyond end-users, the conversation highlighted the role of high-risk populations, NGOs, policymakers, and businesses in the PETs landscape. Messaging must be tailored – migrants, for instance, face urgent harms that don’t always register as “privacy” risks. While PETs can support collective systems like digital commons, structural components and effective messaging are missing. Some companies adopt PETs reactively (e.g. post-GDPR), while others see them as a branding opportunity – but distinguishing meaningful implementations from superficial ones remains difficult. There’s also underused potential in inter-organizational PET deployments and in rethinking how to engage businesses without falling into technical “impossibility” traps. A key takeaway: users shouldn’t bear the burden of privacy, and communicating harm – especially to those at risk – must be better informed, more targeted, and more pragmatic.

The group focused on defining a structured approach to detect fake PETs – privacy-enhancing technologies that mislead through exaggerated claims, poor implementation, or misconfiguration. A central proposal was to create a standardized transparency tool, akin to model cards or data sheets, tentatively referred to as a “privacy card.” This would contain minimal but essential information to assess whether a system is making valid privacy claims. The discussion outlined four key failure categories in PETs: mismatch (between claims and actual threat mitigation), overestimation (inflated protection claims), wrong implementation, and wrong configuration. A foundational requirement is that any privacy claim must specify the threat model, the PET used, and the degree of mitigation. Vague claims without a clear adversarial context were identified as a red flag and sufficient grounds for labeling the system a fake PET.

Each failure mode was elaborated with practical evaluation steps. For mismatch and overestimation, the group emphasized decomposing systems into discrete threat-mitigation claims and validating them with formal or empirical evidence. Identifying implementation failures requires either open-source access or a reproducible protocol, with particular scrutiny on subtleties like side channels, flawed randomness, or deviations from trusted primitives. Configuration errors, such as excessive $ϵ$ values in differential privacy or undersized encryption keys, must be contextualized within both the system’s technical parameters and its deployment environment. The group stressed that privacy guarantees are only meaningful when technical claims are precise, verifiable, and aligned with real-world adversary models – making transparent, auditable documentation a practical necessity to prevent privacy washing.

The discussion highlighted that evaluating Privacy-Enhancing Technologies (PETs) cannot be isolated from the computational infrastructure they rely on, which often embodies extractive or privacy-compromising characteristics. A key challenge identified is the “stack problem”: PETs depend on underlying infrastructures that may themselves lack privacy protections, making truly independent PETs difficult to build and sustain without relying on PET-compatible infrastructure, governance, and funding. This dynamic concentrates power among well-resourced entities capable of controlling infrastructure, raising concerns about exclusionary effects on who can develop or maintain PETs based on existing economic and political incentives.

The group further emphasized that the production environment and infrastructure shape PET design, deployment, and sustainability, often introducing trust relationships and operational vulnerabilities. Coordination among infrastructure providers, deployers, and users creates new power relations, sometimes consolidating rather than distributing it. Ultimately, privacy-washing occurs when infrastructural dependencies and power asymmetries are overlooked, leading to overstated claims about PET’s protections while entrenching systemic privacy risks. Effective evaluation frameworks must therefore assess PETs in a full-stack context, including the socio-technical and governance layers that support or constrain them.

The group explored how privacy-enhancing technologies (PETs) can be co-opted to obscure harm rather than mitigate it, calling for a taxonomy of privacy-washing methods to support clearer evaluation, from the system’s purpose, to its implementation, communication, and resulting consequences. Three key forms of privacy washing were identified: first, when PETs are layered over systems whose underlying purpose is harmful or objectionable, PETs cannot fix this inherent harm; second, when the system’s implementation is harmful, even if its purpose is legitimate, and PETs are used to mask this; and third, when misleading communication about PETs causes harm – such as falsely marketing systems as end-to-end encrypted. In all three cases, PETs risk being used as decorative compliance tools, deflecting attention from structural issues or enabling more sophisticated forms of manipulation, profiling, or opacity.

The group proposed analyzing PETs through the full lifecycle of a system: from purpose, to technical implementation, to communication, and finally to consequences. A system-wide framing was emphasized – assessing not just whether a PET works, but whether it genuinely addresses the privacy risks tied to the system’s function and context. PETs that enable or justify harmful practices and information flows were flagged as particularly concerning. The group cautioned against starting privacy assessments too late in the process (e.g., at the DPIA stage), noting that foundational design choices may already predetermine harm. A meaningful framework, they argued, must account for both intentional and structural misuses of PETs, especially as these tools are increasingly used to manage – not eliminate – power imbalances and information asymmetries.

The group discussed how PETs can be misused to block accountability, particularly by preventing audits, obscuring system behavior, or undermining data access rights. Rather than supporting transparency, some PETs are designed – or framed – as privacy solutions while enabling organizations to evade scrutiny. For example, this includes using PETs to justify blocking data subject’s access rights under the GDPR, offloading privacy responsibilities to third-party service providers, or deploying unverifiable systems that require trust without oversight. The group emphasized the importance of designing PETs with auditability and verifiability in mind to counter these failures and ensure they contribute to, rather than hinder, accountability.

In the context of AI, similar dynamics emerge. Techniques like federated learning – while often cited as privacy-preserving – can be used to obscure data processing practices and resist evaluation due to their complexity. Participants noted that organizations may deploy PETs to legitimize questionable practices or bypass legal requirements, especially in the private sector where business incentives dominate. Overall, the discussion called for evaluation frameworks that address how PETs are used in practice – focusing not just on their technical properties, but also on their role in enabling or obstructing rights, oversight, accountability, and public trust.