Future of Human-Centered Privacy

Software products and services shape societies and peoples’ lives. These software products don’t magically appear – they are created by developers and then used by many more users, potentially by orders of magnitude more. Thus, the design and coding choices of developers determine the quality of software products – including the quality of privacy⁵⁵5We understand “privacy” as mediated via data protection. implementations. This differentiates developers from consumers who usually only make privacy choices for themselves or a small number of others (see Chapter 4.5 on Collective Privacy). Unfortunately, many developers are ill-equipped to shoulder this broad impact: they lack both expertise and advice on privacy, and formal training in software engineering. In addition, being a developer can mean many different things, depending on the roles and the contexts: First, software development is never just about writing code. It encompasses the entire Software Development Lifecycle, from planning and design to deployment and maintenance, as well as all the internal processes, guidelines, and communication that enable and sustain that lifecycle. The resulting jobs to be done include writing code, testing, product and project management, requirements engineering and software architecting – each one associated with specific privacy questions and decisions [3]. In an ideal world, these jobs are distributed among experts who build specialized skills. In reality, governance structures differ and some jobs are combined, skipped, or executed by a “one-person band” [29].

Second, different contexts like the development of a new website or app, software maintenance, ad-hoc scripting to automate admin tasks, or database development all come with their own support needs. In each one of these contexts, developers routinely switch between producing code and consuming code (e.g., embedding third-party services or libraries, reusing code snippets).

This section defines the scope of relevant EU laws applicable to this report and offers conceptual legal clarifications, such as responsibility, accountability, data controller, data processor, liability.

1. 1.

   Responsibility: Who should be responsible for privacy compliance according to their role?

   • $\mathrm{■}$

     How to identify different user groups we consider “developers” in a specific context?

   • $\mathrm{■}$

     Should third-party providers be responsible since they both control the tools and data collection, even if they are not the ones controlling the primary application and therefore may argue that they are not responsible?

   • $\mathrm{■}$

     What organizational or cultural factors influence how developers understand and engage with privacy requirements – or don’t?

2. 2.

   Knowledge: What knowledge gaps and misconceptions do developers have about privacy, and how do these shape their practices?

   • $\mathrm{■}$

     What kinds of legal information do developers need but often lack?

   • $\mathrm{■}$

     What are the common misunderstandings or contradictions developers hold about privacy and user expectations?

   • $\mathrm{■}$

     How do developers’ assumptions about user attitudes influence their privacy-related decisions?

3. 3.

   Delivery of Guidance: What forms of guidance and support mechanisms are most effective in helping developers implement privacy practices?

   • $\mathrm{■}$

     What forms of guidance (lightweight or embedded) are most useful in practice?

   • $\mathrm{■}$

     What prevents developers from consulting privacy experts or using available resources?

   • $\mathrm{■}$

     Is it necessary to build closer relationships between developers and privacy experts?

   • $\mathrm{■}$

     Can AI tools meaningfully support developers in privacy tasks, and how reliable are they?

   • $\mathrm{■}$

     What form of guidance from third-party services would be useful to help developers understand the privacy implications in their decision-making?

4. 4.

   Evaluation: How can we evaluate the effectiveness of privacy guidance and support for developers?

   • $\mathrm{■}$

     What criteria or metrics should be used to measure success (e.g., compliance, usability, adoption)?

   • $\mathrm{■}$

     What kinds of experimental or observational studies can assess whether developers actually use and benefit from guidance?

   • $\mathrm{■}$

     How can we simulate or test real-world developer behavior in privacy-critical scenarios?

5. 5.

   Meta-analysis: In an age where advice seeking is happening in private spaces (chatbots), what are the appropriate ways to study developer behavior and traces they left in the public space (e.g., artifacts/objects – benchmark on correct or wrong)?

   • $\mathrm{■}$

     How to research what chatbot interaction looks like?

   • $\mathrm{■}$

     Can we expect developers to donate their data (e.g., GenAI-prompting history)? We need to consider research ethics in this context.

   • $\mathrm{■}$

     How about intellectual property issues and authorship?

Solutions range from foundational education to increasingly in-the-moment support.

1. 1.

   Build capacity and educate programmers with a privacy-aware mindset:

   • $\mathrm{■}$

     The next generation of computer scientists needs holistic education and effective training, in contrast to generic annual privacy training. These efforts should hone a certain mindset rather than focus on teaching yet another modular skill.

   • $\mathrm{■}$

     This education and training faces uncertainty around the value of skill sets amid the increasing use of generative AI. A large-scale deskilling regarding critical thinking [17] may also endanger democratic agency. The risk of deskilling should be kept in mind, researched, and potentially countered.

2. 2.

   Equip developers with safe software engineering practices:

   • $\mathrm{■}$

     Support mechanisms should deliver clarity of privacy goals through requirements engineering and its translation into software architecture, considering the mental models that developers have of end-users [7].

3. 3.

   Provide contextual support:

   • $\mathrm{■}$

     IDEs should offer easily accessible contextual programming aids for implementing privacy – potentially leveraging AI-based code-generation and AI-based advice (akin to secure development lifecycle and existing tools like Copilot). Barriers to use need to be lowered, and quality assurance of the advice given to developers strengthened – regardless of how and where in the messy software development process people seek advice.

   • $\mathrm{■}$

     Part of the solution might be a repository of privacy-information-seeking behavior of developers that can be studied by researchers (especially important in a present and future in which developers seek advice from chatbots, i.e., not in the open, and every developer potentially getting personalized different kinds of advice).

   • $\mathrm{■}$

     Good privacy must be the easy choice: Dark patterns must not be perpetuated and there should be privacy-friendly options when consuming and updating third-party code [32], such as coarse-grained defaults and code snippets for location APIs [15]. Limited monetisation options can be a struggle towards better privacy [6].

   • $\mathrm{■}$

     Higher transparency regarding data practices (collection, processing, storage etc.) in the market of software products and for third-party libraries is needed, so that developers are empowered to choose wisely which libraries or services they link into their software. Appropriate metrics can support end-to-end accountability aided by transparency along the software supply chain. Similar consumer-oriented portals already exist, such as Mozilla’s “Privacy not included” (which is a “naming and shaming portal for IoT devices”).

Directions for for academia, policy-makers, industry should proceed along the following questions.

1. 1.

   What immediate research questions need to be answered?

   • $\mathrm{■}$

     What motivates developers to care for privacy in different contexts?

   • $\mathrm{■}$

     How can we integrate privacy support in developers’s day-to-day work?

   • $\mathrm{■}$

     What kind of guidance is effective for different kinds of developers?

   • $\mathrm{■}$

     What kinds of privacy-related development tasks/requirements can be supported with existing off-the-shelf GenAI tools or coding aids that are used today? Where do they make mistakes?

   • $\mathrm{■}$

     What could motivate third-party service providers to provide services with privacy-preserving defaults?

2. 2.

   What are challenges and questions to be answered in the next three years?

   • $\mathrm{■}$

     How can GenAI tools be adapted/improved to provide helpful guidance for developers?

   • $\mathrm{■}$

     What role does the current and foreseeable market power of a small number of commercial Big GenAI tools play in the answer to the previous question?

   • $\mathrm{■}$

     How to motivate developers in public bodies (administration/governmental institutions) to care for privacy?

   • $\mathrm{■}$

     How to make privacy a priority task for developers?

   • $\mathrm{■}$

     How to motivate third-party service providers to provide compliant services?

3. 3.

   What are challenges and questions to be answered within the next decade?

   • $\mathrm{■}$

     What are the effects of the quickly evolving GenAI tools, which changes how developers work and who develops software?

4. 4.

   What skills and collaborations are necessary to address them?

   • $\mathrm{■}$

     Collaboration between scholars from computer science and law, psychology, ethics, economics

   • $\mathrm{■}$

     Collaboration with practitioners (controllers, processors) and data protection authorities

   • $\mathrm{■}$

     Collaboration with legal experts that consult companies on data protection requirements

   • $\mathrm{■}$

     Collaboration with industry bodies (like VDI/VDE or IHK/Handwerkskammer in DE) that advise organizations

5. 5.

   What tools, infrastructure, funding, or incentives are needed?

   • $\mathrm{■}$

     Tools that assist developers in making more privacy-friendly decisions

   • $\mathrm{■}$

     Actual testbeds to try some new interventions. (technical support and funding for support staff)

6. 6.

   What roadblocks may slow down or discourage research progress?

   • $\mathrm{■}$

     Perceived poor enforcement of GDPR violations may disincentivize developers to care for privacy.

   • $\mathrm{■}$

     Changes to GDPR that increase or reduce requirements for controllers.

Methodologically, this may work best by working with case studies. Candidate areas for developers are consent management products and tag managers.

The current human-AI interaction paradigm is making computational machines more pervasively trained and deployed on much more data (than previous generations of AI). This raises both the opportunity to use AI tools to enhance societal outcomes, including privacy, and threats from the exacerbation and introduction of risks and harms. We posit that going forward, we must address the question “how does human-centered privacy change in a world where AI-based automation (e.g., AI-agents) is incorporated into the many facets of digital life, for better or worse?”

Risks and harms may come as a result of decisions made by individuals (who use an AI-employing system), by others (collateral privacy through inferences made about you even as a non-user), and by society (to prioritize an AI capability over a threat to some individuals). Therefore, to understand these different sources of harm, we first need to surmise if existing privacy and behavioral decision-making theories, and decision-making consequences, explain and predict such decisions as well as potential privacy issues that consequently emerge. If not, we need to update these theories and frameworks to better explain and predict potential privacy issues that can emerge from individual human-AI interaction and societal decisions about AI deployment. From there we can develop a better understanding of decision making, risks and threats specific to this domain, as well as how AI can be used to benefit society helping rather than hindering privacy in our digital spaces.

Finally, we emphasize that while AI poses new challenges in the space of human-centered privacy, it also offers positive potential for providing usable end-user guidance or even for automated or semi-automated privacy decisions.

Different fields (HCI, AI, SWE, law, and more) have each developed disparate definitions of safety as well as classification taxonomies and evaluation frameworks for privacy risks and harms. Some of these build upon prior theories of privacy, such as using Solove’s framework [18], while others introduce their own structure. However, while these provide organization to the domain, they each use different conceptual terminology, different categorisations, and largely remain at an overview level, aiming to capture the overall field. This creates challenges and limitations in terms of applicability for those creating, deploying, or overseeing these systems as: (i) By trying to cover the breadth of the space, nuance cannot be captured, (ii) these frameworks are often rooted in academic/theoretical/philosophical perspectives such that those wanting to apply these insights need to translate and transfer them to match their requirements. Unfortunately, these can often not transfer well or easily to such industry processes and priorities (e.g. in regards to when or how harms must be mitigated).

We know from usability foundations that adoption rises when “things” are designed with end-user requirements and use cases in mind from the start, not added at the end. Regulators and governments (for instance EU enacted, Canada tabled, UK’s AI Bill tabled) have similarly tried to capture risk as either:

• $\mathrm{■}$

  All AI (which has contentious definitional bounds)

• $\mathrm{■}$

  AI for specific tasks, with some using domain-specific or industry-specific existing guidelines or laws as baselines and others trying to create new ones that specifically target “AI in X”

While we can articulate notions of privacy risk conceptually in this space, which concrete harms will follow from them is less clear, difficult to measure with existing tooling and models, which correspondingly leads to contention as to next steps for all stakeholders and relevant parties.

Following the human-centric approach, to ensure a comprehensive understanding of both the positive and negative effects of AI on people, one must first gain a thorough understanding of how individuals and groups make decisions regarding their privacy and security when using technologies that incorporate AI. Traditional psychological models like the Theory of Reasoned Action (TRA) and Theory of Planned Behavior (TPB) fall short in explaining how people make privacy decisions under uncertainty or cognitive overload [19, 20, 21, 22]. Dual-process theories – System 1 (S1) and System 2 (S2) – provide a more comprehensive framework [22, 21]. S1 is fast, intuitive, and driven by heuristics (e.g., affect heuristic), while S2 is slow, analytical, and effortful. Most everyday decisions rely on S1, which is sensitive to various peripheral cues (e.g., visuals), heuristics and biases, and other factors that might affect cognitive processing of information (e.g., time pressure). Privacy decision-making models build on psychological theories, such as TPB, TRA, or dual process theories, at times combining them into more holistic but theoretical models. One such model, APCO (Antecedents–Privacy Concerns–Outcomes), centers privacy concerns while acknowledging background factors like personality, culture, trust, and skills [23]. This conceptual macro-model incorporates System 1 thinking, recognizing the role of fast, intuitive judgments in privacy choices. Following the model, privacy concerns seem to play a central role to decision-making, yet, they might be hard to assess and various scales to measure them were developed (e.g., Internet Users’s Information Privacy Concerns (IUIPC) [24], Mobile Users Privacy Concerns [25], and even Social Robots Privacy Concerns [26]). AI development at a fast pace resulted in the first attempts to measure concerns in this context with the Privacy Concerns related to AI Misuse (PC-AIM) scale [27], which builds on existing models like the IUIPC but adds dimensions like data permanence, profiling, reduced judgment, and algorithmic bias. However, more studies are needed to validate its accuracy and applicability to various contexts of interactions with AI-based technologies. Considering the complexity of decision-making processes, other measurement instruments/methods for latent factors affecting decisions and behavior in the context of privacy and AI might be needed.

In AI interactions, anthropomorphization can influence privacy decisions, sometimes causing discomfort (the “uncanny valley” effect). However, this can be offset by empathetic responses and immediate implicit feedback (e.g., through multi-turn interactions), which increase user comfort and data disclosure. These effects, though impactful, remain under explored, raising concerns about potential privacy and security risks. AI agents often trigger affective heuristics through emotional cues like personalization and human-like behavior, which can override rational risk assessment. As a result, users may underestimate privacy risks and overshare data, especially under cognitive load or time constraints.

Studies show users are often more willing to share sensitive data with AI than with humans – particularly if AI is perceived as non-judgmental [30, 31]. However, if AI seems too powerful or invasive, privacy concerns intensify. Cultural factors also shape responses: U.S. users tend to trust AI more unless data sensitivity is high, whereas Chinese users are more [32]. It is possible that older adults may accept AI, similarly to how they accept social robots [33], if its utility outweighs privacy risks, but still demand control and transparency; however, research is needed to confirm such assumptions.

Regarding the beneficial use of AI in the context of privacy, i.e., using AI to support stakeholders with privacy-related decision-making, a rapid review of the existing literature sees three main research areas emerging:

(A)

Using AI to assess privacy issues in to-be-shared social media content (e.g., [2, 3, 4, 5]): The key challenges are the lack of real-world deployments, the issue of cold-start, and the social complexity inherent in such sharing decisions. Multi-party privacy conflicts are also difficult to solve, with or without AI.

(B)

Using GenAI/LLMs for privacy policy understanding (e.g., [34, 7] [Tongue and Caragea 2020]): The key challenges here are unreliability of LLMs, the legal validity of consenting to AI-summarized policies, and, of course, the inherent complexity of policies written not for humans but lawyers

(C)

AI-based privacy management (e.g., decision making) (e.g., [10, 6][Bhattacharya, 2024]).

Area (A) has traditionally seen much work due to the importance of social media. Since 2023, area (B) is growing quickly. Area (C) is just emerging.

We identified four key research questions in this domain. This report focuses on the first three. However, we include the fourth as we also need to communicate amongst stakeholders all the aspects of the other three research questions.

• $\mathrm{■}$

  RQ1: How can we use AI to enhance usable privacy?

• $\mathrm{■}$

  RQ2: How does AI affect (positively/negatively) human privacy decision-making and behavior?

• $\mathrm{■}$

  RQ3: How does AI pose novel or exacerbate existing risks to people’s privacy?

• $\mathrm{■}$

  RQ4: How can developers, companies, researchers, or governments (etc.) communicate privacy (cf RQ1-3), both reactively and proactively, (e.g., risks, potential manipulation) in the context of ubiquitous AI-based systems?

Guided by our research questions and the literature, we synthesize the path forward as Table 1.

Interdisciplinary collaboration: the role of social and psychological factors necessitates collaborations with at minimum Psychology and Sociology. Moreover, cooperation with legal experts is needed for eliciting and enforcing legal requirements.

We are bombarded with cookie banners which often violate informed consent requirements [1], privacy policies are not written to be understood [2], and real time bidding – the economic basis of the contemporary web – is “structurally difficult to reconcile with European data protection law” [3]. While it is clear that the transparency and consent model for privacy decisions is broken [4], its adoption as the cornerstone of consumer and data protection regulation means that it is here to stay. This calls for the need to envision and design solutions that address the continuing problems of transparency and consent. At the same time, the continued use and development of artificial intelligence has created a data-oriented world where it seems inevitable that data flows are destined to grow. This development can be beneficial for individuals and societies (e.g., for data-driven medical research), providing new opportunities. At the same time, emerging technologies and novel interaction modalities also pose new threats which the research community must anticipate.

In this context, it is essential that we work together to improve the mechanisms that we have to control how our data is used and how we are asked for consent. In our discussions, we strive to create effective, usable, lawful, and accessible tools for consent, control, and communication around the use of personal data.

We make a distinction between the idea of consent as an explicit decision point allowing for positive and negative choices, and control as the broader category of mechanisms that people have to influence how their data is used. Notably, whilst consent is always something that the data subject negotiates with a data controller, control as used here encompasses resistance, activism, and any other actions taken by an individual, or some other entity on their behalf, that change how their data is used.

Despite decades of research and regulatory attention, privacy notices and consent forms continue to fall short of their intended purposes: attracting user attention, informing individuals about data practices, enabling comprehension, and ensuring meaningful control over personal data. Although during the years privacy policies of service providers have been improved, they are still often compliance-driven rather than user-centred. For example, privacy policy forms, which are one form of privacy notices, have to fulfil requirements from different stakeholders [5, 6]. While users expect to receive clear, simple information about data practices and privacy controls, service providers employ privacy policies to demonstrate compliance with legal requirements.

These shortcomings are further compounded by usability issues such as cluttered layouts, inconsistent placement of privacy controls, or technical barriers that make opting out difficult [7]. In addition, consent mechanisms on websites and digital platforms often rely on design choices nudging users toward acceptance instead of fostering genuine understanding or control. Empirical studies show that interface manipulations such as default buttons, pre-selected options, or the removal and obscuring of rejection choices can raise acceptance [8]. A (perceived) lack of choice and decoupled notices are other reasons why users do not pay attention to privacy notices [5, 6].

This results in notice fatigue, habituation, post-decision regret, and superficial consent, where users agree without fully understanding the implications [9, 10]. Consequently, although many implementations formally satisfy regulatory requirements, they fail to meet the spirit of informed and freely given consent, thereby undermining the user’s ability to exercise meaningful control over personal data.

To partly address the usability issues, researchers have explored ways to improve the usability of privacy notices and consent mechanisms. Proposed solutions include privacy nutrition labels [11], layered and short-form summaries [12, 22], personalised notices [13, 14], icons [15, 16], and even comic-based interfaces [17], among other design patterns [24]. These approaches attempt to make complex data practices more transparent and digestible, but each comes with trade-offs: condensed formats risk oversimplification, while visual representations, such as the meaning behind icons, often require additional user learning. Beyond presentation, other efforts focus on interactive consent, using mechanisms like drag-and-drop, swiping, or checkboxes, to actively engage users with the content, improving both attention and retention [18].

Emerging technologies intensify the challenges in these contexts. Artificial intelligence systems often operate opaquely, making it difficult to communicate how data is processed or how decisions are made, although they have the potential for improving usable personalised privacy and decision-making. In AR/VR and metaverse environments, the traditional notice and choice paradigm is even less effective and sometimes impossible. Immersive systems collect highly sensitive biometric and behavioural data, much of which is invisible to the user. The immersive nature of these platforms also makes interruptions for lengthy consent impractical, raising the risk that consent becomes implicit or illusory.

The working group considered a range of different use cases in order to map out the problem space. Each case study was chosen because it surfaced relevant issues and solutions related to consent, control, and/or communication of data practices.

• $\mathrm{■}$

  Medical data donation for research purposes. This included discussion around the values and reasons behind the donation of medical data, the potential positive and negative consequences of doing so, and how this could be effectively, lawfully, and ethically communicated.

• $\mathrm{■}$

  Consent and control for virtual/mixed reality. We discussed if and when we should design consent decisions to avoid pulling users out of immersive experiences, as well as alternatives such as making choices before/after immersive moments or through other interaction modalities.

• $\mathrm{■}$

  Conversational Agents. We talked about how conversation, particularly when spoken, is a relatively low bandwidth means of interaction, but does present opportunities for more engaging conversations about privacy. It foregrounds a question from multi-layer privacy policies about what should be in the first layer.

• $\mathrm{■}$

  Data collection by smart cars. The discussion covered how people don’t expect this data collection, leading to cases where consent isn’t freely given. There are also tangible consequences that arise from data collected by cars, particularly related to car insurance.

When considering these case studies, two main areas of discussion emerged. The first of these was the lack of communication around consequences for data sharing, making it difficult to answer the question of why people should care about privacy. The second centred on the development of AI agents that (amongst other things) would be tasked with making privacy decisions on our behalf.

This area of discussion was sparked by an observation in the main session that, as privacy experts, we often struggle to effectively communicate the importance of digital privacy. An obvious response would be to convey the tangible consequences (good and bad) and risks of making a privacy decision, but this is something that we don’t currently have a good overview of. To this end, the group considered the specification of a consequences-based and cross-platform privacy decision manager that would help people to make privacy decisions that are aligned with their individual expectations and values. We preliminary call this system 3CM: Consent, Control and Communication Manager and imagine it to be similar to existing password managers. This idea is based on discussions of the group “Consequence-based Privacy Decisions” in Dagstuhl Seminar “My Life, Shared” from 2013⁸⁸8https://www.dagstuhl.de/13312 [19].

A core component of 3CM would be a database of events and outcomes of sharing data, both positive (e.g., personalisation) and negative (e.g., data breaches). Given appropriate context, an LLM (Large Language Model) would be able to provide explanations of these consequences of sharing or not sharing before a decision is made. These decisions would be stored by the manager for future use and could be applied automatically where directed. This would add functionalities to existing tools such as Consent-O-Matic⁹⁹9https://consentomatic.au.dk. The privacy decision manager would learn from the decisions that the user makes, taking into account contextual elements that might impact sharing decisions.

Such a concept is not without challenges. In 2013, one of the main challenges was how the system would present the consequences and learn from past decisions. These challenges remain, but seem to be better manageable with the proliferation of LLMs. There are also implementation difficulties around curating data sources and creating experiences that function across devices. A consequences-based decision manager would not address underlying problems with the notice and consent model, but it would present an opportunity to push back and change the incentives for users (and thus organisations) around the choices they make in relation to their data. Ultimately, it would also need to be easier to use than not use, increasing user acceptance. This kind of system also brings with it the question of when or if to ask users if they want to reconsider their decisions, potentially leveraging so-called ‘teachable moments’ where people have the time and energy to make decisions outside of the task that led to the initial prompt to share data.

Artificial agents are increasingly embedded in our day-to-day activities, whether professional or personal tasks. Especially since the widespread availability of conversational interfaces that enable anyone to interact with Large Language Models (LLMs), such as GPT, it is evident that humans find it useful and beneficial to resort to those instruments, even if their usage is often accompanied by a lack of awareness of the entailed risks. As in other contexts, in this kind of interactions, privacy and security are not the primary task of users. Thus, certain old-standing issues are being re-proposed with emerging technologies, in addition to the new challenges these technologies pose.

The group also discussed possible solutions offered by artificial agents to the so-called “privacy self-management dilemma”. A trained artificial agent could take privacy decisions on behalf of the user while it performs other tasks (e.g., refuse profiling cookies while booking a flight). But this scenario raises additional questions, such as whether and to what extent an agent can make decisions on behalf of the user. Even if this could sometimes be determined contextually (e.g., not in high-risk scenarios), still, to make informed decisions, the agent should understand the contextual norms of using personal information. In this regard, it would be similar to having a human assistant. The participants discussed whether it is possible that the agent learns through a process of personalization or if it should be based on the profiles of others, as is already the case for some privacy assistants. Another open question concerns whether a threshold for the acceptable amount of errors that can be made should be set. Such an agent calls into question the very nature of “informed” decisions, a concept which should probably be reformulated. What is interesting is that taking privacy-friendly decisions not only concerns the type and amount of data that is shared or withheld, but also the capacity to make inferences based on such data.

In addition, the group discussed the types of requirements that should be formalized for the development of such an agent. These should concern the user interface (e.g., in terms of timing and modality of communication) and should envision the situations where a decision needs to be deferred to a human. This setting also emphasizes the central role that trust would play: requirements should concern the establishment and maintenance of user trust in the agent and cover cases where a breakdown of trust may occur. As a conclusion, the participants acknowledged the need to also include experts with AI and legal backgrounds to address all these questions meaningfully.

Bringing together the two focuses, the group was keen to explore the creation of a privacy decision agent that collated and explained the potential consequences of different privacy decisions.

Based on the discussions during the week, the group compiled a list of key research questions for the research community. These are sorted by topic area and timeline:

Privacy has long been conceptualized at the individual level (e.g., [2, 36]). However, with the rise of social and mobile media, as well as the widespread data collection of digital trace data at scale, it is critical to move beyond individual considerations to focus on the privacy needs and risks at the group, organizational, and societal level. For example, in groups or dyads, individuals need to account for and navigate varying privacy preferences and norms; significant research has explored this as it relates to social media (e.g., [15]). Privacy in organizations is likely influenced by the more formalized and hierarchical structures that restrict information flows and create power imbalances between employers and employees. Likewise, technology developers yield significant power in determining the extent to which privacy can–or cannot–be enacted. Finally, privacy evaluations at the societal level run into challenges due to different regulatory landscapes as well as different values and norms.

Beyond these, privacy discussions must now account for the interdependent nature of data creation, use, and ownership. However, accounting for the various entanglements of people and data in a networked world is incredibly challenging and inherently interdisciplinary. Researchers have begun addressing these complexities, with work spanning the computational (e.g., [26]) and social (e.g., [3]) sciences as well as legal scholarship (e.g., [33, 34]).

Researchers have proposed various terms to describe this process, including interdependent privacy [4], networked privacy (e.g., [6]), collective privacy [16], privacy as contextual integrity [24], and comparative privacy [17]; however, these definitions do not fully encompass the complexities involved in a world where both social and technical aspects of data disclosure, collection, and use are constantly evolving and vary based on cultural, community, and contextual factors. Technical solutions alone cannot account for these factors; thus, a more human-centered approach is necessary for imagining and implementing privacy-enhancing solutions for the design of, communication about, and regulation of technologies.

Thus, we propose the following privacy framework to account for these factors:

Socio-collective privacy refers to the idea that privacy is co-constructed and negotiated within social relationships, groups, and society at large. Socio-collective privacy involves the interplay between individual privacy choices and the influence of others – peers, communities, institutions, and socio-cultural norms – in shaping those choices. From this perspective, privacy is not just a personal issue but a shared, social process, embedded in group behavior and societal expectations.

In this working group report, we describe a research roadmap for six aspects of privacy that emerged from the group’s conversations at the seminar. Over multiple days, we identified these six clusters by first reviewing and discussing ideas raised during a brainstorming session with all seminar attendees. We discussed intersections and overlaps across these notes, and then expanded into related areas. We decided to focus on three levels beyond the individual – group, organizational, and societal – and began identifying the areas that had pressing research questions at one or more of these levels. From this discussion, we see this line of research as addressing two overarching goals:

1. 1.

   understanding how the interplay between the dynamics of and across levels – individual, group, organizational, and societal – impacts key privacy outcomes (e.g., perceptions and mental models; awareness and knowledge; decisions and behaviors); and

2. 2.

   supporting communities of people to interact and negotiate around privacy.

In the following sections, we describe each of the six research clusters, including their core challenges, a brief summary of the state of the art, and a set of research questions to address that topic at the group, organizational, and/or societal level. We then share a roadmap that outlines research priorities over the short-, medium-, and long-term.

We already have many of the methods and tools to approach the questions discussed above, and we propose that we should start pursuing those right away. These methods, which we already use, largely came out of collaborations with psychologists, who tend to take an individual-centric approach to questions and methods. Given that the questions we map here are societal/group, we recognize that our existing interdisciplinary connections will need to expand. So, to deepen and expand the questions in this document and envision next steps, we will need to strengthen collaboration with scholars from other fields such as sociology and anthropology, which will require new types of incentives and funding to support that. These collaborations will be slow to come to fruition, and thus should also be started immediately.

While researchers may already be using relevant methods, there is a gap when it comes to measures. There are fewer existing measures of group-level privacy used in our research community. For example, an open research question is how to measure privacy outcomes at a societal level. Foundations for such measures may again be inspired from other fields that research organizational or societal phenomena. Another challenge is that creating interventions suffers from many of the same challenges as any multi-user system, with more complex design requirements, implementations, and evaluation protocols.

Funding for this research needs to support overcoming these challenges, and incentivize researchers towards our overarching goals. Funding agencies can encourage interdisciplinary work through specialized programs, post-doctoral opportunities, interdisciplinary symposia, summer schools, and other novel structures to encourage researchers from different communities to share ideas and collaborate on projects.

A goal of all of the above research is to provide practical and actionable guidance for socio-technical designs that empower people to achieve their own and their community’s privacy needs. Thus, we also discussed the following two questions for the research community:

• $\mathrm{■}$

  How do we help researchers to influence technology design and policy making? In other words, how can we help researchers to communicate and frame results for interested communities, such as regulators, legislators, and technology creators?

• $\mathrm{■}$

  How can we help people choose research questions and study designs that speak to those different audiences?

Finally, we note that our research can also help communities of users contribute to technology and societal outcomes – empowering people to influence designers and regulators towards meetings privacy needs. Thus, we end with two final research questions:

• $\mathrm{■}$

  What privacy advocacy has an impact on regulations?

• $\mathrm{■}$

  How can we support and promote privacy advocacy?