Leveraging AI for Management Decision-Making

Artificial intelligence (AI) has been named a core element of the “fourth industrial revolution” [40]. According to recent estimates by McKinsey & Company, AI has the potential to deliver the added global economic value of $13–20 trillion annually [5].

AI is increasingly being embraced for decision-making in management, both across a wide array of industries (e.g., healthcare, banking, education, manufacturing, retail) and functions (e.g., marketing, accounting, operations, IT). For example, AI can be used for modeling customer behavior [2, 6, 25, 26, 4]. These predictions can also serve as input for better decision-making. Examples include assortment optimization [24, 19], investment decisions [32], scheduling [42], allocation decisions [29, 31, 46], and pricing [1]. AI can predict business failures and thus act as an early warning system for improving service quality [34]. AI can help to locate drivers of low quality and eventually improve product quality [44, 45].

Recent advancements in AI research hold great promise for decision-making in businesses and organizations. Driven by the surge in data availability, computing power, and algorithmic advancement, contemporary AI algorithms are capable of emulating human decision-making and judgment [39]. This places AI in a position to augment and automate a wide range of management decisions within companies and organizations. At the same time, the use of AI for decision making, especially in high-risk applications, poses ethical and legal challenges such as the use of algorithmic risk assessment tools in criminal justice [18]. Likewise, new implications arise from emerging AI acts (e.g., the EU AI Act) with crucial implications for how AI applications must be designed.

A key enabler for data-driven decision-making in business and organizations are new AI technologies [17]. For example, deep learning can empower better decisions in business analytics and operational decision-making [30, 28]. Causal machine learning (ML) allows for optimal targeting (e.g., of customer coupons) by estimating and subsequently leveraging individualized treatment effects [37, 20, 11, 14, 15]. Probabilistic machine learning fuses methods from a statistical foundation with flexible building blocks from neural networks to yield models that are both flexible and explainable for practitioners in risk management [38, 35, 36]. Further, explainable AI (XAI) has emerged as a principled, user-centered tool not only for explaining black-box prediction models but also for explaining the decisions that are made or recommended by AI systems [33, 45, 16, 27, 3], which can be used to identify root causes of bad quality and thereby inform better decision-making in quality management [44]. Likewise, generative AI [12] and AI fairness [9, 7, 10] offer new research opportunities. Importantly, the aforementioned examples can only be solved effectively through new AI technologies that have been developed in recent years. At the same time, the use of advanced AI on digital platforms, especially the marriage of reinforcement learning with behavior modification techniques, has spurred controversy, creating adverse effects to humans and societies, such as addiction, social discord, and political polarization [22]. Existing and envisioned combinations of prediction and causal behavior modification have implications to platforms and their business customers [23]. These technologies have also created new types of barriers for academic researchers [41, 21].

The aim of our Dagstuhl Seminar “Leveraging AI for Management Decision-Making” (24342) was to discuss the future of research on AI/ML in businesses and organizations, and how the field should evolve. We especially sought to focus on “rethinking” the field by discussing the current state of AI/ML in businesses and organizations, discussing thought-provoking questions (e.g., is explainable AI really needed in practice? Where can generative AI actually lead to productivity gains? Does the algorithmic approach to fairness hold much for the future of AI ethics?), and maybe identifying and elevating new important research questions.

Our intended outcome was to reach a joint position (as a group) on what are important and unimportant research directions and what those directions should be going forward. What are the challenges? What are the opportunities? What research questions deserve attention? What questions are getting more attention than they really deserve? Below, we summarize our thoughts where we discuss existing research gaps and make suggestions for the field going forward.

Prior to the seminar, a survey was shared with all participants to identify key topics of interest around which we then designed our discussions. Most participants were primarily interested in topics related to method design and development, as well as the practical applications of AI. Some also expressed interest in evaluating these methods and understanding their impact on organizations. There was a strong focus on exploring the broader implications of AI, particularly in areas like ethics and governance. When asked about specific topics they would like to discuss during the seminar, participants showed the most interest in explainable AI, followed by causal ML, and generative AI. We eventually decided to prioritize the first two – explainable AI and causal ML – thus anticipating that generative AI will naturally arise in all sessions due to its prominence and thus regardless of whether it is a dedicated topic. Many indicated they were also keen to explore how AI can be applied effectively, with discussions centered around overcoming practical challenges in real-world deployments. Other popular topics include the ethics and governance of AI, its economic impact, and the implications for the future of work. However, there was less interest in topics like AI literacy and hybrid work environments. Further, participants also suggested additional topics for discussion. Some highlighted the importance of understanding the behavioral impacts of AI, such as long-term reliance on AI systems and the potential for deskilling human workers.

We designed our Dagstuhl Seminar as an “un-conference”. By following the format of an un-conference, we eliminated the traditional sequence of research presentations from our agenda. Instead, we aimed to focus on interactivity, collaboration, and co-creation, by making space for discussions of different forms regarding how to shape the field in the future. We held discussions with the full group as well as in smaller break-out groups, where subgroups changed from day to day; we obtained information from individuals via surveys; the schedule also encouraged informal one-on-one or small group discussions while socializing. These various modes of interaction were critical, because our seminar attracted participants from a diverse crowd, from academia and industry, from method research to behavioral research, from marketing to operations. Such diverse researchers typically do not meet or interact, and hence we seized the opportunity to foster novel interactions.

We aimed to learn from each other and create more impact. For example, we actively asked each participant in the get-to-know session to provide a summary statement about their current research and where they would like to go. Throughout the seminar, there were many opportunities to potentially start new collaborations. To spur discussion, we organized short “inspiration exchanges”, which were designed as kick-offs to our breakout sessions. Hence, the idea was primarily to discuss the current state of research and point to gaps and needs to elicit forward-thinking. Here, we selected the topics prior to the seminar based on a survey that was sent to the participants. As a result, we identified two important breakout sessions: (1) AI and causality, and (2) AI and responsibility. We summarize the discussions and findings from both breakout sessions below.

The session started with an inspirational short presentation introducing the concept of causal ML [43, 14]. Causal ML is a branch of machine learning focused on identifying and understanding cause-and-effect relationships, rather than simply detecting patterns in data. Traditional machine learning models excel at predicting outcomes by recognizing correlations, but they often struggle to determine whether a specific factor directly causes a change in the target variable. Causal ML attempts to address this limitation by combining statistical techniques from causal inference with modern machine learning approaches. It allows researchers to answer questions like “What will happen if we change this feature?” or “What is the impact of a particular intervention?” This is particularly useful in fields like management where the aim is to to understand how management decisions will influence outcomes, so that the best decision for the business can be made.

In practice, Causal ML uses traditional statistical and econometric methods such as randomized controlled trials, propensity score matching, and instrumental variables to isolate causal effects. Additionally, more advanced ML-based techniques like causal forests and doubly robust estimators have been developed to handle complex, high-dimensional data where traditional methods might fall short. By leveraging these approaches, Causal ML models can go beyond predictions to inform actionable insights, such as making recommendations for business decisions in strategy, marketing, etc. Generally, causal ML follows the same frameworks as traditional causal inference, but the use of ML changes the underlying estimation strategy and thus offers several benefits in business practice, such as making personalized decisions for customers or users. Moreover, the goals of causal inference for decision making can be different from traditional causal-effect estimation [14], motivating the application of alternative machine learning approaches [15].

Defining Concepts and Aligning Terminology:

The discussions highlighted the need for clearer definitions and a shared vocabulary in the field. Participants acknowledged that “causality” is often interpreted differently across disciplines, which can hinder effective collaboration among different fields and between practitioners and researchers. Aligning terminology and notation was seen as a critical step for advancing research and practical applications.

One of the provocative questions debated was whether prediction (causal or non-causal) is sufficient for effective decision-making or if more granular causal inference is necessary. Participants discussed scenarios where accurate predictions might suffice (e.g., short-term business decisions), versus contexts where a deeper understanding of causal relationships is crucial (e.g., long-term strategic planning or policy-making). The consensus was that while predictive models are valuable, they may fall short in areas where understanding the “why” behind outcomes is essential. Yet, beyond that, the participants see a large need for more research and eventually evidence allowing for decision support.

As mentioned above, one of the biggest challenges in causal ML is deciding when to prioritize causal models over traditional predictive models. Predictive ML models, such as those used for classification or regression, are primarily designed to optimize accuracy based on historical patterns, without necessarily understanding the underlying relationships. However, when the goal is to make decisions that could change the environment or influence outcomes (like deciding on a new marketing strategy or a treatment plan in healthcare), causal modeling could be the “go to” approach. Yet, the challenge lies in identifying these scenarios where the extra complexity and effort of causal analysis are justified. Deep causal understanding requires carefully constructed assumptions and a deep understanding of the data-generating process, which can be resource-intensive. On the other hand, causal prediction may be sufficient (and quite effective) for decision making – and is often overlooked [13]. Without clear indicators of when causal methods are needed, organizations may either waste resources on unnecessary complexity or, conversely, miss opportunities to derive actionable insights where causality matters.

From a practical perspective, adopting causal ML techniques is often hindered by non-technical challenges such as the availability of skilled personnel, time, and budget constraints. Traditional ML has broad and accessible toolchains, tutorials, and community support, facilitating rapid training and deployment of models. In contrast, using ML to achieve causal understanding requires additional specialized training, including knowledge of statistical theory and causal inference frameworks, which are not as widely understood or accessible. Moreover, the data requirements for causal models can be more stringent, often requiring richer datasets or careful experimental designs. Together, this may create barriers for companies trying to leverage causal insights, especially those with limited resources.

A recurring question was whether algorithms for automated decision-making always require a causal framework. Opinions varied, with some arguing that in certain applications (e.g., dynamic pricing, recommendation systems), causal understanding is not always necessary, while others stressed that understanding causal relationships is crucial for ensuring fairness and avoiding unintended consequences.

Participants raised concerns about the limitations of causal inference from observational data, particularly when confounding factors and model misspecifications are present. The discussion emphasized the need for developing more robust methodologies to derive reliable causal insights, especially in real-world applications where randomized experiments are often impractical.

The breakout sessions successfully brought to light the complexities and nuances involved in integrating causality with AI, especially around formal definitions, guidelines when causal modeling is beneficial (and when not), and effective approaches. The discussions highlighted the importance of interdisciplinary collaboration to advance research, develop practical tools, and address the societal implications of AI-driven decision-making.

The starting point was an inspirational high-level short introduction and exchange discussing methods for explaining AI algorithms, models, and decisions, as well as the underlying objectives, rationales, and limitations [8]. Explainable AI (XAI) refers to a broad set of techniques and methods used to make the decisions, predictions, and/or inner workings of AI models more transparent and understandable to humans. Many AI systems, especially those that use complex algorithms like deep learning, are often seen as “black boxes” because their decision-making processes are not easily interpretable. This lack of transparency can be problematic, especially in sensitive areas like finance, credit lending, or law enforcement, where understanding how decisions are made is crucial for trust and accountability. Explainable AI aims to bridge this gap by providing insights into how or why models reach their conclusions, which can help users trust and effectively use AI systems.

The goal of XAI is to provide explanations that are not only faithful but also understandable to different stakeholders, such as data scientists, business leaders, or end-users. For example, in quality management, an XAI model might highlight which processes contributed most to a low quality level, helping manufacturing make more informed decisions about where to improve production processes. Techniques like feature importance scores, counterfactual explanations, interpretable models such as rule lists or shallow decision trees, and visualization tools are commonly used to make AI models and decisions more interpretable. By making AI decisions clearer, explainable AI could – in principle – help ensure that systems are not only accurate but also fair, reliable, and aligned with ethical standards, especially when those decisions impact people’s lives. However, as we discuss below, there was large dispute among the participants whether this promise is true in practice.

Despite its benefits, XAI suffers from several limitations. One major limitation in practice is that it is often unclear for whom any particular explanation method was designed, and there is often a mismatch between the capabilities of these methods and the needs in practice. This leads to frequent cases where explanation methods fail to serve the needs of users or stakeholders who would like to use such explanations for decision-making. For example, it is often unclear how certain methods such as feature importance can help decision-makers generate insights that are actionable and that can be translated into better decisions. Another major challenge is that, for highly complex models like deep neural networks, providing simple and intuitive explanations can be difficult. Sometimes, explanations generated by explainable AI methods may oversimplify the decision process, leading to misunderstandings or even incorrect conclusions. Finally, there is little consensus on what a “good” explanation is.

Participants emphasized the importance of XAI for fostering transparency and trust, particularly in sectors like healthcare, finance, and criminal justice. However, the group also critically examined the current state of XAI research and its use in practice, thereby emphasizing salient limitations. A recurring concern was that many so-called “explanations” provided by AI systems are overly simplified, often failing to capture the complexities of the underlying algorithm. Another problem is that the explanations are rarely aligned with the needs of stakeholders in practice [33], so that actions derived from XAI may lead to negative outcomes. Together, the discussions highlighted that XAI can create a false sense of understanding or accountability, particularly when explanations are designed more for compliance than genuine transparency.

The participants also see a large potential for future research and thus for how the field could move forward. The discussion highlighted the need for more rigorous evaluations of the impact of AI explanations on decision-makers, particularly in contexts where human lives or rights are at stake. Currently, the evaluations are rarely aligned with how XAI systems are used in practice. For example, many evaluations are often one-sided and only focus on the role of programmers, while more holistic evaluation approaches that account for the different roles of stakeholders are still scarce. Another point for future research is to clarify what we mean under ‘understandable’ as this is a necessary condition to make assessments as to when one model is more interpretable than another.

The session also touched on the rise of regulatory approaches to AI (e.g., some regulatory frameworks offer a right for explanations in the context of data-driven decision-making). Here, participants emphasized the importance of regulatory and governance frameworks to provide oversight for the development and deployment of AI to prevent harm while enabling innovation.

The session underscored the complexity of ensuring responsible AI in practice. While explainable AI is a step towards greater transparency, participants highlighted that algorithmic explanations alone are not enough. In response to the session, the participants aim to create a commentary that offers critical reflections and guidance. We plan to discuss the following directions in our commentary: Which method should be applied by whom, in what context, and with what goal in mind? We will thus take a step back and will propose a comprehensive framework that shows the translation of real-world business problems into the model world while highlighting the critical dimensions that influence the success of XAI initiatives.