Ethical Preference-Based Decision Support Systems (Invited Paper)

Author Francesca Rossi

Thumbnail PDF


  • Filesize: 285 kB
  • 7 pages

Document Identifiers

Author Details

Francesca Rossi

Cite AsGet BibTex

Francesca Rossi. Ethical Preference-Based Decision Support Systems (Invited Paper). In 27th International Conference on Concurrency Theory (CONCUR 2016). Leibniz International Proceedings in Informatics (LIPIcs), Volume 59, pp. 2:1-2:7, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2016)


The future will see autonomous intelligent systems acting in the same environment as humans, in areas as diverse as driving, assistive technology, and health care. Think of self-driving cars, companion robots, and medical diagnosis support systems. Also, humans and machines will often need to work together and agree on common decisions. Thus hybrid collective decision making systems will be in great need. In these scenarios, both machines and collective decision making systems should follow some form of moral values and ethical principles (appropriate to where they will act but always aligned to humans'). In fact, humans would accept and trust more machines that behave as ethically as other humans in the same environment. Also, these principles would make it easier for machines to determine their actions and explain their behavior in terms understandable by humans. Moreover, often machines and humans will need to make decisions together, either through consensus or by reaching a compromise. This would be facilitated by shared moral values and ethical principles. In this paper we introduce some issues in embedding morality into intelligent systems. A few research questions are defined, with the hope that the discussion raised by the questions will shed some light onto the possible answers.
  • preferences
  • decision making
  • multi-agent systems


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. S. Airiau, U. Endriss, U. Grandi, D. Porello, and J. Uckelman. Aggregating dependency graphs into voting agendas in multi-issue elections. In Proceedings of IJCAI 2011, pages 18-23, 2011. Google Scholar
  2. K. J. Arrow, A. K. Sen, and K. Suzumura. Handbook of Social Choice and Welfare. North-Holland, 2002. Google Scholar
  3. C. Boutilier, R. I. Brafman, C. Domshlak, H. H. Hoos, and D. Poole. CP-nets: A tool for representing and reasoning with conditional ceteris paribus preference statements. JAIR, 21:135-191, 2004. Google Scholar
  4. R. I. Brafman, F. Rossi, D. Salvagnin, K. B. Venable, and T. Walsh. Finding the next solution in constraint- and preference-based knowledge representation formalisms. In Proceedings of KR 2010, 2010. Google Scholar
  5. V. Conitzer, J. Lang, and L. Xia. Hypercubewise preference aggregation in multi-issue domains. In Proceedings of IJCAI 2011, pages 158-163, 2011. Google Scholar
  6. H. Fargier, J. Lang, J. Mengin, and N. Schmidt. Issue-by-issue voting: an experimental evaluation. In Proceedings of MPREF 2012, 2012. Google Scholar
  7. C. Gonzales, P. Perny, and S. Queiroz. Preference aggregation with graphical utility models. In Proceedings of AAAI 2008, pages 1037-1042, 2008. Google Scholar
  8. Joshua Greene. The cognitive neuroscience of moral judgment and decision making. In The Cognitive Neurosciences V (ed. M.S. Cazzaniga). MIT Press, 2014. Google Scholar
  9. Joshua Greene, Francesca Rossi, John Tasioulas, Kristen Brent Venable, and Brian Williams. Embedding ethical principles in collective decision support systems. In Proceedings AAAI 2016. AAAI Press, 2016. Google Scholar
  10. J. Lang, M. S. Pini, F. Rossi, K. B. Venable, and T. Walsh. Winner determination in sequential majority voting. In Proceedings of IJCAI 2007, pages 1372-1377, 2007. Google Scholar
  11. J. Lang and L. Xia. Sequential composition of voting rules in multi-issue domains. Mathematical social sciences, 57:304-324, 2009. Google Scholar
  12. A. Maran, N. Maudet, M. S. Pini, F. Rossi, and K. B. Venable. A framework for aggregating influenced CP-nets and its resistance to bribery. In Proceedings of AAAI 2013, 2013. Google Scholar
  13. P. Meseguer, F. Rossi, and T. Schiex. Soft constraints. In P. Van Beek F. Rossi and T. Walsh, editors, Handbook of Constraint Programming. Elsevier, 2005. Google Scholar
  14. Bert Musschenga and Anton (eds.) van Harskamp. What Makes Us Moral? On the capacities and conditions for being moral. Springer, 2013. Google Scholar
  15. Andrew Y. Ng and Stuart Russell. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann, 2000. Google Scholar
  16. M. S. Pini, F. Rossi, K. B. Venable, and T. Walsh. Incompleteness and incomparability in preference aggregation: Complexity results. Artif. Intell., 175(7-8):1272-1289, 2011. Google Scholar
  17. G. Dalla Pozza, M. S. Pini, F. Rossi, and K. B. Venable. Multi-agent soft constraint aggregation via sequential voting. In Proceedings of IJCAI 2011, pages 172-177, 2011. Google Scholar
  18. K. Purrington and E. H. Durfee. Making social choices from individuals' CP-nets. In Proceedings of AAMAS 2007, pages 1122-1124, 2007. Google Scholar
  19. F. Rossi, P. Van Beek, and T. Walsh. Handbook of Constraint Programming. Elsevier, 2006. Google Scholar
  20. Amartya Sen. Choice, ordering and morality. In Practical Reason, Korner S. (ed). Oxford, 1974. Google Scholar
  21. Wendell Wallach and Colin Allen. Moral Machines. Oxford, 2009. Google Scholar
  22. L. Xia and V. Conitzer. Strategy-proof voting rules over multi-issue domains with restricted preferences. In Proceedings of WINE 2010, pages 402-414, 2010. Google Scholar