Document Open Access Logo

Combining Embeddings and Rules for Fact Prediction (Invited Paper)

Authors Armand Boschin, Nitisha Jain, Gurami Keretchashvili, Fabian Suchanek

PDF
Thumbnail PDF

File

OASIcs.AIB.2022.4.pdf
  • Filesize: 0.83 MB
  • 30 pages

Document Identifiers

Author Details

Armand Boschin
  • Télécom Paris, Institut Polytechnique de Paris, France
Nitisha Jain
  • Hasso Plattner Institute, University of Potsdam, Germany
Gurami Keretchashvili
  • Télécom Paris, Institut Polytechnique de Paris, France
Fabian Suchanek
  • Télécom Paris, Institut Polytechnique de Paris, France

Funding

This work was partially funded by ANR-20-CHIA-0012-01 ("NoRDF").

Cite AsGet BibTex

Armand Boschin, Nitisha Jain, Gurami Keretchashvili, and Fabian Suchanek. Combining Embeddings and Rules for Fact Prediction (Invited Paper). In International Research School in Artificial Intelligence in Bergen (AIB 2022). Open Access Series in Informatics (OASIcs), Volume 99, pp. 4:1-4:30, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022)
https://doi.org/10.4230/OASIcs.AIB.2022.4

Abstract

Knowledge bases are typically incomplete, meaning that they are missing information that we would expect to be there. Recent years have seen two main approaches to guess missing facts: Rule Mining and Knowledge Graph Embeddings. The first approach is symbolic, and finds rules such as "If two people are married, they most likely live in the same city". These rules can then be used to predict missing statements. Knowledge Graph Embeddings, on the other hand, are trained to predict missing facts for a knowledge base by mapping entities to a vector space. Each of these approaches has their strengths and weaknesses, and this article provides a survey of neuro-symbolic works that combine embeddings and rule mining approaches for fact prediction.

Subject Classification

ACM Subject Classification
  • Information systems → Information systems applications
Keywords
  • Rule Mining
  • Embeddings
  • Knowledge Bases
  • Deep Learning

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Naser Ahmadi, Viet-Phi Huynh, Vamsi Meduri, Stefano Ortona, and Paolo Papotti. Mining expressive rules in knowledge graphs. Journal of Data and Information Quality (JDIQ), 12(2):1-27, 2020. Google Scholar
  2. Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li. Realistic re-evaluation of knowledge graph completion methods: An experimental study. In ACM SIGMOD, 2020. Google Scholar
  3. Alessandro Artale, Diego Calvanese, Roman Kontchakov, and Michael Zakharyaschev. The dl-lite family and relations. Journal of artificial intelligence research, 36:1-69, 2009. Google Scholar
  4. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary G. Ives. Dbpedia: A nucleus for a web of open data. In ISWC, 2007. Google Scholar
  5. Ivana Balažević, Carl Allen, and Timothy M Hospedales. Hypernetwork knowledge graph embeddings. In ICANN, 2019. Google Scholar
  6. Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233-259, 2014. Google Scholar
  7. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurIPS, 2013. Google Scholar
  8. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, and Tom M Mitchell. Toward an architecture for never-ending language learning. In AAAI, 2010. Google Scholar
  9. Yuanfei Dai, Shiping Wang, Neal N Xiong, and Wenzhong Guo. A survey on knowledge graph embedding: Approaches, applications and benchmarks. Electronics, 9(5):750, 2020. Google Scholar
  10. Tu Dinh Nguyen Dai Quoc Nguyen, Dat Quoc Nguyen, and Dinh Phung. A novel embedding model for knowledge base completion based on convolutional neural network. In NAACL, 2018. Google Scholar
  11. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In AAAI, 2018. Google Scholar
  12. Boyang Ding, Quan Wang, Bin Wang, and Li Guo. Improving knowledge graph embedding using simple constraints. arXiv preprint, 2018. URL: http://arxiv.org/abs/1805.02408.
  13. Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In ACM SIGKDD, 2014. Google Scholar
  14. Claudia d’Amato, Nicola Flavio Quatraro, and Nicola Fanizzi. Injecting background knowledge into embedding models for predictive tasks on knowledge graphs. In ESWC, 2021. Google Scholar
  15. Bahare Fatemi, Siamak Ravanbakhsh, and David Poole. Improved knowledge graph embedding using background taxonomic information. In AAAI, volume 33, 2019. Google Scholar
  16. Jun Feng, Minlie Huang, Mingdong Wang, Mantong Zhou, Yu Hao, and Xiaoyan Zhu. Knowledge graph embedding by flexible translation. In KR, 2016. Google Scholar
  17. Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian Suchanek. Amie: Association rule mining under incomplete evidence in ontological knowledge bases. In WWW, 2013. Google Scholar
  18. Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. Fast Rule Mining in Ontological Knowledge Bases with AMIE+. In VLDBJ, 2015. Google Scholar
  19. Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. Jointly embedding knowledge graphs and logical rules. In EMNLP, 2016. Google Scholar
  20. Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. Knowledge graph embedding with iterative guidance from soft rules. In AAAI, 2018. Google Scholar
  21. Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. Learning to represent knowledge graphs with gaussian embedding. In CIKM, 2015. Google Scholar
  22. Frank Lauren Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics, 6:164-189, 1927. Google Scholar
  23. Nitisha Jain, Jan-Christoph Kalo, Wolf-Tilo Balke, and Ralf Krestel. Do embeddings actually capture knowledge graph semantics? In ESWC, 2021. Google Scholar
  24. Nitisha Jain, Trung-Kien Tran, Mohamed H Gad-Elrab, and Daria Stepanova. Improving knowledge graph embeddings with ontological reasoning. In ISWC, 2021. Google Scholar
  25. Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dynamic mapping matrix. In ACL, 2015. Google Scholar
  26. Rudolf Kadlec, Ondřej Bajgar, and Jan Kleindienst. Knowledge base completion: Baselines strike back. In RepL4NLP, 2017. Google Scholar
  27. Seyed Mehran Kazemi and David Poole. Simple embedding for link prediction in knowledge graphs. In NeurIPS, 2018. Google Scholar
  28. Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint, 2016. URL: http://arxiv.org/abs/1609.02907.
  29. Bhushan Kotnis and Vivi Nastase. Analysis of the impact of negative sampling on link prediction in knowledge graphs. arXiv preprint, 2017. URL: http://arxiv.org/abs/1708.06816.
  30. Jonathan Lajus, Luis Galárraga, and Fabian M. Suchanek. Fast and Exact Rule Mining with AMIE 3. In ESWC, 2020. Google Scholar
  31. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In AAAI, 2015. Google Scholar
  32. Michael Loster, Davide Mottin, Paolo Papotti, Jan Ehmüller, Benjamin Feldmann, and Felix Naumann. Few-shot knowledge validation using rules. In TheWebConf, 2021. Google Scholar
  33. Farzaneh Mahdisoltani, Joanna Asia Biega, and Fabian M. Suchanek. YAGO3: A Knowledge Base from Multilingual Wikipedias. In CIDR, 2015. Google Scholar
  34. Christian Meilicke, Melisachew Wudage Chekol, Daniel Ruffinelli, and Heiner Stuckenschmidt. An introduction to anyburl. In KI, 2019. Google Scholar
  35. Pasquale Minervini, Luca Costabello, Emir Muñoz, Vít Nováček, and Pierre-Yves Vandenbussche. Regularizing knowledge graph embeddings via equivalence and inversion axioms. In ECML PKDD, 2017. Google Scholar
  36. Sameh K Mohamed, Vít Novácek, Pierre-Yves Vandenbussche, and Emir Muñoz. Loss functions in knowledge graph embedding models. DL4KG@ ESWC, 2377:1-10, 2019. Google Scholar
  37. Brian Murphy, Partha Talukdar, and Tom Mitchell. Learning effective and interpretable semantic models using non-negative sparse embedding. In COLING, 2012. Google Scholar
  38. Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A three-way model for collective learning on multi-relational data. In ICML, 2011. Google Scholar
  39. Pouya Ghiasnezhad Omran, Kewen Wang, and Zhe Wang. An embedding-based approach to rule learning in knowledge graphs. IEEE Transactions on Knowledge and Data Engineering, 33(4):1348-1359, 2021. Google Scholar
  40. Stefano Ortona, Venkata Vamsikrishna Meduri, and Paolo Papotti. Robust discovery of positive and negative rules in knowledge bases. In ICDE, 2018. Google Scholar
  41. Pouya Pezeshkpour, Yifan Tian, and Sameer Singh. Revisiting evaluation of knowledge base completion models. In AKBC, 2020. Google Scholar
  42. Jay Pujara, Eriq Augustine, and Lise Getoor. Sparsity and noise: Where knowledge graph embeddings fall short. In EMNLP, 2017. Google Scholar
  43. Simon Razniewski, Hiba Arnaout, Shrestha Ghosh, and Fabian M. Suchanek. Completeness, Recall, and Negation in Open-World Knowledge Bases. In VLDB, 2021. Google Scholar
  44. Simon Razniewski, Fabian M. Suchanek, and Werner Nutt. But What Do We Actually Know? In AKBC workshop, 2016. Google Scholar
  45. Tim Rocktäschel, Sameer Singh, and Sebastian Riedel. Injecting logical background knowledge into embeddings for relation extraction. In NAACL, 2015. Google Scholar
  46. Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo. Knowledge graph embedding for link prediction: A comparative analysis. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(2):1-49, 2021. Google Scholar
  47. Andrea Rossi and Antonio Matinata. Knowledge graph embeddings: Are relation-learning models learning relations? In EDBT/ICDT, 2020. Google Scholar
  48. Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla. You can teach an old dog new tricks! on training knowledge graph embeddings. In ICLR, 2019. Google Scholar
  49. Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, and Daisy Zhe Wang. Drum: End-to-end differentiable rule mining on knowledge graphs. In NeurIPS, 2019. Google Scholar
  50. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, 2018. Google Scholar
  51. Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, and Bowen Zhou. End-to-end structure-aware convolutional networks for knowledge base completion. In AAAI, 2019. Google Scholar
  52. Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. In NeurIPS, 2013. Google Scholar
  53. Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago - A Core of Semantic Knowledge . In WWW, 2007. Google Scholar
  54. Fabian M. Suchanek, Jonathan Lajus, Armand Boschin, and Gerhard Weikum. Knowledge Representation and Rule Mining in Entity-Centric Knowledge Bases. In RW, 2019. Google Scholar
  55. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR, 2018. Google Scholar
  56. Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, and Yiming Yang. A re-evaluation of knowledge graph completion methods. In ACL, 2020. Google Scholar
  57. Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In CVSC workshop, 2015. Google Scholar
  58. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In ICML, 2016. Google Scholar
  59. Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-based multi-relational graph convolutional networks. In ICLR, 2019. Google Scholar
  60. Denny Vrandecic and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Commun. ACM, 57(10):78-85, 2014. Google Scholar
  61. Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung, et al. A capsule network-based embedding model for knowledge graph completion and search personalization. In NAACL, 2019. Google Scholar
  62. Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 29(12):2724-2743, 2017. Google Scholar
  63. Quan Wang, Bin Wang, and Li Guo. Knowledge base completion using embeddings and rules. In ICOAI, 2015. Google Scholar
  64. Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, Samuel Broscheit, and Christian Meilicke. On evaluating embedding models for knowledge base completion. In RepL4NLP, 2019. Google Scholar
  65. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, 2014. Google Scholar
  66. Gerhard Weikum, Luna Dong, Simon Razniewski, and Fabian M. Suchanek. Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases. In Foundations and Trends in Databases, 2021. Google Scholar
  67. Alfred North Whitehead and Bertrand Russell. Principia mathematica. Cambridge University Press, 1913. Google Scholar
  68. Han Xiao, Minlie Huang, Yu Hao, and Xiaoyan Zhu. Transg: A generative mixture model for knowledge graph embedding. arXiv preprint, 2015. URL: http://arxiv.org/abs/1509.05488.
  69. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint, 2014. URL: http://arxiv.org/abs/1412.6575.
  70. Rui Ye, Xin Li, Yujie Fang, Hongyu Zang, and Mingzhong Wang. A vectorized relational graph convolutional network for multi-relational network alignment. In IJCAI, 2019. Google Scholar
  71. Jindou Zhang and Jing Li. Enhanced knowledge graph embedding by jointly learning soft rules and facts. Algorithms, 12(12):265, 2019. Google Scholar
  72. Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. Learning hierarchy-aware knowledge graph embeddings for link prediction. In AAAI, 2020. Google Scholar
  73. Xiang Zhao, Weixin Zeng, Jiuyang Tang, Wei Wang, and Fabian M. Suchanek. An Experimental Study of State-of-the-Art Entity Alignment Approaches . In TKDE, 2020. Google Scholar
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact