{"@context":"https:\/\/schema.org\/","@type":"ScholarlyArticle","@id":"#article1810","name":"Variational Bayes via Propositionalization","abstract":"We propose a unified approach to VB (variational Bayes) in\r\nsymbolic-statistical modeling via propositionalization.\r\nBy propositionalization we mean, broadly, expressing and\r\ncomputing probabilistic models such as BNs (Bayesian\r\nnetworks) and PCFGs (probabilistic context free grammars)\r\nin terms of propositional logic that considers\r\npropositional variables as binary random variables.\r\n\r\nOur proposal is motivated by three observations. The\r\nfirst one is that PPC (propostionalized probability\r\ncomputation), i.e. probability computation formalized in\r\na propositional setting, has turned out to be general and\r\nefficient when variable values are sparsely\r\ninterdependent. Examples include (discrete) BNs, PCFGs\r\nand more generally PRISM which is a Turing complete logic\r\nprogramming language with EM learning ability we have been\r\ndeveloping, and computes probabilities using graphically\r\nrepresented AND\/OR boolean formulas. Efficiency of PPC is\r\nclassically testified by the Inside-Outside algorithm in\r\nthe case of PCFGs and by recent PPC approaches in the case\r\nof BNs such as the one by Darwiche et al. that exploits\r\n$0$ probability and CSI (context specific independence).\r\nDechter et al. also revealed that PPC is a general\r\ncomputation scheme for BNs by their formulation of AND\/OR\r\nsearch spaces.\r\n\r\nSecond of all, while VB has been around for sometime as a\r\npractically effective approach to Bayesian modeling, it's\r\nuse is still somewhat restricted to simple models such as\r\nBNs and HMMs (hidden Markov models) though its usefulness\r\nis established through a variety of applications from\r\nmodel selection to prediction. On the other hand it is\r\nalready proved that VB can be extended to PCFGs and is\r\nefficiently implementable using dynamic programming. Note\r\nthat PCFGs are just one class of PPC and much more general\r\nPPC is realized by PRISM. Accordingly if VB is extened to\r\nPRISM's PPC, we will obtain VB for general probabilistic\r\nmodels, far wider than BNs and PCFGs.\r\n\r\nThe last observation is that once VB becomes available in\r\nPRISM, it saves us a lot of time and energy. First we do\r\nnot have to derive a new VB algorithm from scratch for\r\neach model and implement it. All we have to do is just to\r\nwrite a probabilistic model at predicate level. The rest\r\nof work will be carried out automatically in a unified\r\nmanner by the PRISM system as it happens in the case of EM\r\nlearning. Deriving and implementing a VB algorithm is a\r\ntedious error-prone process, and ensuring its correctness\r\nwould be difficult beyond PCFGs without formal semantics.\r\n\r\nPRISM augmented with VB will completely eliminate such\r\nneeds and make it easy to explore and test new Bayesian\r\nmodels by helping the user cope with data sparseness and\r\navoid over-fitting.","keywords":["Variational Bayes","propositionalized probability computation","PRISM"],"author":[{"@type":"Person","name":"Sato, Taisuke","givenName":"Taisuke","familyName":"Sato"},{"@type":"Person","name":"Kameya, Yoshitaka","givenName":"Yoshitaka","familyName":"Kameya"},{"@type":"Person","name":"Kurihara, Kenichi","givenName":"Kenichi","familyName":"Kurihara"}],"position":10,"pageStart":1,"pageEnd":8,"dateCreated":"2008-03-06","datePublished":"2008-03-06","isAccessibleForFree":true,"license":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode","copyrightHolder":[{"@type":"Person","name":"Sato, Taisuke","givenName":"Taisuke","familyName":"Sato"},{"@type":"Person","name":"Kameya, Yoshitaka","givenName":"Yoshitaka","familyName":"Kameya"},{"@type":"Person","name":"Kurihara, Kenichi","givenName":"Kenichi","familyName":"Kurihara"}],"copyrightYear":"2008","accessMode":"textual","accessModeSufficient":"textual","creativeWorkStatus":"Published","inLanguage":"en-US","sameAs":"https:\/\/doi.org\/10.4230\/DagSemProc.07161.10","publisher":"Schloss Dagstuhl \u2013 Leibniz-Zentrum f\u00fcr Informatik","isPartOf":{"@type":"PublicationVolume","@id":"#volume650","volumeNumber":7161,"name":"Dagstuhl Seminar Proceedings, Volume 7161","dateCreated":"2008-03-06","datePublished":"2008-03-06","isAccessibleForFree":true,"publisher":"Schloss Dagstuhl \u2013 Leibniz-Zentrum f\u00fcr Informatik","hasPart":"#article1810","isPartOf":{"@type":"Periodical","@id":"#series119","name":"Dagstuhl Seminar Proceedings","issn":"1862-4405","isAccessibleForFree":true,"publisher":"Schloss Dagstuhl \u2013 Leibniz-Zentrum f\u00fcr Informatik","hasPart":"#volume650"}}}