Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Framework to Explain Bayesian Models, published by Jaime Sevilla on December 6, 2021 on The AI Alignment Forum. Bayesian Networks are used to represent uncertainty and probabilistic relations between variables. They have an appealing graphical representation and a clear statistical meaning, which makes them a popular approach to manage uncertainty. I have developed a new mathematical framework, inspired in message passing, to perform content selection and produce explanations of the reasoning in BNs. In this report I will explain what it is possible to do with the framework, obviating the technical details of how it works. Those are available in the technical report Finding, Scoring and Explaining Arguments in Bayesian Networks. Previous work After reviewing past work in Bayesian Networks, I concluded that current frameworks were inadequate to solve some key issues in explaining Bayesian Networks. General explainability approaches like LIME (Ribeiro et al, 2016) and SHAP (Lundberg and Lee, 2017) already do a good job of helping us choose and explain the effect of the evidence that matters most for an explanation. But they cannot help us explain the intricate ways in which information propagates inside a Bayesian Network. So a solution particular to Bayesian Networks was desired. Some existing frameworks addressed how to select the most important paths in a graphical model that linked the evidence to the nodes of interest. For example, (Suerdmont, 1992) described a method to extract such arguments in their INSITE system. The method was later refined in eg (Haddawy et al, 1997) and (Kyrimi et al, 2020). And other work focused on relating Bayesian Networks to argument theory, superficially studying the interactions between arguments. (Vreeswijk, 2005), (Keppens, 2012) and (Timmer et al, 2017) are examples of this line of work. But no previous work that I know of addressed both problems simultaneously. I propose a framework for selecting the most important ways the evidence relates to nodes of interest in a Bayesian Network, while being mindful of interactions in the network. Introducing my framework The best introduction to my work is an example. Consider the ASIA network depicted below: Suppose we learn that the patient’s xray showed an abnormality (xray = yes), that the patient has bronchitis (bronc = yes) and that the patient does not have tuberculosis (tub = no). And we want to learn whether the patient has lung cancer. We can feed this input into the package I developed, and we will get as output two relevant arguments that relate the evidence to the target: We have observed that the lung xray shows an abnormality and the patient does not have tuberculosis.That the lung xray shows an abnormality is evidence that the patient has a lung disease (strong inference).That the patient has a lung disease and the patient does not have tuberculosis is evidence that the patient has lung cancer (strong inference). We have observed that the patient has bronchitis.That the patient has bronchitis is evidence that the patient smokes (moderate inference).That the patient smokes causes that the patient has lung cancer (weak inference). Each of these arguments has associated a quantitative strength, that we can use to prioritize which arguments to show to the user. And in fact, the arguments shown in the example are by default ordered by their importance. The framework already performs argument aggregation where it is appropriate. For example, in the first argument from the example it detected the interaction between the x-ray result and the tuberculosis, and decided to present both observations together in the same argument. Applying sociology to explaining Bayesian Networks I follow (Miller, 2017) to guide some of the choices in my explainability framework. Miller highlights four...