The five works discussed in the previous posts summarized the current state of the research on interpretable neural networks by design. Thus, the analysis of their limitations unveils four open research questions, which we discuss in this new post.

This is the last post of our review on interpretable deep neural networks by design. Previously discussed concept-based methods focused on ranking the most relevant basis concepts to provide a reduced subset of concepts as interpretations. Barbiero et al.[1] addressed the problem that previous approaches fail to provide concise and formal explanations of how the top-ranked concepts are leveraged by the models to make predictions. Therefore, they proposed an entropy-based criterion to generate logical explanations from neural networks.

The approaches discussed so far are set in the context of conventional supervised learning. They use a single prediction model and aim to enhance its interpretability. Conversely, in this fourth part of our review on interpretable deep neural networks by design, we discuss a framework proposed by Parekh, Mozharovskyi, and d’Alche-Bu [1] to learn with interpretation (FLINT) that jointly learns a predictive model and an associated interpretation model. This gives rise to a new generic task the authors called supervised learning with interpretation (SLI).

In this third part of our review on interpretable deep neural networks by design, we discuss a method called Concept Whitening. Recall that both SENNs (Part 1) and ProtoAttend (Part 2) use encoders that transform the original data input into a reduced representation using several neural network layers. However, the encoded information in the latent space is challenging to be interpreted directly. Hence, Chen, Bei, and Rudin [1] introduced a mechanism called concept whitening (CW) that alters the information generated by a layer of the network and constrains the latent space to represent target concepts.

The second part of our review on interpretable deep neural networks by design focuses on a method called ProtoAttend proposed by Arik and Pfister [1]. ProtoAttend aims to model inherently-interpretable deep neural networks based on the principle that “prototypes should constitute a minimal subset of samples with a high interpretable value that can serve as distillation or condensed view of a dataset”. The ProtoAttend mechanism can be integrated into a wide range of neural network architectures. It generates encoded representations and relates them to data samples using attention mechanisms for prototype selection.

Deep learning-based systems have not been widely adopted in critical areas such as healthcare and criminal justice due to their lack of interpretability. In addition to high performance, interpretability is necessary for obtaining an appropriate level of trust in this kind of system. In this five-part post, we discuss five recent works related to the development of interpretable deep neural networks by design; that is, they incorporate the interpretability objective into the learning process. The discussed methods are Self-explaining neural networks, ProtoAttend, concept whitening, a framework to learn with interpretation (FLINT), and Entropy-based logic explanations of neural networks. Their novelty and contributions as well as their potential drawbacks and gaps are presented and analyzed.

Presentation of our paper “Two-dimensional deep regression for early yield prediction of winter wheat” at the SPIE Future Sensing Technologies 2021 (14 November 2021). Pre-print: link

In this post, I will report the best five spectral bands for the Indian Pines (IP) and Salinas (SA) hyperspectral datasets obtained by our hybrid-band selection method presented in our paper “Hyperspectral Dimensionality Reduction based on Inter-Band Redundancy Analysis and Greedy Spectral Selection”. In addition, I will provide code in Python to demonstrate the improvements in classification performance in comparison to other five band selection methods.

Bayesian Networks are directed acyclic graphs used to model beliefs about the world. Abstractly, they represent the chance that a certain effect occurs given a set of possible causes, or the chance that a cause or causes led to an outcome. A Bayesian Network is comprised of variables connected through conditional dependence, which can be thought of as casual relationships. Each variable contains a domain of possible values and a joint probability distribution of the variable taking on specific values given the sets of values for its parents. From this model, one can infer the posterior probability that a set of query variables take on specific values, with or without a set of evidence (i.e. known values for specific variables).

Presentation of our paper “Hyperspectral Band Selection For Multispectral Image Classification With Convolutional Networks” at the International Joint Conference on Neural Networks (IJCNN 2021). Pre-print: link

1. Introduction

Given a set of $n$ independet variables: $X_1, X_2, …, X_n$ and a real-valued dependant variable $Y$ (or response variable), the goal of the linear regression problem is to find a regression function $f$ such that:

Being able to interpret a classifier’s decision has become crucial lately. This ability allows us not only to ensure that a Convolutional Neural Network -for example- has learned the patterns that we expected, but also to discover patterns that were not obvious at first glance. Most of the works related to Layer-wise Relevance Propagation (LRP) so far have been applied to image classification tasks; in that case, we are interested in finding the pixel positions that were more relevant for a given classification result. For example, the following image highlights the most relevant pixels to obtain a prediction of the class “cat”:

Some days ago, I received the Peruvian “RENACYT Researcher” certification by Concytec Perú. This certification takes into account different criteria such as the current academic degree, the generation and relevance of the scientific and/or technological knowledge produced, and the participation in research projects.