2022-03: Deep-learning models with constrained first layer: interpretability and more

Title: Deep-learning models with constrained first layer: interpretability and more

Time: 24th March 2022

Abstract: Interpretability often seeks domain-specific facts, which is understandable to human, from deep-learning (DL) or other machine-learning (ML) models of black-box nature. This is particularly important to establish transparency in ML model’s inner working and decision-making, so that a certain level of trust is achieved when a model is deployed in a sensitive and mission-critical context, such as healthcare. DL model architectures span multiple layers where each layer can be of diverse types. These models were found capable of learning complex mapping between the input and output with automated feature-extraction, but generated features lose interpretability and thus, the understandability of the model’s decision making. The black-box opening literature reveals different philosophies to unveil the black-box nature of DL models, one popular approach being spotting a region in the input data as a heat-map interpreting that the model was paying attention there while making the decision. Another approach is to constrain the first layer to make sure that meaningful information is allowed into the rest of the layers of a model so that the decision-making is based on legitimate information. A decision made by such a constrained model can be interpreted in terms of the constrained layer’s components’ relative significance. This study quantitatively measures the contribution of the convolution kernels of the constrained first layer of a CNN model in the decision-making, which later utilised for domain-specific interpretation and interestingly, to do model optimisation.

Short Bio: Mr Ahsan Habib is a PhD candidate at School of IT in Deakin University. He is currently working on designing explainable deep learning models for physiological time-series signals to solve problems in healthcare domain.