Relative-importance assessment of explanatory variables in generalized linear models: an entropy-based approach

The object of the present paper is to propose a method for relative-importance assessment of explanatory variables in generalized linear models, through an analysis of the variation of entropy of the response variable. First, the problem is reviewed in the ordinary regression model and some criteria to be met by a suitable measure are emphasized. Second, the logic of variation in entropy is introduced, for the assessment both of the predictive power of the whole model and of the relative importance of each variable. Third, the occurrence of a causal order of variables is discussed and a new approach is proposed to deal with cases where this order lacks. Finally, the ability to meet the listed criteria is checked for the proposed measure and two relevant examples (logit model and two-way ANOVA model) are provided, both with numerical applications.