Loading...

Adversarial orthogonal regression: Two non-linear regressions for causal inference

Heydari, M. R ; Sharif University of Technology | 2021

378 Viewed
  1. Type of Document: Article
  2. DOI: 10.1016/j.neunet.2021.05.018
  3. Publisher: Elsevier Ltd , 2021
  4. Abstract:
  5. We propose two nonlinear regression methods, namely, Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors, while putting no assumption on noise distribution. In both methods, two adversarial networks are trained simultaneously where a regression network outputs predictions and a loss network that estimates mutual information (in AdOR) and KL-divergence (in AdOSE). These methods can be formulated as a minimax two-player game; at equilibrium, AdOR finds a deterministic map between inputs and output and estimates mutual information between residual and inputs, while AdOSE estimates a conditional probability distribution of output given inputs. The proposed methods can be used as subroutines to address several learning problems in causality, such as causal direction determination (or more generally, causal structure learning) and causal model estimation. Experimental results on both synthetic and real-world data demonstrate that the proposed methods have remarkable performance with respect to previous solutions. © 2021 Elsevier Ltd
  6. Keywords:
  7. Additives ; Game theory ; Learning systems ; Probability distributions ; Regression analysis ; Additive noise model ; Adversarial model ; Causal inferences ; Mutual informations ; Noise distribution ; Noise models ; Non-linear regression ; Nonlinear regression methods ; Orthogonal regression ; Structural equation models ; Additive noise ; Adversarial orthogonal regression ; Adversarial orthogonal structural equation model ; Causal model ; Generalized regression neural network ; Noise ; Nonhuman ; Nonlinear regression analysis ; Structural equation modeling ; Theoretical model ; Causality ; Models, Theoretical ; Probability
  8. Source: Neural Networks ; Volume 143 , 2021 , Pages 66-73 ; 08936080 (ISSN)
  9. URL: https://www.sciencedirect.com/science/article/pii/S0893608021002148?via%3Dihub