Chuanlong Xie

Bio: Chuanlong Xie is an academic researcher from Huawei. The author has contributed to research in topics: Computer science & Smoothing. The author has an hindex of 5, co-authored 19 publications receiving 102 citations. Previous affiliations of Chuanlong Xie include Jinan University & Hong Kong Baptist University.

Risk Variance Penalization: From Distributional Robustness to Causality.

Abstract: Learning under multi-environments often requires the ability of out-of-distribution generalization for the worst-environment performance guarantee. Some novel algorithms, e.g. Invariant Risk Minimization and Risk Extrapolation, build stable models by extracting invariant (causal) feature. However, it remains unclear how these methods learn to remove the environmental features. In this paper, we focus on the Risk Extrapolation (REx) and make attempts to fill this gap. We first propose a framework, Quasi-Distributional Robustness, to unify the Empirical Risk Minimization (ERM), the Robust Optimization (RO) and the Risk Extrapolation. Then, under this framework, we show that, comparing to ERM and RO, REx has a much larger robust region. Furthermore, based on our analysis, we propose a novel regularization method, Risk Variance Penalization (RVP), which is derived from REx. The proposed method is easy to implement, and has proper degree of penalization, and enjoys an interpretable tuning parameter. Finally, our experiments show that under certain conditions, the regularization strategy that encourages the equality of training risks has ability to discover relationships which do not exist in the training data. This provides important evidence to support that RVP is useful to discover causal models.

Out-of-Distribution Generalization Analysis via Influence Function

Abstract: The mismatch between training dataset and target environment is one major challenge for current machine learning systems When training data is collected from multiple environments and the the evaluation is on any new environment, we are facing an Out-of-Distribution (OOD) generalization problem that aims to find a model with the best OOD accuracy, ie the best worst-environment accuracy However, with limited access to environments, the worst environment may be unseen, and test accuracy is a biased estimate of OOD accuracy In this paper, we show that test accuracy may dramatically fail to identify OOD accuracy and mislead the tuning procedure To this end, we introduce Influence Function, a classical tool from robust statistics, into the OOD generalization problem and suggest the variance of influence function to measure the stability of a model on training environments We show that the proposed index and test accuracy together can help us discern whether OOD algorithms are needed and whether a model achieves good OOD generalization

Sufficient dimension reduction with mixture multivariate skew-elliptical distributions

Abstract: In inverse regression-based methodologies for sufficient dimension reduction, ellipticity (or slightly more generally, the linearity condition) of the predictor vector is a widely used condition, though there is concern over its restrictiveness. In this paper, Stein’s Lemma is generalized to the class of mixture multivariate skewelliptical distributions in different scenarios to identify and estimate the central subspace. Within this class, necessary and sufficient conditions are explored for the simple covariance between the response (or its function) and the predictor vector to identify the central subspace. Further, we provides a way to do adjustments such that the central subspace can still be identifiable when this simple covariance fails to work. Simulations are used to assess the performance of the results and compare with existing methods. A data example is analysed for illustration.

The Risks of Invariant Risk Minimization

Abstract: Invariant Causal Prediction (Peters et al., 2016) is a technique for out-of-distribution generalization which assumes that some aspects of the data distribution vary across the training set but that the underlying causal mechanisms remain constant. Recently, Arjovsky et al. (2019) proposed Invariant Risk Minimization (IRM), an objective based on this idea for learning deep, invariant features of data which are a complex function of latent variables; many alternatives have subsequently been suggested. However, formal guarantees for all of these works are severely lacking. In this paper, we present the first analysis of classification under the IRM objective--as well as these recently proposed alternatives--under a fairly natural and general model. In the linear case, we show simple conditions under which the optimal solution succeeds or, more often, fails to recover the optimal invariant predictor. We furthermore present the very first results in the non-linear regime: we demonstrate that IRM can fail catastrophically unless the test data are sufficiently similar to the training distribution--this is precisely the issue that it was intended to solve. Thus, in this setting we find that IRM and its alternatives fundamentally do not improve over standard Empirical Risk Minimization.

Acupuncture for chronic pain: an individual patient data meta-analysis

Abstract: BACKGROUND Although acupuncture is widely used for chronic pain, there remains considerable controversy as to its value. We aimed to determine the effect size of acupuncture for 4 chronic pain conditions: back and neck pain, osteoarthritis, chronic headache, and shoulder pain. METHODS We conducted a systematic review to identify randomized controlled trials (RCTs) of acupuncture for chronic pain in which allocation concealment was determined unambiguously to be adequate. Individual patient data meta-analyses were conducted using data from 29 of 31 eligible RCTs, with a total of 17 922 patients analyzed. RESULTS In the primary analysis, including all eligible RCTs, acupuncture was superior to both sham and no-acupuncture control for each pain condition (P < .001 for all comparisons). After exclusion of an outlying set of RCTs that strongly favored acupuncture, the effect sizes were similar across pain conditions. Patients receiving acupuncture had less pain, with scores that were 0.23 (95% CI, 0.13-0.33), 0.16 (95% CI, 0.07-0.25), and 0.15 (95% CI, 0.07-0.24) SDs lower than sham controls for back and neck pain, osteoarthritis, and chronic headache, respectively; the effect sizes in comparison to no-acupuncture controls were 0.55 (95% CI, 0.51-0.58), 0.57 (95% CI, 0.50-0.64), and 0.42 (95% CI, 0.37-0.46) SDs. These results were robust to a variety of sensitivity analyses, including those related to publication bias. CONCLUSIONS Acupuncture is effective for the treatment of chronic pain and is therefore a reasonable referral option. Significant differences between true and sham acupuncture indicate that acupuncture is more than a placebo. However, these differences are relatively modest, suggesting that factors in addition to the specific effects of needling are important contributors to the therapeutic effects of acupuncture.

Statistical Regression With Measurement Error

Abstract: Thank you very much for reading statistical regression with measurement error. As you may know, people have look numerous times for their favorite readings like this statistical regression with measurement error, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they cope with some harmful virus inside their computer. statistical regression with measurement error is available in our book collection an online access to it is set as public so you can get it instantly. Our digital library hosts in multiple countries, allowing you to get the most less latency time to download any of our books like this one. Merely said, the statistical regression with measurement error is universally compatible with any devices to read.

Discovering Invariant Rationales for Graph Neural Networks

Abstract: Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features -- rationale -- which guides the model prediction. Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns. Moreover, such data biases easily change outside the training distribution. As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data. In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs. It conducts interventions on the training distribution to create multiple interventional distributions. Then it approaches the causal rationales that are invariant across different distributions while filtering out the spurious patterns that are unstable. Experiments on both synthetic and real-world datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classification over the leading baselines. Code and datasets are available at https://github.com/Wuyxin/DIR-GNN.

Handling Distribution Shifts on Graphs: An Invariance Perspective

Abstract: There is increasing evidence suggesting neural networks' sensitivity to distribution shifts, so that research on out-of-distribution (OOD) generalization comes into the spotlight. Nonetheless, current endeavors mostly focus on Euclidean data, and its formulation for graph-structured data is not clear and remains under-explored, given two-fold fundamental challenges: 1) the inter-connection among nodes in one graph, which induces non-IID generation of data points even under the same environment, and 2) the structural information in the input graph, which is also informative for prediction. In this paper, we formulate the OOD problem on graphs and develop a new invariant learning approach, Explore-to-Extrapolate Risk Minimization (EERM), that facilitates graph neural networks to leverage invariance principles for prediction. EERM resorts to multiple context explorers (specified as graph structure editers in our case) that are adversarially trained to maximize the variance of risks from multiple virtual environments. Such a design enables the model to extrapolate from a single observed environment which is the common case for node-level prediction. We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution and further demonstrate its power on various real-world datasets for handling distribution shifts from artificial spurious features, cross-domain transfers and dynamic graph evolution.