Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

On robust estimation of the location parameter

KEGG(Kyoto Encyclopedia of Genes and Genomes)〔和文〕 (特集 ゲノム医学の現在と未来--基礎と臨床) -- (データベース)

Statistical Foundations of Data ScienceNew Perspectives and Challenges in Econophysics and SociophysicsStatistical Learning with SparsityMathematical Foundations of Infinite-Dimensional Statistical ModelsStatistics and Analysis of ShapesMultivariate StatisticsHigh-Dimensional Covariance EstimationPrinciples and Methods for Data ScienceHigh-Dimensional StatisticsHigh-Dimensional ProbabilityHandbook of Financial Econometrics and StatisticsStatistics for High-Dimensional DataRegularization in High-dimensional StatisticsAnalysis of Multivariate and High-Dimensional DataProbability and ComputingLarge Sample Covariance Matrices and High-Dimensional Data AnalysisHigh-Dimensional Data Analysis in Cancer ResearchHandbook of Big Data AnalyticsIntroduction to Clustering Large and High-Dimensional DataAnalyzing High-Dimensional Gene Expression and DNA Methylation Data with RSufficient Dimension ReductionHandbook of Big DataHandbook of Mixture AnalysisHigh Dimensional Probability VIStatistical Analysis for High-Dimensional DataHigh-Dimensional ProbabilitySpectral Analysis of Large Dimensional Random MatricesBig Data AnalyticsIntroduction to High-Dimensional StatisticsGeometric Structure of High-Dimensional Data and Dimensionality ReductionApplied Biclustering Methods for Big and High-Dimensional Data Using RBig and Complex Data AnalysisHigh-dimensional Data AnalysisFunctional Statistics and Related FieldsHandbook of Data VisualizationNonlinear Dimensionality ReductionModern Statistics for Modern BiologySparse Modeling for Image and Vision ProcessingModeling and Stochastic Learning for Forecasting in High DimensionsContributions to Fault Detection and Diagnosis with High-dimensional Data

Statistics for High-Dimensional Data: Methods, Theory and Applications

We perform a comparative analysis of machine learning methods for the canonical problem of empirical asset pricing: measuring asset risk premia. We demonstrate large economic gains to investors using machine learning forecasts, in some cases doubling the performance of leading regression-based strategies from the literature. We identify the best performing methods (trees and neural networks) and trace their predictive gains to allowance of nonlinear predictor interactions that are missed by other methods. All methods agree on the same set of dominant predictive signals which includes variations on momentum, liquidity, and volatility. Improved risk premium measurement through machine learning simplifies the investigation into economic mechanisms of asset pricing and highlights the value of machine learning in financial innovation.

/pdf/empirical-asset-pricing-via-machine-learning-vq3ztficgc.pdf

Empirical Asset Pricing via Machine Learning

Data subject to heavy-tailed errors are commonly encountered in various scientific fields. To address this problem, procedures based on quantile regression and Least Absolute Deviation (LAD) regression have been developed in recent years. These methods essentially estimate the conditional median (or quantile) function. They can be very different from the conditional mean functions, especially when distributions are asymmetric and heteroscedastic. How can we efficiently estimate the mean regression functions in ultra-high dimensional setting with existence of only the second moment? To solve this problem, we propose a penalized Huber loss with diverging parameter to reduce biases created by the traditional Huber loss. Such a penalized robust approximate quadratic (RA-quadratic) loss will be called RA-Lasso. In the ultra-high dimensional setting, where the dimensionality can grow exponentially with the sample size, our results reveal that the RA-lasso estimator produces a consistent estimator at the same rate as the optimal rate under the light-tail situation. We further study the computational convergence of RA-Lasso and show that the composite gradient descent algorithm indeed produces a solution that admits the same optimal rate after sufficient iterations. As a byproduct, we also establish the concentration inequality for estimating population mean when there exists only the second moment. We compare RA-Lasso with other regularized robust estimators based on quantile regression and LAD regression. Extensive simulation studies demonstrate the satisfactory finite-sample performance of RA-Lasso.

Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions

A progression of advanced statistical methods is applied to investigate the dependence of the 6-h tropical cyclone (TC) intensity change on various environmental variables, including the recently developed ventilation index (VI). The North Atlantic (NA) and western North Pacific (WNP) observations from 1979 to 2014 are used. As a first step, a model of the intensity change is developed as a linear function of 13 variables used in operational models, obtaining statistical R2 values of 0.26 for NA and 0.3 for WNP. Statistical variable selection techniques are then applied to significantly reduce the number of predictors (to 5–11), while keeping similar R2 values with linear or nonlinear models. Further reduction of the number of predictors (to 5–7) and significant improvement of R2 (0.41–0.53) are obtained with mixture modeling, indicating that the dependence of TC intensification on the environment is nonhomogeneous. Applying VI as the environmental predictor in the mixture modeling gives R2 result...

A Statistical Investigation of the Dependence of Tropical Cyclone Intensity Change on the Surrounding Environment

Factor modeling is an essential tool for exploring intrinsic dependence structures among high-dimensional random variables Much progress has been made for estimating the covariance matrix from a high-dimensional factor model However, the blessing of dimensionality has not yet been fully embraced in the literature: much of the available data are often ignored in constructing covariance matrix estimates If our goal is to accurately estimate a covariance matrix of a set of targeted variables, shall we employ additional data, which are beyond the variables of interest, in the estimation? In this article, we provide sufficient conditions for an affirmative answer, and further quantify its gain in terms of Fisher information and convergence rate In fact, even an oracle-like result (as if all the factors were known) can be achieved when a sufficiently large number of variables is used The idea of using data as much as possible brings computational challenges A divide-and-conquer algorithm is thus proposed to alleviate the computational burden, and also shown not to sacrifice any statistical accuracy in comparison with a pooled analysis Simulation studies further confirm our advocacy for the use of full data, and demonstrate the effectiveness of the above algorithm Our proposal is applied to a microarray data example that shows empirical benefits of using more data Supplementary materials for this article are available online

/pdf/embracing-the-blessing-of-dimensionality-in-factor-models-4v46xwt52i.pdf

Embracing the Blessing of Dimensionality in Factor Models.

Data subject to heavy-tailed errors are commonly encountered in various scientific fields, especially in the modern era with explosion of massive data. To address this problem, procedures based on quantile regression and Least Absolute Deviation (LAD) regression have been devel- oped in recent years. These methods essentially estimate the conditional median (or quantile) function. They can be very different from the conditional mean functions when distributions are asymmetric and heteroscedastic. How can we efficiently estimate the mean regression functions in ultra-high dimensional setting with existence of only the second moment? To solve this problem, we propose a penalized Huber loss with diverging parameter to reduce biases created by the traditional Huber loss. Such a penalized robust approximate quadratic (RA-quadratic) loss will be called RA-Lasso. In the ultra-high dimensional setting, where the dimensionality can grow exponentially with the sample size, our results reveal that the RA-lasso estimator produces a consistent estimator at the same rate as the optimal rate under the light-tail situation. We further study the computational convergence of RA-Lasso and show that the composite gradient descent algorithm indeed produces a solution that admits the same optimal rate after sufficient iterations. As a byproduct, we also establish the concentration inequality for estimat- ing population mean when there exists only the second moment. We compare RA-Lasso with other regularized robust estimators based on quantile regression and LAD regression. Extensive simulation studies demonstrate the satisfactory finite-sample performance of RA-Lasso.

Robust Estimation of High-Dimensional Mean Regression

As multi-task models gain popularity in a wider range of machine learning applications, it is becoming increasingly important for practitioners to understand the fairness implications associated with those models. Most existing fairness literature focuses on learning a single task more fairly, while how ML fairness interacts with multiple tasks in the joint learning setting is largely under-explored. In this paper, we are concerned with how group fairness (e.g., equal opportunity, equalized odds) as an ML fairness concept plays out in the multi-task scenario. In multi-task learning, several tasks are learned jointly to exploit task correlations for a more efficient inductive transfer. This presents a multi-dimensional Pareto frontier on (1) the trade-off between group fairness and accuracy with respect to each task, as well as (2) the trade-offs across multiple tasks. We aim to provide a deeper understanding on how group fairness interacts with accuracy in multi-task learning, and we show that traditional approaches that mainly focus on optimizing the Pareto frontier of multi-task accuracy might not perform well on fairness goals. We propose a new set of metrics to better capture the multi-dimensional Pareto frontier of fairness-accuracy trade-offs uniquely presented in a multi-task learning setting. We further propose a Multi-Task-Aware Fairness (MTA-F) approach to improve fairness in multi-task learning. Experiments on several real-world datasets demonstrate the effectiveness of our proposed approach.

Yuyan Wang

Papers

Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions

A Statistical Investigation of the Dependence of Tropical Cyclone Intensity Change on the Surrounding Environment

Embracing the Blessing of Dimensionality in Factor Models.

Robust Estimation of High-Dimensional Mean Regression

Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning