Geographically neural network weighted regression for the accurate estimation of spatial non-stationarity
03 Jan 2020-International Journal of Geographical Information Science (Taylor & Francis)-Vol. 34, Iss: 7, pp 1353-1377
TL;DR: A geographically neural network weighted regression model that combines ordinary least squares (OLS) and neural networks to estimate spatial non-stationarity based on a concept similar to GWR is proposed and achieved better fitting accuracy and more adequate prediction than OLS and GWR.
Abstract: Geographically weighted regression (GWR) is a classic and widely used approach to model spatial non-stationarity. However, the approach makes no precise expressions of its weighting kernels and is ...
Citations
More filters
TL;DR: In this paper , the authors use SHAP to interpret XGBoost (eXtreme Gradient Boosting) as an example to demonstrate how to extract spatial effects from machine learning models.
38Â citations
TL;DR: GWANN combines geographical weighting with artificial neural networks, which are able to learn complex nonlinear relationships in a data-driven manner without assumptions, and shows that the performance of GWANN can also be superior in a practical setting.
Abstract: While recent developments have extended geographically weighted regression (GWR) in many directions, it is usually assumed that the relationships between the dependent and the independent variables
34Â citations
Cites methods from "Geographically neural network weigh..."
...Another extension is geographically neural network weighted regression (Du et al. 2020), which utilizes an artificial neural network (ANN) to find appropriate geographical weights when estimating the coefficients of a GWR model....
[...]
TL;DR: To address complex non-linear interactions between time and space, a spatiotemporal proximity neural network (STPNN) is proposed in this paper to accurately generate space-time distance and has the potential to handle complex spatiotmporal non-stationarity in various geographical processes and environmental phenomena.
Abstract: Geographically weighted regression (GWR) and geographically and temporally weighted regression (GTWR) are classic methods for estimating non-stationary relationships. Although these methods have be...
30Â citations
TL;DR: The proposed GTWNN model was proposed by integrating artificial neural network into geographically and temporally weighted regression (GTWR) using publicly available data sources, including satellite imagery and climate data to provide evidence for the existence of spatial non-stationarity and temporal non- stationarity in winter wheat yield prediction.
26Â citations
TL;DR: Wang et al. as mentioned in this paper developed a water quality assessment method based on a newly proposed geographically neural network weighted regression (GNNWR) model to address that challenge and obtained a highly accurate and realistic water quality distribution on the basis of the comprehensive index of Chinese Water Quality Classification Standards.
Abstract: The accurate assessment of large-scale and complex coastal waters is a grand challenge due to the spatial nonstationarity and complex nonlinearity involved in integrating remote sensing and in situ data. We developed a water quality assessment method based on a newly proposed geographically neural network weighted regression (GNNWR) model to address that challenge and obtained a highly accurate and realistic water quality distribution on the basis of the comprehensive index of Chinese Water Quality Classification Standards. Using geostationary ocean color imager (GOCI) data and observations from 1240 water quality sampling sites, we conducted experiments for a typical large-scale coastal area of the Zhejiang Coastal Sea (ZCS), People's Republic of China. The GNNWR model achieved higher prediction performance (average R2 = 84%) in comparison to the widely used models, and the obtained water quality classification (WQC) maps in May of 2015-2017 and August 2015 can depict intuitively reasonable spatiotemporal patterns of water quality in the ZCS. Furthermore, an analysis of WQC maps successfully illustrated how terrestrial discharges, anthropogenic activities, and seasonal changes influenced the coastal environment in the ZCS. Finally, we identified essential regions and provided targeted regulatory interventions for them to facilitate the management and restoration of large-scale and complex coastal environments.
15Â citations
References
More filters
Journal Article•
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Abstract: Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
33,597Â citations
"Geographically neural network weigh..." refers background in this paper
...The neural network adopts a fully connected layer and dropout technologies to enhance the generalization capability, as suggested by Srivastava et al. (2014)....
[...]
Proceedings Article•
06 Jul 2015TL;DR: Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
Abstract: Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.
30,843Â citations
"Geographically neural network weigh..." refers methods in this paper
...Moreover, we adopt the batch normalization technique proposed by Ioffe and Szegedy (2015) to reduce the influence of the internal covariate transformation problem so that the model can set a larger learning rate and further enhance the computing power of the GNNWR model....
[...]
Posted Content•
TL;DR: Batch Normalization as mentioned in this paper normalizes layer inputs for each training mini-batch to reduce the internal covariate shift in deep neural networks, and achieves state-of-the-art performance on ImageNet.
Abstract: Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.
17,184Â citations
Posted Content•
TL;DR: This work proposes a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit and derives a robust initialization method that particularly considers the rectifier nonlinearities.
Abstract: Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass human-level performance (5.1%, Russakovsky et al.) on this visual recognition challenge.
11,866Â citations
"Geographically neural network weigh..." refers methods in this paper
...In addition, we use the parameter initialization method and activation function developed by He et al. (2015) in each hidden layer to improve the optimization efficiency....
[...]
07 Dec 2015
TL;DR: In this paper, a Parametric Rectified Linear Unit (PReLU) was proposed to improve model fitting with nearly zero extra computational cost and little overfitting risk, which achieved a 4.94% top-5 test error on ImageNet 2012 classification dataset.
Abstract: Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on the learnable activation and advanced initialization, we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66% [33]). To our knowledge, our result is the first to surpass the reported human-level performance (5.1%, [26]) on this dataset.
11,732Â citations