scispace - formally typeset
Search or ask a question
Topic

Sigmoid function

About: Sigmoid function is a research topic. Over the lifetime, 2228 publications have been published within this topic receiving 59557 citations. The topic is also known as: S curve.


Papers
More filters
Journal ArticleDOI
TL;DR: Since the ultimate goal is accuracy in the prediction, it is found that sigmoid networks trained with the weight-elimination algorithm outperform traditional nonlinear statistical approaches.
Abstract: We investigate the effectiveness of connectionist architectures for predicting the future behavior of nonlinear dynamical systems. We focus on real-world time series of limited record length. Two examples are analyzed: the benchmark sunspot series and chaotic data from a computational ecosystem. The problem of overfitting, particularly serious for short records of noisy data, is addressed both by using the statistical method of validation and by adding a complexity term to the cost function ("back-propagation with weight-elimination"). The dimension of the dynamics underlying the time series, its Liapunov coefficient, and its nonlinearity can be determined via the network. We also show why sigmoid units are superior in performance to radial basis functions for high-dimensional input spaces. Furthermore, since the ultimate goal is accuracy in the prediction, we find that sigmoid networks trained with the weight-elimination algorithm outperform traditional nonlinear statistical approaches.

775 citations

Book ChapterDOI
07 Jun 1995
TL;DR: A variant sigmoid function with three parameters that denote the dynamic range, symmetry and slope of the function respectively is discussed to illustrate how these parameters influence the speed of backpropagation learning and a hybrid sigmoidal network with different parameter configuration in different layers is introduced.
Abstract: Sigmoid function is the most commonly known function used in feed forward neural networks because of its nonlinearity and the computational simplicity of its derivative. In this paper we discuss a variant sigmoid function with three parameters that denote the dynamic range, symmetry and slope of the function respectively. We illustrate how these parameters influence the speed of backpropagation learning and introduce a hybrid sigmoidal network with different parameter configuration in different layers. By regulating and modifying the sigmoid function parameter configuration in different layers the error signal problem, oscillation problem and asymmetrical input problem can be reduced. To compare the learning capabilities and the learning rate of the hybrid sigmoidal networks with the conventional networks we have tested the two-spirals benchmark that is known to be a very difficult task for backpropagation and their relatives.

729 citations

Journal ArticleDOI
TL;DR: Inflow of the dam reservoir in the 12 past months shows that ARIMA model had a less error compared with the ARMA model, and dynamic artificial neural network model was chosen as the best model for forecasting inflow of the Dez dam reservoir.

704 citations

Journal ArticleDOI
TL;DR: This study proposes two activation functions for neural network function approximation in reinforcement learning: the sigmoid-weighted linear unit (SiLU) and its derivative function (dSiLU), and suggests the more traditional approach of using on-policy learning with eligibility traces, instead of experience replay, and softmax action selection can be competitive with DQN, without the need for a separate target network.

696 citations

01 Jan 2005
TL;DR: This paper discusses non-PSD kernels through the viewpoint of separability, and shows that the sigmoid kernel matrix is conditionally positive definite (CPD) in certain parameters and thus are valid kernels there.
Abstract: The sigmoid kernel was quite popular for support vector machines due to its origin from neural networks. Although it is known that the kernel matrix may not be positive semi-definite (PSD), other properties are not fully studied. In this paper, we discuss such non-PSD kernels through the viewpoint of separability. Results help to validate the possible use of non-PSD kernels. One example shows that the sigmoid kernel matrix is conditionally positive definite (CPD) in certain parameters and thus are valid kernels there. However, we also explain that the sigmoid kernel is not better than the RBF kernel in general. Experiments are given to illustrate our analysis. Finally, we discuss how to solve the non-convex dual problems by SMO-type decomposition methods. Suitable modifications for any symmetric non-PSD kernel matrices are proposed with convergence proofs.

632 citations


Network Information
Related Topics (5)
Artificial neural network
207K papers, 4.5M citations
69% related
Deep learning
79.8K papers, 2.1M citations
68% related
Linear regression
21.3K papers, 1.2M citations
68% related
Convolutional neural network
74.7K papers, 2M citations
67% related
Sampling (statistics)
65.3K papers, 1.2M citations
67% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
2023253
2022674
2021121
2020158
2019167
2018134