Swarm intelligence algorithms are a subset of the artificial intelligence (AI) field, which is increasing popularity in resolving different optimization problems and has been widely utilized in various applications. In the past decades, numerous swarm intelligence algorithms have been developed, including ant colony optimization (ACO), particle swarm optimization (PSO), artificial fish swarm (AFS), bacterial foraging optimization (BFO), and artificial bee colony (ABC). This review tries to review the most representative swarm intelligence algorithms in chronological order by highlighting the functions and strengths from 127 research literatures. It provides an overview of the various swarm intelligence algorithms and their advanced developments, and briefly provides the description of their successful applications in optimization problems of engineering fields. Finally, opinions and perspectives on the trends and prospects in this relatively new research domain are represented to support future developments.

/pdf/a-review-on-representative-swarm-intelligence-algorithms-for-4f005x62n2.pdf

A Review on Representative Swarm Intelligence Algorithms for Solving Optimization Problems: Applications and Trends

Swarm intelligence (SI), an integral part in the field of artificial intelligence, is gradually gaining prominence, as more and more high complexity problems require solutions which may be sub-optimal but yet achievable within a reasonable period of time. Mostly inspired by biological systems, swarm intelligence adopts the collective behaviour of an organized group of animals, as they strive to survive. This study aims to discuss the governing idea, identify the potential application areas and present a detailed survey of eight SI algorithms. The newly developed algorithms discussed in the study are the insect-based algorithms and animal-based algorithms in minute detail. More specifically, we focus on the algorithms inspired by ants, bees, fireflies, glow-worms, bats, monkeys, lions and wolves. The inspiration analyses on these algorithms highlight the way these algorithms operate. Variants of these algorithms have been introduced after the inspiration analysis. Specific areas for the application of such algorithms have also been highlighted for researchers interested in the domain. The study attempts to provide an initial understanding for the exploration of the technical aspects of the algorithms and their future scope by the academia and practice.

Swarm Intelligence: A Review of Algorithms

This tutorial clarifies the axiomatic definition of (v(α); i(β)) circuit elements via a lookup table dubbed an A-pad, of admissible (v; i) signals measured via Gedanken probing circuits. The (v(α); i(β)) elements are ordered via a complexity metric. Under this metric, the memristor emerges naturally as the fourth element, characterized by a state-dependent Ohm's law. A logical generalization to memristive devices reveals a common fingerprint consisting of a dense continuum of pinched hysteresis loops whose area decreases with the frequency ω and tends to a straight line as ω ~ ∞, for all bipolar periodic signals and for all initial conditions. This common fingerprint suggests that the term memristor be used hence-forth as a moniker for memristive devices.

The Fourth Element.

To determine the location of the potential dust source regions and to evaluate the dust emission frequencies over the arid and semiarid areas of China and Mongolia (35.5°N-47°N; 73°E-125°E), we established a map of Z 0 from the composition of protrusion coefficient (PC) derived from the POLDER-1 bidirectional reflectance distribution function (BRDF). Using a 1/4° x 1/4° resolution Z 0 data set, we derived a map of the 10-m erosion threshold wind velocity for the Chinese and Mongolian deserts. The retrieved erosion thresholds range from 7 m s -1 in the sandy deserts (Taklimakan, Badain Jaran, and Tengger deserts) to up to 20 m s -1 in the Gobi desert. They are comparable to the minimum wind velocity measured in meteorological stations during dust storms in the Taklimakan (6-8 m s -1 ) and in the Gobi desert (11 -20 m s -1 ). These erosion thresholds were combined with surface wind fields, soil moisture, and snow cover to simulate the dust emission frequencies of the eastern Asian deserts over 3 years (1997-1999). The simulations point out the Taklimakan desert and the deserts of northern China as the most frequent sources of dust emissions. The simulated seasonal cycle is characterized by a maximum in late spring and a minimum in late autumn and winter. The comparison with climatologic studies of dust storms derived from synoptic observations confirms the importance of these two source areas and the reliability of the simulated seasonal cycle. It reveals an underestimation of the dust emission frequency in the Gobi desert, because of a low frequency of high wind velocities. Our results also suggest that soil moisture and snow cover are not the main factors controlling the seasonal cycle or the interannual variability of the dust emission frequencies. We finally compared the simulated dust event frequencies to occurrences of Total Ozone Mapping Spectrometer (TOMS) Absorbing Aerosol Index (AAI) higher than 0.7 over the Taklimakan desert, where mineral dust is expected to be the dominant absorbing aerosol. A very good agreement is obtained between the simulated frequencies and the TOMS AAI frequencies on monthly and seasonal timescales.

Simulation of the mineral dust emission frequencies from desert areas of China and Mongolia using an aerodynamic roughness length map derived from the POLDER/ADEOS 1 surface products : Quantifying the radiative and biogeochemical impacts of mineral dust

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

On the Opportunities and Risks of Foundation Models.

This paper introduces a new optimization technique termed as Recursive Ant Colony Optimization (RACO), a modified form of ant colony method (ACO), for finding best probable solution to a combinatorial problem. ACO simulates the social behavior of ants, optimizing their path from the nest to food source. The movement of an ant is random and the shortest path is found on the basis of the pheromone laid on the path by other ants. RACO applies ACO recursively introducing an additional term ‘depth’ which decides the extent of recursion. Each depth is a usual ACO with three steps per iteration, ‘pheromone tracking’, ‘pheromone updating’ and ‘city selection’. Results of each depth contribute towards constructing models for the following depth and the range of values for each parameter is reduced around the actual solution. The algorithm is tested on two simple functions and to further test its efficiency and stability in real world, it has been applied to a geophysical problem of self-potential anomaly due to a inclined sheet like body buried inside the earth. The results are found to be very good and they describe the effectiveness and practicability of this method.

Recursive Ant Colony Optimization for estimation of parameters of a function

In the era of pre-trained language models, Transformers are the de facto choice of model architectures. While recent research has shown promise in entirely convolutional, or CNN, architectures, they have not been explored using the pre-train-fine-tune paradigm. In the context of language models, are convolutional models competitive to Transformers when pre-trained? This paper investigates this research question and presents several interesting findings. Across an extensive set of experiments on 8 datasets/tasks, we find that CNN-based pre-trained models are competitive and outperform their Transformer counterpart in certain scenarios, albeit with caveats. Overall, the findings outlined in this paper suggest that conflating pre-training and architectural advances is misguided and that both advances should be considered independently. We believe our research paves the way for a healthy amount of optimism in alternative architectures.

Are Pre-trained Convolutions Better than Pre-trained Transformers?

State-of-the-art models in natural language processing rely on separate rigid subword tokenization algorithms, which limit their generalization ability and adaptation to new settings. In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model. To this end, we introduce a soft gradient-based subword tokenization module (GBST) that automatically learns latent subword representations from characters in a data-driven fashion. Concretely, GBST enumerates candidate subword blocks and learns to score them in a position-wise fashion using a block scoring network. We additionally introduce Charformer, a deep Transformer model that integrates GBST and operates on the byte level. Via extensive experiments on English GLUE, multilingual, and noisy text datasets, we show that Charformer outperforms a series of competitive byte-level baselines while generally performing on par and sometimes outperforming subword-based models. Additionally, Charformer is fast, improving the speed of both vanilla byte-level and subword-level Transformers by 28%-100% while maintaining competitive quality. We believe this work paves the way for highly performant token-free models that are trained completely end-to-end.

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization.

Spell correction is a must-have feature for any modern search engine in applications such as web or e-commerce search. Typical spell correction solutions used in production systems consist of large indexed lookup tables based on a global model trained across many users over a large scale web corpus or a query log. For search over personal corpora, such as email, this global solution is not sufficient, as it ignores the user's personal lexicon. Without personalization, global spelling fails to correct tail queries drawn from a user's own, often idiosyncratic, lexicon. Personalization using existing algorithms is difficult due to resource constraints and unavailability of sufficient data to build per-user models. In this work, we propose a simple and effective personalized spell correction solution that augments existing global solutions for search over private corpora. Our event driven spell correction candidate generation method is specifically designed with personalization as the key construct. Our novel spell correction and query completion algorithms do not require complex model training and is highly efficient. The proposed solution has shown over 30% click-through rate gain on affected queries when evaluated against a range of strong commercial personal search baselines - Google's Gmail, Drive, and Calendar search production systems.

https://dl.acm.org/doi/pdf/10.1145/3308558.3313706

Personalized Online Spell Correction for Personal Search

We introduce Recursive Ant Colony Optimization (RACO), a new technique for the estimation of function parameters from field data obtained from various geophysical surveys. RACO is a modified form of the ant colony method (ACO) and can be used to determine the best solution for various geophysical problems. As RACO is an extension of ACO, it simulates the social behaviour of ants, optimizing their path from the nest to the food source. RACO applies ACO recursively, introducing an additional term ‘depth’ that decides the extent of the recursion. The results of each depth contribute towards the construction of models for the following depth and the range of values for each parameter is condensed around the actual solution. The algorithm is tested on a two parameter mathematical function and to further test its efficiency and stability on real world problems, we applied it on a few geophysical inversion problems. We show its application on synthetic data sets for the self-potential anomaly (noise free and 10% noise) caused in the case of an inclined sheet like a body buried inside the earth, synthetic 1D vertical electrical sounding (VES) geo-electric data sets for three-layer and five-layer earth models, a real 1D VES data set obtained from the Tangasol region, India and P-wave logs for two sites located in the Krishna-Godavari (KG) Basin, India. The results generated by RACO inversions are found to be in good agreement with previous studies conducted using these data sets.

Jai Gupta

Papers

Recursive Ant Colony Optimization for estimation of parameters of a function

Are Pre-trained Convolutions Better than Pre-trained Transformers?

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization.

Personalized Online Spell Correction for Personal Search

Recursive ant colony optimization: a new technique for the estimation of function parameters from geophysical field data