Spark (mathematics)

About: Spark (mathematics) is a research topic. Over the lifetime, 7304 publications have been published within this topic receiving 63322 citations.

PAMAE: Parallel k -Medoids Clustering with High Accuracy and Efficiency

Abstract: The k-medoids algorithm is one of the best-known clustering algorithms. Despite this, however, it is not as widely used for big data analytics as the k-means algorithm, mainly because of its high computational complexity. Many studies have attempted to solve the efficiency problem of the k-medoids algorithm, but all such studies have improved efficiency at the expense of accuracy. In this paper, we propose a novel parallel k-medoids algorithm, which we call PAMAE, that achieves both high accuracy and high efficiency. We identify two factors---"global search" and "entire data"---that are essential to achieving high accuracy, but are also very time-consuming if considered simultaneously. Thus, our key idea is to apply them individually through two phases: parallel seeding and parallel refinement, neither of which is costly. The first phase performs global search over sampled data, and the second phase performs local search over entire data. Our theoretical analysis proves that this serial execution of the two phases leads to an accurate solution that would be achieved by global search over entire data. In order to validate the merit of our approach, we implement PAMAE on Spark as well as Hadoop and conduct extensive experiments using various real-world data sets on 12 Microsoft Azure machines (48 cores). The results show that PAMAE significantly outperforms most of recent parallel algorithms and, at the same time, produces a clustering quality as comparable as the previous most-accurate algorithm. The source code and data are available at https://github.com/jaegil/k-Medoid.

Electronic ignition system

Abstract: A distributorless electronic ignition system for replacing the entire traditional spark distributor system in an internal combustion engine First timing pulses, generated in a first pulse generator by electromagnetically sensing the magnetic discontinuities of one or more lobed discs fixed to the engine crankshaft, are coupled to a second pulse generator which generates a second timing signal The second timing signals are coupled to an electronic shaft means which generates a shaft position signal and shaft rate signal which are added to form a composite signal whose amplitude increases as the speed of the engine increases An advance signal pulse is initiated each time the composite signal amplitude increases to a value equal to a reference voltage defining the basic idle timing The pulse width of each advance signal pulse is then modified in a pulse width control circuit to form a spark enable signal having a time duration equal to the time duration desired for the sparks The spark enable signal as well as the first timing signal are combined in an electronic distributor to sequentially generate ignition spark signals on a plurality of leads, each coupled to spark plugs positioned in a pair of cylinders having the same relative position with one being in the combustion phase and the other being in the exhaust phase of the combustion cycle Spark modulation to provide multiple ignition sparks during each spark enable pulse is also provided by ANDing the spark enable pulse with a high frequency pulse signal from a signal generator

Towards Near-Real-Time Intrusion Detection for IoT Devices using Supervised Learning and Apache Spark

Abstract: In the fields of Internet of Things (IoT) infrastructures, attack and anomaly detection are rising concerns. With the increased use of IoT infrastructure in every domain, threats and attacks in these infrastructures are also growing proportionally. In this paper the performances of several machine learning algorithms in identifying cyber-attacks (namely SYN-DOS attacks) to IoT systems are compared both in terms of application performances, and in training/application times. We use supervised machine learning algorithms included in the MLlib library of Apache Spark, a fast and general engine for big data processing. We show the implementation details and the performance of those algorithms on public datasets using a training set of up to 2 million instances. We adopt a Cloud environment, emphasizing the importance of the scalability and of the elasticity of use. Results show that all the Spark algorithms used result in a very good identification accuracy (>99%). Overall, one of them, Random Forest, achieves an accuracy of 1. We also report a very short training time (23.22 sec for Decision Tree with 2 million rows). The experiments also show a very low application time (0.13 sec for over than 600,000 instances for Random Forest) using Apache Spark in the Cloud. Furthermore, the explicit model generated by Random Forest is very easy-to-implement using high- or low-level programming languages. In light of the results obtained, both in terms of computation times and identification performance, a hybrid approach for the detection of SYN-DOS cyber-attacks on IoT devices is proposed: the application of an explicit Random Forest model, implemented directly on the IoT device, along with a second level analysis (training) performed in the Cloud.

Chaotic Combustion in Spark Ignition Engines

Abstract: We analyse the combustion process in a spark ignition engine using the experimental data of an internal pressure during the combustion process and show that the system can be driven to chaotic behaviour. Our conclusion is based on the observation of unperiodicity in the time series, suitable stroboscopic maps and a complex structure of a reconstructed strange attractor. This analysis can explain that in some circumstances the level of noise in spark ignition engines increases considerably due to nonlinear dynamics of a combustion process.