Showing papers on "Spark (mathematics) published in 2022"

Scan-specific artifact reduction in k-space (SPARK) neural networks synergize with physics-based reconstruction to accelerate MRI.

Abstract: Purpose To develop a scan-specific model that estimates and corrects k-space errors made when reconstructing accelerated MRI data. Methods Scan-specific artifact reduction in k-space (SPARK) trains a convolutional-neural-network to estimate and correct k-space errors made by an input reconstruction technique by back-propagating from the mean-squared-error loss between an auto-calibration signal (ACS) and the input technique's reconstructed ACS. First, SPARK is applied to generalized autocalibrating partially parallel acquisitions (GRAPPA) and demonstrates improved robustness over other scan-specific models, such as robust artificial-neural-networks for k-space interpolation (RAKI) and residual-RAKI. Subsequent experiments demonstrate that SPARK synergizes with residual-RAKI to improve reconstruction performance. SPARK also improves reconstruction quality when applied to advanced acquisition and reconstruction techniques like 2D virtual coil (VC-) GRAPPA, 2D LORAKS, 3D GRAPPA without an integrated ACS region, and 2D/3D wave-encoded imaging. Results SPARK yields SSIM improvement and 1.5 - 2× root mean squared error (RMSE) reduction when applied to GRAPPA and improves robustness to ACS size for various acceleration rates in comparison to other scan-specific techniques. When applied to advanced reconstruction techniques such as residual-RAKI, 2D VC-GRAPPA and LORAKS, SPARK achieves up to 20% RMSE improvement. SPARK with 3D GRAPPA also improves RMSE performance by ~2×, SSIM performance, and perceived image quality without a fully sampled ACS region. Finally, SPARK synergizes with non-Cartesian, 2D and 3D wave-encoding imaging by reducing RMSE between 20% and 25% and providing qualitative improvements. Conclusion SPARK synergizes with physics-based acquisition and reconstruction techniques to improve accelerated MRI by training scan-specific models to estimate and correct reconstruction errors in k-space.

Optimizing Network Transfers for Data Analytic Jobs Across Geo-Distributed Datacenters

Abstract: It has become a recent trend that large volumes of data are generated, stored, and processed across geographically distributed datacenters. When popular data parallel frameworks, such as MapReduce and Spark, are employed to process such geo-distributed data, optimizing the network transfer in communication stages becomes increasingly crucial to application performance, as the inter-datacenter links have much lower bandwidth than intra-datacenter links. In this article, we focus on exploiting the flexibility of multi-path routing for inter-datacenter flows of data analytic jobs, with the hope of better utilizing inter-datacenter links and thus improve job performance. We design an optimal multi-path routing and scheduling strategy to achieve the best possible network performance for all concurrent jobs, based on our formulation of an optimization problem that can be transformed into an equivalent linear programming (LP) problem to be efficiently solved. As a highlight of this article, we have implemented our proposed algorithm in the controller of an application-layer software-defined inter-datacenter overlay testbed, designed to provide transfer optimization service for Spark jobs. With extensive evaluations of our real-world implementation on Google Cloud, we have shown convincing evidence that our optimal multi-path routing and scheduling strategies have achieved significant improvements in terms of job performance.

Towards Spark-Based Deep Learning Approach for Fraud Detection Analysis

Abstract: Fraud activity is a major concern in the telecommunication business domain. Thus, it’s very important to analyze the huge amount of data available in the network in order to detect early potential fraud behaviors and take countermeasures accordingly. In this paper, we used a novel approach using deep learning techniques to analyze datasets from a real mobile communication career and to extract and analyze features from labeled/unlabeled data. Our dataset was extracted from the business support system (BSS) containing 2.5 million call details records (CDR) of active subscribers. The proposed method integrates deep learning techniques with Apache Spark framework for parallel training execution. We found that our approach performed better than traditional machine learning with an F1 score of 91% and enhance the training speed of our deep learning model significantly when using the spark platform. Thus, the use of this proposed model can remarkably decrease the cost related to the unauthorized use of telecommunication services.