scispace - formally typeset
Search or ask a question

How to reset spark context? 

Spark (mathematics)
Abstraction (linguistics)
Data modeling
Decision tree
Data structure

Answers from top 10 papers

More filters
Papers (10)Insight
Proceedings Article•DOI
A Novel Method for Tuning Configuration Parameters of Spark Based on Machine Learning
Guolu Wang, Jungang Xu, Ben He 
01 Dec 2016
79 Citations
This method can be used to auto-tune the configuration parameters of Spark.
Proceedings Article•DOI
GPU in-memory processing using Spark for iterative computation
Sumin Hong, Woohyuk Choi, Won-Ki Jeong 
14 May 2017
14 Citations
Having discovered that the main bottleneck in the current Spark system for GPU computing is data communication on a Java virtual machine, we propose a modification of the current Spark implementation to bypass expensive data management for iterative task offloading to GPUs.
Open access•Journal Article•DOI
Combustion variation control strategy with thermal efficiency optimization for lean combustion in spark-ignition engines
Zidan Xu, Yahui Zhang, Huanyu Di, Tielong Shen 
01 Oct 2019-Applied Energy
23 Citations
In addition, the self-optimization performance of spark timing under environmental changes is proven to be effective.
Proceedings Article•DOI
A Novel Method for Tuning Configuration Parameters of Spark Based on Machine Learning
Guolu Wang, Jungang Xu, Ben He 
01 Dec 2016
79 Citations
In this paper, in order to make the parameter tuning process of Spark more effective, a novel method for tuning configuration of Spark based on machine learning is proposed, which is composed of binary classification and multi-classification.
Proceedings Article•DOI
High-performance design of apache spark with RDMA and its benefits on various workloads
Xiaoyi Lu, Dipti Shankar, Shashank Gugnani, Dhabaleswar K. Panda 
01 Dec 2016
54 Citations
Our RDMA-based Spark design is implemented as a pluggable module and it does not change any Spark APIs, which means that it can be combined with other existing enhanced designs for Apache Spark and Hadoop proposed in the community.
Journal Article•DOI
Deep Learning Frameworks on Apache Spark: A Review
Nikitha Johnsirani Venkatesan, Choon-Sung Nam, Dong-Ryeol Shin 
04 Mar 2019-Iete Technical Review
14 Citations
Spark has an advantage of in-memory and fast processing of data.
Open access•Proceedings Article•DOI
Parallelizing ListNet training using spark
Shilpa Shukla, Matthew Lease, Ambuj Tewari 
12 Aug 2012
14 Citations
Unlike MapReduce, Spark is especially suited for iterative and interactive algorithms.
Book Chapter•DOI
Distributed Sequential Pattern Mining in Large Scale Uncertain Databases
Jiaqi Ge, Yuni Xia 
19 Apr 2016
16 Citations
Directly applying the DP method to Spark is impractical because its memory-consuming characteristic may cause heavy JVM garbage collection overhead in Spark.
Open access•Journal Article•DOI
A Mathematical Analysis of the Generation and Termination of Calcium Sparks
Robert Hinch
01 Mar 2004-Biophysical Journal
78 Citations
The analysis calculates both spark frequencies and spark duration distributions, and shows under what circumstances stochastic transitions are important.
Open access•Journal Article•DOI
Mammoth: Gearing Hadoop Towards Memory-Intensive MapReduce Applications
Xuanhua Shi, Chen Ming, Ligang He, Xu Xie, Lu Lu, Hai Jin, Yong Chen, Song Wu 
01 Aug 2015-IEEE Transactions on Parallel and Distributed Systems
38 Citations
Although Spark can achieve better performance than Mammoth for interactive and iterative applications when the memory is sufficient, our experimental results show that for batch processing applications, Mammoth can adapt better to various memory environments and outperform Spark when the memory is insufficient, and can obtain similar performance as Spark when the memory is sufficient.

Related Questions

How do I change my twitter context?10 answers
How do I reset Internet Explorer as my default browser?10 answers
How do I reset my Kidde carbon monoxide detector?5 answers
How to Clear Spring security context?10 answers
How to change database context in SQL Server?10 answers
How to reset cannabinoid receptors faster?10 answers

See what other people are reading

What are the factors that affect the frequent use of gadgets among humss students?
5 answers
The frequent use of gadgets among HUMSS (Humanities and Social Sciences) students can be influenced by various factors. Factors contributing to this phenomenon include the appeal of smartphone features for accessing the internet and social media, entertainment purposes, and aiding in school assignments. Additionally, the addictive nature of electronic gadgets, especially smartphones, can lead to increased usage among students, impacting their mental and physical health. Furthermore, low self-control, feelings of loneliness, and sensation-seeking behavior are reasons why teenagers may excessively use gadgets. It is essential for parents, teachers, government agencies, and society to play pivotal roles in guiding students towards balanced gadget usage to prevent negative consequences on academic performance and overall well-being.What is unemployment?
5 answers
Unemployment is a significant economic and social issue characterized by individuals unable to secure employment despite their willingness to work at prevailing wage levels. It leads to adverse effects on both the economy and individuals, causing poverty, depression, and health problems. Various factors contribute to unemployment, such as lack of awareness about job opportunities, mismatch between skilled labor and available positions, and external events like the COVID-19 pandemic. Studies have explored the relationship between unemployment and economic growth, with findings indicating a stable but complex dynamic. Research has also utilized regression analysis and machine learning algorithms to identify regions with low employment rates, aiming to guide policy interventions for enhancing job opportunities and reducing unemployment.Are there any potential negative effects of gadgets on students' academic performance that should be considered?
5 answers
The impact of gadgets on students' academic performance can have both positive and negative aspects. Research suggests that excessive use of electronic devices, such as smartphones and laptops, during study time and lectures can negatively affect academic performance. Additionally, the addictive nature of electronic gadgets among students can lead to distractions, affecting mental and physical health, ultimately impacting academic success. While gadgets can be beneficial for learning, their misuse, like using them for non-educational purposes or excessive screen time, may hinder academic progress. It is crucial to promote positive gadget use and implement regulations to mitigate the potential negative effects on students' academic performance.Each food's effect for human body depend on weather condition?
4 answers
The effects of food on the human body can be influenced by weather conditions. Climate and weather play crucial roles in food production, affecting farming outcomes and food availability. In rural Uganda, higher temperatures have been found to have an adverse impact on food consumption, leading to a decline in crop yields and overall food intake. Additionally, a specific food designed to condition human metabolism includes ingredients like black sesame seeds, black beans, and peanuts, aiming to strengthen various bodily functions and prevent diseases, showcasing how food composition can be tailored to counteract specific health issues. Therefore, weather conditions can indirectly impact the efficacy of certain foods on the human body by influencing agricultural productivity and nutritional intake.What is M3Fair?
5 answers
M3Fair is a novel bias mitigation method proposed to address limitations in existing methods like Reweighting (RW) in healthcare AI. It extends the RW method to multiple sensitive attributes at multiple levels, aiming to tackle intersectional bias more effectively. By considering various sensitive attributes such as race, gender, age, and medical condition, M3Fair enhances the fairness and equity in healthcare data analysis. The method is designed to detect and mitigate bias in healthcare datasets, ultimately improving the quality of care received by individuals, especially from minority groups who often face disparities in medical treatment. M3Fair's effectiveness, simplicity, and generalizability have been demonstrated through experiments on real-world datasets, highlighting its potential to address healthcare fairness issues.How the excessive use of gadgets affect the students performance?
5 answers
The excessive use of gadgets, particularly smartphones, can significantly impact students' academic performance. Research indicates that smartphone addiction leads to increased procrastination, reduced self-regulation, and ultimately affects online academic performance negatively. Moreover, the dependency on electronic gadgets can have adverse effects on students' mental and physical health, leading to addiction and potential distractions from academic activities. Studies also show that the positive use of gadgets can enhance learning capabilities and contribute positively to academic performance, especially when regulated effectively. Machine learning techniques have been employed to analyze the addictiveness of gadgets among students and predict the impact on academic performance, highlighting the need for interventions to promote responsible gadget usage for improved educational outcomes.How is lionel messi that good?
5 answers
Lionel Messi's exceptional performance can be attributed to his unique motor skills and laterality in individual attacking actions resulting in goals. His use of lower limbs, foot contact zones, turn direction, wings, and body orientation towards the goal showcases his mastery in soccer. Moreover, Messi's abilities have made him a standout player, with studies emphasizing the significance of analyzing motor skills in soccer to understand what sets certain players apart. Additionally, Messi's influence extends beyond the field, serving as a role model for young athletes who aim to emulate his movements and aspire to replicate his success. By selecting the right role models and critically assessing and reflecting on the portrayal of athletes in the media, individuals can benefit from learning and practicing with digitalized idols to enhance their self-perception and skill development.How do open science frameworks compare to traditional data management systems in terms of accessibility, interoperability, and cost?
5 answers
Open science frameworks, as advocated in various research papers, emphasize accessibility, interoperability, and cost-effectiveness compared to traditional data management systems. These frameworks aim to make research data findable, accessible, interoperable, and reusable (FAIR principles). They promote sharing data widely across research domains, enhancing interoperability through core vocabulary sharing and adapting metadata schemata. Open science culture prioritizes transparency and broad accessibility of scholarly work, facilitating immediate sharing with a wide audience. In contrast, traditional data management systems may lack the flexibility and efficiency required for seamless data integration and sharing. By focusing on open science principles, frameworks can enhance data accessibility, promote interoperability through standardized practices, and potentially reduce costs associated with proprietary systems, thus offering a more efficient and collaborative approach to data management.Are ensemble learning methods good with complex decision boundaries?
4 answers
Ensemble learning methods excel in handling complex decision boundaries by combining multiple models to enhance predictive performance. Various approaches have been proposed to address this, such as modifying contrastive learning objectives to automatically train self-complementing ensembles, utilizing a novel voting model based on ensemble learning for threat detection in complex network environments, and designing a dynamic ensemble selection decision-making method to tackle class-imbalance and other data-complexity issues effectively. Additionally, a novel ensemble method called BEBS leverages ensemble models to capture robust decision boundaries in imbalanced data learning scenarios. Furthermore, an ensemble learning method based on deep neural networks and group decision making has been developed to generate diverse individual learners and combine them effectively for improved classification performance.How does ml is used in cancer?
5 answers
Machine learning (ML) is extensively utilized in cancer research for various applications. ML models have been developed to predict cancer types and stages, determine IDH and ATRX mutation status in gliomas, and classify cancer based on expression data for visualization and dimension reduction. These models employ supervised learning techniques like Decision Tree, Random Forest, and XG Boost algorithms to achieve high accuracy and sensitivity in cancer detection. Additionally, novel ML techniques have been introduced to incorporate class memberships, enabling efficient classification of unseen data and improved cancer prediction. AI techniques are also employed for early detection and accurate prediction of oral cancer progression. Overall, ML plays a crucial role in enhancing cancer diagnostics, treatment decisions, and prognosis.What are the trends in software and programming?
4 answers
Software engineering and programming are witnessing various trends in the modern technological landscape. These trends include the integration of new technologies like Cloud Computing, machine learning, deep learning, big Data, and mobile Computing into software development processes. The constant evolution of programming languages to address the challenges posed by multi-core processors, cloud programming, and portable devices has become a significant trend in the field. Moreover, the software industry is experiencing a shift towards cutting-edge technology due to the increasing demand for cloud-based solutions, leading to a surge in software development market growth. Additionally, the quality of software engineering education is being influenced by modern industrial trends, IT trends, and software development methodologies, especially in the context of Industry 4.0, which introduces new challenges and the need for updated educational approaches.