Abstract: This paper presents an assessment of the implications of climate change for global river flood risk. It is based on the estimation of flood frequency relationships at a grid resolution of 0.5 × 0.5°, using a global hydrological model with climate scenarios derived from 21 climate models, together with projections of future population. Four indicators of the flood hazard are calculated; change in the magnitude and return period of flood peaks, flood-prone population and cropland exposed to substantial change in flood frequency, and a generalised measure of regional flood risk based on combining frequency curves with generic flood damage functions. Under one climate model, emissions and socioeconomic scenario (HadCM3 and SRES A1b), in 2050 the current 100-year flood would occur at least twice as frequently across 40 % of the globe, approximately 450 million flood-prone people and 430 thousand km2 of flood-prone cropland would be exposed to a doubling of flood frequency, and global flood risk would increase by approximately 187 % over the risk in 2050 in the absence of climate change. There is strong regional variability (most adverse impacts would be in Asia), and considerable variability between climate models. In 2050, the range in increased exposure across 21 climate models under SRES A1b is 31–450 million people and 59 to 430 thousand km2 of cropland, and the change in risk varies between −9 and +376 %. The paper presents impacts by region, and also presents relationships between change in global mean surface temperature and impacts on the global flood hazard. There are a number of caveats with the analysis; it is based on one global hydrological model only, the climate scenarios are constructed using pattern-scaling, and the precise impacts are sensitive to some of the assumptions in the definition and application.

Abstract: Background:High-quality epidemiologic data worldwide are needed to improve our understanding of disease risk, support health policy to meet the diverse needs of people with multiple sclerosis (MS) ...

Abstract: We present a simple and effective scheme for dependency parsing which is based on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with a BiLSTM vector representing the token in its sentential context, and feature vectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is trained jointly with the parser objective, resulting in very effective feature extractors for parsing. We demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser as well as to a globally optimized graph-based parser. The resulting parsers have very simple architectures, and match or surpass the state-of-the-art accuracies on English and Chinese.

Abstract: Summary Background Almost a third of patients with epilepsy have a treatment-resistant form, which is associated with severe morbidity and increased mortality. Cannabis-based treatments for epilepsy have generated much interest, but scientific data are scarce. We aimed to establish whether addition of cannabidiol to existing anti-epileptic regimens would be safe, tolerated, and efficacious in children and young adults with treatment-resistant epilepsy. Methods In this open-label trial, patients (aged 1–30 years) with severe, intractable, childhood-onset, treatment-resistant epilepsy, who were receiving stable doses of antiepileptic drugs before study entry, were enrolled in an expanded-access programme at 11 epilepsy centres across the USA. Patients were given oral cannabidiol at 2–5 mg/kg per day, up-titrated until intolerance or to a maximum dose of 25 mg/kg or 50 mg/kg per day (dependent on study site). The primary objective was to establish the safety and tolerability of cannabidiol and the primary efficacy endpoint was median percentage change in the mean monthly frequency of motor seizures at 12 weeks. The efficacy analysis was by modified intention to treat. Comparisons of the percentage change in frequency of motor seizures were done with a Mann-Whitney U test. Results Between Jan 15, 2014, and Jan 15, 2015, 214 patients were enrolled; 162 (76%) patients who had at least 12 weeks of follow-up after the first dose of cannabidiol were included in the safety and tolerability analysis, and 137 (64%) patients were included in the efficacy analysis. In the safety group, 33 (20%) patients had Dravet syndrome and 31 (19%) patients had Lennox-Gastaut syndrome. The remaining patients had intractable epilepsies of different causes and type. Adverse events were reported in 128 (79%) of the 162 patients within the safety group. Adverse events reported in more than 10% of patients were somnolence (n=41 [25%]), decreased appetite (n=31 [19%]), diarrhoea (n=31 [19%]), fatigue (n=21 [13%]), and convulsion (n=18 [11%]). Five (3%) patients discontinued treatment because of an adverse event. Serious adverse events were reported in 48 (30%) patients, including one death—a sudden unexpected death in epilepsy regarded as unrelated to study drug. 20 (12%) patients had severe adverse events possibly related to cannabidiol use, the most common of which was status epilepticus (n=9 [6%]). The median monthly frequency of motor seizures was 30·0 (IQR 11·0–96·0) at baseline and 15·8 (5·6–57·6) over the 12 week treatment period. The median reduction in monthly motor seizures was 36·5% (IQR 0–64·7). Interpretation Our findings suggest that cannabidiol might reduce seizure frequency and might have an adequate safety profile in children and young adults with highly treatment-resistant epilepsy. Randomised controlled trials are warranted to characterise the safety profile and true efficacy of this compound. Funding GW Pharmaceuticals, Epilepsy Therapy Project of the Epilepsy Foundation, Finding A Cure for Epilepsy and Seizures.

Abstract: This paper presents the Deep Convolution Inverse Graphics Network (DC-IGN), a model that learns an interpretable representation of images. This representation is disentangled with respect to transformations such as out-of-plane rotations and lighting variations. The DC-IGN model is composed of multiple layers of convolution and de-convolution operators and is trained using the Stochastic Gradient Variational Bayes (SGVB) algorithm. We propose a training procedure to encourage neurons in the graphics code layer to represent a specific transformation (e.g. pose or light). Given a single input image, our model can generate new images of the same object with variations in pose and lighting. We present qualitative and quantitative results of the model's efficacy at learning a 3D rendering engine.

Abstract: MTC are expected to play an essential role within future 5G systems. In the FP7 project METIS, MTC has been further classified into mMTC and uMTC. While mMTC is about wireless connectivity to tens of billions of machinetype terminals, uMTC is about availability, low latency, and high reliability. The main challenge in mMTC is scalable and efficient connectivity for a massive number of devices sending very short packets, which is not done adequately in cellular systems designed for human-type communications. Furthermore, mMTC solutions need to enable wide area coverage and deep indoor penetration while having low cost and being energy-efficient. In this article, we introduce the PHY and MAC layer solutions developed within METIS to address this challenge.

Abstract: The large number of potential applications from bridging web data with knowledge bases have led to an increase in the entity linking research. Entity linking is the task to link entity mentions in text with their corresponding entities in a knowledge base. Potential applications include information extraction, information retrieval, and knowledge base population. However, this task is challenging due to name variations and entity ambiguity. In this survey, we present a thorough overview and analysis of the main approaches to entity linking, and discuss various applications, the evaluation of entity linking systems, and future directions.

Abstract: Background The effects of coronavirus disease 2019 (COVID-19) on the population's mental health and well-being are likely to be profound and long lasting. Aims To investigate the trajectory of mental health and well-being during the first 6 weeks of lockdown in adults in the UK. Method A quota survey design and a sampling frame that permitted recruitment of a national sample was employed. Findings for waves 1 (31 March to 9 April 2020), 2 (10 April to 27 April 2020) and 3 (28 April to 11 May 2020) are reported here. A range of mental health factors was assessed: pre-existing mental health problems, suicide attempts and self-harm, suicidal ideation, depression, anxiety, defeat, entrapment, mental well-being and loneliness. Results A total of 3077 adults in the UK completed the survey at wave 1. Suicidal ideation increased over time. Symptoms of anxiety, and levels of defeat and entrapment decreased across waves whereas levels of depressive symptoms did not change significantly. Positive well-being also increased. Levels of loneliness did not change significantly over waves. Subgroup analyses showed that women, young people (18–29 years), those from more socially disadvantaged backgrounds and those with pre-existing mental health problems have worse mental health outcomes during the pandemic across most factors. Conclusions The mental health and well-being of the UK adult population appears to have been affected in the initial phase of the COVID-19 pandemic. The increasing rates of suicidal thoughts across waves, especially among young adults, are concerning.

Abstract: Importance Detailed information about the association of COVID-19 with outcomes in pregnant individuals compared with not-infected pregnant individuals is much needed. Objective To evaluate the risks associated with COVID-19 in pregnancy on maternal and neonatal outcomes compared with not-infected, concomitant pregnant individuals. Design, Setting, and Participants In this cohort study that took place from March to October 2020, involving 43 institutions in 18 countries, 2 unmatched, consecutive, not-infected women were concomitantly enrolled immediately after each infected woman was identified, at any stage of pregnancy or delivery, and at the same level of care to minimize bias. Women and neonates were followed up until hospital discharge. Exposures COVID-19 in pregnancy determined by laboratory confirmation of COVID-19 and/or radiological pulmonary findings or 2 or more predefined COVID-19 symptoms. Main Outcomes and Measures The primary outcome measures were indices of (maternal and severe neonatal/perinatal) morbidity and mortality; the individual components of these indices were secondary outcomes. Models for these outcomes were adjusted for country, month entering study, maternal age, and history of morbidity. Results A total of 706 pregnant women with COVID-19 diagnosis and 1424 pregnant women without COVID-19 diagnosis were enrolled, all with broadly similar demographic characteristics (mean [SD] age, 30.2 [6.1] years). Overweight early in pregnancy occurred in 323 women (48.6%) with COVID-19 diagnosis and 554 women (40.2%) without. Women with COVID-19 diagnosis were at higher risk for preeclampsia/eclampsia (relative risk [RR], 1.76; 95% CI, 1.27-2.43), severe infections (RR, 3.38; 95% CI, 1.63-7.01), intensive care unit admission (RR, 5.04; 95% CI, 3.13-8.10), maternal mortality (RR, 22.3; 95% CI, 2.88-172), preterm birth (RR, 1.59; 95% CI, 1.30-1.94), medically indicated preterm birth (RR, 1.97; 95% CI, 1.56-2.51), severe neonatal morbidity index (RR, 2.66; 95% CI, 1.69-4.18), and severe perinatal morbidity and mortality index (RR, 2.14; 95% CI, 1.66-2.75). Fever and shortness of breath for any duration was associated with increased risk of severe maternal complications (RR, 2.56; 95% CI, 1.92-3.40) and neonatal complications (RR, 4.97; 95% CI, 2.11-11.69). Asymptomatic women with COVID-19 diagnosis remained at higher risk only for maternal morbidity (RR, 1.24; 95% CI, 1.00-1.54) and preeclampsia (RR, 1.63; 95% CI, 1.01-2.63). Among women who tested positive (98.1% by real-time polymerase chain reaction), 54 (13%) of their neonates tested positive. Cesarean delivery (RR, 2.15; 95% CI, 1.18-3.91) but not breastfeeding (RR, 1.10; 95% CI, 0.66-1.85) was associated with increased risk for neonatal test positivity. Conclusions and Relevance In this multinational cohort study, COVID-19 in pregnancy was associated with consistent and substantial increases in severe maternal morbidity and mortality and neonatal complications when pregnant women with and without COVID-19 diagnosis were compared. The findings should alert pregnant individuals and clinicians to implement strictly all the recommended COVID-19 preventive measures.

Abstract: Magnetic skyrmions are promising candidates for future spintronic applications such as skyrmion racetrack memories and logic devices. They exhibit exotic and complex dynamics governed by topology and are less influenced by defects, such as edge roughness, than conventionally used domain walls. In particular, their non-zero topological charge leads to a predicted ‘skyrmion Hall effect’, in which current-driven skyrmions acquire a transverse velocity component analogous to charged particles in the conventional Hall effect. Here, we use nanoscale pump–probe imaging to reveal the real-time dynamics of skyrmions driven by current-induced spin–orbit torques. We find that skyrmions move at a well-defined angle ΘSkH that can exceed 30° with respect to the current flow, but in contrast to conventional theoretical expectations, ΘSkH increases linearly with velocity up to at least 100 ms−1. We qualitatively explain our observation based on internal mode excitations in combination with a field-like spin–orbit torque, showing that one must go beyond the usual rigid skyrmion description to understand the dynamics. Experiments show that when driven by electric currents, magnetic skyrmions experience transverse motion due to their topological charge — similar to the conventional Hall effect experienced by charged particles in a perpendicular magnetic field.

Abstract: Skeleton based action recognition distinguishes human actions using the trajectories of skeleton joints, which provide a very good representation for describing actions. Considering that recurrent neural networks (RNNs) with Long Short-Term Memory (LSTM) can learn feature representations and model long-term temporal dependencies automatically, we propose an end-to-end fully connected deep LSTM network for skeleton based action recognition. Inspired by the observation that the co-occurrences of the joints intrinsically characterize human actions, we take the skeleton as the input at each time slot and introduce a novel regularization scheme to learn the co-occurrence features of skeleton joints. To train the deep LSTM network effectively, we propose a new dropout algorithm which simultaneously operates on the gates, cells, and output responses of the LSTM neurons. Experimental results on three human action recognition datasets consistently demonstrate the effectiveness of the proposed model.

Abstract: Mendelian randomization (MR) uses genetic data to probe questions of causality in epidemiological research, by invoking the Instrumental Variable (IV) assumptions. In recent years, it has become commonplace to attempt MR analyses by synthesising summary data estimates of genetic association gleaned from large and independent study populations. This is referred to as two-sample summary data MR. Unfortunately, due to the sheer number of variants that can be easily included into summary data MR analyses, it is increasingly likely that some do not meet the IV assumptions due to pleiotropy. There is a pressing need to develop methods that can both detect and correct for pleiotropy, in order to preserve the validity of the MR approach in this context. In this paper, we aim to clarify how established methods of meta-regression and random effects modelling from mainstream meta-analysis are being adapted to perform this task. Specifically, we focus on two contrastin g approaches: the Inverse Variance Weighted (IVW) method which assumes in its simplest form that all genetic variants are valid IVs, and the method of MR-Egger regression that allows all variants to violate the IV assumptions, albeit in a specific way. We investigate the ability of two popular random effects models to provide robustness to pleiotropy under the IVW approach, and propose statistics to quantify the relative goodness-of-fit of the IVW approach over MR-Egger regression. © 2017 The Authors. Statistics in Medicine Published by JohnWiley & Sons Ltd.

Abstract: #### Summary points Healthcare decision making is complex. Decision-making processes and the factors (criteria) that decision makers should consider vary for different types of decisions, including clinical recommendations, coverage decisions, and health system or public health recommendations or decisions.1 2 3 4 However, some criteria are relevant for all of these decisions, including the anticipated effects of the options being considered, the certainty of the evidence for those effects (also referred to as quality of evidence or confidence in effect estimates), and the costs and feasibility of the options. Decision makers must make judgments about each relevant factor, informed by the best evidence that is available to them. Often, the processes that decision makers use, the criteria that they consider and the evidence that they …

Abstract: High ionic conductivity solid polymer electrolyte (SPE) has long been desired for the next generation high energy and safe rechargeable lithium batteries. Among all of the SPEs, composite polymer electrolyte (CPE) with ceramic fillers has garnered great interest due to the enhancement of ionic conductivity. However, the high degree of polymer crystallinity, agglomeration of ceramic fillers, and weak polymer–ceramic interaction limit the further improvement of ionic conductivity. Different from the existing methods of blending preformed ceramic particles with polymers, here we introduce an in situ synthesis of ceramic filler particles in polymer electrolyte. Much stronger chemical/mechanical interactions between monodispersed 12 nm diameter SiO2 nanospheres and poly(ethylene oxide) (PEO) chains were produced by in situ hydrolysis, which significantly suppresses the crystallization of PEO and thus facilitates polymer segmental motion for ionic conduction. In addition, an improved degree of LiClO4 dissociati...

Abstract: This paper discusses the fourth year of the ”Sentiment Analysis in Twitter Task”. SemEval-2016 Task 4 comprises five subtasks, three of which represent a significant departure from previous editions. The first two subtasks are reruns from prior years and ask to predict the overall sentiment, and the sentiment towards a topic in a tweet. The three new subtasks focus on two variants of the basic “sentiment classification in Twitter” task. The first variant adopts a five-point scale, which confers an ordinal character to the classification task. The second variant focuses on the correct estimation of the prevalence of each class of interest, a task which has been called quantification in the supervised learning literature. The task continues to be very popular, attracting a total of 43 teams.

Abstract: Clinical and translational research has played a major role in advancing our understanding of pulmonary hypertension (PH), including pulmonary arterial hypertension and other forms of PH with severe vascular remodelling (e.g. chronic thromboembolic PH and pulmonary veno-occlusive disease). However, PH remains an incurable condition with a high mortality rate, underscoring the need for a better transfer of novel scientific knowledge into healthcare interventions. Herein, we review recent findings in pathology (with the questioning of the strict morphological categorisation of various forms of PH into pre- or post-capillary involvement of pulmonary vessels) and cellular mechanisms contributing to the onset and progression of pulmonary vascular remodelling associated with various forms of PH. We also discuss ways to improve management and to support and optimise drug development in this research field.

Abstract: The FDA approved pembrolizumab on May 23, 2017, for the treatment of adult and pediatric patients with unresectable or metastatic, microsatellite instability-high (MSI-H), or mismatch repair deficient (dMMR) solid tumors that have progressed following prior treatment and who have no satisfactory alternative treatment options and for the treatment of unresectable or metastatic MSI-H or dMMR colorectal cancer that has progressed following treatment with a fluoropyrimidine, oxaliplatin, and irinotecan. The FDA granted the approval based on an understanding of the biology of MSI-H/dMMR across different tumors along with the clinically important effects on overall response rate (ORR) observed in patients who were enrolled in 1 of 5 single-arm clinical trials. The ORR was 39.6% among 149 patients with 15 different tumor types (95% confidence interval, 31.7–47.9), with a 7% complete response rate. The duration of response ranged from 1.6+ months to 22.7+ months, with 78% of responses lasting ≥6 months. Overall, the adverse event profile of pembrolizumab was similar to the adverse event profile observed across prior trials that supported the approval of pembrolizumab in other indications. This approval of pembrolizumab is the first time that the FDA has approved a cancer treatment for an indication based on a common biomarker rather than the primary site of origin.

Abstract: Current generative frameworks use end-to-end learning and generate images by sampling from uniform noise distribution. However, these approaches ignore the most basic principle of image formation: images are product of: (a) Structure: the underlying 3D model; (b) Style: the texture mapped onto structure. In this paper, we factorize the image generation process and propose Style and Structure Generative Adversarial Network (\({\text {S}^2}\)-GAN). Our \({\text {S}^2}\)-GAN has two components: the Structure-GAN generates a surface normal map; the Style-GAN takes the surface normal map as input and generates the 2D image. Apart from a real vs. generated loss function, we use an additional loss with computed surface normals from generated images. The two GANs are first trained independently, and then merged together via joint learning. We show our \({\text {S}^2}\)-GAN model is interpretable, generates more realistic images and can be used to learn unsupervised RGBD representations.

Abstract: Charge carriers in transition metal dichalcogenides have an extra degree of freedom known as valley pseudospin, which is associated with the shape of the energy bands. Experiments show that this pseudospin can be manipulated using magnetic fields.

Abstract: Over the last decade, Convolutional Neural Network (CNN) models have been highly successful in solving complex vision problems. However, these deep models are perceived as "black box" methods considering the lack of understanding of their internal functioning. There has been a significant recent interest in developing explainable deep learning models, and this paper is an effort in this direction. Building on a recently proposed method called Grad-CAM, we propose a generalized method called Grad-CAM++ that can provide better visual explanations of CNN model predictions, in terms of better object localization as well as explaining occurrences of multiple object instances in a single image, when compared to state-of-the-art. We provide a mathematical derivation for the proposed method, which uses a weighted combination of the positive partial derivatives of the last convolutional layer feature maps with respect to a specific class score as weights to generate a visual explanation for the corresponding class label. Our extensive experiments and evaluations, both subjective and objective, on standard datasets showed that Grad-CAM++ provides promising human-interpretable visual explanations for a given CNN architecture across multiple tasks including classification, image caption generation and 3D action recognition; as well as in new settings such as knowledge distillation.

Abstract: The composition of the intestinal microbiota influences the development of inflammatory disorders. However, associating inflammatory diseases with specific microbial members of the microbiota is challenging, because clinically detectable inflammation and its treatment can alter the microbiota's composition. Immunologic checkpoint blockade with ipilimumab, a monoclonal antibody that blocks cytotoxic T-lymphocyte-associated antigen-4 (CTLA-4) signalling, is associated with new-onset, immune-mediated colitis. Here we conduct a prospective study of patients with metastatic melanoma undergoing ipilimumab treatment and correlate the pre-inflammation faecal microbiota and microbiome composition with subsequent colitis development. We demonstrate that increased representation of bacteria belonging to the Bacteroidetes phylum is correlated with resistance to the development of checkpoint-blockade-induced colitis. Furthermore, a paucity of genetic pathways involved in polyamine transport and B vitamin biosynthesis is associated with an increased risk of colitis. Identification of these biomarkers may enable interventions to reduce the risk of inflammatory complications following cancer immunotherapy.

Abstract: Among collaborative recommendation approaches, methods based on nearest-neighbors still enjoy a huge amount of popularity, due to their simplicity, their efficiency, and their ability to produce accurate and personalized recommendations. This chapter presents a comprehensive survey of neighborhood-based methods for the item recommendation problem. In particular, the main benefits of such methods, as well as their principal characteristics, are described. Furthermore, this document addresses the essential decisions that are required while implementing a neighborhood-based recommender system, and gives practical information on how to make such decisions. Finally, the problems of sparsity and limited coverage, often observed in large commercial recommender systems, are discussed, and a few solutions to overcome these problems are presented.

Abstract: In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications. Traditional cloudbased Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL

Abstract: All-polymer solar cells have shown great potential as flexible and portable power generators. These devices should offer good mechanical endurance with high power-conversion efficiency for viability in commercial applications. In this work, we develop highly efficient and mechanically robust all-polymer solar cells that are based on the PBDTTTPD polymer donor and the P(NDI2HD-T) polymer acceptor. These systems exhibit high power-conversion efficiency of 6.64%. Also, the proposed all-polymer solar cells have even better performance than the control polymer-fullerene devices with phenyl-C61-butyric acid methyl ester (PCBM) as the electron acceptor (6.12%). More importantly, our all-polymer solar cells exhibit dramatically enhanced strength and flexibility compared with polymer/PCBM devices, with 60- and 470-fold improvements in elongation at break and toughness, respectively. The superior mechanical properties of all-polymer solar cells afford greater tolerance to severe deformations than conventional polymer-fullerene solar cells, making them much better candidates for applications in flexible and portable devices.

Abstract: The theory of electric polarization in crystals defines the dipole moment of an insulator in terms of a Berry phase (geometric phase) associated with its electronic ground state. This concept not only solves the long-standing puzzle of how to calculate dipole moments in crystals, but also explains topological band structures in insulators and superconductors, including the quantum anomalous Hall insulator and the quantum spin Hall insulator, as well as quantized adiabatic pumping processes. A recent theoretical study has extended the Berry phase framework to also account for higher electric multipole moments, revealing the existence of higher-order topological phases that have not previously been observed. Here we demonstrate experimentally a member of this predicted class of materials-a quantized quadrupole topological insulator-produced using a gigahertz-frequency reconfigurable microwave circuit. We confirm the non-trivial topological phase using spectroscopic measurements and by identifying corner states that result from the bulk topology. In addition, we test the critical prediction that these corner states are protected by the topology of the bulk, and are not due to surface artefacts, by deforming the edges of the crystal lattice from the topological to the trivial regime. Our results provide conclusive evidence of a unique form of robustness against disorder and deformation, which is characteristic of higher-order topological insulators.

Abstract: Large pre-trained neural networks such as BERT have had great recent success in NLP, motivating a growing body of research investigating what aspects of language they are able to learn from unlabeled data. Most recent analysis has focused on model outputs (e.g., language model surprisal) or internal vector representations (e.g., probing classifiers). Complementary to these works, we propose methods for analyzing the attention mechanisms of pre-trained models and apply them to BERT. BERT's attention heads exhibit patterns such as attending to delimiter tokens, specific positional offsets, or broadly attending over the whole sentence, with heads in the same layer often exhibiting similar behaviors. We further show that certain attention heads correspond well to linguistic notions of syntax and coreference. For example, we find heads that attend to the direct objects of verbs, determiners of nouns, objects of prepositions, and coreferent mentions with remarkably high accuracy. Lastly, we propose an attention-based probing classifier and use it to further demonstrate that substantial syntactic information is captured in BERT's attention.

Abstract: This review article summarizes the last few decades of research on nickel hydroxide, an important material in physics and chemistry, that has many applications in engineering including, significantly, batteries. First, the structures of the two known polymorphs, denoted as α-Ni(OH)2 and β-Ni(OH)2, are described. The various types of disorder, which are frequently present in nickel hydroxide materials, are discussed including hydration, stacking fault disorder, mechanical stresses and the incorporation of ionic impurities. Several related materials are discussed, including intercalated α-derivatives and basic nickel salts. Next, a number of methods to prepare, or synthesize, nickel hydroxides are summarized, including chemical precipitation, electrochemical precipitation, sol-gel synthesis, chemical ageing, hydrothermal and solvothermal synthesis, electrochemical oxidation, microwave-assisted synthesis, and sonochemical methods. Finally, the known physical properties of the nickel hydroxides are reviewed, including their magnetic, vibrational, optical, electrical and mechanical properties. The last section in this paper is intended to serve as a summary of both the potentially useful properties of these materials and the methods for the identification and characterization of 'unknown' nickel hydroxide-based samples.

Abstract: Superconducting qubits are leading candidates in the race to build a quantum computer capable of realizing computations beyond the reach of modern supercomputers. The superconducting qubit modality...