Facebook

About: Facebook is a company organization based out in Tel Aviv, Israel. It is known for research contribution in the topics: Computer science & Artificial neural network. The organization has 7856 authors who have published 10906 publications receiving 570123 citations. The organization is also known as: facebook.com & FB.

Distilling Knowledge from Reader to Retriever for Question Answering

Abstract: The task of information retrieval is an important component of many natural language processing systems, such as open domain question answering. While traditional methods were based on hand-crafted features, continuous representations based on neural networks recently obtained competitive results. A challenge of using such methods is to obtain supervised data to train the retriever model, corresponding to pairs of query and support documents. In this paper, we propose a technique to learn retriever models for downstream tasks, inspired by knowledge distillation, and which does not require annotated pairs of query and documents. Our approach leverages attention scores of a reader model, used to solve the task based on retrieved documents, to obtain synthetic labels for the retriever. We evaluate our method on question answering, obtaining state-of-the-art results.

Phrase-based Image Captioning

Abstract: Generating a novel textual description of an image is an interesting problem that connects computer vision and natural language processing. In this paper, we present a simple model that is able to generate descriptive sentences given a sample image. This model has a strong focus on the syntax of the descriptions. We train a purely bilinear model that learns a metric between an image representation (generated from a previously trained Convolutional Neural Network) and phrases that are used to described them. The system is then able to infer phrases from a given image sample. Based on caption syntax statistics, we propose a simple language model that can produce relevant descriptions for a given test image using the phrases inferred. Our approach, which is considerably simpler than state-of-the-art models, achieves comparable results in two popular datasets for the task: Flickr30k and the recently proposed Microsoft COCO.

Flash storage disaggregation

Abstract: PCIe-based Flash is commonly deployed to provide datacenter applications with high IO rates. However, its capacity and bandwidth are often underutilized as it is difficult to design servers with the right balance of CPU, memory and Flash resources over time and for multiple applications. This work examines Flash disaggregation as a way to deal with Flash overprovisioning. We tune remote access to Flash over commodity networks and analyze its impact on workloads sampled from real datacenter applications. We show that, while remote Flash access introduces a 20% throughput drop at the application level, disaggregation allows us to make up for these overheads through resource-efficient scale-out. Hence, we show that Flash disaggregation allows scaling CPU and Flash resources independently in a cost effective manner. We use our analysis to draw conclusions about data and control plane issues in remote storage.

20 years of research on the Alcator C-Mod tokamak

Abstract: The object of this review is to summarize the achievements of research on the Alcator C-Mod tokamak [Hutchinson et al., Phys. Plasmas 1, 1511 (1994) and Marmar, Fusion Sci. Technol. 51, 261 (2007)] and to place that research in the context of the quest for practical fusion energy. C-Mod is a compact, high-field tokamak, whose unique design and operating parameters have produced a wealth of new and important results since it began operation in 1993, contributing data that extends tests of critical physical models into new parameter ranges and into new regimes. Using only high-power radio frequency (RF) waves for heating and current drive with innovative launching structures, C-Mod operates routinely at reactor level power densities and achieves plasma pressures higher than any other toroidal confinement device. C-Mod spearheaded the development of the vertical-target divertor and has always operated with high-Z metal plasma facing components—approaches subsequently adopted for ITER. C-Mod has made ground-breaking discoveries in divertor physics and plasma-material interactions at reactor-like power and particle fluxes and elucidated the critical role of cross-field transport in divertor operation, edge flows and the tokamak density limit. C-Mod developed the I-mode and the Enhanced Dα H-mode regimes, which have high performance without large edge localized modes and with pedestal transport self-regulated by short-wavelength electromagnetic waves. C-Mod has carried out pioneering studies of intrinsic rotation and demonstrated that self-generated flow shear can be strong enough in some cases to significantly modify transport. C-Mod made the first quantitative link between the pedestal temperature and the H-mode's performance, showing that the observed self-similar temperature profiles were consistent with critical-gradient-length theories and followed up with quantitative tests of nonlinear gyrokinetic models. RF research highlights include direct experimental observation of ion cyclotron range of frequency (ICRF) mode-conversion, ICRF flow drive, demonstration of lower-hybrid current drive at ITER-like densities and fields and, using a set of novel diagnostics, extensive validation of advanced RF codes. Disruption studies on C-Mod provided the first observation of non-axisymmetric halo currents and non-axisymmetric radiation in mitigated disruptions. A summary of important achievements and discoveries are included.

Realtime Data Processing at Facebook

Abstract: Realtime data processing powers many use cases at Facebook, including realtime reporting of the aggregated, anonymized voice of Facebook users, analytics for mobile applications, and insights for Facebook page administrators. Many companies have developed their own systems; we have a realtime data processing ecosystem at Facebook that handles hundreds of Gigabytes per second across hundreds of data pipelines. Many decisions must be made while designing a realtime stream processing system. In this paper, we identify five important design decisions that affect their ease of use, performance, fault tolerance, scalability, and correctness. We compare the alternative choices for each decision and contrast what we built at Facebook to other published systems. Our main decision was targeting seconds of latency, not milliseconds. Seconds is fast enough for all of the use cases we support and it allows us to use a persistent message bus for data transport. This data transport mechanism then paved the way for fault tolerance, scalability, and multiple options for correctness in our stream processing systems Puma, Swift, and Stylus. We then illustrate how our decisions and systems satisfy our requirements for multiple use cases at Facebook. Finally, we reflect on the lessons we learned as we built and operated these systems.