Showing papers by "NEC published in 2019"

Simplifying Graph Convolutional Networks

Abstract: Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

Pay Less Attention with Lightweight and Dynamic Convolutions

Abstract: Self-attention is a useful mechanism to build generative models for language and images. It determines the importance of context elements by comparing each element to the current time step. In this paper, we show that a very lightweight convolution can perform competitively to the best reported self-attention results. Next, we introduce dynamic convolutions which are simpler and more efficient than self-attention. We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements. The number of operations required by this approach scales linearly in the input length, whereas self-attention is quadratic. Experiments on large-scale machine translation, language modeling and abstractive summarization show that dynamic convolutions improve over strong self-attention models. On the WMT'14 English-German test set dynamic convolutions achieve a new state of the art of 29.7 BLEU.

Pay Less Attention with Lightweight and Dynamic Convolutions

Abstract: Self-attention is a useful mechanism to build generative models for language and images. It determines the importance of context elements by comparing each element to the current time step. In this paper, we show that a very lightweight convolution can perform competitively to the best reported self-attention results. Next, we introduce dynamic convolutions which are simpler and more efficient than self-attention. We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements. The number of operations required by this approach scales linearly in the input length, whereas self-attention is quadratic. Experiments on large-scale machine translation, language modeling and abstractive summarization show that dynamic convolutions improve over strong self-attention models. On the WMT'14 English-German test set dynamic convolutions achieve a new state of the art of 29.7 BLEU.

Learning Discrete Structures for Graph Neural Networks

Abstract: Graph neural networks (GNNs) are a popular class of machine learning models whose major advantage is their ability to incorporate a sparse and discrete dependency structure between data points. Unfortunately, GNNs can only be used when such a graph-structure is available. In practice, however, real-world graphs are often noisy and incomplete or might not be available at all. With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph. This allows one to apply GCNs not only in scenarios where the given graph is incomplete or corrupted but also in those where a graph is not available. We conduct a series of experiments that analyze the behavior of the proposed method and demonstrate that it outperforms related methods by a significant margin.

Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy.

Abstract: Gaps in colonoscopy skills among endoscopists, primarily due to experience, have been identified, and solutions are critically needed. Hence, the development of a real-time robust detection system for colorectal neoplasms is considered to significantly reduce the risk of missed lesions during colonoscopy. Here, we develop an artificial intelligence (AI) system that automatically detects early signs of colorectal cancer during colonoscopy; the AI system shows the sensitivity and specificity are 97.3% (95% confidence interval [CI] = 95.9%–98.4%) and 99.0% (95% CI = 98.6%–99.2%), respectively, and the area under the curve is 0.975 (95% CI = 0.964–0.986) in the validation set. Moreover, the sensitivities are 98.0% (95% CI = 96.6%–98.8%) in the polypoid subgroup and 93.7% (95% CI = 87.6%–96.9%) in the non-polypoid subgroup; To accelerate the detection, tensor metrics in the trained model was decomposed, and the system can predict cancerous regions 21.9 ms/image on average. These findings suggest that the system is sufficient to support endoscopists in the high detection against non-polypoid lesions, which are frequently missed by optical colonoscopy. This AI system can alert endoscopists in real-time to avoid missing abnormalities such as non-polypoid polyps during colonoscopy, improving the early detection of this disease.

Sereum: Protecting Existing Smart Contracts Against Re-Entrancy Attacks

Abstract: Recently, a number of existing blockchain systems have witnessed major bugs and vulnerabilities within smart contracts. Although the literature features a number of proposals for securing smart contracts, these proposals mostly focus on proving the correctness or absence of a certain type of vulnerability within a contract, but cannot protect deployed (legacy) contracts from being exploited. In this paper, we address this problem in the context of re-entrancy exploits and propose a novel smart contract security technology, dubbed Sereum (Secure Ethereum), which protects existing, deployed contracts against re-entrancy attacks in a backwards compatible way based on run-time monitoring and validation. Sereum does neither require any modification nor any semantic knowledge of existing contracts. By means of implementation and evaluation using the Ethereum blockchain, we show that Sereum covers the actual execution flow of a smart contract to accurately detect and prevent attacks with a false positive rate as small as 0.06% and with negligible run-time overhead. As a by-product, we develop three advanced re-entrancy attacks to demonstrate the limitations of existing offline vulnerability analysis tools.

A 28-GHz CMOS Phased-Array Transceiver Based on LO Phase-Shifting Architecture With Gain Invariant Phase Tuning for 5G New Radio

Abstract: This paper presents a 28-GHz CMOS four-element phased-array transceiver chip for the fifth-generation mobile network (5G) new radio (NR). The proposed transceiver is based on the local-oscillator (LO) phase-shifting architecture, and it achieves quasi-continuous phase tuning with less than 0.2-dB radio frequency (RF) gain variation and 0.3°C phase error. Accurate beam control with suppressed sidelobe level during beam steering could be supported by this work. At 28 GHz, a single-element transmitter-mode output ${{\mathrm {P}}_{\mathrm {1\,dB}}}$ of 15.7 dBm and a receiver-mode noise figure (NF) of 4.1 dB are achieved. The eight-element transceiver modules developed in this work are capable of scanning the beam from −50° to +50° with less than −9-dB sidelobe level. A saturated equivalent isotropic radiated power (EIRP) of 39.8 dBm is achieved at 0° scan. In a 5-m over-the-air measurement, the proposed module demonstrates the first 512 quadrature amplitude modulation (QAM) constellation in the 28-GHz band. A data stream of 6.4 Gb/s in 256-QAM could be supported within a beam angle of ±50°. The achieved maximum data rate is 15 Gb/s in 64-QAM. The proposed transceiver chip consumes 1.2 W/chip in transmitter mode and 0.59 W/chip in receiver mode.

NoDoze: Combatting Threat Alert Fatigue with Automated Provenance Triage

Abstract: Large enterprises are increasingly relying on threat detection softwares (e.g., Intrusion Detection Systems) to allow them to spot suspicious activities. These softwares generate alerts which must be investigated by cyber analysts to figure out if they are true attacks. Unfortunately, in practice, there are more alerts than cyber analysts can properly investigate. This leads to a “threat alert fatigue” or information overload problem where cyber analysts miss true attack alerts in the noise of false alarms. In this paper, we present NoDoze to combat this challenge using contextual and historical information of generated threat alert in an enterprise. NoDoze first generates a causal dependency graph of an alert event. Then, it assigns an anomaly score to each event in the dependency graph based on the frequency with which related events have happened before in the enterprise. NoDoze then propagates those scores along the edges of the graph using a novel network diffusion algorithm and generates a subgraph with an aggregate anomaly score which is used to triage alerts. Evaluation on our dataset of 364 threat alerts shows that NoDoze decreases the volume of false alarms by 86%, saving more than 90 hours of analysts’ time, which was required to investigate those false alarms. Furthermore, NoDoze generated dependency graphs of true alerts are 2 orders of magnitude smaller than those generated by traditional tools without sacrificing the vital information needed for the investigation. Our system has a low average runtime overhead and can be deployed with any threat detection software.

Field and lab experimental demonstration of nonlinear impairment compensation using neural networks

Abstract: Fiber nonlinearity is one of the major limitations to the achievable capacity in long distance fiber optic transmission systems. Nonlinear impairments are determined by the signal pattern and the transmission system parameters. Deterministic algorithms based on approximating the nonlinear Schrodinger equation through digital back propagation, or a single step approach based on perturbation methods have been demonstrated, however, their implementation demands excessive signal processing resources, and accurate knowledge of the transmission system. A completely different approach uses machine learning algorithms to learn from the received data itself to figure out the nonlinear impairment. In this work, a single-step, system agnostic nonlinearity compensation algorithm based on a neural network is proposed to pre-distort symbols at transmitter side to demonstrate ~0.6 dB Q improvement after 2800 km standard single-mode fiber transmission using 32 Gbaud signal. Without prior knowledge of the transmission system, the neural network tensor weights are constructed from training data thanks to the intra-channel cross-phase modulation and intra-channel four-wave mixing triplets used as input features. Long-distance fiber communications still face many fundamental challenges in capacity due to nonlinearities. The authors develop a neural-network based tool to compensate nonlinearities, without prior knowledge of the transmission link, with low complexity.

Learning Discrete Structures for Graph Neural Networks

Abstract: Graph neural networks (GNNs) are a popular class of machine learning models whose major advantage is their ability to incorporate a sparse and discrete dependency structure between data points. Unfortunately, GNNs can only be used when such a graph-structure is available. In practice, however, real-world graphs are often noisy and incomplete or might not be available at all. With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph. This allows one to apply GCNs not only in scenarios where the given graph is incomplete or corrupted but also in those where a graph is not available. We conduct a series of experiments that analyze the behavior of the proposed method and demonstrate that it outperforms related methods by a significant margin.

Machine-learning guided discovery of a new thermoelectric material

Abstract: Thermoelectric technologies are becoming indispensable in the quest for a sustainable future. Recently, an emerging phenomenon, the spin-driven thermoelectric effect (STE), has garnered much attention as a promising path towards low cost and versatile thermoelectric technology with easily scalable manufacturing. However, progress in development of STE devices is hindered by the lack of understanding of the fundamental physics and materials properties responsible for the effect. In such nascent scientific field, data-driven approaches relying on statistics and machine learning, instead of more traditional modeling methods, can exhibit their full potential. Here, we use machine learning modeling to establish the key physical parameters controlling STE. Guided by the models, we have carried out actual material synthesis which led to the identification of a novel STE material with a thermopower an order of magnitude larger than that of the current generation of STE devices.

Anonymization of Sensitive Quasi-Identifiers for l-Diversity and t-Closeness

Abstract: A number of studies on privacy-preserving data mining have been proposed. Most of them assume that they can separate quasi-identifiers (QIDs) from sensitive attributes. For instance, they assume that address, job, and age are QIDs but are not sensitive attributes and that a disease name is a sensitive attribute but is not a QID. However, all of these attributes can have features that are both sensitive attributes and QIDs in practice. In this paper, we refer to these attributes as sensitive QIDs and we propose novel privacy models, namely, (l1, …, lq)-diversity and (t1, …, tq)-closeness, and a method that can treat sensitive QIDs. Our method is composed of two algorithms: An anonymization algorithm and a reconstruction algorithm. The anonymization algorithm, which is conducted by data holders, is simple but effective, whereas the reconstruction algorithm, which is conducted by data analyzers, can be conducted according to each data analyzer's objective. Our proposed method was experimentally evaluated using real data sets.

A Standard-Based Open Source IoT Platform: FIWARE

Abstract: The ever-increasing acceleration of technology evolution in all fields is rapidly changing the architectures of data-driven systems toward the Internet-of-Things concept. Many general and specific-purpose IoT platforms are already available. This article introduces the capabilities of the FIWARE platform, which is transitioning from a research to a commercial level. We base our exposition on the analysis of three real-world use cases (global IoT market, analytics in smart cities, and IoT augmented autonomous driving) and their requirements that are addressed with the usage of FIWARE. We highlight the lessons learned during the design, implementation, and deployment phases for each of the use cases and their critical issues. Finally, we give two examples showing that FIWARE still maintains openness to innovation: semantics and privacy.

Simulating 6TiSCH Networks

Abstract: 6TiSCH is a working group at the IETF which is standardizing how to combine IEEE802.15.4 Time-Slotted Channel Hopping (TSCH) with IPv6. The result is a solution which offers both industrial performance and seamless integration into the Internet, and is therefore seen as a key technology for the Industrial Internet of Things. This article presents the 6TiSCH Simulator, created as part of the standardization activity, and which has been used extensively by the working group. The goal of the simulator is to benchmark 6TiSCH against realistic scenarios, something which is hard to do using formal models or real-world deployments. This article discusses the overall architecture of the simulator, details the different models it uses (i.e. energy, propagation), compares it to other simulation/emulation platforms, and presents 5 published examples of how the 6TiSCH Simulator has been used.

Introduction to Voice Presentation Attack Detection and Recent Advances

Abstract: Over the past few years, significant progress has been made in the field of presentation attack detection (PAD) for automatic speaker recognition (ASV). This includes the development of new speech corpora, standard evaluation protocols and advancements in front-end feature extraction and back-end classifiers. The use of standard databases and evaluation protocols has enabled for the first time the meaningful benchmarking of different PAD solutions. This chapter summarises the progress, with a focus on studies completed in the last 3 years. The article presents a summary of findings and lessons learned from two ASVspoof challenges, the first community-led benchmarking efforts. These show that ASV PAD remains an unsolved problem and that further attention is required to develop generalised PAD solutions which have potential to detect diverse and previously unseen spoofing attacks.

Priority Inheritance with Backtracking for Iterative Multi-agent Path Finding

Abstract: The Multi-agent Path Finding (MAPF) problem consists in all agents having to move to their own destinations while avoiding collisions. In practical applications to the problem, such as for navigation in an automated warehouse, MAPF must be solved iteratively. We present here a novel approach to iterative MAPF, that we call Priority Inheritance with Backtracking (PIBT). PIBT gives a unique priority to each agent every timestep, so that all movements are prioritized. Priority inheritance, which aims at dealing effectively with priority inversion in path adjustment within a small time window, can be applied iteratively and a backtracking protocol prevents agents from being stuck. We prove that, regardless of their number, all agents are guaranteed to reach their destination within finite time, when the environment is a graph such that all pairs of adjacent nodes belong to a simple cycle of length 3 or more (e.g., biconnected). Our implementation of PIBT can be fully decentralized without global communication. Experimental results over various scenarios confirm that PIBT is adequate both for finding paths in large environments with many agents, as well as for conveying packages in an automated warehouse.

Identification of advanced spin-driven thermoelectric materials via interpretable machine learning

Abstract: Machine learning is becoming a valuable tool for scientific discovery. Particularly attractive is the application of machine learning methods to the field of materials development, which enables innovations by discovering new and better functional materials. To apply machine learning to actual materials development, close collaboration between scientists and machine learning tools is necessary. However, such collaboration has been so far impeded by the black box nature of many machine learning algorithms. It is often difficult for scientists to interpret the data-driven models from the viewpoint of material science and physics. Here, we demonstrate the development of spin-driven thermoelectric materials with anomalous Nernst effect by using an interpretable machine learning method called factorized asymptotic Bayesian inference hierarchical mixture of experts (FAB/HMEs). Based on prior knowledge of material science and physics, we were able to extract from the interpretable machine learning some surprising correlations and new knowledge about spin-driven thermoelectric materials. Guided by this, we carried out an actual material synthesis that led to the identification of a novel spin-driven thermoelectric material. This material shows the largest thermopower to date.

The CORAL+ Algorithm for Unsupervised Domain Adaptation of PLDA

Abstract: State-of-the-art speaker recognition systems comprise an x-vector (or i-vector) speaker embedding front-end followed by a probabilistic linear discriminant analysis (PLDA) backend. The effectiveness of these components relies on the availability of a large collection of labeled training data. In practice, it is common that the domains (e.g., language, demographic) in which the system is deployed differ from that we trained the system. To close the gap due to the domain mismatch, we propose an unsupervised PLDA adaptation algorithm to learn from a small amount of unlabeled in-domain data. The proposed method was inspired by a prior work on feature-based domain adaptation technique known as the correlation alignment (CORAL). We refer to the model-based adaptation technique proposed in this paper as CORAL+. The efficacy of the proposed technique is experimentally validated on the recent NIST 2016 and 2018 Speaker Recognition Evaluation (SRE’16, SRE’18) datasets.

Improving Students' Daily Life Stress Forecasting using LSTM Neural Networks

Abstract: Accurately forecasting stress may enable people to make behavioral changes that could improve their future health. For example, accurate stress forecasting might inspire people to make changes to their schedule to get more sleep or exercise, in order to reduce excessive stress tomorrow night. In this paper, we examine how accurately the previous $N$ -days of multi-modal data can forecast tomorrow evening's high/low binary stress levels using long short-term memory neural network models (LSTM), logistic regression (LR), and support vector machines (SVM). Using a total of 2,276 days, with 1,231 overlapping 8-day sequences of data from 142 participants (including physiological signals, mobile phone usage, location, and behavioral surveys), we find the LSTM significantly outperforms LR and SVM with the best results reaching 83.6% using 7 days of prior data. Using time-series models improves the forecasting of stress even when considering only subsets of the multi-modal data set, e.g., using only physiology data. In particular, the LSTM model reaches 81.4% accuracy using only objective and passive data, i.e., not including subjective reports from a daily survey.

Positional Normalization.

Abstract: A popular method to reduce the training time of deep neural networks is to normalize activations at each layer. Although various normalization schemes have been proposed, they all follow a common theme: normalize across spatial dimensions and discard the extracted statistics. In this paper, we propose an alternative normalization method that noticeably departs from this convention and normalizes exclusively across channels. We argue that the channel dimension is naturally appealing as it allows us to extract the first and second moments of features extracted at a particular image position. These moments capture structural information about the input image and extracted features, which opens a new avenue along which a network can benefit from feature normalization: Instead of disregarding the normalization constants, we propose to re-inject them into later layers to preserve or transfer structural information in generative networks. Codes are available at https://github.com/Boyiliee/PONO.

Radio access network node, radio terminal, core network node, and method therefor

Abstract: A target RAN node (3) is configured to: receive, directly from a core network (5), core network context information about a handover of a radio terminal (1) from a first network to the second network; and control communication of the radio terminal (1) based on the core network context information. The target RAN node (3) is further configured to transfer a handover signaling message to a source RAN node on a direct interface (101) in response to receiving the core network context information. The core network context information includes at least one of flow information, slice information, and security-related information. It is thus possible, for example, to provide an inter-RAT handover procedure involving transfer of handover signaling messages on a direct inter-base-station interface.

Embedded sim management system, node device, embedded sim management method, program, and information registrant device

Abstract: A node device configuring a peer-to-peer network includes: a network interface; and a blockchain management part configured to receive, through the network interface, an information registration request transaction that includes embedded Subscriber Identity Module, SIM, information including SIM identification information, an electronic signature put on the embedded SIM information by using a private key of an information registrant, and a public key paired with the private key, and accumulate the received information registration request transaction into a blockchain based on a consensus building algorithm executed in cooperation with another node device configuring the peer-to-peer network.

Attending to Future Tokens for Bidirectional Sequence Generation

Abstract: Neural sequence generation is typically performed token-by-token and left-to-right. Whenever a token is generated only previously produced tokens are taken into consideration. In contrast, for problems such as sequence classification, bidirectional attention, which takes both past and future tokens into consideration, has been shown to perform much better. We propose to make the sequence generation process bidirectional by employing special placeholder tokens. Treated as a node in a fully connected graph, a placeholder token can take past and future tokens into consideration when generating the actual output token. We verify the effectiveness of our approach experimentally on two conversational tasks where the proposed bidirectional model outperforms competitive baselines by a large margin.

Meimei: An Efficient Probabilistic Approach for Semantically Annotating Tables

Abstract: Given a large amount of table data, how can we find the tables that contain the contents we want? A naive search fails when the column names are ambiguous, such as if columns containing stock price information are named “Close” in one table and named “P” in another table.One way of dealing with this problem that has been gaining attention is the semantic annotation of table data columns by using canonical knowledge. While previous studies successfully dealt with this problem for specific types of table data such as web tables, it still remains for various other types of table data: (1) most approaches do not handle table data with numerical values, and (2) their predictive performance is not satisfactory.This paper presents a novel approach for table data annotation that combines a latent probabilistic model with multilabel classifiers. It features three advantages over previous approaches due to using highly predictive multi-label classifiers in the probabilistic computation of semantic annotation. (1) It is more versatile due to using multi-label classifiers in the probabilistic model, which enables various types of data such as numerical values to be supported. (2) It is more accurate due to the multi-label classifiers and probabilistic model working together to improve predictive performance. (3) It is more efficient due to potential functions based on multi-label classifiers reducing the computational cost for annotation.Extensive experiments demonstrated the superiority of the proposed approach over state-of-the-art approaches for semantic annotation of real data (183 human-annotated tables obtained from the UCI Machine Learning Repository).

Positional Normalization

Abstract: A widely deployed method for reducing the training time of deep neural networks is to normalize activations at each layer. Although various normalization schemes have been proposed, they all follow a common theme: normalize across spatial dimensions and discard the extracted statistics. In this paper, we propose a novel normalization method that deviates from this theme. Our approach, which we refer to as Positional Normalization (PONO), normalizes exclusively across channels, which allows us to capture structural information of the input image in the first and second moments. Instead of disregarding this information, we inject it into later layers to preserve or transfer structural information in generative networks. We show that PONO significantly improves the performance of deep networks across a wide range of model architectures and image generation tasks.

On-demand generation of traveling cat states using a parametric oscillator

Abstract: We theoretically propose a method for on-demand generation of traveling Schr\"odinger cat states, namely, quantum superpositions of distinct coherent states of traveling fields. This method is based on deterministic generation of intracavity cat states using a Kerr-nonlinear parametric oscillator (KPO) via quantum adiabatic evolution. We show that the cat states generated inside a KPO can be released into an output mode by dynamically controlling the parametric pump amplitude. We further show that the quality of the traveling cat states can be improved by using a shortcut-to-adiabaticity technique.

Speaker detection in the wild: Lessons learned from JSALT 2019.

Abstract: This paper presents the problems and solutions addressed at the JSALT workshop when using a single microphone for speaker detection in adverse scenarios. The main focus was to tackle a wide range of conditions that go from meetings to wild speech. We describe the research threads we explored and a set of modules that was successful for these scenarios. The ultimate goal was to explore speaker detection; but our first finding was that an effective diarization improves detection, and not having a diarization stage impoverishes the performance. All the different configurations of our research agree on this fact and follow a main backbone that includes diarization as a previous stage. With this backbone, we analyzed the following problems: voice activity detection, how to deal with noisy signals, domain mismatch, how to improve the clustering; and the overall impact of previous stages in the final speaker detection. In this paper, we show partial results for speaker diarizarion to have a better understanding of the problem and we present the final results for speaker detection.

Gradient-Based Low-Light Image Enhancement

Abstract: A low-light image enhancement is a highly demanded image processing technique, especially for consumer digital cameras and cameras on mobile phones. In this paper, a gradient-based low-light image enhancement algorithm is proposed. The key is to enhance the gradients of dark region, because the gradients are more sensitive for human visual system than absolute values. In addition, we involve the intensity-range constraints for the image integration. By using the intensity-range constraints, we can integrate the output image with enhanced gradients preserving the given gradient information while enforcing the intensity range of the output image within a certain intensity range. Experiments demonstrate that the proposed gradient-based low-light image enhancement can effectively enhance the low-light images.