Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Knowledge graph (KG) embedding is to embed components of a KG including entities and relations into continuous vector spaces, so as to simplify the manipulation while preserving the inherent structure of the KG. It can benefit a variety of downstream tasks such as KG completion and relation extraction, and hence has quickly gained massive attention. In this article, we provide a systematic review of existing techniques, including not only the state-of-the-arts but also those with latest trends. Particularly, we make the review based on the type of information used in the embedding task. Techniques that conduct embedding using only facts observed in the KG are first introduced. We describe the overall framework, specific model design, typical training procedures, as well as pros and cons of such techniques. After that, we discuss techniques that further incorporate additional information besides facts. We focus specifically on the use of entity types, relation paths, textual descriptions, and logical rules. Finally, we briefly introduce how KG embedding can be applied to and benefit a wide variety of downstream tasks such as KG completion, relation extraction, question answering, and so forth.

Knowledge Graph Embedding: A Survey of Approaches and Applications

Human knowledge provides a formal understanding of the world. Knowledge graphs that represent structural relations between entities have become an increasingly popular research direction toward cognition and human-level intelligence. In this survey, we provide a comprehensive review of the knowledge graph covering overall research topics about: 1) knowledge graph representation learning; 2) knowledge acquisition and completion; 3) temporal knowledge graph; and 4) knowledge-aware applications and summarize recent breakthroughs and perspective directions to facilitate future research. We propose a full-view categorization and new taxonomies on these topics. Knowledge graph embedding is organized from four aspects of representation space, scoring function, encoding models, and auxiliary information. For knowledge acquisition, especially knowledge graph completion, embedding methods, path inference, and logical rule reasoning are reviewed. We further explore several emerging topics, including metarelational learning, commonsense reasoning, and temporal knowledge graphs. To facilitate future research on knowledge graphs, we also provide a curated collection of data sets and open-source libraries on different tasks. In the end, we have a thorough outlook on several promising research directions.

A Survey on Knowledge Graphs: Representation, Acquisition and Applications

The phase-shifting mask consists of a normal transmission mask that has been coated with a transparent layer patterned to ensure that the optical phases of nearest apertures are opposite. Destructive interference between waves from adjacent apertures cancels some diffraction effects and increases the spatial resolution with which such patterns can be projected. A simple theory predicts a near doubling of resolution for illumination with partial incoherence σ < 0.3, and substantial improvements in resolution for σ < 0.7. Initial results obtained with a phase-shifting mask patterned with typical device structures by electron-beam lithography and exposed using a Mann 4800 10× tool reveals a 40-percent increase in usuable resolution with some structures printed at a resolution of 1000 lines/mm. Phase-shifting mask structures can be used to facilitate proximity printing with larger gaps between mask and wafer. Theory indicates that the increase in resolution is accompanied by a minimal decrease in depth of focus. Thus the phase-shifting mask may be the most desirable device for enhancing optical lithography resolution in the VLSI/VHSIC era.

Improving resolution in photolithography with a phase-shifting mask

The theory of image formation is formulated in terms of the coherence function in the object plane, the diffraction distribution function of the image-forming system and a function describing the structure of the object. There results a four-fold integral involving these functions, and the complex conjugate functions of the latter two. This integral is evaluated in terms of the Fourier transforms of the coherence function, the diffraction distribution function and its complex conjugate. In fact, these transforms are respectively the distribution of intensity in an 'effective source', and the complex transmission of the optical system-they are the data initially known and are generally of simple form. A generalized 'transmission factor' is found which reduces to the known results in the simple cases of perfect coherence and complete incoherence. The procedure may be varied in a manner more suited to non-periodic objects. The theory is applied to study inter alia the influence of the method of illumination on the images of simple periodic structures and of an isolated line.

On the diffraction theory of optical images

With the advent of Internet, people actively express their opinions about products, services, events, political parties, etc., in social media, blogs, and website comments. The amount of research work on sentiment analysis is growing explosively. However, the majority of research efforts are devoted to English-language data, while a great share of information is available in other languages. We present a state-of-the-art review on multilingual sentiment analysis. More importantly, we compare our own implementation of existing approaches on common data. Precision observed in our experiments is typically lower than the one reported by the original authors, which we attribute to the lack of detail in the original presentation of those approaches. Thus, we compare the existing works by what they really offer to the reader, including whether they allow for accurate implementation and for reliable reproduction of the reported results.

/pdf/multilingual-sentiment-analysis-state-of-the-art-and-2qxlxx29em.pdf

Multilingual Sentiment Analysis: State of the Art and Independent Comparison of Techniques.

Many knowledge repositories nowadays contain billions of triplets, i.e. (head-entity, relationship, tail-entity), as relation instances. These triplets form a directed graph with entities as nodes and relationships as edges. However, this kind of symbolic and discrete storage structure makes it difficult for us to exploit the knowledge to enhance other intelligenceacquired applications (e.g. the QuestionAnswering System), as many AI-related algorithms prefer conducting computation on continuous data. Therefore, a series of emerging approaches have been proposed to facilitate knowledge computing via encoding the knowledge graph into a low-dimensional embedding space. TransE is the latest and most promising approach among them, and can achieve a higher performance with fewer parameters by modeling the relationship as a transitional vector from the head entity to the tail entity. Unfortunately, it is not flexible enough to tackle well with the various mapping properties of triplets, even though its authors spot the harm on performance. In this paper, we thus propose a superior model called TransM to leverage the structure of the knowledge graph via pre-calculating the distinct weight for each training triplet according to its relational mapping property. In this way, the optimal function deals with each triplet depending on its own weight. We carry out extensive experiments to compare TransM with the state-of-the-art method TransE and other prior arts. The performance of each approach is evaluated within two different application scenarios on several benchmark datasets. Results show that the model we proposed significantly outperforms the former ones with lower parameter complexity as TransE.

/pdf/transition-based-knowledge-graph-embedding-with-relational-28ivhoi4l2.pdf

Transition-based Knowledge Graph Embedding with Relational Mapping Properties

Silicon physical unclonable function (PUF) is a popular hardware security primitive that exploits the intrinsic variation of IC manufacturing process to generate chip-unique information for various security related applications. For example, the PUF information can be used as a chip identifier, a secret key, the seed for a random number generator, or the response to a given challenge. Due to the unpredictability and irreplicability of IC manufacturing variation, silicon PUF has emerged as a promising hardware security primitive and gained a lot of attention over the past few years. In this article, we first give a survey on the current state-of-the-art of silicon PUFs, then analyze known attacks to PUFs and the countermeasures. After that we discuss PUF-based applications, highlight some recent research advances in ring oscillator PUFs, and conclude with some challenges and opportunities in PUF research and applications.

A Survey on Silicon PUFs and Recent Advances in Ring Oscillator PUFs

The essence of distantly supervised relation extraction is that it is an incomplete multi-label classification problem with sparse and noisy features. To tackle the sparsity and noise challenges, we propose solving the classification problem using matrix completion on factorized matrix of minimized rank. We formulate relation classification as completing the unknown labels of testing items (entity pairs) in a sparse matrix that concatenates training and testing textual features with training labels. Our algorithmic framework is based on the assumption that the rank of item-byfeature and item-by-label joint matrix is low. We apply two optimization models to recover the underlying low-rank matrix leveraging the sparsity of feature-label matrix. The matrix completion problem is then solved by the fixed point continuation (FPC) algorithm, which can find the global optimum. Experiments on two widely used datasets with different dimensions of textual features demonstrate that our low-rank matrix completion approach significantly outperforms the baseline and the state-of-the-art methods.

http://cslt.riit.tsinghua.edu.cn/~fzheng/PAPERS/2014/1406E_ACL_Relation-Extraction_FM.pdf

Distant Supervision for Relation Extraction with Matrix Completion

The essence of distantly supervised relation extraction is that it is an incomplete multi-label classification problem with sparse and noisy features. To tackle the sparsity and noise challenges, we propose solving the classification problem using matrix completion on factorized matrix of minimized rank. We formulate relation classification as completing the unknown labels of testing items (entity pairs) in a sparse matrix that concatenates training and testing textual features with training labels. Our algorithmic framework is based on the assumption that the rank of item-by-feature and item-by-label joint matrix is low. We apply two optimization models to recover the underlying low-rank matrix leveraging the sparsity of feature-label matrix. The matrix completion problem is then solved by the fixed point continuation (FPC) algorithm, which can find the global optimum. Experiments on two widely used datasets with different dimensions of textual features demonstrate that our low-rank matrix completion approach significantly outperforms the baseline and the state-of-the-art methods.

Qiang Zhou

Papers

Multilingual Sentiment Analysis: State of the Art and Independent Comparison of Techniques.

Transition-based Knowledge Graph Embedding with Relational Mapping Properties

A Survey on Silicon PUFs and Recent Advances in Ring Oscillator PUFs

Distant Supervision for Relation Extraction with Matrix Completion

Errata: Distant Supervision for Relation Extraction with Matrix Completion