We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source code is at this https URL

YOLOv4: Optimal Speed and Accuracy of Object Detection

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

We report a continuous-wave (CW) KTA (KTiOAsO4)-OPO (optical parametric oscillator) with a compact linear cavity utilizing an LD pumped mixed crystal Nd:Lu0.5Y0.5VO4 laser as the pump source for the first time. A singly resonant oscillator with low signal light loss was designed to reduce the OPO’s threshold. Maximum output powers of 630 mW at 3475 nm and 190 mW at 1536 nm were obtained at a pump power of 13.2 W. A total conversion efficiency of 6.2% was achieved with respect to the incident diode pump power. The performance of this work demonstrates that a CW KTA-OPO derived by an LD pumped Nd3+ doped solid laser can also produce efficient mid-infrared light.

http://iopscience.iop.org/article/10.1088/1612-2011/10/5/055803/pdf

A compact, CW mid-infrared intra-cavity Nd:Lu0.5Y0.5VO4∖KTA-OPO at 3.5 μm

Deep convolutional neural networks (CNNs) are usually over-parameterized, which cannot be easily deployed on edge devices such as mobile phones and smart cameras. Existing works used to decrease the number or size of requested convolution filters for a minimum viable CNN on edge devices. In contrast, this paper introduces filters that are full-stack and can be used to generate many more sub-filters. Weights of these sub-filters are inherited from full-stack filters with the help of different binary masks. Orthogonal constraints are applied over binary masks to decrease their correlation and promote the diversity of generated sub-filters. To preserve the same volume of output feature maps, we can naturally reduce the number of established filters by only maintaining a few full-stack filters and a set of binary masks. We also conduct theoretical analysis on the memory cost and an efficient implementation is introduced for the convolution of the proposed filters. Experiments on several benchmark datasets and CNN models demonstrate that the proposed method is able to construct minimum viable convolution networks of comparable performance.

Full-Stack Filters to Build Minimum Viable CNNs.

Federated learning (FL) enables data owners to train a joint global model without sharing private data. However, it is vulnerable to Byzantine attackers that can launch poisoning attacks to destroy model training. Existing defense strategies rely on the additional datasets to train trustable server models or trusted execution environments to mitigate attacks. Besides, these strategies can only tolerate a small number of malicious users or resist a few types of poisoning attacks. To address these challenges, we design a novel federated learning method <bold>TDFL</bold>, <bold>T</bold>ruth <bold>D</bold>iscovery based <bold>F</bold>ederated <bold>L</bold>earning, which can defend against multiple poisoning attacks without additional datasets even when the Byzantine users are <inline-formula><tex-math notation="LaTeX">$\geq 50\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mo>≥</mml:mo><mml:mn>50</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq1-3205714.gif"/></alternatives></inline-formula>. Specifically, the TDFL considers different scenarios with different malicious proportions. For Honest-majority setting (Byzantine <inline-formula><tex-math notation="LaTeX">$< 50\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mo><</mml:mo><mml:mn>50</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq2-3205714.gif"/></alternatives></inline-formula>), we design a special robust truth discovery aggregation scheme to remove malicious model updates, which can assign weights according to users’ contribution; for Byzantine-majority setting (Byzantine <inline-formula><tex-math notation="LaTeX">$\geq 50\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mo>≥</mml:mo><mml:mn>50</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="xu-ieq3-3205714.gif"/></alternatives></inline-formula>), we use maximum clique-based filter to guarantee global model quality. To the best of our knowledge, this is the first study that uses truth discovery to defend against poisoning attacks. It is also the first scheme which can achieve strong robustness under multiple kinds of attacks launched by high proportion attackers without root datasets. Extensive comparative experiments are designed with five state-of-the-art aggregation rules under five types of classical poisoning attacks on different datasets. The experimental results demonstrate that TDFL is practical and achieves reasonable Byzantine-robustness.

TDFL: Truth Discovery Based Byzantine Robust Federated Learning

Abstract Eleven celadons with different degrees of degradation excavated from the Dalian Island shipwreck of the Yuan Dynasty (1271–1368 AD) were selected, and the degradation mechanism of the ancient porcelain at marine environment was investigated. The chemical composition, microstructure, and corrosion morphology of the samples were characterized. The results showed that these celadons can be divided into two types: transparent glazes and matt-opaque glazes. The glazes are subject to the combined effects of physical damage and chemical corrosion. The glaze surface became rougher through physical impact, and the increased surface area may also promote chemical corrosion. The mutual promotion of the two effects usually leads to more serious degradation. There are two possible corrosion mechanisms, depending on the glaze microstructure. Transparent glazes may experience the process of dissolution and reprecipitation. The crystallization-phase-separation structure in the matte glaze has lower chemical stability, which can contribute to more extensive corrosion. 

/pdf/research-on-the-degradation-of-ancient-longquan-celadons-in-2keekmod.pdf

Research on the degradation of ancient Longquan celadons in the Dalian Island shipwreck

As modern neural machine translation (NMT) systems have been widely deployed, their security vulnerabilities require close scrutiny. Most recently, NMT systems have been shown to be vulnerable to targeted attacks which cause them to produce specific, unsolicited, and even harmful translations. These attacks are usually exploited in a white-box setting, where adversarial inputs causing targeted translations are discovered for a known target system. However, this approach is less useful when the target system is black-box and unknown to the adversary (e.g., secured commercial systems). In this paper, we show that targeted attacks on black-box NMT systems are feasible, based on poisoning a small fraction of their parallel training data. We demonstrate that this attack can be realised practically via targeted corruption of web documents crawled to form the system’s training data. We then analyse the effectiveness of the targeted poisoning in two common NMT training scenarios, which are the one-off training and pre-train & fine-tune paradigms. Our results are alarming: even on the state-of-the-art systems trained with massive parallel data (tens of millions), the attacks are still successful (over 50% success rate) under surprisingly low poisoning rates (e.g., 0.006%). Lastly, we discuss potential defences to counter such attacks.

Chang Xu

Papers

A compact, CW mid-infrared intra-cavity Nd:Lu0.5Y0.5VO4∖KTA-OPO at 3.5 μm

Full-Stack Filters to Build Minimum Viable CNNs.

TDFL: Truth Discovery Based Byzantine Robust Federated Learning

Research on the degradation of ancient Longquan celadons in the Dalian Island shipwreck

Targeted Poisoning Attacks on Black-Box Neural Machine Translation.