Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source code is at this https URL

YOLOv4: Optimal Speed and Accuracy of Object Detection

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Automatic estimation of salient object regions across images, without any prior assumption or knowledge of the contents of the corresponding scenes, enhances many computer vision and computer graphics applications. We introduce a regional contrast based salient object detection algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherence scores. The proposed algorithm is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps. These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervised salient object segmentation. We extensively evaluated our algorithm using traditional salient object detection datasets, as well as a more challenging Internet image dataset. Our experimental results demonstrate that our algorithm consistently outperforms 15 existing salient object detection and segmentation methods, yielding higher precision and better recall rates. We also show that our algorithm can be used to efficiently extract salient object masks from Internet images, enabling effective sketch-based image retrieval (SBIR) via simple shape comparisons. Despite such noisy internet images, where the saliency regions are ambiguous, our saliency guided image retrieval achieves a superior retrieval rate compared with state-of-the-art SBIR methods, and additionally provides important target object region information.

https://mmcheng.net/mftp/SalObj/PosterSaliency.pdf

Global contrast based salient region detection

The PASCAL Visual Object Classes Challenge

It is known that media outlets, such as CNN and FOX, have intrinsic political bias that is reflected in their news reports. The computational prediction of such bias has broad application prospects. However, the prediction is difficult via directly analyzing the news content without high-level context. In contrast, social signals (e.g., the network structure of media followers) provide inspiring cues to uncover such bias. In this article, we realize the first attempt of predicting the latent bias of media outlets by analyzing their social network structures. In particular, we address two key challenges: network sparsity and label sparsity. The network sparsity refers to the partial sampling of the entire follower network in practical analysis and computing, whereas the label sparsity refers to the difficulty of annotating sufficient labels to train the prediction model. To cope with the network sparsity, we propose a hybrid sampling strategy to construct a training corpus that contains network information from micro to macro views. Based on this training corpus, a semi-supervised network embedding approach is proposed to learn low-dimensional yet effective network representations. To deal with the label sparsity, we adopt a graph-based label propagation scheme to supplement the missing links and augment label information for model training. The preceding two steps are iteratively optimized to reinforce each other. We further collect a large-scale dataset containing social networks of 10 media outlets together with about 300,000 followers and more than 5 million connections. Over this dataset, we compare our model to a range of state of the art. Superior performance gains demonstrate the merits of the proposed approach. More importantly, the experimental results and analyses confirm the validity of our approach for the computerized prediction of media bias.

https://dl.acm.org/doi/pdf/10.1145/3422181

Uncovering Media Bias via Social Network Learning

In this demo, a cross-media public sentiment analysis system is presented. The system presents and visualizes the sentiments of microblog data by organizing the results by region, topic, and content, respectively. Such sentiment is obtained by fusing of sentiment classification scores from both visual and textual channel. In such a way, social multimedia sentiment is shown in a multi-level and user-friendly form.

A Cross-media Sentiment Analytics Platform For Microblog

The proliferation of mobile devices has led to a bright new stage in which multimedia search and analysis are increasingly moving from the desktop to the cloud. Nowadays, it has become convenient to capture images and videos on the mobile end and associate them with social and contextual metadata such as comments and Global Positioning System (GPS) tags. Such a hybrid data structure can benefit a wide variety of potential multimedia applications on the mobile end, such as location recognition, landmark search, augmented reality, and commercial recommendations. One intrinsic potential is to explore large-scale social multimedia to assist and facilitate location-related applications, which is further promoted by the evolution of mobile devices. With the combination of social and mobile cues, several problems that previously had been difficult to tackle in multimedia content analysis have become more tractable. For example, multiple images can be captured for accurate object research and recognition. And in the case of visual search, large-scale image repository can be significantly pruned by employing GPS tags to filter out irrelevant images with respect to the query. It has become ever emerging when the analytics of mobile social multimedia face the big data era. Several challenges have been raised, including, for instance, how to design a suitable cloud/distributed architecture, how to carry out online computing efficiently, and how to handle the scalability issue. When being applied to big data applications, many of the existing tools and systems for mobile social multimedia analytics would become questionable. On the other hand, the big data itself has provided a unique opportunity to accomplish the above tasks. The goals of this special issue will be threefold: (1) present new theories and new application scenarios on mobile social multimedia computing, communication, and application; (2) survey the recent progress in this area; (3) discuss new applications and problems.

Mobile Social Multimedia Analytics in the Big Data Era: An Introduction to the Special Issue

Does there exist a compact set of keywords that can completely and effectively cover the image annotation problem by expanding from it? In this paper, we answer this question by presenting a complete set framework for image annotation, which is motivated by the existence of semantic ontology. To generate this set, we propose a cross model optimization strategy from both textual and visual information for topic decomposition, based on a so-called Bipartite LSA model, which minimize multimodal error energy functions in a probabilistic Latent Semantic Analysis model. To achieve complete set based annotation, we present a Gaussian-Kernel-Generative process based keyword generation procedure, which analogizes keyword annotation in a probabilistic generative manner. A group of experiments is performed on Washington University image database and 80,000 Flickr images with comparisons to the state-of-the-arts. Finally, potential advantages and future improvements of our framework are discussed outside the scope of topic modeling.

What is a complete set of keywords for image description & annotation on the web

In this paper, we present a hypergraph neural networks (HGNN) framework for data representation learning, which can encode high-order data correlation in a hypergraph structure. Confronting the challenges of learning representation for complex data in real practice, we propose to incorporate such data structure in a hypergraph, which is more flexible on data modeling, especially when dealing with complex data. In this method, a hyperedge convolution operation is designed to handle the data correlation during representation learning. In this way, traditional hypergraph learning procedure can be conducted using hyperedge convolution operations efficiently. HGNN is able to learn the hidden layer representation considering the high-order data structure, which is a general framework considering the complex data correlations. We have conducted experiments on citation network classification and visual object recognition tasks and compared HGNN with graph convolutional networks and other traditional methods. Experimental results demonstrate that the proposed HGNN method outperforms recent state-of-the-art methods. We can also reveal from the results that the proposed HGNN is superior when dealing with multi-modal data compared with existing methods.

Rongrong Ji

Papers

Uncovering Media Bias via Social Network Learning

A Cross-media Sentiment Analytics Platform For Microblog

Mobile Social Multimedia Analytics in the Big Data Era: An Introduction to the Special Issue

What is a complete set of keywords for image description & annotation on the web

Hypergraph Neural Networks