Showing papers in "Information Fusion in 2015"
TL;DR: A general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods is presented and experimental results demonstrate that the proposed fusion framework can obtain state-of-the-art performance.
Abstract: Includes discussion on multi-scale transform (MST) based image fusion methods.Includes discussion on sparse representation (SR) based image fusion methods.Presents a general image fusion framework with MST and SR.Introduces several promising image fusion methods under the proposed framework.Provides a new image fusion toolbox. In image fusion literature, multi-scale transform (MST) and sparse representation (SR) are two most widely used signal/image representation theories. This paper presents a general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods. In our fusion framework, the MST is firstly performed on each of the pre-registered source images to obtain their low-pass and high-pass coefficients. Then, the low-pass bands are merged with a SR-based fusion approach while the high-pass bands are fused using the absolute values of coefficients as activity level measurement. The fused image is finally obtained by performing the inverse MST on the merged coefficients. The advantages of the proposed fusion framework over individual MST- or SR-based method are first exhibited in detail from a theoretical point of view, and then experimentally verified with multi-focus, visible-infrared and medical image fusion. In particular, six popular multi-scale transforms, which are Laplacian pyramid (LP), ratio of low-pass pyramid (RP), discrete wavelet transform (DWT), dual-tree complex wavelet transform (DTCWT), curvelet transform (CVT) and nonsubsampled contourlet transform (NSCT), with different decomposition levels ranging from one to four are tested in our experiments. By comparing the fused results subjectively and objectively, we give the best-performed fusion method under the proposed framework for each category of image fusion. The effect of the sliding window's step length is also investigated. Furthermore, experimental results demonstrate that the proposed fusion framework can obtain state-of-the-art performance, especially for the fusion of multimodal images.
952 citations
TL;DR: A novel image fusion method for multi-focus images with dense scale invariant feature transform (SIFT) that shows the great potential of image local features such as the dense SIFT used for image fusion.
Abstract: Multi-focus image fusion technique is an important approach to obtain a composite image with all objects in focus The key point of multi-focus image fusion is to develop an effective activity level measurement to evaluate the clarity of source images This paper proposes a novel image fusion method for multi-focus images with dense scale invariant feature transform (SIFT) The main novelty of this work is that it shows the great potential of image local features such as the dense SIFT used for image fusion Particularly, the local feature descriptor can not only be employed as the activity level measurement, but also be used to match the mis-registered pixels between multiple source images to improve the quality of the fused image In our algorithm, via the sliding window technique, the dense SIFT descriptor is first used to measure the activity level of source image patches to obtain an initial decision map, and then the decision map is refined with feature matching and local focus measure comparison Experimental results demonstrate that the proposed method can be competitive with or even outperform the state-of-the-art fusion methods in terms of both subjective visual perception and objective evaluation metrics
359 citations
TL;DR: This paper presents a novel multi-focus image fusion method in spatial domain that utilizes a dictionary which is learned from local patches of source images and outperforms existing state-of-the-art methods, in terms of visual and quantitative evaluations.
Abstract: Multi-focus image fusion has emerged as a major topic in image processing to generate all-focus images with increased depth-of-field from multi-focus photographs. Different approaches have been used in spatial or transform domain for this purpose. But most of them are subject to one or more of image fusion quality degradations such as blocking artifacts, ringing effects, artificial edges, halo artifacts, contrast decrease, sharpness reduction, and misalignment of decision map with object boundaries. In this paper we present a novel multi-focus image fusion method in spatial domain that utilizes a dictionary which is learned from local patches of source images. Sparse representation of relative sharpness measure over this trained dictionary are pooled together to get the corresponding pooled features. Correlation of the pooled features with sparse representations of input images produces a pixel level score for decision map of fusion. Final regularized decision map is obtained using Markov Random Field (MRF) optimization. We also gathered a new color multi-focus image dataset which has more variety than traditional multi-focus image sets. Experimental results demonstrate that our proposed method outperforms existing state-of-the-art methods, in terms of visual and quantitative evaluations.
343 citations
TL;DR: C-SPINE, a framework for Collaborative BSNs (CBSNs), is proposed and natively supports multi-sensor data fusion among CBSNs to enable joint data analysis such as filtering, time-dependent data integration and classification.
Abstract: Body Sensor Networks (BSNs) have emerged as the most effective technology enabling not only new e-Health methods and systems but also novel applications in human-centered areas such as electronic health care, fitness/welness systems, sport performance monitoring, interactive games, factory workers monitoring, and social physical interaction. Despite their enormous potential, they are currently mostly used only to monitor single individuals. Indeed, BSNs can proactively interact and collaborate to foster novel BSN applications centered on collaborative groups of individuals. In this paper, C-SPINE, a framework for Collaborative BSNs (CBSNs), is proposed. CBSNs are BSNs able to collaborate with each other to fulfill a common goal. They can support the development of novel smart wearable systems for cyberphysical pervasive computing environments. Collaboration therefore relies on interaction and synchronization among the CBSNs and on collaborative distributed computing atop the collaborating CBSNs. Specifically, collaboration is triggered upon CBSN proximity and relies on service-specific protocols allowing for managing services among the collaborating CBSNs. C-SPINE also natively supports multi-sensor data fusion among CBSNs to enable joint data analysis such as filtering, time-dependent data integration and classification. To demonstrate its effectiveness, C-SPINE is used to implement e-Shake, a collaborative CBSN system for the detection of emotions. The system is based on a multi-sensor data fusion schema to perform automatic detection of handshakes between two individuals and capture of possible heart-rate-based emotion reactions due to the individuals’ meeting.
260 citations
TL;DR: This survey focuses on the multi-source domain adaptation problem where there is more than one source domain available together with only one target domain, and examines how to select good sources and samples for the adaptation.
Abstract: Theoretical developments on multi-source domain adaptation are reviewed.Well developed algorithms on multi-source domain adaptation are reviewed and categorized.Performance measurements and benchmark data for multi-source domain adaptation are summarized.Interesting open problems that can be explored in future are discussed. In many machine learning algorithms, a major assumption is that the training and the test samples are in the same feature space and have the same distribution. However, for many real applications this assumption does not hold. In this paper, we survey the problem where the training samples and the test samples are from different distributions. This problem can be referred as domain adaptation. The training samples, always with labels, are obtained from what is called source domains, while the test samples, which usually have no labels or only a few labels, are obtained from what is called target domains. The source domains and the target domains are different but related to some extent; the learners can learn some information from the source domains for the learning of the target domains. We focus on the multi-source domain adaptation problem where there is more than one source domain available together with only one target domain. A key issue is how to select good sources and samples for the adaptation. In this survey, we review some theoretical results and well developed algorithms for the multi-source domain adaptation problem. We also discuss some open problems which can be explored in future work.
244 citations
TL;DR: Three types of fusion approaches are presented: the indirect approach, the optimization-based approach and the direct approach to solve GDM problems with heterogeneous preference structures.
Abstract: We review the fusion process with heterogeneous preference structures in GDM.We mainly summary and discuss three existing types of fusion approaches.We propose some open problems regarding the different fusion approaches. In group decision making (GDM), decision makers who have different experiential, cultural and educational backgrounds will naturally provide their preference information by heterogeneous preference structures (e.g., utility values, preference orderings, numerical preference relations and multigranular linguistic preference relations). To date, many studies have discussed GDM problems with heterogeneous preference structures. To provide a clear perspective on the fusion process with heterogeneous preference structures in GDM, this paper presents a review of three types of fusion approaches: the indirect approach, the optimization-based approach and the direct approach. Moreover, with respect to insights gained from prior researches, several open problems are proposed for the future research.
188 citations
TL;DR: An effective quadtree decomposition strategy is presented and the new weighted focus-measure performs better than the commonly used focus-measures on the detection of the focused regions, since it is sensitive to the homogeneous regions.
Abstract: The purpose of multi-focus image fusion is integrating the partially focused images into one single image which is focused everywhere. To achieve this purpose, we propose a new quadtree-based algorithm for multi-focus image fusion. In this work, an effective quadtree decomposition strategy is presented. According to the proposed decomposition strategy, the source images are decomposed into blocks with optimal sizes in a quadtree structure. And in this tree structure, the focused regions are detected by using a new weighted focus-measure, named as the sum of the weighted modified Laplacian. Finally, the focused regions could be well extracted from the source images and reconstructed to produce one fully focused image. Moreover, the new weighted focus-measure performs better than the commonly used focus-measures on the detection of the focused regions, since it is sensitive to the homogeneous regions. The proposed algorithm is simple yet effective, because of the quadtree decomposition strategy and the new weighted focus-measure. To do the comparison, the proposed algorithm is compared with several existing fusion algorithms, in both the qualitative and quantitative ways. The experimental results show that the proposed algorithm yields good results.
188 citations
TL;DR: Simulation results are presented to show that ROL/NDC gives a higher network lifetime than other similar schemes, such Mires++.
Abstract: One of the key challenges for research in wireless sensor networks is the development of routing protocols that provide application-specific service guarantees. This paper presents a new cluster-based Route Optimisation and Load-balancing protocol, called ROL, that uses various Quality of Service (QoS) metrics to meet application requirements. ROL combines several application requirements, specifically it attempts to provide an inclusive solution to prolong network life, provide timely message delivery and improve network robustness. It uses a combination of routing metrics that can be configured according to the priorities of user-level applications to improve overall network performance. To this end, an optimisation tool for balancing the communication resources for the constraints and priorities of user applications has been developed and Nutrient-flow-based Distributed Clustering (NDC), an algorithm for load balancing is proposed. NDC works seamlessly with any clustering algorithm to equalise, as far as possible, the diameter and the membership of clusters. This paper presents simulation results to show that ROL/NDC gives a higher network lifetime than other similar schemes, such Mires++. In simulation, ROL/NDC maintains a maximum of 7% variation from the optimal cluster population, reduces the total number of set-up messages by up to 60%, reduces the end-to-end delay by up to 56%, and enhances the data delivery ratio by up to 0.98% compared to Mires++.
152 citations
[...]
TL;DR: A path forward is proposed to advance the research on ocular recognition by improving the sensing technology, heterogeneous recognition for addressing interoperability, utilizing advanced machine learning algorithms for better representation and classification, and developing algorithms for ocular Recognition at a distance.
Abstract: Display Omitted A literature review of ocular modalities such as iris and periocular is presented.Information fusion approaches that combine ocular modalities with other modalities are reviewed.Future research directions are presented on sensing technologies, algorithms, and fusion approaches. Biometrics, an integral component of Identity Science, is widely used in several large-scale-county-wide projects to provide a meaningful way of recognizing individuals. Among existing modalities, ocular biometric traits such as iris, periocular, retina, and eye movement have received significant attention in the recent past. Iris recognition is used in Unique Identification Authority of India's Aadhaar Program and the United Arab Emirate's border security programs, whereas the periocular recognition is used to augment the performance of face or iris when only ocular region is present in the image. This paper reviews the research progression in these modalities. The paper discusses existing algorithms and the limitations of each of the biometric traits and information fusion approaches which combine ocular modalities with other modalities. We also propose a path forward to advance the research on ocular recognition by (i) improving the sensing technology, (ii) heterogeneous recognition for addressing interoperability, (iii) utilizing advanced machine learning algorithms for better representation and classification, (iv) developing algorithms for ocular recognition at a distance, (v) using multimodal ocular biometrics for recognition, and (vi) encouraging benchmarking standards and open-source software development.
138 citations
TL;DR: This survey aims to provide a comprehensive status of recent and current research on context-based Information Fusion (IF) systems, tracing back the roots of the original thinking behind the development of the concept of "context" and discussing the current strategies and techniques.
Abstract: Historical analysis of the evolution of context-based approaches in different research communities.Survey and discussion of context representation and techniques in mobile and pervasive computing, image processing, AI.Survey and discussion of context-based approaches in Information Fusion from a JDL perspective.Discussion and insights on key context concepts across different domains and their impacts on fusion systems.Proposal of novel architectural design aspects for context-aware fusion systems. This survey aims to provide a comprehensive status of recent and current research on context-based Information Fusion (IF) systems, tracing back the roots of the original thinking behind the development of the concept of "context". It shows how its fortune in the distributed computing world eventually permeated in the world of IF, discussing the current strategies and techniques, and hinting possible future trends. IF processes can represent context at different levels (structural and physical constraints of the scenario, a priori known operational rules between entities and environment, dynamic relationships modelled to interpret the system output, etc.). In addition to the survey, several novel context exploitation dynamics and architectural aspects peculiar to the fusion domain are presented and discussed.
131 citations
TL;DR: A Minimized Variance Model and an Entropy Weight Model are proposed to determine the expert weights in the cluster and the cluster weights, respectively, and synthesize these two types of weights into the final objective weights of the CMALGDM experts.
Abstract: We propose a two-layer weight determination model in a linguistic environment, when all the clustering results of the experts are known, to objectively obtain expert weights in complex multi-attribute large-group decision-making (CMALGDM) problems. The linguistic information considered in this paper involves both linguistic terms and linguistic intervals. We assume that, for CMALGDM problems, the final expert weights should be determined based on the expert weight in the cluster and on the cluster weights. This is mainly because experts in the same cluster will certainly make varying contributions to the cluster's overall consensus, and different clusters will also obtain the distinctive "cluster information quality". Hence, a Minimized Variance Model and an Entropy Weight Model are proposed to determine the expert weights in the cluster and the cluster weights, respectively. We then synthesize these two types of weights into the final objective weights of the CMALGDM experts. The feasibility of the two-layer weight determination model method for the CMALGDM problems is illustrated using a case study of salary reform for professors at a university.
TL;DR: A novel interval-valued intuitionistic fuzzy (IVIF) mathematical programming method for hybrid MCGDM considering alternative comparisons with hesitancy degrees, which is solved by the technically developed linear goal programming approach.
Abstract: As an important component of group decision making, the hybrid multi-criteria group decision making (MCGDM) is very complex and interesting in real applications. The purpose of this paper is to develop a novel interval-valued intuitionistic fuzzy (IVIF) mathematical programming method for hybrid MCGDM considering alternative comparisons with hesitancy degrees. The subjective preference relations between alternatives given by each decision maker (DM) are formulated as an IVIF set (IVIFS). The IVIFSs, intuitionistic fuzzy sets (IFSs), trapezoidal fuzzy numbers (TrFNs), linguistic variables, intervals and real numbers are used to represent the multiple types of criteria values. The information of criteria weights is incomplete. The IVIFS-type consistency and inconsistency indices are defined through considering the fuzzy positive and negative ideal solutions simultaneously. To determine the criteria weights, we construct a novel bi-objective IVIF mathematical programming of minimizing the inconsistency index and meanwhile maximizing the consistency index, which is solved by the technically developed linear goal programming approach. The individual ranking order of alternatives furnished by each DM is subsequently obtained according to the comprehensive relative closeness degrees of alternatives to the fuzzy positive ideal solution. The collective ranking order of alternatives is derived through establishing a new multi-objective assignment model. A real example of critical infrastructure evaluation is provided to demonstrate the applicability and effectiveness of this method.
TL;DR: The concept of a human-centric wireless sensor network is introduced, as an infrastructure that supports the capture and delivery of shared information in the field, and helps increase the information availability, and therefore, the efficiency and effectiveness of the emergency response process.
Abstract: When a natural disaster hits an urban area, the first 72 h after are the most critical. After that period the probability of finding survivors falls dramatically, therefore the search and rescue activities in that area must be conducted as quickly and effectively as possible. These activities are often improvised by first responders, stemming from the lack of communication and information support needed for making decisions in the field. Unfortunately, improvisations reduce the effectiveness and efficiency of the activities, in turn, affecting the number of people that can be rescued. To address this challenge, this article introduces the concept of a human-centric wireless sensor network, as an infrastructure that supports the capture and delivery of shared information in the field. These networks help increase the information availability, and therefore, the efficiency and effectiveness of the emergency response process. The use of these networks, which is complimentary to the currently used VHF/UHF radio systems, was evaluated using a simulated scenario and also through the feedback provided by an expert in urban search and rescue. The obtained results are highly encouraging.
TL;DR: The proposed approach is applied to solve the practical decision making problem concerned with the selection of Strategic Freight Forwarder of China Southern Airlines, and a comparison analysis with a similar approach is conducted to demonstrate the advantages of the proposed method.
Abstract: We present a method based on the distances to the ideal solutions to manage the heterogeneous decision data.We construct a maximizing deviation model to determine the optimal weights of criteria for each expert.We establish a minimizing deviation model to determine the weights of experts and identify the optimal alternative. Multiple criteria group decision making (MCGDM) problems with multiple formats of decision information, which are called heterogeneous MCGDM problems, have broad applications in the fields of natural science, social science, economy and management, etc. It is quite common that in heterogeneous MCGDM problems both the weights of the decision makers/experts and the criteria are partially known or completely unknown, but few studies focus on this issue. The purpose of this paper is to develop a deviation modeling method to deal with the heterogeneous MCGDM problems with incomplete weight information in which the decision information is expressed as real numbers, interval numbers, linguistic variables, intuitionistic fuzzy numbers, hesitant fuzzy elements and hesitant fuzzy linguistic term sets. There are three key issues being addressed in this approach, the first one is to construct a maximizing deviation optimal model in order to determine the optimal weights of criteria for each expert. Borrowing the idea of TOPSIS, the second one is to calculate the relative closeness indices of the alternatives for each expert. The third one is to establish a minimizing deviation optimal model based on the idea that the opinion of the individual expert should be consistent with that of the group to the greatest extent, which is used to determine the weights of experts and identify the optimal alternative. The proposed approach is applied to solve the practical decision making problem concerned with the selection of Strategic Freight Forwarder of China Southern Airlines, and a comparison analysis with a similar approach is conducted to demonstrate the advantages of the proposed method.
TL;DR: This paper proposes a novel technique which is a joint of pixel-level and feature-level fusion at the top-level’s wavelet sub-bands for face recognition, and proposes two alternating direction methods to solve the corresponding optimization problems for finding transformation matrices of dimension reduction and optimal fusion coefficients of the high frequency waveletSub-bands.
Abstract: The traditional wavelet-based approaches directly use the low frequency sub-band of wavelet transform to extract facial features However, the high frequency sub-bands also contain some important information corresponding to the edge and contour of face, reflecting the details of face, especially the top-level’s high frequency sub-bands In this paper, we propose a novel technique which is a joint of pixel-level and feature-level fusion at the top-level’s wavelet sub-bands for face recognition We convert the problem of finding the best pixel-level fusion coefficients of high frequency wavelet sub-bands to two optimization problems with the help of principal component analysis and linear discriminant analysis, respectively; and propose two alternating direction methods to solve the corresponding optimization problems for finding transformation matrices of dimension reduction and optimal fusion coefficients of the high frequency wavelet sub-bands The proposed methods make full use of four top-level’s wavelet sub-bands rather than the low frequency sub-band only Experiments are carried out on the FERET, ORL and AR face databases, which indicate that our methods are effective and robust
TL;DR: A personalized travel planning system that simultaneously considers all categories of user requirements and provides users with a travel schedule planning service that approximates automation and has better performance on the schedule adjustment, personalization, and feedback giving.
Abstract: Recently, the Internet has made a lot of services and products appear online provided by many tourism sectors. By this way, many information such as timetables, routes, accommodations, and restaurants are easily available to help travelers plan their travels. However, how to plan the most appropriate travel schedule under simultaneously considering several factors such as tourist attractions visiting, local hotels selecting, and travel budget calculation is a challenge. This gives rise to our interest in exploring the recommendation systems with relation to schedule recommendation. Additionally, the personalized concept is not implemented completely in most of travel recommendation systems. One notable problem is that they simply recommended the most popular travel routes or projects, and cannot plan the travel schedule. Moreover, the existing travel planning systems have limits in their capabilities to adapt to the changes based on users' requirements and planning results. To tackle these problems, we develop a personalized travel planning system that simultaneously considers all categories of user requirements and provides users with a travel schedule planning service that approximates automation. A novel travel schedule planning algorithm is embedded to plan travel schedules based on users' need. Through the user-adapted interface and adjustable results design, users can replace any unsatisfied travel unit to specific one. The feedback mechanism provides a better accuracy rate for next travel schedule to new users. An experiment was conducted to examine the satisfaction and use intention of the system. The results showed that participants who used the system with schedule planning have statistical significant on user satisfaction and use intention. We also analyzed the validity of applying the proposed algorithm to a user preference travel schedule through a number of practical system tests. In addition, comparing with other travel recommendation systems, our system had better performance on the schedule adjustment, personalization, and feedback giving.
TL;DR: A new MAS specially designed to manage data from WSNs, which was tested in a residential home for the elderly, and is based on virtual organizations, and incorporates social behaviors to improve the information fusion processes.
Abstract: With the increase of intelligent systems based on Multi-Agent Systems (MAS) and the use of Wireless Sensor Networks (WSN) in context-aware scenarios, information fusion has become an essential part of this kind of systems where the information is distributed among nodes or agents. This paper presents a new MAS specially designed to manage data from WSNs, which was tested in a residential home for the elderly. The proposed MAS architecture is based on virtual organizations, and incorporates social behaviors to improve the information fusion processes. The data that the system manages and analyzes correspond to the actual data of the activities of a resident. Data is collected as the information event counts detected by the sensors in a specific time interval, typically one day. We have designed a system that improves the quality of life of dependant people, especially elderly, by fusioning data obtained by multiple sensors and information of their daily activities. The high development of systems that extract and store information make essential to improve the mechanisms to deal with the avalanche of context data. In our case, the MAS approach results appropriated because each agent can represent an autonomous entity with different capabilities and offering different services but collaborating among them. Several tests have been performed to evaluate this platform and preliminary results and the conclusions are presented in this paper.
TL;DR: In this article, Markov Logic Networks (MLNs) are used for encoding uncertain knowledge and compute inferences according to observed evidence, and a mechanism to evaluate the level of completion of complex events is presented.
Abstract: The concepts of event and anomaly are important building blocks for developing a situational picture of the observed environment. We here relate these concepts to the JDL fusion model and demonstrate the power of Markov Logic Networks (MLNs) for encoding uncertain knowledge and compute inferences according to observed evidence. MLNs combine the expressive power of first-order logic and the probabilistic uncertainty management of Markov networks. Within this framework, different types of knowledge (e.g. a priori, contextual) with associated uncertainty can be fused together for situation assessment by expressing unobservable complex events as a logical combination of simpler evidences. We also develop a mechanism to evaluate the level of completion of complex events and show how, along with event probability, it could provide additional useful information to the operator. Examples are demonstrated on two maritime scenarios of rules for event and anomaly detection.
TL;DR: The proposed method uses the approach of likelihood-based outranking comparisons to address multiple criteria decision analysis (MCDA) problems based on interval type-2 trapezoidal fuzzy numbers and introduces the concepts of lower and upper likelihoods for acquiring the likelihood of an IT2TrF binary relationship.
Abstract: We develop an interval type-2 fuzzy PROMETHEE method to address MCDA problems.The proposed method uses the approach of likelihood-based outranking comparisons.We present novel likelihood-based preference functions based on outranking indices.We develop two algorithmic procedures to acquire partial and complete rankings.Comparative analysis validates the effectiveness of the proposed method. Based on the preference ranking organization method for enrichment evaluations (PROMETHEE), the purpose of this paper is to develop a new multiple criteria decision-making method that uses the approach of likelihood-based outranking comparisons within the environment of interval type-2 fuzzy sets. Uncertain and imprecise assessment of information often occurs in multiple criteria decision analysis (MCDA). The theory of interval type-2 fuzzy sets is useful and convenient for modeling impressions and quantifying the ambiguous nature of subjective judgments. Using the approach of likelihood-based outranking comparisons, this paper presents an interval type-2 fuzzy PROMETHEE method designed to address MCDA problems based on interval type-2 trapezoidal fuzzy (IT2TrF) numbers. This paper introduces the concepts of lower and upper likelihoods for acquiring the likelihood of an IT2TrF binary relationship and defines a likelihood-based outranking index to develop certain likelihood-based preference functions that correspond to several generalized criteria. The concept of comprehensive preference measures is proposed to determine IT2TrF exiting, entering, and net flows in the valued outranking relationships. In addition, this work establishes the concepts of a comprehensive outranking index, a comprehensive outranked index, and a comprehensive dominance index to induce partial and total preorders for the purpose of acquiring partial ranking and complete ranking, respectively, of the alternative actions. The feasibility and applicability of the proposed method are illustrated with two practical applications to the problem of landfill site selection and a car evaluation problem. Finally, a comparison with other relevant methods is conducted to validate the effectiveness of the proposed method.
TL;DR: This paper proposes a robust solution based on the use of multimodal sensor fusion that considerably improves the effectiveness of the algorithm and reduces computation time when compared with the classical inverse perspective mapping.
Abstract: Over the past years, inverse perspective mapping has been successfully applied to several problems in the field of Intelligent Transportation Systems. In brief, the method consists of mapping images to a new coordinate system where perspective effects are removed. The removal of perspective associated effects facilitates road and obstacle detection and also assists in free space estimation. There is, however, a significant limitation in the inverse perspective mapping: the presence of obstacles on the road disrupts the effectiveness of the mapping. The current paper proposes a robust solution based on the use of multimodal sensor fusion. Data from a laser range finder is fused with images from the cameras, so that the mapping is not computed in the regions where obstacles are present. As shown in the results, this considerably improves the effectiveness of the algorithm and reduces computation time when compared with the classical inverse perspective mapping. Furthermore, the proposed approach is also able to cope with several cameras with different lenses or image resolutions, as well as dynamic viewpoints.
TL;DR: Trajectory-level analysis indicates that the proposed adaptive MCS for partially-supervised learning of facial models over time allows for robust spatio-temporal video-to-video FR, and may therefore enhance security and situation analysis in video surveillance.
Abstract: Face recognition (FR) is employed in several video surveillance applications to determine if facial regions captured over a network of cameras correspond to a target individuals. To enroll target individuals, it is often costly or unfeasible to capture enough high quality reference facial samples a priori to design representative facial models. Furthermore, changes in capture conditions and physiology contribute to a growing divergence between these models and faces captured during operations. Adaptive biometrics seek to maintain a high level of performance by updating facial models over time using operational data. Adaptive multiple classifier systems (MCSs) have been successfully applied to video-to-video FR, where the face of each target individual is modeled using an ensemble of 2-class classifiers (trained using target vs. non-target samples). In this paper, a new adaptive MCS is proposed for partially-supervised learning of facial models over time based on facial trajectories. During operations, information from a face tracker and individual-specific ensembles is integrated for robust spatio-temporal recognition and for self-update of facial models. The tracker defines a facial trajectory for each individual that appears in a video, which leads to the recognition of a target individual if the positive predictions accumulated along a trajectory surpass a detection threshold for an ensemble. When the number of positive ensemble predictions surpasses a higher update threshold, then all target face samples from the trajectory are combined with non-target samples (selected from the cohort and universal models) to update the corresponding facial model. A learn-and-combine strategy is employed to avoid knowledge corruption during self-update of ensembles. In addition, a memory management strategy based on Kullback-Leibler divergence is proposed to rank and select the most relevant target and non-target reference samples to be stored in memory as the ensembles evolves. For proof-of-concept, a particular realization of the proposed system was validated with videos from Face in Action dataset. Initially, trajectories captured from enrollment videos are used for supervised learning of ensembles, and then videos from various operational sessions are presented to the system for FR and self-update with high-confidence trajectories. At a transaction level, the proposed approach outperforms baseline systems that do not adapt to new trajectories, and provides comparable performance to ideal systems that adapt to all relevant target trajectories, through supervised learning. Subject-level analysis reveals the existence of individuals for which self-updating ensembles with unlabeled facial trajectories provides a considerable benefit. Trajectory-level analysis indicates that the proposed system allows for robust spatio-temporal video-to-video FR, and may therefore enhance security and situation analysis in video surveillance.
TL;DR: An energy-efficient image prioritization framework is presented to cope with the fragility of traditional WVSNs and demonstrates the usefulness of the proposed method in terms of salient event coverage and reduced computational and transmission costs, as well as in helping analysts find semantically relevant visual information.
Abstract: In wireless visual sensor networks (WVSNs), streaming all imaging data is impractical due to resource constraints. Moreover, the sheer volume of surveillance videos inhibits the ability of analysts to extract actionable intelligence. In this work, an energy-efficient image prioritization framework is presented to cope with the fragility of traditional WVSNs. The proposed framework selects semantically relevant information before it is transmitted to a sink node. This is based on salient motion detection, which works on the principle of human cognitive processes. Each camera node estimates the background by a bootstrapping procedure, thus increasing the efficiency of salient motion detection. Based on the salient motion, each sensor node is classified as being high or low priority. This classification is dynamic, such that camera nodes toggle between high-priority and low-priority status depending on the coverage of the region of interest. High-priority camera nodes are allowed to access reliable radio channels to ensure the timely and reliable transmission of data. We compare the performance of this framework with other state-of-the-art methods for both single and multi-camera monitoring. The results demonstrate the usefulness of the proposed method in terms of salient event coverage and reduced computational and transmission costs, as well as in helping analysts find semantically relevant visual information.
TL;DR: This paper introduces various Nystrom methods, reviews different sampling methods for the Nystrom method and summarize them from the perspectives of both theoretical analysis and practical performance, and discusses some open machine learning problems related to Nystrom Methods.
Abstract: Nystrom methods are state-of-the-art techniques for large scale machine learning.Both the standard and enhanced Nystrom methods are reviewed.Different sampling methods are also reviewed and compared.Typical machine learning applications are summarized.Interesting open problems are discussed. Generating a low-rank matrix approximation is very important in large-scale machine learning applications. The standard Nystrom method is one of the state-of-the-art techniques to generate such an approximation. It has got rapid developments since being applied to Gaussian process regression. Several enhanced Nystrom methods such as ensemble Nystrom, modified Nystrom and SS-Nystrom have been proposed. In addition, many sampling methods have been developed. In this paper, we review the Nystrom methods for large-scale machine learning. First, we introduce various Nystrom methods. Second, we review different sampling methods for the Nystrom methods and summarize them from the perspectives of both theoretical analysis and practical performance. Then, we list several typical machine learning applications that utilize the Nystrom methods. Finally, we make our conclusions after discussing some open machine learning problems related to Nystrom methods.
TL;DR: A new hierarchical routing algorithm with high energy efficiency named EESSC is proposed which is based on the improved HAC clustering approach, and a re-cluster mechanism is designed to dynamic adjust the result of clustering to make sensor nodes organization more reasonable.
Abstract: In Wireless Sensor Networks (WSNs), energy efficiency is one of the most important factors influencing the networks' performance. Through a well designed routing algorithm, WSNs' energy efficiency can be improved evidently. Among various routing algorithms, hierarchical routing algorithms have advantages in improving nets' robustness and flexibility, and it is more appropriate for large scale of networks. In this paper, some typical hierarchical routing algorithms are introduced, and their advantages and defects are analyzed. Based on these analyses, a new hierarchical routing algorithm with high energy efficiency named EESSC is proposed which is based on the improved HAC clustering approach. In EESSC, the sensor nodes' residual energy would be taken into account in clustering operation, and a special packet head is defined to help update nodes' energy information when transmitting message among the nodes. When the clusters have been formed, the nodes in cluster would be arrayed in a list and cluster head would be rotated automatically by the order of list. And a re-cluster mechanism is designed to dynamic adjust the result of clustering to make sensor nodes organization more reasonable. At last, EESSC is compared to other typical hierarchical routing algorithms in a series of experiments, and the experiments' result which proves that EESSC has obviously improved the WSNs' energy efficiency has been analyzed.
TL;DR: Seven knowledge fusion patterns have been discovered: simple fusion, extension, instantiated fusion, configured fusion, adaptation, flat fusion, and historical fusion.
Abstract: The here presented research focuses on the context-based knowledge fusion patterns Patterns are discovered based on an analysis and investigation of knowledge fusion processes in a context aware decision support system at the operational stage of the system functioning At this stage the context-based knowledge fusion processes are manifested around the context The patterns are generalized in regard to the following three aspects: (1) the effects that the knowledge fusion processes produce in the system; (2) the preservation of internal structures for the context and multiple sources the information/knowledge is fused from; and (3) the preservation of multiple sources and the context autonomies At that, seven knowledge fusion patterns have been discovered: simple fusion, extension, instantiated fusion, configured fusion, adaptation, flat fusion, and historical fusion
TL;DR: This paper introduces the semi-uninorm based ordered weighted averaging (SUOWA) operators, a new class of aggregation functions that, as WOWA operators, simultaneously generalize weighted means and OWA operators.
Abstract: A new class of aggregation operators is proposed to generalize weighted means and OWA operators.SUOWA operators are defined by using Choquet integral.These operators are continuous, monotonic, idempotent, compensative and homogeneous of degree 1 functions. In this paper we introduce the semi-uninorm based ordered weighted averaging (SUOWA) operators, a new class of aggregation functions that, as WOWA operators, simultaneously generalize weighted means and OWA operators. To do this we take into account that weighted means and OWA operators are particular cases of Choquet integral. So, SUOWA operators are Choquet integral-based operators where their capacities are constructed by using semi-uninorms and the values of the capacities associated to the weighted means and the OWA operators. We also show some interesting properties of these new operators and provide examples showing that SUOWA and WOWA operators are different classes of aggregation operators.
TL;DR: This paper proposes a low-complexity distributed data replication mechanism to increase the resilience of WSN-based distributed storage at large scale and proposes a simple, yet accurate, analytical modeling framework and an extensive simulation campaign, which complement experimental results on the SensLab testbed.
Abstract: In the emerging field of the Internet of Things (IoT), Wireless Sensor Networks (WSNs) have a key role to play in sensing and collecting measures on the surrounding environment. In the deployment of large scale observation systems in remote areas, when there is not a permanent connection with the Internet, WSNs are calling for replication and distributed storage techniques that increase the amount of data stored within the WSN and reduce the probability of data loss. Unlike conventional network data storage, WSN-based distributed storage is constrained by the limited resources of the sensors. In this paper, we propose a low-complexity distributed data replication mechanism to increase the resilience of WSN-based distributed storage at large scale. In particular, we propose a simple, yet accurate, analytical modeling framework and an extensive simulation campaign, which complement experimental results on the SensLab testbed. The impact of several key parameters on the system performance is investigated.
TL;DR: The results demonstrate that the generative model is able to model both spatial and temporal datasets and may be used for the purpose of developing and evaluating counter-piracy methods and algorithms.
Abstract: A generative model for modelling maritime vessel behaviour is proposed. The model is a novel variant of the dynamic Bayesian network (DBN). The proposed DBN is in the form of a switching linear dynamic system (SLDS) that has been extended into a larger DBN. The application of synthetic data fabrication of maritime vessel behaviour is considered. Behaviour of various vessels in a maritime piracy situation is simulated. A means to integrate information from context based external factors that influence behaviour is provided. Simulated observations of the vessels kinematic states are generated. The generated data may be used for the purpose of developing and evaluating counter-piracy methods and algorithms. A novel methodology for evaluating and optimising behavioural models such as the proposed model is presented. The log-likelihood, cross entropy, Bayes factor and the Bhattacharyya distance measures are applied for evaluation. The results demonstrate that the generative model is able to model both spatial and temporal datasets.
TL;DR: A novel approach for crowd density measure, in which local information at pixel level substitutes a global crowd level or a number of people per-frame, is proposed, which demonstrates good performances for detection, tracking, behavior analysis, and privacy preservation.
Abstract: Crowd density analysis is a crucial component in visual surveillance mainly for security monitoring. This paper proposes a novel approach for crowd density measure, in which local information at pixel level substitutes a global crowd level or a number of people per-frame. The proposed approach consists of generating automatic crowd density maps using local features as an observation of a probabilistic density function. It also involves a feature tracking step which excludes feature points belonging to the background. This process is favorable for the later density estimation as the influence of features irrelevant to the underlying crowd density is removed. Since the proposed crowd density conveys rich information about the local distributions of persons in the scene, we employ it as a side information to complement other tasks related to video surveillance in crowded scenes. First, since conventional detection and tracking methods are hard to be scalable to crowds, we use the proposed crowd density to enhance detection and tracking in videos of high density crowds. Second, we employ the local density together with regular motion patterns as crowd attributes for high level applications such as crowd change detection and event recognition. Third, we investigate the concept of crowd context-aware privacy protection by adjusting the obfuscation level according to the crowd density. In the experimental results, our proposed approach for crowd density estimation is evaluated on videos from different datasets, and the results demonstrate the effectiveness of feature tracks for crowd measurements. Moreover, the employment of crowd density in other applications demonstrate good performances for detection, tracking, behavior analysis, and privacy preservation.
TL;DR: A fusion system for context-based situation and threat assessment with application to harbor surveillance and Belief-based Argumentation to evaluate the threat posed by suspicious vessels is proposed.
Abstract: Harbor surveillance is a critical and challenging part of maritime security procedures. Building a surveillance picture to support decision makers in detection of potential threats requires the integration of data and information coming from heterogeneous sources. Context plays a key role in achieving this task by providing expectations, constraints and additional information for inference about the items of interest. This paper proposes a fusion system for context-based situation and threat assessment with application to harbor surveillance. The architecture of the system is organized in two levels. The lowest level uses an ontological model to formally represent input data and to classify harbor objects and basic situations by deductive reasoning according to the harbor regulations. The higher level applies Belief-based Argumentation to evaluate the threat posed by suspicious vessels. The functioning of the system is illustrated with several examples that reproduce common harbor scenarios.