Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

Real and Complex Analysis

Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network

Convex Analysis: (pms-28)

SUMMARY This survey provides a brief introduction to compressed sensing as well as several major algorithms to solve it and its various applications to communications systems. We firstly review linear simultaneous equations as ill-posed inverse problems, since the idea of compressed sensing could be best understood in the context of the linear equations. Then, we consider the problem of compressed sensing as an underdetermined linear system with a prior information that the true solution is sparse, and explain the sparse signal recovery based on � 1 optimization, which plays the central role in compressed sensing, with some intuitive explanations on the optimization problem. Moreover, we introduce some important properties of the sensing matrix in order to establish the guarantee of the exact recovery of sparse signals from the underdetermined system. After summarizing several major algorithms to obtain a sparse solution focusing on the � 1 optimization and the greedy approaches, we introduce applications of compressed sensing to communications systems, such as wireless channel estimation, wireless sensor network, network tomography, cognitive radio,

/pdf/a-user-s-guide-to-compressed-sensing-for-communications-4kovoff7iw.pdf

A User's Guide to Compressed Sensing for Communications Systems

In this paper, we propose a paradigm of control, called a maximum hands-off control. A hands-off control is defined as a control that has a short support per unit time. The maximum hands-off control is the minimum support (or sparsest) per unit time among all controls that achieve control objectives. For finite horizon continuous-time control, we show the equivalence between the maximum hands-off control and $L^{1}$ -optimal control under a uniqueness assumption called normality. This result rationalizes the use of $L^{1}$ optimality in computing a maximum hands-off control. The same result is obtained for discrete-time hands-off control. We also propose an $L^{1}/L^{2}$ -optimal control to obtain a smooth hands-off control. Furthermore, we give a self-triggered feedback control algorithm for linear time-invariant systems, which achieves a given sparsity rate and practical stability in the case of plant disturbances. An example is included to illustrate the effectiveness of the proposed control.

Maximum Hands-Off Control: A Paradigm of Control Effort Minimization

We study feedback control over erasure channels with packet-dropouts. To achieve robustness with respect to packet-dropouts, the controller transmits data packets containing plant input predictions, which minimize a finite horizon cost function. To reduce the data size of packets, we propose to adopt sparsity-promoting optimizations, namely, $ \ell ^{1} {\rm \mathchar"702D }\ell ^{2}$ and $\ell ^{2}$ -constrained $\ell ^{0}$ optimizations, for which efficient algorithms exist. We show how to design the tuning parameters to ensure (practical) stability of the resulting feedback control systems when the number of consecutive packet-dropouts is bounded.

/pdf/sparse-packetized-predictive-control-for-networked-control-3dlvtocbi8.pdf

Sparse Packetized Predictive Control for Networked Control Over Erasure Channels

This paper proposes a min-max design of noise-shaping delta-sigma modulators. We first characterize the all stabilizing loop-filters for a linearized modulator model. By this characterization, we formulate the design problem of lowpass, bandpass, and multi-band modulators as minimization of the maximum magnitude of the noise transfer function (NTF) in fixed frequency band(s). We show that this optimization minimizes the worst-case reconstruction error, and hence improves the SNR (signal-to-noise ratio) of the modulator. The optimization is reduced to an optimization with a linear matrix inequality (LMI) via the generalized KYP (Kalman-Yakubovich-Popov) lemma. The obtained NTF is an FIR (finite-impulse-response) filter, which is favorable in view of implementation. We also derive a stability condition for the nonlinear model of delta-sigma modulators with general quantizers including uniform ones. This condition is described as an norm condition, which is reduced to an LMI via the KYP lemma. Design examples show advantages of our design.

/pdf/frequency-domain-min-max-optimization-of-noise-shaping-delta-335hlgmeeq.pdf

Frequency Domain Min-Max Optimization of Noise-Shaping Delta-Sigma Modulators

In this letter, we consider a problem of reconstructing an unknown discrete signal taking values in a finite alphabet from incomplete linear measurements. The difficulty of this problem is that the computational complexity of the reconstruction is exponential as it is. To overcome this difficulty, we extend the idea of compressed sensing, and propose to solve the problem by minimizing the sum of weighted absolute values. We assume that the probability distribution defined on an alphabet is known, and formulate the reconstruction problem as linear programming. Examples are shown to illustrate that the proposed method is effective.

/pdf/discrete-signal-reconstruction-by-sum-of-absolute-values-1o4ubfqpcr.pdf

Masaaki Nagahara

Papers

A User's Guide to Compressed Sensing for Communications Systems

Maximum Hands-Off Control: A Paradigm of Control Effort Minimization

Sparse Packetized Predictive Control for Networked Control Over Erasure Channels

Frequency Domain Min-Max Optimization of Noise-Shaping Delta-Sigma Modulators

Discrete Signal Reconstruction by Sum of Absolute Values