/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

Matrix Factorization Techniques for Recommender Systems

Christensen, R. (2002), Plane Answers to Complex Questions: The Theory of Linear Models (3rd ed.), New York: Springer-Verlag. Crocker, D. C. (1980), Review of Linear Regression Analysis, by G. A. F. Seber, Technometrics, 22, 130. Datta, B. N. (1995), Numerical Linear Algebra and Applications, Paci c Grove, CA: Brooks/Cole. Draper, N. R. (2002), “Applied Regression Analysis Bibliography Update 2000–2001,” Communications in Statistics: Theory and Methods, 2051– 2075. Golub, G. H., and Van Loan, C. F. (1996), Matrix Computations (3rd ed.), Baltimore, MD: Johns Hopkins University Press. Graybill, F. A. (2000), Theory and Application of the Linear Model, Paci c Grove, CA: Brooks/Cole. Hocking, R. R. (2003), Methods and Applications of Linear Models: Regression and the Analysis of Variance (2nd ed.), New York: Wiley. Porat, B. (1993), Digital Processing of Random Signals, Englewood Cliffs, NJ: Prentice-Hall. Ravishanker, N., and Dey, D. K. (2002), A First Course in Linear Model Theory, Boca Raton, FL: Chapman and Hall/CRC. White, H. (1984), Asymptotic Theory for Econometricians, Orlando, FL: Academic Press.

Generalized Estimating Equations

Weak Convergence and Empirical Processes - A. W. van der Vaart; J. A. Wellner.

1 Introduction.- 2 Experiments, Deficiencies, Distances v.- 2.1 Comparing Risk Functions.- 2.2 Deficiency and Distance between Experiments.- 2.3 Likelihood Ratios and Blackwell's Representation.- 2.4 Further Remarks on the Convergence of Distri butions of Likelihood Ratios.- 2.5 Historical Remarks.- 3 Contiguity - Hellinger Transforms.- 3.1 Contiguity.- 3.2 Hellinger Distances, Hellinger Transforms.- 3.3 Historical Remarks.- 4 Gaussian Shift and Poisson Experiments.- 4.1 Introduction.- 4.2 Gaussian Experiments.- 4.3 Poisson Experiments.- 4.4 Historical Remarks.- 5 Limit Laws for Likelihood Ratios.- 5.1 Introduction.- 5.2 Auxiliary Results.- 5.2.1 Lindeberg's Procedure.- 5.2.2 Levy Splittings.- 5.2.3 Paul Levy's Symmetrization Inequalities.- 5.2.4 Conditions for Shift-Compactness.- 5.2.5 A Central Limit Theorem for Infinitesimal Arrays.- 5.2.6 The Special Case of Gaussian Limits.- 5.2.7 Peano Differentiable Functions.- 5.3 Limits for Binary Experiments.- 5.4 Gaussian Limits.- 5.5 Historical Remarks.- 6 Local Asymptotic Normality.- 6.1 Introduction.- 6.2 Locally Asymptotically Quadratic Families.- 6.3 A Method of Construction of Estimates.- 6.4 Some Local Bayes Properties.- 6.5 Invariance and Regularity.- 6.6 The LAMN and LAN Conditions.- 6.7 Additional Remarks on the LAN Conditions.- 6.8 Wald's Tests and Confidence Ellipsoids.- 6.9 Possible Extensions.- 6.10 Historical Remarks.- 7 Independent, Identically Distributed Observations.- 7.1 Introduction.- 7.2 The Standard i.i.d. Case: Differentiability in Quadratic Mean.- 7.3 Some Examples.- 7.4 Some Nonparametric Considerations.- 7.5 Bounds on the Risk of Estimates.- 7.6 Some Cases Where the Number of Observations Is Random.- 7.7 Historical Remarks.- 8 On Bayes Procedures.- 8.1 Introduction.- 8.2 Bayes Procedures Behave Nicely.- 8.3 The Bernstein-von Mises Phenomenon.- 8.4 A Bernstein-von Mises Result for the i.i.d. Case.- 8.5 Bayes Procedures Behave Miserably.- 8.6 Historical Remarks.- Author Index.

Asymptotics in Statistics–Some Basic Concepts

We introduce a family of adaptive estimators on graphs, based on penalizing the l1 norm of discrete graph differences. This generalizes the idea of trend filtering (Kim et al., 2009; Tibshirani, 2014), used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filtering exhibits a level of local adaptivity unmatched by the usual l2-based graph smoothers. It is also defined by a convex minimization problem that is readily solved (e.g., by fast ADMM or Newton algorithms). We demonstrate the merits of graph trend filtering through both examples and theory.

/pdf/trend-filtering-on-graphs-zyta2s5lne.pdf

Trend Filtering on Graphs

Temporal information is crucial for recommendation problems because user preferences are naturally dynamic in the real world. Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed for better use of the temporal ordering of items that each user has engaged with. In particular, the SASRec model, inspired by the popular Transformer model in natural languages processing, has achieved state-of-the-art results. However, SASRec, just like the original Transformer model, is inherently an un-personalized model and does not include personalized user embeddings. To overcome this limitation, we propose a Personalized Transformer (SSE-PT) model, outperforming SASRec by almost 5% in terms of NDCG@10 on 5 real-world datasets. Furthermore, after examining some random users’ engagement history, we find our model not only more interpretable but also able to focus on recent engagement patterns for each user. Moreover, our SSE-PT model with a slight modification, which we call SSE-PT++, can handle extremely long sequences and outperform SASRec in ranking results with comparable training speed, striking a balance between performance and speed requirements. Our novel application of the Stochastic Shared Embeddings (SSE) regularization is essential to the success of personalization. Code and data are open-sourced at https://github.com/wuliwei9278/SSE-PT.

/pdf/sse-pt-sequential-recommendation-via-personalized-21blgkutsh.pdf

SSE-PT: Sequential Recommendation Via Personalized Transformer

The fused lasso was proposed recently to enable recovery of high-dimensional patterns which are piece-wise constant on a graph, by penalizing the ‘1-norm of dierences of measurements at vertices that share an edge. While there have been some attempts at coming up with ecient algorithms for solving the fused lasso optimization, a theoretical analysis of its performance is mostly lacking except for the simple linear graph topology. In this paper, we investigate sparsistency of fused lasso for general graph structures, i.e. its ability to correctly recover the exact support of piece-wise constant graphstructured patterns asymptotically (for largescale graphs). To emphasize this distinction over previous work, we will refer to it as Edge Lasso. We focus on the (structured) normal means setting, and our results provide necessary and sucient conditions on the graph properties as well as the signal-to-noise ratio needed to ensure sparsistency. We examplify our results using simple graph-structured patterns, and demonstrate that in some cases fused lasso is sparsistent at very weak signal-to-noise ratios (scaling as p (logn)=jAj, where n is the number of vertices in the graph and A is the smallest set of vertices with constant activation). In other cases, it performs no better than thresholding the dierence

/pdf/sparsistency-of-the-edge-lasso-over-graphs-1hvv5osfie.pdf

Sparsistency of the Edge Lasso over Graphs

/pdf/trend-filtering-on-graphs-4g4svlnzhp.pdf

Trend filtering on graphs

We consider the change-point detection problem of deciding, based on noisy measurements, whether an unknown signal over a given graph is constant or is instead piecewise constant over two connected induced subgraphs of relatively low cut size. We analyze the corresponding generalized likelihood ratio (GLR) statistics and relate it to the problem of finding a sparsest cut in a graph. We develop a tractable relaxation of the GLR statistic based on the combinatorial Laplacian of the graph, which we call the spectral scan statistic, and analyze its properties. We show how its performance as a testing procedure depends directly on the spectrum of the graph, and use this result to explicitly derive its asymptotic properties on few significant graph topologies. Finally, we demonstrate both theoretically and by simulations that the spectral scan statistic can outperform naive testing procedures based on edge thresholding and $\chi^2$ testing.

James Sharpnack

Papers

Trend Filtering on Graphs

SSE-PT: Sequential Recommendation Via Personalized Transformer

Sparsistency of the Edge Lasso over Graphs

Trend filtering on graphs

Changepoint Detection over Graphs with the Spectral Scan Statistic