A simple cosmological model with only six parameters (matter density, Omega_m h^2, baryon density, Omega_b h^2, Hubble Constant, H_0, amplitude of fluctuations, sigma_8, optical depth, tau, and a slope for the scalar perturbation spectrum, n_s) fits not only the three year WMAP temperature and polarization data, but also small scale CMB data, light element abundances, large-scale structure observations, and the supernova luminosity/distance relationship. Using WMAP data only, the best fit values for cosmological parameters for the power-law flat LCDM model are (Omega_m h^2, Omega_b h^2, h, n_s, tau, sigma_8) = 0.1277+0.0080-0.0079, 0.02229+-0.00073, 0.732+0.031-0.032, 0.958+-0.016, 0.089+-0.030, 0.761+0.049-0.048). The three year data dramatically shrink the allowed volume in this six dimensional parameter space. Assuming that the primordial fluctuations are adiabatic with a power law spectrum, the WMAP data_alone_ require dark matter, and favor a spectral index that is significantly less than the Harrison-Zel'dovich-Peebles scale-invariant spectrum (n_s=1, r=0). Models that suppress large-scale power through a running spectral index or a large-scale cut-off in the power spectrum are a better fit to the WMAP and small scale CMB data than the power-law LCDM model: however, the improvement in the fit to the WMAP data is only Delta chi^2 = 3 for 1 extra degree of freedom. The combination of WMAP and other astronomical data yields significant constraints on the geometry of the universe, the equation of state of the dark energy, the gravitational wave energy density, and neutrino properties. Consistent with the predictions of simple inflationary theories, we detect no significant deviations from Gaussianity in the CMB maps.

/pdf/wilkinson-microwave-anisotropy-probe-wmap-three-year-results-2l0o1lq78p.pdf

Wilkinson Microwave Anisotropy Probe (WMAP) three year results: implications for cosmology

The Wilkinson Microwave Anisotropy Probe (WMAP) 5-year data provide stringent limits on deviations from the minimal, six-parameter Λ cold dark matter model. We report these limits and use them to constrain the physics of cosmic inflation via Gaussianity, adiabaticity, the power spectrum of primordial fluctuations, gravitational waves, and spatial curvature. We also constrain models of dark energy via its equation of state, parity-violating interaction, and neutrino properties, such as mass and the number of species. We detect no convincing deviations from the minimal model. The six parameters and the corresponding 68% uncertainties, derived from the WMAP data combined with the distance measurements from the Type Ia supernovae (SN) and the Baryon Acoustic Oscillations (BAO) in the distribution of galaxies, are: Ω b h 2 = 0.02267+0.00058 –0.00059, Ω c h 2 = 0.1131 ± 0.0034, ΩΛ = 0.726 ± 0.015, ns = 0.960 ± 0.013, τ = 0.084 ± 0.016, and at k = 0.002 Mpc-1. From these, we derive σ8 = 0.812 ± 0.026, H 0 = 70.5 ± 1.3 km s-1 Mpc–1, Ω b = 0.0456 ± 0.0015, Ω c = 0.228 ± 0.013, Ω m h 2 = 0.1358+0.0037 –0.0036, z reion = 10.9 ± 1.4, and t 0 = 13.72 ± 0.12 Gyr. With the WMAP data combined with BAO and SN, we find the limit on the tensor-to-scalar ratio of r 1 is disfavored even when gravitational waves are included, which constrains the models of inflation that can produce significant gravitational waves, such as chaotic or power-law inflation models, or a blue spectrum, such as hybrid inflation models. We obtain tight, simultaneous limits on the (constant) equation of state of dark energy and the spatial curvature of the universe: –0.14 < 1 + w < 0.12(95%CL) and –0.0179 < Ω k < 0.0081(95%CL). We provide a set of WMAP distance priors, to test a variety of dark energy models with spatial curvature. We test a time-dependent w with a present value constrained as –0.33 < 1 + w 0 < 0.21 (95% CL). Temperature and dark matter fluctuations are found to obey the adiabatic relation to within 8.9% and 2.1% for the axion-type and curvaton-type dark matter, respectively. The power spectra of TB and EB correlations constrain a parity-violating interaction, which rotates the polarization angle and converts E to B. The polarization angle could not be rotated more than –59 < Δα < 24 (95% CL) between the decoupling and the present epoch. We find the limit on the total mass of massive neutrinos of ∑m ν < 0.67 eV(95%CL), which is free from the uncertainty in the normalization of the large-scale structure data. The number of relativistic degrees of freedom (dof), expressed in units of the effective number of neutrino species, is constrained as N eff = 4.4 ± 1.5 (68%), consistent with the standard value of 3.04. Finally, quantitative limits on physically-motivated primordial non-Gaussianity parameters are –9 < f local NL < 111 (95% CL) and –151 < f equil NL < 253 (95% CL) for the local and equilateral models, respectively.

https://iopscience.iop.org/article/10.1088/0067-0049/180/2/330/pdf

Five-year wilkinson microwave anisotropy probe observations: cosmological interpretation

[Crohn's disease].

The collection of digital information by governments, corporations, and individuals has created tremendous opportunities for knowledge- and information-based decision making. Driven by mutual benefits, or by regulations that require certain data to be published, there is a demand for the exchange and publication of data among various parties. Data in its original form, however, typically contains sensitive information about individuals, and publishing such data will violate individual privacy. The current practice in data publishing relies mainly on policies and guidelines as to what types of data can be published and on agreements on the use of published data. This approach alone may lead to excessive data distortion or insufficient protection. Privacy-preserving data publishing (PPDP) provides methods and tools for publishing useful information while preserving data privacy. Recently, PPDP has received considerable attention in research communities, and many approaches have been proposed for different data publishing scenarios. In this survey, we will systematically summarize and evaluate different approaches to PPDP, study the challenges in practical data publishing, clarify the differences and requirements that distinguish PPDP from other related problems, and propose future research directions.

/pdf/privacy-preserving-data-publishing-a-survey-of-recent-4o22vc981u.pdf

Privacy-preserving data publishing: A survey of recent developments

Generalized Linear Models (2nd ed.)

Higher criticism, or second-level significance testing, is a multiple-comparisons concept mentioned in passing by Tukey. It concerns a situation where there are many independent tests of significance and one is interested in rejecting the joint null hypothesis. Tukey suggested comparing the fraction of observed significances at a given α-level to the expected fraction under the joint null. In fact, he suggested standardizing the difference of the two quantities and forming a z-score; the resulting z-score tests the significance of the body of significance tests. We consider a generalization, where we maximize this z-score over a range of significance levels 0<α≤α0. We are able to show that the resulting higher criticism statistic is effective at resolving a very subtle testing problem: testing whether n normal means are all zero versus the alternative that a small fraction is nonzero. The subtlety of this “sparse normal means” testing problem can be seen from work of Ingster and Jin, who studied such problems in great detail. In their studies, they identified an interesting range of cases where the small fraction of nonzero means is so small that the alternative hypothesis exhibits little noticeable effect on the distribution of the p-values either for the bulk of the tests or for the few most highly significant tests. In this range, when the amplitude of nonzero means is calibrated with the fraction of nonzero means, the likelihood ratio test for a precisely specified alternative would still succeed in separating the two hypotheses. We show that the higher criticism is successful throughout the same region of amplitude sparsity where the likelihood ratio test would succeed. Since it does not require a specification of the alternative, this shows that higher criticism is in a sense optimally adaptive to unknown sparsity and size of the nonnull effects. While our theoretical work is largely asymptotic, we provide simulations in finite samples and suggest some possible applications. We also show that higher critcism works well over a range of non-Gaussian cases.

/pdf/higher-criticism-for-detecting-sparse-heterogeneous-mixtures-1q93uhjbcp.pdf

Higher criticism for detecting sparse heterogeneous mixtures

Consider a network where the nodes split into $K$ different communities. The community labels for the nodes are unknown and it is of major interest to estimate them (i.e., community detection). Degree Corrected Block Model (DCBM) is a popular network model. How to detect communities with the DCBM is an interesting problem, where the main challenge lies in the degree heterogeneity. We propose a new approach to community detection which we call the Spectral Clustering On Ratios-of-Eigenvectors (SCORE). Compared to classical spectral methods, the main innovation is to use the entry-wise ratios between the first leading eigenvector and each of the other leading eigenvectors for clustering. Let $A$ be the adjacency matrix of the network. We first obtain the $K$ leading eigenvectors of $A$, say, $\hat{\eta}_{1},\ldots,\hat{\eta}_{K}$, and let $\hat{R}$ be the $n\times(K-1)$ matrix such that $\hat{R}(i,k)=\hat{\eta}_{k+1}(i)/\hat{\eta}_{1}(i)$, $1\leq i\leq n$, $1\leq k\leq K-1$. We then use $\hat{R}$ for clustering by applying the $k$-means method. The central surprise is, the effect of degree heterogeneity is largely ancillary, and can be effectively removed by taking entry-wise ratios between $\hat{\eta}_{k+1}$ and $\hat{\eta}_{1}$, $1\leq k\leq K-1$. The method is successfully applied to the web blogs data and the karate club data, with error rates of $58/1222$ and $1/34$, respectively. These results are more satisfactory than those by the classical spectral methods. Additionally, compared to modularity methods, SCORE is easier to implement, computationally faster, and also has smaller error rates. We develop a theoretic framework where we show that under mild conditions, the SCORE stably yields consistent community detection. In the core of the analysis is the recent development on Random Matrix Theory (RMT), where the matrix-form Bernstein inequality is especially helpful.

/pdf/fast-community-detection-by-score-dfzigwubk0.pdf

Fast community detection by score

The non-Gaussian cold spot detected in wavelet space in the WMAP 1–year data, is detected again in the coadded WMAP 3–year data at the same position (b = 57 ◦ ,l = 209 ◦ ) and size in the sky (� 10 ◦ ). The present analysis is based on several statistical methods: kurtosis, maximum absolute temperature, number of pixels below a given threshold, volume and Higher Criticism. All these methods detect deviations from Gaussianity in the 3–year data set at a slightly higher confidence level than in the WMAP 1–year data. These small differences are mainly due to the new foreground reduction technique and not to the reduction of the noise level, which is negligible at the scale of the spot. In order to avoid a posteriori analyses, we recalculate for the WMAP 3–year data the significance of the deviation in the kurtosis. The skewness and kurtosis tests were the first tests performed with wavelets for the WMAP data. We obtain that the probability of finding an at least as high deviation in Gaussian simulations is 1.85%. The frequency dependence of the spot is shown to be extremely flat. Galactic foreground emissions are not likely to be responsible for the detected deviation from Gaussianity. Subject headings: methods: data analysis – cosmic microwave background

http://www.stat.cmu.edu/~jiashun/Research/Year/Coldspot.pdf

The Non-Gaussian Cold Spot in the 3 Year Wilkinson Microwave Anisotropy Probe Data

In important application fields today-genomics and proteomics are examples-selecting a small subset of useful features is crucial for success of Linear Classification Analysis. We study feature selection by thresholding of feature Z-scores and introduce a principle of threshold selection, based on the notion of higher criticism (HC). For i = 1, 2, ..., p, let pi(i) denote the two-sided P-value associated with the ith feature Z-score and pi((i)) denote the ith order statistic of the collection of P-values. The HC threshold is the absolute Z-score corresponding to the P-value maximizing the HC objective (i/p - pi((i)))/sqrt{i/p(1-i/p)}. We consider a rare/weak (RW) feature model, where the fraction of useful features is small and the useful features are each too weak to be of much use on their own. HC thresholding (HCT) has interesting behavior in this setting, with an intimate link between maximizing the HC objective and minimizing the error rate of the designed classifier, and very different behavior from popular threshold selection procedures such as false discovery rate thresholding (FDRT). In the most challenging RW settings, HCT uses an unconventionally low threshold; this keeps the missed-feature detection rate under better control than FDRT and yields a classifier with improved misclassification performance. Replacing cross-validated threshold selection in the popular Shrunken Centroid classifier with the computationally less expensive and simpler HCT reduces the variance of the selected threshold and the error rate of the constructed classifier. Results on standard real datasets and in asymptotic theory confirm the advantages of HCT.

https://www.pnas.org/content/pnas/105/39/14790.full.pdf

Higher criticism thresholding: Optimal feature selection when useful features are rare and weak.

Higher criticism is a method for detecting signals that are both sparse and weak. Although first proposed in cases where the noise variables are independent, higher criticism also has reasonable performance in settings where those variables are correlated. In this paper we show that, by exploiting the nature of the correlation, performance can be improved by using a modified approach which exploits the potential advantages that correlation has to offer. Indeed, it turns out that the case of independent noise is the most difficult of all, from a statistical viewpoint, and that more accurate signal detection (for a given level of signal sparsity and strength) can be obtained when correlation is present. We characterize the advantages of correlation by showing how to incorporate them into the definition of an optimal detection boundary. The boundary has particularly attractive properties when correlation decays at a polynomial rate or the correlation matrix is Toeplitz.

/pdf/innovated-higher-criticism-for-detecting-sparse-signals-in-2b3klnvoxf.pdf

Jiashun Jin

Papers

Higher criticism for detecting sparse heterogeneous mixtures

Fast community detection by score

The Non-Gaussian Cold Spot in the 3 Year Wilkinson Microwave Anisotropy Probe Data

Higher criticism thresholding: Optimal feature selection when useful features are rare and weak.

Innovated Higher Criticism for Detecting Sparse Signals in Correlated Noise