Classifying sequential data as early as and as accurately as possible is a challenging yet critical problem, especially when a sampling cost is high. One algorithm that achieves this goal is the sequential probability ratio test (SPRT), which is known as Bayes-optimal: it can keep the expected number of data samples as small as possible, given the desired error upper-bound. The SPRT has recently been found to be the best model that explains the activities of the neurons in the primate parietal cortex that are thought to mediate our complex decision-making processes. However, the original SPRT makes two critical assumptions that limit its application in real-world scenarios: (i) samples are independently and identically distributed, and (ii) the likelihood of the data being derived from each class can be calculated precisely. Here, we propose the SPRT-TANDEM, a deep neural network-based SPRT algorithm that overcomes the above two obstacles. The SPRT-TANDEM estimates the log-likelihood ratio of two alternative hypotheses by leveraging a novel Loss function for Log-Likelihood Ratio estimation (LLLR), while allowing for correlations up to $N (\in \mathbb{N})$ preceding samples. In tests on one original and two public video databases, Nosaic MNIST, UCF101, and SiW, the SPRT-TANDEM achieves statistically significantly better classification accuracy than other baseline classifiers, with a smaller number of data samples. The code and Nosaic MNIST are publicly available at this https URL.

Deep Neural Networks for the Sequential Probability Ratio Test on Non-i.i.d. Data Series.

In this article Wald’s sequential probability ratio test (SPRT) is implemented for multivariate normal distribution, for independent and autocorrelated data and known covariance matrix. The methodo...

On Wald’s sequential test for vector mean under multivariate normal distribution and correlated data

/pdf/evaluation-of-the-stream-function-assessment-methodology-ieoz0n4grv.pdf

Evaluation of the Stream Function Assessment Methodology (SFAM) in watersheds of the Puget Sound lowlands

Classifying sequential data as early and as accurately as possible is a challenging yet critical problem, especially when a sampling cost is high. One algorithm that achieves this goal is the sequential probability ratio test (SPRT), which is known as Bayes-optimal: it can keep the expected number of data samples as small as possible, given the desired error upper-bound. However, the original SPRT makes two critical assumptions that limit its application in real-world scenarios: (i) samples are independently and identically distributed, and (ii) the likelihood of the data being derived from each class can be calculated precisely. Here, we propose the SPRT-TANDEM, a deep neural network-based SPRT algorithm that overcomes the above two obstacles. The SPRT-TANDEM sequentially estimates the log-likelihood ratio of two alternative hypotheses by leveraging a novel Loss function for Log-Likelihood Ratio estimation (LLLR) while allowing correlations up to $N (\in \mathbb{N})$ preceding samples. In tests on one original and two public video databases, Nosaic MNIST, UCF101, and SiW, the SPRT-TANDEM achieves statistically significantly better classification accuracy than other baseline classifiers, with a smaller number of data samples. The code and Nosaic MNIST are publicly available at this https URL

Sequential Density Ratio Estimation for Simultaneous Optimization of Speed and Accuracy.

The Clean Water Act requires states to develop methods for assessing water quality. Assessment methods serve as decision-making procedures for including waterbodies on the Section 303(d) List of Impaired Waters. We used 17 years of ambient water quality data to explore statistical analyses for assessment methods that represent New York's waterbodies. Power analyses were performed to determine how many samples are needed to evaluate exceedances of water quality criteria using one-sample t-tests in lakes and flowing waters. Results suggest six samples for lakes and eight samples for flowing waters are needed to obtain at least 80% power, which is fewer samples than most other types of statistical assessment methodologies. This smaller number was possible because the power analysis was applied to the actual variability found in monitoring data to calculate the effect size as opposed to more conservative statistical estimates based on random data. Water quality criteria can have different analysis requirements such as single samples or means above the threshold, so we compared how many impairments would occur in the dataset if the six or eight samples were assessed as two single exceedances or a mean above the water quality criteria. Because the power analysis gives no indication of the time frame of when samples should be collected, the intra- and interannual variability of the data was assessed to determine whether sampling over a growing season in one year or sampling over multiple years is more representative of the water quality status. Results demonstrated that data collected over the growing season captured more variability in water quality data than data collected over multiple years in both waterbody types. With the prevalence of regulatory agencies having large, historical datasets rising, it would be possible for other agencies to apply these types of analyses to their assessment methodologies. Integr Environ Assess Manag 2022;18:1621-1628. © 2021 SETAC. 

Use of power analysis to determine the number of samples needed to assess water quality in lakes and flowing waters

The Sequential Probability Ratio Test: An efficient alternative to exact binomial testing for Clean Water Act 303(d) evaluation.

Environmental epidemiology has proven critical to study various associations between environmental exposures and adverse human health effects. However, there is a perception that it often does not sufficiently inform quantitative risk assessment. To help address this concern, in 2017, the Health and Environmental Sciences Institute initiated a project engaging the epidemiology, exposure science, and risk assessment communities with tripartite representation from government agencies, industry, and academia, in a dialogue on the use of environmental epidemiology for quantitative risk assessment and public health decision making. As part of this project, four meetings attended by experts in epidemiology, exposure science, toxicology, statistics, and risk assessment, as well as one additional meeting engaging funding agencies, were organized to explore incentives and barriers to realizing the full potential of epidemiological data in quantitative risk assessment. A set of questions was shared with workshop participants prior to the meetings, and two case studies were used to support the discussion. Five key ideas emerged from these meetings as areas of desired improvement to ensure that human data can more consistently become an integral part of quantitative risk assessment: 1) reducing confirmation and publication bias, 2) increasing communication with funding agencies to raise awareness of research needs, 3) developing alternative funding channels targeted to support quantitative risk assessment, 4) making data available for reuse and analysis, and 5) developing cross-disciplinary and cross-sectoral interactions, collaborations, and training. We explored and integrated these themes into a roadmap illustrating the need for a multi-stakeholder effort to ensure that epidemiological data can fully contribute to the quantitative evaluation of human health risks, and to build confidence in a reliable decision-making process that leverages the totality of scientific evidence. 

Connie Chen

Papers

The Sequential Probability Ratio Test: An efficient alternative to exact binomial testing for Clean Water Act 303(d) evaluation.

Considerations towards the better integration of epidemiology into quantitative risk assessment