Showing papers on "Reliability (statistics) published in 2003"
TL;DR: The aim of the present study was to develop and validate an instrument that can be used to determine the methodological quality of observational or non‐randomized studies in surgical research.
Abstract: Background: Because of specific methodological difficulties in conducting randomized trials, surgical research remains dependent predominantly on observational or non-randomized studies. Few validated instruments are available to determine the methodological quality of such studies either from the reader's perspective or for the purpose of meta-analysis. The aim of the present study was to develop and validate such an instrument.
Methods: After an initial conceptualization phase of a methodological index for non-randomized studies (MINORS), a list of 12 potential items was sent to 100 experts from different surgical specialities for evaluation and was also assessed by 10 clinical methodologists. Subsequent testing involved the assessment of inter-reviewer agreement, test-retest reliability at 2 months, internal consistency reliability and external validity.
Results: The final version of MINORS contained 12 items, the first eight being specifically for non-comparative studies. Reliability was established on the basis of good inter-reviewer agreement, high test-retest reliability by the κ-coefficient and good internal consistency by a high Cronbach's α-coefficient. External validity was established in terms of the ability of MINORS to identify excellent trials.
Conclusions: MINORS is a valid instrument designed to assess the methodological quality of non-randomized surgical studies, whether comparative or non-comparative. The next step will be to determine its external validity when used in a large number of studies and to compare it with other existing instruments.
4,522 citations
Book•
05 Dec 2003
TL;DR: The aim of this book is to provide a Discussion of Reliability of Maintained Systems, Specifically, the Foundations of Bayesian Reliability Analysis, and its Applications to Life Data Analysis.
Abstract: Preface to the Second Edition. Preface to the First Edition. Acknowledgments. 1. Introduction. 2. Failure Models. 3. Qualitative Systems Analysis. 4. Systems of Independent Components. 5. Component Importance. 6. Dependent Failures. 7. Counting Processes. 8. Markov Models. 9. Reliability of Maintained Systems. 10. Reliability of Safety Systems. 11. Life Data Analysis. 12. Accelerated Life Testing. 13. Bayesian Reliability Analysis. 14. Reliability Data Sources. Appendix A. The Gamma and Beta Functions. Appendix B. Laplace Transforms. Appendix C. Kronecker Products. Appendix D. Distribution Theorems. Appendix E. Maximum Likelihood Estimation. Appendix F. Statistical Tables. Acronyms. Glossary. References. Author Index. Subject Index.
1,559 citations
Book•
01 Jan 2003
TL;DR: Basic concepts of Multi-State Systems (MSS) Boolean methods extension for MSS reliability analysis basic random process methods for M SS reliability assessment Universal Generating Function (UGF) models MSSreliability optimization application problems.
Abstract: Basic concepts of Multi-State Systems (MSS) Boolean methods extension for MSS reliability analysis basic random process methods for MSS reliability assessment Universal Generating Function (UGF) models MSS reliability optimization application problems.
850 citations
TL;DR: It is important for researchers who use pedometers to assess physical activity to be aware of their accuracy and reliability, due to the variation that exists among models in regard to the internal mechanism and sensitivity.
Abstract: SCHNEIDER, P. L., S. E. CROUTER, O. LUKAJIC, and D. R. BASSETT, JR. Accuracy and Reliability of 10 Pedometers for Measuring Steps over a 400-m Walk. Med. Sci. Sports Exerc., Vol. 35, No. 10, pp. 1779–1784, 2003.PurposeThe purpose of this study was to determine the accuracy and reliability of
659 citations
TL;DR: A systematic review of job satisfaction instruments of adequate reliability and validity for use as evaluative tools in hospital environments found seven instruments met the defined reliability andvalidity criteria.
Abstract: Background Although job satisfaction research has been carried out for decades, no recent overview of job satisfaction instruments and their quality is available. Aim The aim of this systematic review is to select job satisfaction instruments of adequate reliability and validity for use as evaluative tools in hospital environments. Methods Systematic literature searches were performed in the Medline and PsycInfo databases. First, the construct of job satisfaction was operationalized by generating work factors from both theoretical studies and meta-analyses or reviews of empirical studies on job satisfaction. Secondly, emphasis was placed on the internal consistency, construct validity and responsiveness of these instruments.Twenty-nine job satisfaction instruments were retrieved in total. Results Seven instruments met the defined reliability and validity criteria. Of the seven, the 'Measure of Job Satisfaction' had an adequate content validity. Only the 'Job in General Scale' provided data about 'responsiveness' to change. Conclusion Few instruments have shown both high reliability and high validity, but little is known about their evaluative potential.
514 citations
01 Jan 2003
TL;DR: The OutcomeRating Scale (ORS) as mentioned in this paper is an ultra-brief outcome measure, which has been developed and validated for both clinical and non-clinical samples and has been evaluated at various sites in the US.
Abstract: Industry-wide, there is a trend toward making outcome evaluation a routine part of ther-apeutic services. Although various multidimensional assessments of outcome are validand reliable, their methodological complexity, length of administration, and cost oftenrender them infeasible for many service providers and settings. The present articledescribes the development and validation of an ultra-brief outcome measure, the OutcomeRating Scale (ORS). The instrument’s psychometric properties are examined and reportedfor both clinical and nonclinical samples. Based on experience with the instrument at thevarious sites in the study, the feasibility of the scale is considered. Results indicate that theORS represents a balanced trade-off between the reliability and validity of the longer meas-ures, and the feasibility of this brief scale. Results and implications for clinical practice andfuture research are discussed.
436 citations
TL;DR: The measure is efficient and faithful in characterizing spike timing reliability and produces smaller errors in the reliability estimate than the histogram-based measure based on the same number of trials.
360 citations
TL;DR: The authors argue that the likelihood of inappropriately restricting the domain of the construct and threatening construct validity tends to be greater when formative indicators are eliminated than when reflective indicators (of a unidimensional construct) are eliminated.
Abstract: level and is expected to have several distinct subdimensions, it is important to think about not only the relationships between the measures and these first-order subdimensions but also about the relationships between the first-order subdimensions and the second-order construct they measure. My colleagues and I (Jarvis et al. forthcoming) provide several examples of marketing constructs conceptualized at this level of abstraction (e.g., market orientation, trust, helping behavior, perceived risk, etc.) and call attention to the fact that the measurement model relating the first-order subdimensions to the measures need not be the same as the measurement model relating the second-order construct to its first-order subdimensions. Although one could reasonably argue that all constructs should be unidimensional, our review suggests that such a view is often inconsistent with the way constructs are defined in the marketing literature. So as a practical matter, this is something authors should think about carefully. Defend the Construct Domain and Insist on the Conceptually Appropriate Measurement Model Do not sacrifice construct validity at the altar of internal consistency reliability. Although this is good advice regardless of whether the measures are reflective or formative, it is particularly important to remember when your construct has formative measures, because formative indicator measurement models do not imply high levels of internal consistency reliability (Bollen and Lennox 1991). Therefore, when your measures are formative, it is important to resist the temptation to delete items as a means of improving Cronbach’s alpha (internal consistency reliability). Following this advice may be difficult if the temptation comes from a reviewer in the form of a recommendation. However, you must be vigilant, because the likelihood of inappropriately restricting the domain of the construct and threatening construct validity tends to be greater when formative indicators are eliminated than when reflective indicators (of a unidimensional construct) are eliminated (cf. Bollen and Lennox 1991). The best way to avoid this unpleasant situation is to head it off by carefully discussing the construct domain and the hypothesized relations between the construct and its measures, and explicitly noting the implications of your measurement model for how it should be evaluated. If internal consistency reliability is irrelevant, consider providing other evidence of reliability (e.g., item test-retest reliability) and/or empirically examining the sensitivity of your findings to different assumptions of reliability. In other words, try to gently educate the reviewers of your manuscript as a means of avoiding problems in the review process, always remembering that your arguments will be more convincing if they come in advance rather than in response to a reviewer’s criticisms. In conclusion, the problems of poor construct validity and statistical conclusion validity that plague many manuscripts can be minimized if you carefully define the focal constructs in your research, make sure that your measures fully represent them, correctly specify the relations between the measures and constructs in your measurement model, and stick to it. I believe that following this advice will greatly improve your chances of publication success— perhaps more than anything else I might recommend.
345 citations
TL;DR: Retest reliability of the HRQoL Core is moderate to excellent, and scaling options will require future attention, as will research into appropriate metrics for what constitutes important population group differences and change in HRZoL.
Abstract: Study objectives: Health related quality of life (HRQoL) is an important surveillance measure for monitoring the health of populations, as proposed in the American public health plan, Healthy People 2010. The authors investigated the retest reliability of four HRQoL questions from the US Behavioral Risk Factor Surveillance System (BRFSS).
Design: Randomly sampled BRFSS respondents from the state of Missouri were re-contacted for a retest of the HRQoL questions. Reliability was estimated by κ statistics for categorical questions and intraclass correlation coefficients for continuous questions.
Setting: Missouri, United States.
Participants: 868 respondents were re-interviewed by telephone about two weeks after the initial interview (mean 13.5 days). Participants represented the adult, non-institutionalised population of Missouri: 59.1% women; mean age 49.5 years; 93.2% white race.
Main results: Retest reliability was excellent (0.75 or higher) for Self-Reported Health and Healthy Days measures, and moderate (0.58 to 0.71) for other measures. Reliability was lower for older adults. Other demographic subgroups (for example, gender) showed no regular pattern of differing reliability and there was very little change in reliability by the time interval between the first and second interview.
Conclusions: Retest reliability of the HRQoL Core is moderate to excellent. Scaling options will require future attention, as will research into appropriate metrics for what constitutes important population group differences and change in HRQoL.
296 citations
TL;DR: This paper presents an approach based on rich architecture definition language (RADL) oriented towards modem industrial middleware platforms, such as Microsoft's NET and Sun's EJB that allows software architects to predict component reliability through compositional analysis of usage profiles and of environment component reliability.
TL;DR: A selective review of recent findings bearing on the conceptualization and measurement of cardiovascular reactivity to psychological challenge, with a focus on several issues relevant to the reliability, content validity, construct validity, and criterion validity of these measures.
Abstract: Objective and MethodsThis article is a selective review of recent findings bearing on the conceptualization and measurement of cardiovascular reactivity to psychological challenge, with a focus on several issues relevant to the reliability, content validity, construct validity, and criterion validit
TL;DR: Regression analyses suggest that age and assistive device use are important factors in performance on functional tests and may be useful in describing functional limitations and monitoring change in physical performance of older adults.
Abstract: Purpose:Routinely, physical therapists use a variety of physical performance tests to determine functional status of older adults. Whereas many commonly used instruments have been evaluated for some aspects of reliability and validity, few studies report typical performance for community liv
TL;DR: In this article, a Tabu search meta-heuristic has been developed and successfully demonstrated to provide solutions to the system reliability optimization problem of redundancy allocation, which generally involves the selection of components and redundancy levels to maximize system reliability given various system-level constraints.
Abstract: A tabu search meta-heuristic has been developed and successfully demonstrated to provide solutions to the system reliability optimization problem of redundancy allocation. Tabu search is particularly well-suited to this problem and it offers distinct advantages compared to alternative optimization methods. While there are many forms of the problem, the redundancy allocation problem generally involves the selection of components and redundancy levels to maximize system reliability given various system-level constraints. This is a common and extensively studied problem involving system design, reliability engineering and operations research. It is becoming increasingly important to develop efficient solutions to this reliability optimization problem because many telecommunications (and other) systems are becoming more complex, yet with short development schedules and very stringent reliability requirements. Tabu search can be applied to a more diverse problem domain compared to mathematical programming meth...
01 Jan 2003
TL;DR: In this paper, a method of RBD with the mixture of random variables with distributions and uncertain variables with intervals is proposed, where the reliability is considered under the condition of the worst combination of interval variables.
Abstract: In Reliability-Based Design (RBD), uncertainties usually imply for randomness. Nondeterministic variables are assumed to follow certain probability distributions. However, in real engineering applications, some of distributions may not be precisely known or uncertainties associated with some uncertain variables are not from randomness. These nondeterministic variables are only known within intervals. In this paper, a method of RBD with the mixture of random variables with distributions and uncertain variables with intervals is proposed. The reliability is considered under the condition of the worst combination of interval variables. In comparison with traditional RBD, the computational demand of RBD with the mixture of random and interval variables increases dramatically. To alleviate the computational burden, a sequential single-loop procedure is developed to replace the computationally expensive double-loop procedure when the worst case scenario is applied directly. With the proposed method, the RBD is conducted within a series of cycles of deterministic optimization and reliability analysis. The optimization model in each cycle is built based on the Most Probable Point (MPP) and the worst case combination obtained in the reliability analysis in previous cycle. Since the optimization is decoupled from the reliability analysis, the computational amount for MPP search is decreased to the minimum extent. The proposed method is demonstrated with a structural design example.Copyright © 2003 by ASME
01 Feb 2003
TL;DR: A comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems shows that the radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time.
Abstract: This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.
01 May 2003
TL;DR: In this paper, the authors discuss the background for developing reliability measures as a component of mobility performance metrics, the factors to consider before selecting a measure and defines the calculation procedures for typical reliability analyses.
Abstract: Reliability and variability in transportation are being discussed for a variety of reasons. The two terms are related, but different in their focus, how they are measured, how they are communicated and, in some respects, how they might frame the discussion about potential solutions. Reliability is commonly used in reference to the level of consistency in transportation service for a mode, trip, route or corridor for a time period. Typically, reliability is viewed by travelers in relation to their experience. Variability might be thought of as the amount of inconsistency in operating conditions. This definition takes more of a facility perspective and, therefore, relates to the concerns of transportation agencies. Both of these concepts are useful, but the term reliability may have a more "marketable" connotation for the purposes of reporting performance measures to the public because it relates to an "outcome" of transportation--the quality of the service provided. Variability seems to be more related to the change in transportation system operations. The traveling public and a variety of companies or product sectors use the term "reliability" in their goal statements and it would seem this is the term that should be used with a performance measure. This report discusses the background for developing reliability measures as a component of mobility performance metrics, the factors to consider before selecting a measure and defines the calculation procedures for typical reliability analyses. This might involve a change in thinking about how mobility service is conceptualized and communicated. The terms "recurring" and "non-recurring" congestion are misnomers--variations in weather, traffic volume and a range of other factors can be termed "recurring" but have traditionally been included outside the "regular" congestion label.
TL;DR: In this article, the authors test the potential value of McVay's (2000) Readiness for Online Learning questionnaire for research and practice, the instrument was administered to 107 undergraduate university students drawn from a range of courses in the United States and Australia.
Abstract: To test the potential value of McVay's (2000) Readiness for Online Learning questionnaire for research and practice, the instrument was administered to 107 undergraduate university students drawn from a range of courses in the United States and Australia. The questionnaire was subjected to a reliability analysis and a factor analysis. The instrument fared well in the reliability analysis, and yielded a two-factor structure that was readily interpretable in a framework of existing theory and research. Factors identified were "Comfort with e-learning" and "Self-management of learning." It is suggested that the instrument is useful for both research and practice, but would be enhanced through further work on 5 of the 13 items. Additionally, further work is required to establish predictive validity.
TL;DR: In this article, the authors present a brief review of the life-cycle reliability-based optimization field with emphasis on civil and aerospace structures, focusing on the recent developments in life cycle engineering of civil and aircraft structures based on system reliability, time-dependent reliability, life cycle maintenance, and optimization.
01 Jan 2003
TL;DR: In this article, the authors present a brief review of the life-cycle reliability-based optimization field with emphasis on civil and aerospace structures, with a focus on the use of life cycle optimization techniques to obtain a cost-effective design during a specified time horizon.
Abstract: Today, it is widely recognized that optimization methodologies should account for the stochastic nature of engineering systems and that concepts and methods of life-cycle engineering should be used to obtain a cost-effective design during a specified time horizon. The recent developments in life-cycle engineering of civil and aerospace structures based on system reliability, time-dependent reliability, life-cycle maintenance, life-cycle cost and optimization constitute an important progress. The objective of this study is to present a brief review of the life-cycle reliability-based optimization field with emphasis on civil and aerospace structures. � 2003 Published by Elsevier Science Ltd.
TL;DR: In this paper, the authors propose a response surface method named CQ2RS (Complete Quadratic Response Surface with ReSampling) allowing to take into account the knowledge of the engineer on one hand and to reduce the cost of the reliability analysis using a statistical formulation of the RSM problem on the other hand.
Posted Content•
16 Dec 2003
TL;DR: In this article, the authors examine reliability in public railway systems and propose to decrease the propagation of delays due to the interdependencies between trains by reducing the running time differences per track section.
Abstract: textReliability is one of the key factors in transportation, both for passengers and for cargo. This paper examines reliability in public railway systems. Reliability of railway services is a complex matter, since there are many causes for disruptions and at least as many causes for delays to spread around in space and time. One way to increase the reliability is to reduce the propagation of delays due to the interdependencies between trains. In this paper we attempt to decrease these interdependencies by reducing the running time differences per track section, i.e. by creating more homogeneous timetables. Because of the complexity of railway systems, we use network wide simulation for the analysis of the alternative timetables. We report on both theoretical and practical cases. Besides a comparison of different timetables, also general timetabling principles are deduced.
TL;DR: Correlations showed that over the past several decades there has been neither a rise nor a decline in the value of internal consistency, retest, or interjudge reliability coefficients.
Abstract: The author presented descriptive statistics for 937 reliability coefficients for various reliability methods (e.g., alpha) and test types (e.g., intelligence). He compared the average reliability coefficients with the reliability standards that are suggested by experts and found that most average reliabilities were less than ideal. Correlations showed that over the past several decades there has been neither a rise nor a decline in the value of internal consistency, retest, or interjudge reliability coefficients. Of the internal consistency approaches, there has been an increase in the use of coefficient alpha, whereas use of the split-half method has decreased over time. Decision analysis and true-score confidence intervals showed how low reliability can result in clinical decision errors.
TL;DR: None of the tested measures exhibited adequate reliability in young children, and, with less-than-desirable reliability being demonstrated, most measures need to be interpreted accordingly when repeated measures are involved.
Abstract: Repeatable measures are essential for clinicians and researchers alike. Both need baseline measures that are reliable, as intervention effects cannot be accurately identified without consistent measures. The intrarater and interrater reliability of the new Foot Posture Index and current podiatric measures of foot position were assessed using a same-subject, repeated-measures study design across three age groups. The Foot Posture Index total score showed moderate reliability overall, demonstrating better reliability than most other current measures, although navicular height (normalized for foot length) was the single most reliable measure in adults. None of the tested measures exhibited adequate reliability in young children, and, with less-than-desirable reliability being demonstrated, most measures need to be interpreted accordingly when repeated measures are involved.
21 Feb 2003
TL;DR: Simulations using real traces show that the cost of overlay maintenance in realistic dynamic environments and novel techniques to reduce this cost by adapting to the operating conditions enable high reliability and performance even in very adverse conditions with low maintenance cost.
Abstract: Structured peer-to-peer overlay networks provide a useful substrate for building distributed applications but there are general concerns over the cost of maintaining these overlays. The current approach is to configure the overlays statically and conservatively to achieve the desired reliability even under uncommon adverse conditions. This results in high cost in the common case, or poor reliability in worse than expected conditions. We analyze the cost of overlay maintenance in realistic dynamic environments and design novel techniques to reduce this cost by adapting to the operating conditions. With our techniques, the concerns over the overlay maintenance cost are no longer warranted. Simulations using real traces show that they enable high reliability and performance even in very adverse conditions with low maintenance cost.
Posted Content•
TL;DR: In this paper, a structural equation approach was used to estimate the Italian shadow economy by means of a model approach and verify the reliability of the MIMIC method for this kind of analysis.
Abstract: The aims of this paper are, firstly, to estimate the Italian shadow economy by means of a structural equation approach and, secondly, to verify the generality of the main criticisms about the reliability of the "MIMIC method" (or model approach) for this kind of analysis. Using the Italian shadow economy, I will show how only some of these are confirmed, others exist as a consequence of the model implementation and sample and some more will exposed to question this methodology. Particular attention is paid to detect the assumptions that should be respected to use appropriately this technique (test of multinormality, independence between measurement and structural errors, unit root detection, etc.). According to the obtained results, is confirmed a sufficient reliability of this approach for the estimate of the size of underground economy.
TL;DR: Some evidence is provided for the reliability and validity of the IQT, QS and DISCERN, although this needs testing in further research with different types of Internet information and larger sample sizes.
TL;DR: The reliability-based design (RBD) initiative sponsored by the Electric Power Research Institute for transmission line structure foundations is presented in this paper, where the role of RBD is presented within the context of geotechnical limit state design.
Abstract: This paper presents the reliability-based design (RBD) initiative sponsored by the Electric Power Research Institute for transmission line structure foundations. The role of RBD is presented within the context of geotechnical limit state design. Design parameters that are amenable to statistical description can be propagated systematically to a consistent measure of design risk using reliability techniques such as the first-order reliability method (FORM). Less quantifiable factors are incorporated approximately into RBD by judicious selection of the target reliability level. A simplified RBD approach based on the load and resistance factor design (LRFD) and multiple resistance factor design (MRFD) formats is proposed for practical implementation. The resistance factors for the LRFD and MRFD formats are calibrated rigorously using FORM to produce designs that can achieve a known level of reliability consistently.
03 May 2003
TL;DR: The study provides strong support for the validity of the framework and suggests that it is likely to be adopted in practice, but raises questions about its reliability and the ability of participants to use it to accurately identify defects.
Abstract: This paper conducts an empirical analysis of a semiotics-based quality framework for quality assuring information models. 192 participants were trained in the concepts of the quality framework, and used it to evaluate models represented in an extended Entity Relationship (ER) language. A randomised, double-blind design was used, in which each participant independently reviewed multiple models and each model was evaluated by multiple reviewers. A combination of quantitative and qualitative analysis techniques were used to evaluate the results, including reliability analysis, validity analysis, interaction analysis, influence analysis, defect pattern analysis and task accuracy analysis.. An analysis was also conducted of the framework's likelihood of adoption in practice. The study provides strong support for the validity of the framework and suggests that it is likely to be adopted in practice, but raises questions about its reliability and the ability of participants to use it to accurately identify defects. The research findings provide clear directions for improvement of the framework. The research methodology used provides a general approach to empirical validation of quality frameworks.
TL;DR: State-of-the-art survey chapters by leading researchers covering geometric algebra-a powerful mathematical tool for solving problems in computer science, engineering, physics, and mathematics.
Abstract: State-of-the-art survey chapters by leading researchers covering geometric algebra-a powerful mathematical tool for solving problems in computer science, engineering, physics, and mathematics. Focus is on interdisciplinary applications and techniques. Self-contained assuming only a knowledge of li... Accurate and efficient computer algorithms for factoring matrices, solving linear systems of equations, and extracting eigenvalues and eigenvectors. Regardless of the software system used, the book describes and gives examples of the use of modern computer software for numerical linear algebra. It b...