Showing papers on "Reliability (statistics) published in 2010"

Toward Robust Information: Data Quality and Inter-Rater Reliability in the American College of Surgeons National Surgical Quality Improvement Program

Abstract: Background Data used for evaluating quality of medical care need to be of high reliability to ensure valid quality assessment and benchmarking. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) has continually emphasized the collection of highly reliable clinical data through its program infrastructure. Study Design We provide a detailed description of the various mechanisms used in ACS NSQIP to assure collection of high quality data, including training of data collectors (surgical clinical reviewers) and ongoing audits of data reliability. For the 2005 through 2008 calendar years, inter-rater reliability was calculated overall and for individual variables using percentages of agreement between the data collector and the auditor. Variables with > 5% disagreement are flagged for educational efforts to improve accurate collection. Cohen's kappa was estimated for selected variables from the 2007 audit year. Results Inter-rater reliability audits show that overall disagreement rates on variables have fallen from 3.15% in 2005 (the first year of public enrollment in ACS NSQIP) to 1.56% in 2008. In addition, disagreement levels for individual variables have continually improved, with 26 individual variables demonstrating > 5% disagreement in 2005, to only 2 such variables in 2008. Estimated kappa values suggest substantial or almost perfect agreement for most variables. Conclusions The ACS NSQIP has implemented training and audit procedures for its hospital participants that are highly effective in collecting robust data. Audit results show that data have been reliable since the program's inception and that reliability has improved every year.

Reliability and relative validity of an FFQ for nutrients in the Tehran lipid and glucose study

Abstract: Objective: To describe the relative validity and reliability of the FFQ used for assessing nutrient intakes of participants in the Tehran Lipid and Glucose Study (TLGS). Design: A total of 132 subjects (sixty-one males and seventy-one females) were included in the study. Dietary data were collected monthly by means of twelve 24h dietary recalls (24hDR). Subjects completed two, 168-item semi-quantitative FFQ. Blood and urine samples were taken every season for measurement of plasma biomarkers and urinary N and K. Results: Mean age and BMI of the participants were 35? 5( SD 16?8) years and 25? 5( SD 5?2) kg/m 2 , respectively. The mean energy-adjusted and deattenuated correlation coefficients for overall nutrient intake between the 24hDR and FFQ2 were 0?44 and 0?37 in #35-year-olds and .35-year-olds, respectively, and for individual nutrients ranged from 0?24 to 0?71 in men (mean r 50?53) and from 0?11 to 0?60 in women (mean r 50?39). The mean energy-adjusted reliability coefficients varied from 0?48 in #35-year-olds to 0?65 in .35-year-olds, and ranged from 0?41 to 0?79 in men (mean r 50?59) and from 0?39 to 0?74 in women (mean r 50?60). The FFQ2 and 24hDR produced exact agreement rates ranging between 39?6% and 68?3% in men and between 39?6% and 54?1% in women. The ranges of questionnaire validity coefficients, with the sample correlation between the questionnaires and biochemical marker as the lower limit and the estimate obtained by the method of triads as the upper limit, were 0?21‐0?56 (protein) and 0?37‐0?61 (K). Conclusions: The FFQ developed for the TLGS has reasonable relative validity and reliability for nutrient intakes in Tehranian adults.

The ‘‘What’’ and ‘‘How’’ of Case Study Rigor: Three Strategies Based on Published Work:

Abstract: To provide evidence-based strategies for ensuring rigor of case studies, the authors examine what rigor types authors report and how they report them by content analyzing all case studies published 1995—2000 in 10 management journals. Comparing practices in articles addressing rigor extensively and less extensively, the authors reveal three strategies for insuring rigor. First, very few case study authors explicitly label the rigor criteria in terms of the concepts commonly used in the positivist tradition (construct, internal, and external validity, as well as reliability). Despite this, papers addressing rigor extensively do report concrete research actions taken to ensure methodological rigor. Second, papers addressing rigor extensively prioritized rigor types: more, and more detailed, strategies were reported for ensuring internal and construct validity than for external validity. Third, emergent strategies used in the field were reported, such as setbacks and serendipities, that necessitated changes ...

How reliable are the results from functional magnetic resonance imaging

Abstract: Functional magnetic resonance imaging (fMRI) is one of the most important methods for in vivo investigation of cognitive processes in the human brain. Within the last two decades, an explosion of research has emerged using fMRI, revealing the underpinnings of everything from motor and sensory processes to the foundations of social cognition. While these results have revealed the potential of neuroimaging, important questions regarding the reliability of these results remain unanswered. In this paper, we take a close look at what is currently known about the reliability of fMRI findings. First, we examine the many factors that influence the quality of acquired fMRI data. We also conduct a review of the existing literature to determine if some measure of agreement has emerged regarding the reliability of fMRI. Finally, we provide commentary on ways to improve fMRI reliability and what questions remain unanswered. Reliability is the foundation on which scientific investigation is based. How reliable are the results from fMRI?

Reliability for Content Analysis

Abstract: One of the critical standards in quantitative, scientific research is the reliability of measures. Most basically, reliability is the extent to which measurement error is absent from the data (Nunnally, 1978). A widely accepted definition of reliability is that of Carmines and Zeller (1979): the extent to which a measurement procedure yields the same results on repeated trials. This chapter considers a type of reliability known as inter-coder reliability, which is a central concern in most content analysis research utilizing human coders. Inter-coder reliability assesses the consistency among human raters involved in a content analysis of messages. For such human coding, reliability is paramount (Neuendorf, 2002). If a content analytic measure is dependent upon the skills of a particular individual, the investigation has not met the standards of scientific inquiry.

System and method

Abstract: A system and method providing for communication and resolution of utility functions between participants, wherein the utility function is evaluated based on local information at the recipient to determine a cost value thereof. A user interface having express representation of both information elements, and associated reliability of the information. An automated system for optimally conveying information based on relevance and reliability.

Mixed Method Research: Fundamental Issues of Design, Validity, and Reliability in Construction Research

Abstract: The fact that people play key roles in nearly all aspects of construction suggests that effective construction research requires proper application of social science research methods. This is particularly true for researchers studying topics that involve human actions or behavior in construction processes, such as leadership, innovation, and planning. In social science research, no single method of data collection survey, experiment, participant observation, or unobtrusive research is ideal. Each method has inherent strengths and weak- nesses. Careful attention to the methodological ABCs of the design process, as discussed here, can enhance the validity and reliability of a given study. Combining quantitative and qualitative approaches in research design and data collection, however, should be considered whenever possible. Such mixed-methods research is more expensive than a single method approach, in terms of time, money, and energy, but improves the validity and reliability of the resulting data and strengthens causal inferences by providing the opportunity to observe data convergence or divergence in hypothesis testing. DOI: 10.1061/ASCECO.1943-7862.0000026 CE Database subject headings: Research; Methodology; Measurement; Data analysis; Construction management. Author keywords: Research methods; Mixed methods; Social science; Concept measurement; Data analysis.

Reliability and validity of the 2002 and 2006 Chapel Hill expert surveys on party positioning

Abstract: This research note reports on the 2002 and 2006 Chapel Hill expert surveys (CHES), which measure national party positioning on European integration, ideology, and several European Union (EU) and non-EU policies. The reliability of expert judgments is examined and the CHES data are cross-validated with data from the Comparative Mani- festo Project, the 2003 Benoit-Laver expert survey and the 2002 Rohrschneider-Whitefield survey. The dataset is available on the CHES website.

Willingness to pay for travel time reliability in passenger transport: A review and some new empirical evidence

Abstract: This paper reviews and critiques the modelling frameworks and empirical measurement paradigms used to obtain willingness to pay (WTP) for improved travel time reliability, suggesting new directions for ongoing research. We also estimate models to derive values of reliability, scheduling costs and reliability ratios in the context of Australian toll roads and use the new evidence to highlight the important influence of the way that trip time variability is included in stated preference studies in deriving WTP estimates of reliability in absolute terms, and relative to the value of travel time savings.

Feasibility, reliability, and validity of the EQ-5D-Y: Results from a multinational study

Abstract: Purpose To examine the feasibility, reliability, and validity of the newly developed EQ-5D-Y.

A survey on approaches for reliability-based optimization

Abstract: Reliability-based Optimization is a most appropriate and advantageous methodology for structural design. Its main feature is that it allows determining the best design solution (with respect to prescribed criteria) while explicitly considering the unavoidable effects of uncertainty. In general, the application of this methodology is numerically involved, as it implies the simultaneous solution of an optimization problem and also the use of specialized algorithms for quantifying the effects of uncertainties. In view of this fact, several approaches have been developed in the literature for applying this methodology in problems of practical interest. This contribution provides a survey on approaches for performing Reliability-based Optimization, with emphasis on the theoretical foundations and the main assumptions involved. Early approaches as well as the most recently developed methods are covered. In addition, a qualitative comparison is performed in order to provide some general guidelines on the range of applicability on the different approaches discussed in this contribution.

Multi-state System Reliability Analysis and Optimization for Engineers and Industrial Managers

Abstract: Multi-state Systems in Nature and Engineering Modern Stochastic Process Methods for MSS Reliability Assessment Statistical Analysis of Reliability Data for Real-world MSS's Universal Generating Function (UGF) Models Combined UGF and Stochastic Process Technique Aging Multi-state Systems Reliability Associated Costs for MSS and Optimal Management Decisions Fuzzy Multi-state System

Why cloud computing will never be free

Abstract: The last time the IT industry delivered outsourced shared-resource computing to the enterprise was with timesharing in the 1980s, when it evolved to a high art, delivering the reliability, performa...

Reliability and maintenance modeling for systems subjected to multiple dependent competing failure precesses

Abstract: For complex systems that experience Multiple Dependent Competing Failure Processes (MDCFP), the dependency among the failure processes presents challenging issues in reliability modeling. This article, develops reliability models and preventive maintenance policies for systems subject to MDCFP. Specifically, two dependent/correlated failure processes are considered: soft failures caused jointly by continuous smooth degradation and additional abrupt degradation damage due to a shock process and catastrophic failures caused by an abrupt and sudden stress from the same shock process. A general reliability model is developed based on degradation and random shock modeling (i.e., extreme and cumulative shock models), which is then extended to a specific model for a linear degradation path and normally distributed shock load sizes and damage sizes. A preventive maintenance policy using periodic inspection is also developed by minimizing the average long-run maintenance cost rate. The developed reliability and ma...

Adaptive Designs of Experiments for Accurate Approximation of a Target Region

Abstract: This paper addresses the issue of designing experiments for a metamodel that needs to be accurate for a certain level of the response value. Such situation is encountered in particular in constrained optimization and reliability analysis. Here, we propose an iterative strategy to build designs of experiments, which is based on an explicit trade-off between reduction of global uncertainty and exploration of the regions of interest. The method is illustrated on several test-problems. It is shown that a substantial reduction of error can be achieved in the crucial regions, with reasonable loss on the global accuracy. The method is finally applied to a reliability analysis problem; it is found that the adaptive designs significantly outperform classical space-filling designs.

Research Methods for Sports Performance Analysis

Abstract: Modern techniques of sports performance analysis enable the sport scientist, coach and athlete to objectively assess, and therefore improve upon, sporting performance. They are an important tool for any serious practitioner in sport and, as a result, performance analysis has become a key component of degree programmes in sport science and sports coaching. Research Methods for Sports Performance Analysis explains how to undertake a research project in performance analysis including: selection and specification of a research topic the research proposal gaining ethical approval for a study developing a performance analysis system testing a system for reliability analysing and discussing data writing up results. Covering the full research cycle and clearly introducing the key themes and issues in contemporary performance analysis, this is the only book that sports students will need to support a research project in performance analysis, from undergraduate dissertation to doctoral thesis. Including case studies, examples and data throughout, this book is essential reading for any student or practitioner with an interest in performance analysis, sports coaching or applied sport science.

Collaborative reliability prediction of service-oriented systems

Abstract: Service-oriented architecture (SOA) is becoming a major software framework for building complex distributed systems. Reliability of the service-oriented systems heavily depends on the remote Web services as well as the unpredictable Internet. Designing effective and accurate reliability prediction approaches for the service-oriented systems has become an important research issue. In this paper, we propose a collaborative reliability prediction approach, which employs the past failure data of other similar users to predict the Web service reliability for the current user, without requiring real-world Web service invocations. We also present a user-collaborative failure data sharing mechanism and a reliability composition model for the service-oriented systems. Large-scale real-world experiments are conducted and the experimental results show that our collaborative reliability prediction approach obtains better reliability prediction accuracy than other approaches.

Quality of Psychology Test Between Likert Scale 5 and 6 Points

Abstract: Problem statement: The purposes of this research were (1) to study the quality evaluation psychology test Likert's scale 5 and 6 points (atti tude test, locus of control test and achievement motive test) focusing on construct validity, discri mination and reliability, (2) to compare the study of quality psychology test between Likert's scale 5 an d 6 points and (3) to compare the study for persona l decision making level between Likert's scale 5 and 6 points. Approach: The subjects were 180 (60 for each test) undergraduate students from Mahasarakham University who were selected by purposive sampling. Results: The research tools were attitude test, locus of co ntrol test and achievement motive test comprised of measurement patterns developed for different variables. Means, standard deviation, Factors Analysis, Alpha Coefficient and t-test were used for data analysis. Conclusion/Recommendations: The research revealed that (1) higher component an d Initial Eigenvalues cumulative percent of psychology test L ikert's scale 5 and 6 points, but psychology test Likert's scale 6 points had more higher trend of di scrimination and reliability than Likert's scale 5 points, (2) construct validity, discrimination and reliability among Likert's scale 5 and 6 points wer e compared. It was found different reliability at 0.0 5 level on achievement motive test only and (3) personal decision making level psychology test among Likert's scale 5 and 6 points was different at 0.05 level.

Reliability and Validity of the Korean Version of the Connor-Davidson Resilience Scale

Abstract: ObjectiveaaThe Connor-Davidson Resilience Scale (CD-RISC) measures various aspects of psychological resilience in patients with posttraumatic stress disorder (PTSD) and other psychiatric ailments. This study sought to assess the reliability and validity of the Korean version of the Connor-Davidson Resilience Scale (K-CD-RISC). MethodsaaIn total, 576 participants were enrolled (497 females and 79 males), including hospital nurses, university students, and firefighters. Subjects were evaluated using the K-CD-RISC, the Beck Depression Inventory (BDI), the Impact of Event Scale-Revised (IES-R), the Rosenberg Self-Esteem Scale (RSES), and the Perceived Stress Scale (PSS). Test-retest reliability and internal consistency were examined as a measure of reliability, and convergent validity and factor analysis were also performed to evaluate validity. ResultsaaCronbach’s α coefficient and test-retest reliability were 0.93 and 0.93, respectively. The total score on the K-CD-RISC was positively correlated with the RSES (r=0.56, p<0.01). Conversely, BDI (r=-0.46, p<0.01), PSS (r=-0.32, p<0.01), and IES-R scores (r=-0.26, p<0.01) were negatively correlated with the K-CD-RISC. The K-CD-RISC showed a five-factor structure that explained 57.2% of the variance. ConclusionaaThe K-CD-RISC showed good reliability and validity for measurement of resilience among Korean subjects. Psychiatry Investig 2010;7:109-115

Construction of Optimal Prediction Intervals for Load Forecasting Problems

Abstract: Short-term load forecasting is fundamental for the reliable and efficient operation of power systems. Despite its importance, accurate prediction of loads is problematic and far remote. Often uncertainties significantly degrade performance of load forecasting models. Besides, there is no index available indicating reliability of predicted values. The objective of this study is to construct prediction intervals for future loads instead of forecasting their exact values. The delta technique is applied for constructing prediction intervals for outcomes of neural network models. Some statistical measures are developed for quantitative and comprehensive evaluation of prediction intervals. According to these measures, a new cost function is designed for shortening length of prediction intervals without compromising their coverage probability. Simulated annealing is used for minimization of this cost function and adjustment of neural network parameters. Demonstrated results clearly show that the proposed methods for constructing prediction interval outperforms the traditional delta technique. Besides, it yields prediction intervals that are practically more reliable and useful than exact point predictions.

Owner reports of attention, activity, and impulsivity in dogs: a replication study

Abstract: When developing behaviour measurement tools that use third party assessments, such as parent report, it is important to demonstrate reliability of resulting scales through replication using novel cohorts. The domestic dog has been suggested as a model to investigate normal variation in attention, hyperactivity, and impulsive behaviours impaired in Attention Deficit Hyperactive Disorder (ADHD). The human ADHD Rating Scale, modified for dogs and using owner-directed surveys, was applied in a European sample. We asked whether findings would be replicated utilizing an Internet survey in a novel sample, where unassisted survey completion, participant attitudes and breeds might affect previous findings. Using a slightly modified version of the prior survey, we collected responses (n = 1030, 118 breeds representing 7 breed groups) primarily in the United States and Canada. This study was conducted using an Internet survey mechanism. Reliability analyses confirmed two scales previously identified for dogs (inattention [IA], hyperactivity-impulsivity [HA-IM]). Models including age, training status, and breed group accounted for very little variance in subscales, with no effect of gender. The factor invariance demonstrated in these findings confirms that owner report, using this modified human questionnaire, provides dog scores according to "inattention" and "hyperactivity-impulsivity" axes. Further characterization of naturally occurring variability of attention, activity, and impulsivity in domestic dogs may provide insight into genetic backgrounds underlying behaviours impaired in attention and associated disorders.

The MIQ-RS: A Suitable Option for Examining Movement Imagery Ability

Abstract: Within rehabilitation settings, mental imagery helps to promote long-term recovery and facilitates compliance to rehabilitation exercises. Individuals who are able to effectively engage in imagery practice are likely to gain the most benefit from imagery training. Thus, a suitable imagery ability measurement tool for individuals with movement limitations is needed. The purpose of the present study was to evaluate the Movement Imagery Questionnaire—Revised second version (MIQ-RS), and compare the results of this new version with Hall and Martin's (1997) MIQ-R. Three-hundred and twenty participants from a variety of sports and performance levels agreed to take part. Results showed the internal consistency and test–retest reliability of the MIQ-RS were satisfactory, the two-factor structure of the MIQ-RS was supported by confirmatory factor analysis, and Pearson correlations indicated a strong relationship between the MIQ-R and MIQ-RS. It appears the MIQ-RS is a suitable option for examining movement imagery ability primarily aimed at the upper extremity.

Measuring internet skills

Abstract: Research that considers Internet skills often lacks theoretical justifications and does not go beyond basic button knowledge. There is a strong need for a measurement framework that can guide future research. In this article, operational definitions for measuring Internet skills are proposed, applied in two large-scale performance tests, and tested for reliability and validity. The framework consists of four Internet skills: operational, formal, information, and strategic Internet skills. The framework proves to be a powerful means for understanding the complexity of the Internet skills that people employ when they use the Internet. The reliability of the framework is supported by obtaining similar results from two studies focusing on different contexts. The validity of the framework is investigated by comparing the results with external standards that also provide an indication of Internet skill levels

Smart Grid - a reliability perspective

Abstract: Increasing complexity of power grids, growing demand, and requirement for greater grid reliability, security and efficiency as well as environmental and energy sustainability concerns continue to highlight the need for a quantum leap in harnessing communication and information technologies. This leap toward a “smarter” grid is now widely referred to as “smart grid”. A framework for cohesive integration of these technologies facilitates convergence of acutely needed standards and protocols, and implementation of necessary analytical capabilities. The paper critically reviews the reliability impacts of major smart grid resources such as renewables, demand response, storage. We observe that an ideal mix of these resources leads to a flatter net demand that eventually accentuates reliability issues further. We then present a grid-wide IT architectural framework to meet the reliability challenges. This architecture supports a multitude of geographically and temporally coordinated hierarchical monitoring and control actions over time scales from milliseconds and up.