Why representativeness should be avoided
01 Aug 2013-International Journal of Epidemiology (Oxford University Press)-Vol. 42, Iss: 4, pp 1012-1014
TL;DR: The essence of knowledge is generalisation; that rubbing wood in a certain way can produce fire is a knowledge derived by generalisation from individual experiences.
Abstract: The essence of knowledge is generalisation. That rubbing wood in a certain way can produce fire is a knowledge derived by generalisation from individual experiences; the statement means that rubbing wood in this way will always produce fire. The art of discovery is therefore the art of correct generalisation. What is irrelevant, such as the particular shape or size of the piece of wood used, is to be excluded from the generalisation; what is relevant, for example, the dryness of the wood, is to be included in it. The meaning of the term relevant can thus be defined: that is relevant which must be mentioned for the generalisation to be valid. The separation of relevant from irrelevant factors is the beginning of knowledge. —Hans Reichenbach
Citations
More filters
TL;DR: UK Biobank is not representative of the sampling population; there is evidence of a “healthy volunteer” selection bias; valid assessment of exposure-disease relationships may be widely generalizable and does not require participants to be Representative of the population at large.
Abstract: The UK Biobank cohort is a population-based cohort of 500,000 participants recruited in the United Kingdom (UK) between 2006 and 2010. Approximately 9.2 million individuals aged 40-69 years who lived within 25 miles (40 km) of one of 22 assessment centers in England, Wales, and Scotland were invited to enter the cohort, and 5.5% participated in the baseline assessment. The representativeness of the UK Biobank cohort was investigated by comparing demographic characteristics between nonresponders and responders. Sociodemographic, physical, lifestyle, and health-related characteristics of the cohort were compared with nationally representative data sources. UK Biobank participants were more likely to be older, to be female, and to live in less socioeconomically deprived areas than nonparticipants. Compared with the general population, participants were less likely to be obese, to smoke, and to drink alcohol on a daily basis and had fewer self-reported health conditions. At age 70-74 years, rates of all-cause mortality and total cancer incidence were 46.2% and 11.8% lower, respectively, in men and 55.5% and 18.1% lower, respectively, in women than in the general population of the same age. UK Biobank is not representative of the sampling population; there is evidence of a "healthy volunteer" selection bias. Nonetheless, valid assessment of exposure-disease relationships may be widely generalizable and does not require participants to be representative of the population at large.
1,896 citations
Clinical Trial Service Unit1, Johns Hopkins University School of Medicine2, University of Cambridge3, University of Bristol4, University of Wisconsin-Madison5, University of London6, Queen Mary University of London7, The George Institute for Global Health8, Imperial College London9, Brigham and Women's Hospital10, University of Edinburgh11, University of North Carolina at Chapel Hill12, National Institutes of Health13, National Health and Medical Research Council14, Population Health Research Institute15
TL;DR: The large-scale evidence from randomised trials indicates that it is unlikely that large absolute excesses in other serious adverse events still await discovery, and any further findings that emerge about the effects of statin therapy would not be expected to alter materially the balance of benefits and harms.
1,245 citations
TL;DR: Randomized Controlled Trials (RCTs) are increasingly popular in the social sciences, not only in medicine as discussed by the authors, and they can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not 'what works', but 'why things work'.
874 citations
TL;DR: In simulations, it is shown that even modest influences on selection into, or attrition from, a study can generate biased and potentially misleading estimates of both phenotypic and genotypic associations.
Abstract: Large-scale cross-sectional and cohort studies have transformed our understanding of the genetic and environmental determinants of health outcomes. However, the representativeness of these samples may be limited-either through selection into studies, or by attrition from studies over time. Here we explore the potential impact of this selection bias on results obtained from these studies, from the perspective that this amounts to conditioning on a collider (i.e. a form of collider bias). Whereas it is acknowledged that selection bias will have a strong effect on representativeness and prevalence estimates, it is often assumed that it should not have a strong impact on estimates of associations. We argue that because selection can induce collider bias (which occurs when two variables independently influence a third variable, and that third variable is conditioned upon), selection can lead to substantially biased estimates of associations. In particular, selection related to phenotypes can bias associations with genetic variants associated with those phenotypes. In simulations, we show that even modest influences on selection into, or attrition from, a study can generate biased and potentially misleading estimates of both phenotypic and genotypic associations. Our results highlight the value of knowing which population your study sample is representative of. If the factors influencing selection and attrition are known, they can be adjusted for. For example, having DNA available on most participants in a birth cohort study offers the possibility of investigating the extent to which polygenic scores predict subsequent participation, which in turn would enable sensitivity analyses of the extent to which bias might distort estimates.
608 citations
Cites background from "Why representativeness should be av..."
...However, it is often assumed that although selection bias will have a strong effect on prevalence estimates, it should not have a strong impact on observed associations between variables.(8) This overlooks the fact that selection can induce collider bias (see Figure 1), which can lead to biased observational and genetic associations....
[...]
Posted Content•
TL;DR: RCTs are valuable tools whose use is spreading in economics and in other social sciences as mentioned in this paper. But some of the enthusiasm for RCTs appears to be based on misunderstandings: that randomization provides a fair test by equalizing everything but the treatment and so allows a precise estimate of the treatment alone.
Abstract: RCTs are valuable tools whose use is spreading in economics and in other social sciences. They are seen as desirable aids in scientific discovery and for generating evidence for policy. Yet some of the enthusiasm for RCTs appears to be based on misunderstandings: that randomization provides a fair test by equalizing everything but the treatment and so allows a precise estimate of the treatment alone; that randomization is required to solve selection problems; that lack of blinding does little to compromise inference; and that statistical inference in RCTs is straightforward, because it requires only the comparison of two means. None of these statements is true. RCTs do indeed require minimal assumptions and can operate with little prior knowledge, an advantage when persuading distrustful audiences, but a crucial disadvantage for cumulative scientific progress, where randomization adds noise and undermines precision. The lack of connection between RCTs and other scientific knowledge makes it hard to use them outside of the exact context in which they are conducted. Yet, once they are seen as part of a cumulative program, they can play a role in building general knowledge and useful predictions, provided they are combined with other methods, including conceptual and theoretical development, to discover not "what works," but why things work. Unless we are prepared to make assumptions, and to stand on what we know, making statements that will be incredible to some, all the credibility of RCTs is for naught.
591 citations
References
More filters
TL;DR: Benefit was greatest for cancers of the proximal colon, which are not otherwise prevented effectively by screening with sigmoidoscopy or colonoscopy, and benefit increased with scheduled duration of treatment.
1,208 citations
748 citations
Book•
01 Jan 1951
TL;DR: In this article, Reichenbach traces the failures of the traditional philosophical systems to psychological causes and finds answers to the age-old questions of space, time, causality, and life; of the human observer and the external world.
Abstract: This book represents a new approach to philosophy. It treats philosophy as not a collection of systems, but as a study of problems. It recognizes in traditional philosophical systems the historical function of having asked questions rather than having given solutions. Professor Reichenbach traces the failures of the systems to psychological causes. Speculative philosophers offered answers at a time when science had not yet provided the means to give true answers. Their search for certainty and for moral directives led them to accept pseudo-solutions. Plato, Descartes, Spinoza, Kant, and many others are cited to illustrate the rationalist fallacy: reason, unaided by observation, was regarded as a source of knowledge, revealing the physical world and 'moral truth'. The empiricists could not disprove this thesis, for they could not give a valid account of mathematical knowledge. Mathematical discoveries in the early nineteenth century cleared the way for modern scientific philosophy. Its advance was furthered by discoveries in modern physics, chemistry, biology, and psychology. These findings have made possible a new conception of the universe and of the atom. The work of scientists thus altered philosophy completely and brought into being a philosopher with a new attitude and training. Instead of dictating so-called laws of reason to the scientist, this modern philosopher proceeds by analyzing scientific methods and results. He finds answers to the age-old questions of space, time, causality, and life; of the human observer and the external world. He tells us how to find our way through this world without resorting to unjustifiable beliefs or assuming a supernatural origin for moral standards. Philosophy thus is no longer a battleground of contradictory opinions, but a science discovering truth step by step. Professor Reichenbach, known for his many contributions to logic and the philosophy of science, addresses this book to a wider audience. He writes for those who do not have the leisure or preparation to read in the fields of mathematics, symbolic logic, or physics. Besides showing the principal foundations of the new philosophy, he has been careful to provide the necessary factual background. He has written a philosophical study, not a mere popularization. It contains within its chapters all the necessary scientific material in an understandable form - and, therefore, conveys all the information indispensable to a modern world-view. The late Hans Reichenbach was Professor of Philosophy at the University of California, Los Angeles.
553 citations
TL;DR: A 52-year-old man with no personal or family history of colon cancer, colonic polyps, or inflammatory bowel disease underwent a screening colonoscopy that showed no abnormalities except for a 1.5-cm pedunculated polyp at the hepatic flexure that was removed by means of a snare with cautery.
Abstract: A 52-year-old man with no personal or family history of colon cancer, colonic polyps, or inflammatory bowel disease underwent a screening colonoscopy that showed no abnormalities except for a 1.5-cm pedunculated polyp at the hepatic flexure that was removed by means of a snare with cautery. The polyp was a tubulovillous adenoma without high-grade dysplasia. How should his care be managed?
156 citations