Portal de Periódicos da CAPES

Rationality in psychological research: The good-enough principle.

1985; American Psychological Association; Volume: 40; Issue: 1 Linguagem: Inglês

10.1037/0003-066x.40.1.73

1935-990X

Ronald C. Serlin, Daniel K. Lapsley,

Social Representations and Identity

This article reexamines a number of methodological and procedural issues raised by Meehl (1967, 1978) seem to question rationality of psychological inquiry. The first issue concerns asymmetry in theory testing between psychology and physics and resulting paradox that, because psychological null hypothesis is always false, increases in precision in psychology always lead to weaker tests of a theory, whereas converse is true in physics. The second issue, related to first, regards slow progress observed in psychological research and seeming unwillingness of social scientists to take seriously Popperian requirements for intellectual honesty. We propose a principle to resolve Meehl's methodological paradox and appeal to a more powerful reconstruction of science developed by Lakatos (1978a, 1978b) to account for actual practice of psychological researchers. From time to time every research discipline must reevaluate its method for generating and certifying knowledge. The actual practice of working scientists in a discipline must continually be subjected to severe criticism and be held accountable to standards of intellectual honesty, standards are themselves revised in light of critical appraisal (Lakatos, 1978a). If, on a metatheoretical level, scientific methodology cannot be defended on rational grounds, then metatheory must be reconstructed so as to make science rationally justifiable. The history of science is replete with numerous such reconstructions, from portrayal of science as being inductive and justificationist, to more recent reconstructions favored by (naive and sophisticated) methodological falsificationists, such as Popper (1959), Lakatos (1978a), and Zahar (1973). In last two decades psychology, too, has been subjected to criticism for its research methodology. Of increasing concern is empirical psychology's use of inferential hypothesis-testing techniques and way in which information derived from these procedures is used to help us make decisions about theories under test (e.g., Bakan, 1966; Lykken, 1968; Rozeboom, 1960). In two penetrating essays, Meehl (1967, 1978) has cogently and effectively faulted use of traditional null-hypothesis significance test in psychological research. According to Meehl (1978, p. 817), the almost universal reliance on merely refuting null hypothesis as standard method for corroborating substantive theories [in psychology] is a terrible mistake, is basically unsound, poor scientific strategy, and one of worst things ever happened in history of He maintained it leads to a methodological paradox when compared to theory testing in physics. In addition, Meehl (1978) pointed to apparently slow progress in psychological research and deleterious effect null-hypothesis testing has had on detection of progress in accumulation of psychological knowledge. The cumulative effect of this criticism is to do nothing less than call into question rational character of our empirical inquiries. As yet there has been no attempt to deal with problems raised by Meehl by reconstructing actual practice of psychologists into a logically defensible form. This is purpose of present article. The two articles by Meehl seem to deal with two disparate issues--null-hypothesis testing and slow progress. Both issues, however, are linked in methodological falsificationist reconstruction of science to necessity for scientists to agree on what experimental outcomes are to be considered as disconfirming instances. We will argue methodological paradox can be ameliorated with help of a good-enough principle, to be proposed here, so hypothesis testing in psychology is not rationally disadvantaged when compared to physics. We will also account for apparent slow progress in psychological research, and we will take issue with certain (though not all) claims made by Meehl (1978) in this regard. Both methodological and progress issues will be resolved by an appeal to (sophisticated) methodological falsificationist reconstruction of science developed by Lakatos (1978a), an approach with which Meehl is familiar but one he did not apply to psychology in his articles. January 1985 • American Psychologist Copyright 1985 by American Psychological Association, Inc. 0003-066X/85/$00.75 Vol. 40, No. 1, 73-83 73 Meehl's Asymmetry Argument Let us develop Meehl's argument. It is his contention improved measurement precision has widely different effects in psychology and physics on success of a theory in overcoming an observational hurdle. Perfect precision in behavioral provides an easier hurdle for theories, whereas such accuracy in physics makes it much more difficult for a theory to survive. According to Popperian reconstruction of science (Popper, 1959), scientific theories must be continually subjected to severe tests. But if social are immanently incapable of generating such tests, if they cannot expose their theories to strongest possible threat of refutation, even with ever-increasing measurement precision, then their claim to scientific status might reasonably be questioned. Further, according to this view of research in social sciences, there can be no question of scientific progress based on rational consideration of experimental outcomes. Instead, progress is more a matter of psychological conversion (Kuhn, 1962). Let us look more closely at standard practice in psychology. On basis of some theory T we derive conclusion a parameter 6 will differ for two populations. In order to examine this conclusion, we can set up a point-null hypothesis, Ho: = 0, and test this hypothesis against predicted outcome, H~: 6 4: 0. However, it has also been recognized (Kaiser, 1960; Kimmel, 1957) another question of interest is whether difference is in a certain direction, and so we could instead test directional null hypothesis, I-I~: 6 ~ 0, against directional alternative, H*: ~ > 0. In such tests, we can make two types of errors. The Type I error would lead to rejecting Ho or H~ when they are indeed true, whereas Type II error involves not rejecting Ho or HJ when they are false. The conventional methodology sets Type I (or alpha) error rate at 5% and seeks to reduce frequency of Type II errors. Such a reduction in Type II error rate can be achieved by improving logical structure of experiment, reducing measurement errors, or increasing sample size. Meehl pointed out in behavioral sciences, because of large number of factors affecting variables, we would never expect two populations to have literally equal means. Hence, he concluded An earlier version of this article was read at 1983 meeting of American Educational Research Association. The authors are grateful to Robbie Case, Joel R. Lcvin, and Leonard Marascuilo for reading earlier drafts, and to Crescent L. Kringle for her help with manuscript. Requests for reprints should be sent to Ronald C. Serlin, Department of Educational Psychology, University of Wisconsin, Madison, Wisconsin 53706. point-null hypothesis is always false. With infinite precision, we would always reject Ho. This is perhaps one reason to prefer directional null hypothesis H~. But Meehl then conducted a thought experiment in which direction predicted by T was assigned at random. In such an experiment, T provides no logical connection to predicted direction and so is totally without merit. Because H0 is always false, two populations will always differ, but because direction in H~ is assigned at random, with infinite precision we will reject HJ half of time. Hence, Meehl concluded that effect of increased p r e c i s i o n . . , is to yield a probability approaching 1/2 of corroborating our substantive theory by a significance test, even i f theory is totally without merit (Meehl, 1967, p. 111, emphasis in original). Meehl contrasted this state of affairs with in physics, wherein usual situation involves prediction of a point value. That which corresponds to point-null hypothesis is value flowing as a consequence of a substantive theory T. An increase in statistical power in physics has effect of stiffening experimental hurdle by 'decreasing prior probability of a successful experimental outcome if theory lacks verisimilitude, is, precisely reverse of situation obtaining in social sciences (Meehl, 1967, p. 113). With infinite precision, and if theory has no merit, logical probability of it surviving such a test in physics is negligible; in social sciences, this logical probability for H~ is one half. Perhaps another way of describing asymmetry in hypothesis testing between psychology and physics is to note that, in psychology, point-null hypothesis is not what is derived from a substantive theory. Rather, it is a straw-man competitor whose rejection we interpret as increasing plausibility of T. In physics, on other hand, theories entail point-null statistical hypotheses are very ones physicists take seriously and hope to confirm. If 0 is a predicted outcome of interest, and 0 is its logical complement, then depiction of null and alternative statistical hypotheses in two disciplines can be written as follows: