This is the html version of the file http://www.amazinginfoonhomosexuals.com/nobasis.pdf.
G o o g l e automatically generates html versions of documents as we crawl the web.

Google is neither affiliated with the authors of this page nor responsible for its content.
These search terms have been highlighted:  no  basis  studies  don  t  tell  us  same  sex  parenting 
Page 1
1
No Basis:
What the Studies Don’t Tell Us
About Same-Sex Parenting
By
Robert Lerner, Ph.D., and Althea K. Nagai, Ph.D.
Marriage Law Project, Washington, D.C.
January 2001
Page 2
2
The Good, the Bad, or the Ugly:
Formulating the Hypothesis
Compared to What?
Methods to Control for Unrelated Effects
Does it Measure Up?
Bias, Reliability, and Validity
It all Depends on Who You Ask:
Sampling
Just by Chance?
Statistical Testing
Give Me More Power:
How the Studies Find False Negatives
Bibliography
Evaluation of the Studies
Same-Sex Parenting Studies
and the Law
No Balance: Same-Sex Parenting
Studies in the News
Table of Contents
Executive Summary
3
Foreword
4
Introduction
6
Chapter
1
11
Chapter
2
26
Chapter
3
61
Chapter
4
69
Chapter
5
83
Chapter
6
95
Appendix
1
111
Appendix
2
118
Appendix
3
124
Appendix
4
145
About the Authors
149
Page 3
3
Executive Summary
It is routinely asserted in courts, journals and the media that it makes
“no difference” whether a child has a mother and a father, two fathers,
or two mothers. Reference is often made to social-scientific studies that
are claimed to have “demonstrated” this.
An objective analysis, however, demonstrates that there is no basis for this
assertion. The studies on which such claims are based are all gravely deficient.
Robert Lerner, Ph.D., and Althea Nagai, Ph.D., professionals in the
field of quantitative analysis, evaluated 49 empirical studies on same-sex
(or homosexual) parenting.
The evaluation looks at how each study carries out six key research
tasks: (1) formulating a hypothesis and research design; (2) controlling
for unrelated effects; (3) measuring concepts (bias, reliability and valid-
ity); (4) sampling; (5) statistical testing; and (6) addressing the problem
of false negatives (statistical power).
Each chapter of the evaluation describes and evaluates how the studies
utilized one of these research steps. Along the way, Lerner and Nagai
offer pointers for how future studies can be more competently done.
Some major problems uncovered in the studies include the following:
Unclear hypotheses and research designs
Missing or inadequate comparison groups
Self-constructed, unreliable and invalid measurements
Non-random samples, including participants who recruit
other participants
Samples too small to yield meaningful results
Missing or inadequate statistical analysis
Lerner and Nagai found at least one fatal research flaw in all forty-
nine studies. As a result, they conclude that no generalizations can reli-
ably be made based on any of these studies. For these reasons the studies
are no basis for good science or good public policy.
Four Appendices follow. Appendix 1 is a bibliography of the studies
and related publications. Appendix 2 is a table that summarizes the
evaluation of each of the studies with regard to each research step. Ap-
pendix 3 (by William C. Duncan) is an overview of how these studies
have been used in the law. Appendix 4 (by Kristina Mirus) describes
how the media has covered these studies.
Page 4
4
Foreword
By David Orgon Coolidge
Director, Marriage Law Project
What do existing studies tell us about the impact of same-sex
parenting on children?
Nothing.
That’s right, nothing.
You would never know that, however, if you were to read most
court decisions, law review articles, commission reports or newspa-
per articles. You would hear the opposite.
The point of the study which follows is not to try to answer the
question, “Why is this?” Instead, Robert Lerner and Althea Nagai
have simply evaluated the studies themselves. They have asked: What
are their hypotheses? How do they set about to prove them? What
do they conclude? In formulating, executing and analyzing their re-
search, do these studies get it right?
The results are not pretty. Lerner and Nagai identified 49 empiri-
cal studies on the subject of same-sex parenting.* After going
through them all, inch-by-inch, they found…nothing.
I first saw the need for such an evaluation back in 1996, in Ho-
nolulu, Hawaii. I sat through two weeks of testimony in the same-
sex “marriage” case, Baehr v. Miike. Almost all of the testimony was
* The terms “homosexual” (on the one hand) and “gay and les-
bian” (on the other) are both loaded. The studies evaluated here ex-
amine parenting by same-sex couples in sexual relationships. To
avoid distraction I have used the term “same-sex.”
Page 5
5
by social scientists. It raised questions I could not shake.
Many of those questions are larger ones, such as how science and
morality relate. But other questions were more straightforward: Are
these studies well-done by normal standards? Should journals pub-
lish them? Should policymakers rely on them?
The fact of the matter is that many people, including
policymakers, are relying upon these studies in litigation, legisla-
tion, scholarly writing, and in the larger public debate. (To confirm
this, see Appendices Three and Four by Bill Duncan and Kristina
Mirus.)
The least that should be done is to take a serious look at the
methodology of the studies. That is what Robert Lerner and Althea
Nagai have done. At the risk of damaging their professional and
academic reputations, they have done this full-scale evaluation. Here
you have the results. You will learn more than you ever wanted to
know about how studies should be designed, implemented and
evaluated. And you will learn how even the best studies of same-sex
parenting fall far short of these standards.
Lerner and Nagai have not only taken apart existing studies, how-
ever. By setting their evaluation in the context of a broader discus-
sion of social-scientific research, they have pointed the way toward
better studies. They are clearing ground so others can go forward.
In the meantime, the rest of us have decisions to make. How
shall we proceed? Lerner and Nagai make no attempt to answer this
question. They have only one point to make: Whatever you do,
don’t do it based on these studies.
Take the time to see what Lerner and Nagai discovered about the
same-sex parenting studies. These authors know a better or worse
study when they see it, and they tell it like it is. Whether we like it
or not, we are all in their debt.
Page 6
6
“[C]hildren with two parents of the same gender are as well ad-
justed as children with one of each kind.”
1
This view, revolutionary in its implications, and unheard of five
years ago, is now commonly asserted by social scientists, lawyers,
policymakers and the media. Numerous studies are routinely offered
to show that the sexual orientation of a couple makes “no differ-
ence” to the well-being of children. The obvious implication of this
view is that two gay “dads” or two lesbian “moms” can raise a child
as well as can two married biological parents. Simply being sur-
rounded by two caring adults is thought to be enough to raise most
children to be healthy, well-adjusted adults.
2
Is this claim true? Does
the research supporting it stand up to scientific scrutiny? These are
the questions discussed in this study. Our approach to this question
concentrates on an analysis of the methodologies used to carry out
existing same-sex parenting studies. We conclude that the methods
used in these studies are so flawed that these studies prove nothing.
Therefore, they should not be used in legal cases to make any argu-
ment about “homosexual vs. heterosexual” parenting.
3
Their claims
have no basis.
4
What Social Science Requires
Social science research is a complex process, but it follows a series
of well-defined steps. Each of these steps must be carried out prop-
erly to obtain valid conclusions. Like a chain is only as strong as its
No Basis: What the Studies Don’t Tell Us
About Same-Sex Parenting
By Robert Lerner, Ph.D., and Althea K. Nagai, Ph.D.
Introduction
Notes for this section begin on Page 9
Page 7
7
weakest link, the conclusions derived from any research study are
only as reliable as its weakest part.
5
The typical sequence of social-scientific research involves:
Formulating concepts and research hypotheses
Creating the research design
Establishing measurements for important concepts
Defining the sample and its selection procedures
Collecting the data
Performing statistical tests on the data analysis, and
Based on the above, hopefully reaching valid conclusions.
The studies discussed here will be analyzed by following the typi-
cal sequence of social science research methods textbooks. Under
each heading we will analyze all the studies to see how well they
meet accepted social science standards. Any failures in the process-
failure to properly design the study, failure to properly measure the
relevant variables, failure to properly control for extraneous variables,
and failure to use the proper statistical tests-make a study scientifi-
cally invalid. Most importantly of all, if a study claims to find no dif-
ference i.e. “non-significant results,” and that study failed to carry
out one or more of these research links in the proper manner, its
conclusions are purely and simply invalid. Why? Because failing to
carry out correctly one or more of these essential elements, in and of
itself increases the chances of finding non-significant results. In
other words, if you look for wrong findings using wrong methods,
it is even more likely you’ll get wrong results.
Social Science and Public Policy
With one exception, the authors of these studies wish to influence
public policy to support same-sex marriage and the adoption of
children by homosexual couples. While the authors of these studies
have every right to advocate this point of view, as do those who dis-
agree with them, their wish means that the stakes in obtaining valid
answers to these research questions are very high. It is not enough
for a study to be interesting, or raise important questions about a
subject, or to be provocative. While these criteria may be enough to
Page 8
8
get a study published, they are not strong enough to justify dramatic
alterations in long-established public policies. To justify changes in
public policy, studies should be strong enough that policy makers
have faith in the study’s reliability, and confidence that more research
is unlikely to overturn its findings.
This is not an unreasonable requirement. The public policy con-
sequences of relying on inadequate or insufficient studies can be dev-
astating.
In 1973, a literature review undertaken by social scientists Eliza-
beth Hertzog and Cecelia Sudia purported to find that the effects of
growing up in fatherless homes are at most minimal and likely to be
due to other factors. The authors did not stop here. They stated it
might be a good idea to increase community support for single par-
ents,
6
rather than developing policies that forestall the absence of fa-
thers, or that oppose easy divorce. This study was part of a larger
current of expert opinion proclaiming that growing up in a one-
parent family had no negative consequences for the children living in
these arrangements.
With more rigorous research, these interpretations were chal-
lenged and eventually overthrown. Research has demonstrated that
divorce is not the costless exercise for children that many had pro-
claimed it to be. The newer research demonstrated that children
growing up in fatherless families do not do as well financially, in
school, and emotionally both as children and as adults, as those in
families with their married biological parents.
7
Therefore, the stan-
dards used here, to investigate studies on the impact of same-sex
parenting on children, are necessarily demanding. We owe ourselves
nothing less.
How These Studies Were Selected
All of the articles used in this review deal with same-sex couples
and/or their children. We excluded dissertations, review articles, and
articles in the nonscientific press.
8
We have only analyzed reports of
original research studies (i.e., real social science). We have tried to be
as exhaustive as possible, although research is exploding in this field.
Page 9
9
Working from a variety of angles,
9
we arrived at a final list of 49
studies for analysis that have been either published in professional
journals or as chapters in a book.
10
All present the results of original
research on homosexual parents and/or their children.
Do these 49 studies offer conclusive proof that there is “no dif-
ference” between heterosexual and homosexual households? We be-
lieve that these studies offer no basis for that conclusion—because
they are so deeply flawed pieces of research. The reader is invited to
make his or her own judgment.
1. Harris, 1998, p. 51. Harris cites this body of studies in her controversial book
on child development.
2. For example, sociologist Judith Stacey, writing in a recent issue of Contempo-
rary Sociology, a book review journal, that focuses on sociology and public
policy, writes that “thus far the research on the effects of lesbian parenting on
child development is remarkably positive and therefore challenging [the status
quo] . . .” Stacey, 1999, p. 21.
3. This is not the same as concluding that traditional family arrangements are
better. It simply states that the evidence presented above does not justify the
opposite conclusion.
4. Since vocabulary related to homosexuality is extremely contentious, we
should explain our terminology. We have tried to generally use the term “same-
sex,” since the terms “gay and lesbian” (on the one hand) and “homosexual
and heterosexual” (on the other hand) are so ideologically polarized. However,
the studies themselves use one set of terms or the other, so the reader should
expect a variety of terms.
5. For example, one can have a perfectly selected sample, but concepts that are
so badly defined and poorly measured that one is unable to conclude anything
from the results of the study.
6. Cited and discussed in Popenoe, 1998, pp. 59-61; McLanahan and Sandefur,
1994, pp. 13-14. A well-known study in the same vein was sociologist Jessie
Bernard’s The Future of Marriage (1972), which became famous or infamous
for its comment that to be happy in a traditional marriage a woman must be
mentally ill (quoted in Whitehead, 1998, p. 51).
7. For detailed discussion of the extensive research literature see the following
Notes to Introduction
Page 10
10
works: Waite, 1995; McLanahan and Sandefur, 1994; Popenoe, 1998; Amato
and Booth, 1997; and Whitehead, 1996.
8. Dissertations are original studies, not review articles; but if they go unpub-
lished, the most one can say is that they met the minimum standards for receiv-
ing a degree from the university that granted them, and nothing more. Review
articles were excluded because they present no original data for assessment. Ar-
ticles found in the nonscientific press were excluded because their criteria for
publication (e.g., popular interest, immediate policy relevance) are not the same
as those for assessing the scientific credibility of a study.
9. Graciela Ortiz, M.S.W., conducted initial bibliographic research in the summer
of 1998. Additional studies were identified by examining law review articles
published by Wardle (1997) and Ball and Pea (1998), briefs filed in Baker v.
State, the Vermont same-sex “marriage” lawsuit, and Lesbian and Gay Parenting:
A Resource for Psychologists, Washington, D.C.: American Psychological Associa-
tion, 1995.
10. There is also one book, Tasker and Golombok (1997), which is part of the
study.
Page 11
11
We’ve all heard the slogans: “If you don’t know where you’re go-
ing, you can count on getting there,” or “If you aim for nothing,
you’re sure to hit it.” The same is true for formulating the hypoth-
esis of a research study: If your goal is to prove no differences, you’re
bound to reach it. But you won’t have proved “no difference,” only
no basis.
All good studies begin with careful definitions of key concepts
and careful delineation of the relationship between these concepts.
Formulating the hypothesis is the crux of any scientific design, and
its development requires special care. The hypothesis determines the
main focus of the study, and frames all subsequent research endeav-
ors.
1
Hypotheses can be Good (affirmative), Bad (fuzzy), or Ugly
(null). Of the 49 studies, two are Good, 29 are Bad, and 18 are
Ugly. Understanding why requires Social Science Research Methods
101, which we will sprinkle throughout this and other chapters.
What is a Good Hypothesis?
All good social science studies have at their core a positive hy-
pothesis statement. This takes the form of an explicit conceptual re-
lationship between two variables whereby something (an
independent variable) “causes” something else (a dependent vari-
able).
2
The researcher posits a direct relationship between the inde-
pendent and the dependent variables.
3
The hypothesis can and
Chapter 1
The Good, the Bad, or the Ugly:
Formulating the Hypothesis
Notes for this section begin on Page 22
Page 12
12
should be stated as a proposition that takes the following form: “the
greater the a, the greater the b,” where “a” is the independent vari-
able and “b” is the dependent variable.
4
Hypothesis statements may be either quantitative or qualitative.
Consider the following example. A study group of children is en-
rolled in a social program such as Head Start, while a control group
of children is not enrolled there. The independent variable here
therefore, is “enrolled versus not enrolled.” The dependent variable
might be something like “readiness for school.” Assuming that
“readiness for school” is a quantitative variable (i.e., it can be scored
on a three or more point scale), then the research hypothesis would
compare mean levels of school readiness of those in Head Start with
those who are not. Assuming “readiness for school” is a qualitative
(i.e., “yes” or “no”) variable, the research hypothesis would compare
the proportion of Head Start children who are ready for school
with the proportion who are not ready.
5
There are many different
possible hypotheses a reseacher might have, depending upon the na-
ture of the problem studied and the level of measurement assumed
in the independent and dependent variables.
6
Applying this view to
studying a parent’s sexual identity and its possible relationship to
child outcomes, the investigator should define conceptually the in-
dependent variable (“homosexual versus heterosexual” identity),
7
the
dependent variable (such as a child’s sexual identity, child’s psycho-
logical adjustment, or the child’s sexual behavior), and the posited
relationship between the independent variable and the dependent
variable(s).
An example of such a properly stated research hypothesis is: “the
children of homosexual parents are more likely to grow up to be ho-
mosexual than are the children of heterosexual parents.”
Good: The Affirmative Research Hypothesis
Only two studies among the 49 studies we examined actually
contain an explicit positive hypothesis statement of this sort
(Pagelow, 1980 and Miller, 1979).
8
Pagelow (1980) hypothesizes
that lesbian mothers are more oppressed than heterosexual mothers.
The researcher then seeks to measure this by the concept of perceived
oppression in the areas of freedom of association, employment,
housing, and child custody.
9
Page 13
13
Miller (1979) comes closest to presenting a hypothesis in
the proper format: Miller asks, A.) “Do gay fathers have children to
cover their homosexuality?” B.) “Do they molest their children?” C.)
“Do their children turn out to be disproportionately homosexual?”
D.) “Do they expose their children to homophobic harassment?”
10
While Miller does not put his hypotheses in precisely the y=(f)x for-
mat, the hypothesis statements are both specific and decisional (i.e.,
they can be answered as either “yes” or “no” regarding the homo-
sexuality of the father).
Thus Miller’s statements can be easily rephrased into the follow-
ing testable hypotheses: A.) The reason for gay men having children
is to cover their homosexuality (as opposed to other choices pro-
vided by the investigator, such as he loved the woman, he was con-
fused, he just wanted children, don’t know); B.) Gay fathers are
more likely to molest children than are straight fathers; C.) Children
of gay fathers are more likely to be homosexual than are children of
straight fathers; and D.) Children of gay fathers are more likely to be
exposed to homophobic harassment than are children of straight fa-
thers.
11
Stated in this form, the hypotheses can then be verified or re-
futed by empirical research.
Bad: The Fuzzy Hypothesis
A majority of the studies we examined (29 of them or 59 per-
cent) failed to produce a testable hypothesis. Of these, 12 studies
rendered their statement of the research problem in the form of “Are
there differences between homosexual and heterosexual parents?”
12
For example, Bigner and Jacobsen, 1989a state their research prob-
lem as, “an examination of factors that may motivate gay men to be-
come parents, and to explore whether gay fathers may differ from
heterosexual fathers regarding the value of children in their life as an
adult.”
13
Brewaeys et al. (1997) poses the problem as an examina-
tion of “family relationships and emotional/behavioral and gender
role development of 4-8 year old children”
14
in lesbian donor-in-
seminated families, compared to heterosexual families who conceived
their child also by donor insemination and heterosexual families who
conceived their child naturally. Hypotheses that are stated in the
form of looking for possible differences do not suffice as statements
of research hypotheses. Formulation of a hypothesis in terms of pos-
Page 14
14
sible differences fails to address any of the causal questions that
guide hypothesis formation. Such a formulation is purely descriptive
in nature and is not “an explicit conceptual relationship between
two variables whereby something (an independent variable) “causes”
something else (a dependent variable).”
15
This kind of formulation, which may seem commendable in its
caution, fails the “so what?” test. A proper research hypothesis re-
quires the hypothesizing of some kind of causal mechanism operat-
ing in the real world so that some kind of tentative causal
conclusion can be drawn from the research results if they are valid
and the hypothesis test is successful.
16
Seventeen studies present the research problem in the form of,
“what are the characteristics?”
17
For example, Gartrell states, “The
aim [of this study] was to learn about the homes, families, and com-
munities into which the children were to be born.”
18
McCandish
writes, “The family dynamics and developmental changes within
these families and the implications for the psychotherapeutic treat-
ment of lesbian mother families are the subject of this chapter.”
19
Pennington declares, “The purpose of this chapter is to discuss the
major issues confronted by children living in lesbian mother house-
holds.”
20
Hypotheses that take the form of descriptions of characteristics
face a different problem from that faced by statements of possible
differences. A focus on what is “characteristic” of a population (e.g.,
the mean, median, or mode) can obscure causal relations that are
not “characteristic” of the populations studied, but are nonetheless
causal in nature.
21
For example, sociologists Sara McLanahan and
Gary Sandefur report that 29 percent of young adults from one-par-
ent families dropped out of high school while only 13 percent of
those from two-parent families dropped out. Dropping out is not
“characteristic” of children from either type of family structure, yet
there is little doubt that a causal relationship between the variables
of type of family structure and the propensity to drop out of school
exists (1994, Figure 1, p. 41). This problem can be put in more
general terms. Focusing on characteristics of populations obscures
the necessity for a proper research hypothesis to focus on the rela-
tionship between two variables and not the properties of each of
Page 15
15
them considered separately. In this respect, focusing on the charac-
teristics of an attribute is misleading and hinders the scientific re-
search enterprise. All of these studies are not a priori invalid as
instances of exploratory research. Compared to studies that state and
test the research hypothesis properly, however, they are much inferior
in their level of research sophistication and precision. It tells us to
look for and expect other problems in later research steps. Authors
with such weaknesses in their formulation of hypotheses are unlikely
to produce any conclusions sufficiently robust so to inform public
policy debates with any degree of dependability.
The Ugly: Affirming The Null Hypothesis
The remaining 18 studies explicitly seek to find no differences
between heterosexual and homosexual parents in child outcomes
and to make this formulation a kind of hypothesis statement. While
this procedure is superior in some way to those used in the other
studies, it is also highly problematic because of the difficulties asso-
ciated with testing hypotheses purporting to affirm the null
hypothesis.
The authors of the null hypothesis-affirming studies seek to show
either that children raised by homosexual couples are not more likely
to grow up to be homosexual themselves than are those raised by
heterosexual couples, and/or that they are not more likely to grow
up with psychological problems than are children raised by hetero-
sexual couples, or both.
Eighteen studies explicitly seek to find no differences between
heterosexuals and homosexuals.
22
For example, Flaks et al, in their
study of 15 lesbian couples, 15 heterosexual couples and their chil-
dren, state, “On the basis of prior research, we expected [to find] no
differences between the children of lesbian and heterosexual parents
in any of the areas evaluated.”
23
In another case, Huggins studied adolescent children of lesbians,
expecting that a parent’s homosexuality would not result in confu-
sion of the child’s sexual identity, inappropriate gender role behav-
ior, sexual orientation, and overall psychopathology.
24
Likewise, Patterson’s studies of donor-inseminated lesbian fami-
lies all start with the expectation of finding no differences between
Page 16
16
the children of lesbian and heterosexual parents.
25
The same is also
the case with Tasker and Golombok (1995, 1997).
The “no difference” hypothesis used in the 18 studies discussed
above inverts the usual social science quantitative research procedure,
which would use a positive research hypothesis in the form described
above. This creates two major methodological problems that are un-
recognized by the all the authors of these studies save one (Chan et
al, 1998).
1) Failing to reject the null hypothesis necessarily leads to an
indeterminate result because one cannot validly “confirm”
the null hypothesis, and
2) Inverting the normal hypothesis testing situation makes it
too easy to fail to reject the null hypothesis, which is the
outcome favored by these researchers.
This results in an undue partiality in interpreting their research
findings and in carrying out the research itself. To see all of this
clearly, it is necessary to review the usual statistical testing procedure
in quantitative social science. This procedure requires statistical test-
ing of a positive research hypothesis.
A simplified example may help to visualize what is involved. For
example, suppose a researcher hypothesizes that political liberalism
leads to greater support for abortion rights than does political con-
servatism. One way to test this research hypothesis might be to use a
national sample survey of the American public (e.g., data from the
General Social Survey produced by the National Opinion Research
Center). With this body of data, which consists of individual re-
sponses to questions on a questionnaire, one computes the mean
“support for abortion” score of liberals and the mean “support for
abortion” score for conservatives.
26
One can assume that if this pro-
cedure is carried out, liberals will have a higher average score than do
conservatives (and in fact, they do). Since it is extremely unlikely that
such a comparison will yield exactly the same average score for both
liberals and conservatives, one must question whether this finding is
a real difference or whether it could be due to chance factors. The
difference in averages alone does not provide sufficient information
to determining the likelihood. To answer the question, statisticians
Page 17
17
have developed ways of distinguishing between statistically signifi-
cant and statistically insignificant differences. Insignificant differ-
ences might be due to sampling error, measurement error, or just
random fluctuations. In fact, these are competing claims that ought
to be considered in as possible explanations for any research finding.
Another way to state the null claim is, that if it is true, any difference
found in the data is due solely to random variation. Chance occur-
rences of this kind do happen; individuals do win the lottery and
draw royal flushes in honest poker games.
To ascertain whether in a given instance random variation explains
the findings in the data, the researcher carries out a statistical hypothesis
test. This requires a research hypothesis
27
and a null hypothesis.
The research hypothesis is of the kind already discussed. The null
hypothesis is a hypothesis of no difference or no effect. In the abor-
tion example, the research hypothesis is that liberals are more pro-
abortion than are conservatives.
28
The null hypothesis is that there is
no difference between liberals and conservatives in their support for
abortion rights. Both of these cannot be true. In carrying out statis-
tical hypothesis testing, the null hypothesis is a statistical device that
allows for calculation of the value of a test statistic. The test statistic
is calculated to determine the probability that the null hypothesis is
true given the data at hand. After the calculations are carried out,
the test statistic yields some number. If the number calculated from
the test statistic is greater than a certain preset value, which is called
the critical value (e.g., t>=2.00), the null hypothesis is rejected at
the associated level of statistical significance (e.g., p<.05)
29
and the
research hypothesis is accepted.
For example, suppose we find that the mean abortion rights score
for liberals is greater than it is for conservatives. In our example, the
relevant test statistic is calculated and the results are checked to see
whether they are statistically significant or not at the preset level of
statistical significance (in fact, there is a substantial statistically sig-
nificant difference between liberals and conservatives when this test
is carried out). Then we reject the null hypothesis and accept the al-
ternative hypothesis that “liberalism” is correlated with “support for
abortion rights.” Of course, carrying out this one hypothesis test
does not end the researcher’s task. In fact, the formal hypothesis test
Page 18
18
is just the initial step in analyzing the data. The social scientist then
has to show that this difference is not due to other factors (for ex-
ample, due to differences in education among sample members or to
selection biases in the sample). However, at least he has a statistical
relationship to work with, to try to either explain or explain away in
terms of broader substantive considerations.
There are two subsidiary points that need to be made here. First,
the reason the statistical test situation is conceived in the above man-
ner is to yield a determinate outcome. If the null hypothesis is re-
jected, then the alternative hypothesis is accepted. In our example
above, we reject the null hypothesis of no difference between liberals
and conservative on their support for abortion rights and accept the
alternative hypothesis that liberals have a higher average support for
abortion rights scores than do conservatives which is statistically sig-
nificant (and they really do). Also, statistical tests of this kind place
the burden of proof on the investigator to show support for his re-
search hypothesis. That is why the criterion for rejecting the null hy-
pothesis is difficult. Thus, social science researchers conventionally
use the p<.05 level of statistical significance. It does not have to be
this stringent (1 out of 20), but the practice has evolved so that it
has become the standard social science research convention in the
standard positive hypothesis social science research situation.
30
What happens if the null hypothesis cannot be rejected? In this
situation there are always two competing explanations for this result.
The first possible explanation for the failure to reject the null hy-
pothesis is that whatever differences are found really are due to
chance factors, so that no statistical, let alone causal, relationship be-
tween the two variables really exists. A statistician might say that if
the test were repeated an infinite number of times, a zero correlation
or a zero difference between the two groups studied would result.
The examples of the honest poker game and the honest lottery are
relevant here. The second possibility is that the research hypothesis is
true but its truth cannot be ascertained by the research results be-
cause there is some flaw in the study design or in the statistical test
itself, which causes the test statistic to yield a statistically insignifi-
cant result or p value.
31
Therefore, failing to reject the null hypoth-
esis by itself does not lead to a determinate result. Since every failure
to reject the null hypothesis has two possible explanations, one can-
Page 19
19
not simply “accept” the null hypothesis in the same way that one
can “reject” the null hypothesis and “accept” the alternative hypoth-
esis. Further investigation or conducting a new study is always in or-
der.
This is the problem with the 18 studies that explicitly sought to
confirm the null hypothesis as their research hypothesis. These stud-
ies sought to prove the null hypothesis, which, as we have shown, is
not the same thing as failing to reject the null hypothesis. In sub-
stantive terms, their authors seek to show that homosexual parents
produce the same child outcomes as do heterosexual parents. This
means that they desire to be able to “accept” the null hypothesis as
showing that homosexual parenting has no effect on child outcomes
simply on the basis of failing to reject the null hypothesis. This vio-
lates the standard statistical hypothesis testing procedure. It is
wrong because, as we show above, failing to reject the null hypoth-
esis does not necessarily mean that the null is true.
32
This is not merely a technical flaw in these studies. These investi-
gators report their failure to reject the null hypothesis and falsely
conclude that there is no difference between homosexual and hetero-
sexual parents in child outcomes.
33
This false conclusion invalidates
the “findings” of no difference between heterosexual and homo-
sexual parents as reported in the research literature that we have sur-
veyed. Only the authors of one study (Chan et. al, 1998) showed
any awareness of the problem, but they did nothing to correct for it
or to alter their interpretations of their results because of it. If the
null hypothesis itself becomes the research hypothesis, and some
kind of research hypothesis is to become the new null hypothesis,
then the standard testing situation must be radically altered to ac-
commodate this situation and non-standard statistical tools are
needed in order to reach defensible results.
34
The studies we surveyed
all failed to do this or even to indicate that they saw the need for do-
ing it. This indicates that their authors’ understanding of the logic
of quantitative social scientific research is suspect. When the hypoth-
esis statement is properly conceptualized, the null hypothesis is used
in conducting statistical tests as the comparison hypothesis to the
one under investigation. It is no substitute for a properly formulated
affirmative hypothesis. It is the objective of properly stated hypoth-
eses, proper design, and proper execution of an empirical research
Page 20
20
study to decrease the probability that the relationship uncovered by
the investigator is due to chance.
35
The goal of genuine social-scientific research, in short, is to make
the null hypothesis less, not more, likely.
36
Properly speaking, then,
one can never prove the validity of the “null hypothesis.” When you
hear the statement that a study found “no significant difference,”
what this actually means is that, having done some tests, the investi-
gator can only say, “I looked for differences, and haven’t found any-
thing significant yet. But who knows?” In social-scientific terms, the
study “failed to reject the null hypothesis.” It proved nothing.
37
In summary, in conducting a statistical test of a hypothesis there
are two possible outcomes. The first is to be able to reject the null
hypothesis and accept the research hypothesis that a difference be-
tween the groups does exist that is not likely to be due to chance
factors. The researcher then proceeds to see if his or her hypothesis
can stand up to other tests of its validity, by introducing controls for
extraneous and confounding factors and the like. These are all the
subsequent research steps we will be discussing below.
The second possible outcome is to fail to be able to reject the
null hypothesis. This is NOT the same as showing that no effect ex-
ists. There are many possible reasons why one may fail to reject the
null hypothesis yet be in error in doing so. For example, the sample
used in the study may be too small to reach the appropriate level of
statistical significance for a given effect, the significance level used in
the significance test itself may be set too high, or the research instru-
ments used to measure the independent and dependent variables
may so highly unreliable that no stable results are possible. Even if
none of these factors can explain the absence of positive results, this
still does not show that no effect exists. The researcher then pro-
ceeds to see if his or her non-finding can stand up to additional tests
of its validity, by introducing controls for extraneous and confound-
ing factors that might cause a spurious non-correlation (see more
below).
Precisely because the usual and correct research procedure is to try
to reject the null hypothesis, projects that aim to demonstrate no
sig