2/4/2014
Calfee and Sperling (2010) argue
that a major purpose of mixed methods researchers is “to present a valid ‘story
of reality’ (Luyten, Blatt, & Corveleyn, 2007)” (p. 15). Particular versions
of reality, it seems to me, divide researchers into various methodological
traditions. In regard to language and literacy, positivists are concerned with
the true score of students; while constructivists are focused on diversity of
literacy. Which of the two is a truer story of reality? Granted, statistics
that claim to be generalizable to a large population inevitably flatten
representation of what might be the reality. But, is there any value in this
type of representation? A large, state-mandated test that finds a particular
subgroup of the population to be insufficient in literacy is an easy target of
constructivist criticism.
I’d suggest caution here for two
reasons: 1) statistics or any other type of experimental, positivist research
is by definition limited in its generalizability. A correlation study, for example,
claims and sets out to find only a partial account of the whole picture. Correlation
between teachers’ comments and their effect on students’ writing explains only one
category of factors, with other known or unknown factors being controlled, that
might be at work. The research has neither an intention to objectify students who
are diverse human beings nor an ability to overgeneralize the contextualized. Correlation
researchers will be content to find that an identified factor accounts for a
certain percentage of a phenomenon.
2) What counts as “a story of
reality?” A satisfactory answer to this question should not stop at offering an
overall claim that diversity is the reality. What if a large-scale,
standardized test finds a less-privileged subgroup to score lower than a
traditionally privileged subgroup? Should we take the result as the
representation of a reality or criticize it for creating that reality? The
latter standpoint is ethically unassailable, but methodologically problematic. As
a technology, testing may be intentionally or inherently discriminant. If
designed, administered, or interpreted intentionally to the disadvantage of a
particular group of people, a test is no doubt discriminant. However, in some
cases tests may represent differentiated performances and achievements of test
takers. If any subgroup falls within the low-achieving section of all
test-takers, should researchers at least consider if this is an existing, rather
than created reality? Before jumping to conclude that differentiated test
scores are racially, socially, economically, or culturally discriminating,
should researchers at least see if discrimination has already created differentiated
achievements that are only accurately represented in tests and test scores?
It is just too easy to problematize
standardized tests by politicizing any such tests, and along with them, any
quantitative methods and measurements. Calfee & Sperling’s mixed methods
should be more important than an attempt to bridge qualitative and quantitative
methodologies; they should bring changes to our epistemology. Seeing knowledge
as socially constructed has the benefit of foregrounding diversity and complexity
in literacy, but runs the risk of losing sight of patterns and trends. Similarly,
search for patterns and causality would necessarily overlook individual cases. What
may serve the purpose of bringing qualitative and quantitative communities
together entails an ideological and cultural overhaul, if we are not content
with mere talk.
More specifically, a major issue at
hand is whether or not researchers see any commonality in their individualist,
democratic practices of literacy instruction and research. Let me offer an
extreme case to illustrate the issue. Suppose a literacy instructor or
researcher is fully aware of cultural, racial, and social diversity in his or
her class and has developed a successful coping strategy. What then? Should he
or she promote the strategy to a larger population or constrain it to the
specific context of a class? Should he or she not choose to generalize the
practice in fear of making a quantitative mistake, is this not exactly what we
mean by discrimination? Or, should the diversity-driven instructor or
researcher move further to erase diversity by promoting the practice in his or
her district, state, and the entire nation? Isn’t this exactly what a
standardized test is doing? Finally, is diversity a merit to celebrate or a
status to change? These questions compelled me to believe that the real worth
of Calfee & Sperling (2010) lies in a proposal for revolutionizing our
epistemology.
Thinking of the qualitative and the
quantitative not as a binary but as a dialectical continuum has the benefit of
a fuller understanding of accountability. Everyone, instructors, students, and
policy-makers, should be held accountable for progress, not diversity, in
literacy. Please correct me if I am too bold: diversity is as irresponsible and
discriminatory as homogeneity.
Arthur, you bring forward a lot of points to consider, so please forgive the use of bullet points to help me construct my response.
ReplyDelete- I do agree that people with the responsibility of literacy sponsorship (instructors, policy-makers, etc.) should be held responsible for what we consider as progress. However, I would not hold diversity as irresponsible in of itself, but rather, irresponsible when we consider diversity as the opposition in the diversity/homogeneity binary. In a post modern sense, diversity and homogeneity occur at the same time - we are individuals who experience collective ideas as well.
- The problem you seem to be getting at is the concern of using tests to seek out problematic issues in diversity. Tests are constructed tools; they can be created to discriminate or can alternatively cause unintended discrimination.
- We often revise tests under the concern of diversity and related issues instead of looking to the root cause. To illustrate this, consider how we try to build cars with alternative fuels to "fix" the problem with environmental/sustainability concerns. However, in doing this, we ignore the connected problem of the fact that not all people can afford these vehicle. Instead, considering the creation of a sustainable mass public transit system such as trains would look at the concern from a perspective that does not cause the issue of the previous solution. We need a way to review problems from multiple perspectives, keeping us away from disingenuous use of diversity.
- Calfree & Sperling (2010) suggest the use of mixed methods as triangulation; I think this works as checks and balances so we can consider the problem at its root concerns, nuanced and on a continuum.
Ginny,
DeleteThank you very much for helping me clarify so many confusions. Like you, I find diversity to be often treated as a given, a god word, a hammer for all nails. For that reason, I agree with you that diversity goes hand in hand with homogeneity and that triangulation should be put in place to ensure some balance. But so much as tests are social constructs, diversity should in itself be a constructed concept and therefore be treated as a transient state that calls for change. I suspect collectivism and diversity each has its own role in literacy progress.