Archive for the ‘Qualitative research’ Category

Triangulation and Disconfirmation

January 17, 2010

Triangulation and confirmation

Methodological triangulation is the use of different kinds of method in a single investigation. In the nursing literature, there was a debate over triangulation in the nineteen eighties and nineties. While less febrile than it was, the debate continues today. The crux of the debate is whether both “qualitative” and “quantitative” methods could support a single result (theory). Those who argued against confirming triangulation thought that different methods could be only complementary, each supporting a distinct and independent part of a study. In “A New Foundation for Methodological Triangulation” (Journal of Nursing Scholarship, 34: 269-275, 2002) and “Methodological Triangulation in Nursing Research” (Philosophy of the Social Sciences, 313: 40-59, 2001), Sandra Dunbar, Margret Moloney, and I argued in favor of the use of qualitative and quantitative methods to support a single study. (See my web page for links to these and other essays.)

In our discussion, we noted that results of the different methods might conflict, but we did not pursue the further questions that such conflict raises. How are investigators to handle inconsistent results from different methods? Can the results of a quantitative study show that a qualitative result is incorrect, or vice versa? Does one kind of method trump the other? Published literature normally touts successes, not failures. Hence, it is difficult to find concrete examples that help us think about how one kind of method might undermine or disconfirm results obtained by another.

Heat wave

An interesting example for reflecting on triangulation comes from the sociological literature on the 1995 Chicago heat wave. 739 people died in a week of record heat and humidity. In Heat Wave: A Social Autopsy of Disaster in Chicago (University of Chicago Press, 2002), Eric Klinenberg argued that differences in death rates among neighborhoods were explained by their differing “social ecologies.” Death rates were higher in North Lawndale, a poor and predominantly African-American neighborhood, than in South Lawndale, a Latino neighborhood. Analysis by the CDC showed that throughout Chicago, being elderly and living alone were risk factors for heat-related deaths. Klinenberg appeals to factors like crime rates and population density as contributing to social isolation of the elderly, which would make them vulnerable.

The data for Klinenberg’s research were primarily demographic, and would count as “quantitative,” if we apply the category as used in the nursing literature. An interesting challenge to Klinenberg’s work comes from Mitchell Duneier, who conducted a series of interviews with residents of North Lawndale and nearby neighborhoods (“Ethnography, the Ecological Fallacy, and the 1995 Chicago Heat Wave” American Sociological Review, 2006). 16 people died in North Lawndale. Interviewing neighbors and family members, Duneier found that of the 16, “at least 12 were living with families and had domestic relationships” (p. 682). 14 of the 16 were said to have gone out of the house regularly. Duneier’s interviews thus directly undermine Klinenberg’s explanation for the heat wave deaths in North Lawndale.

Text and numbers in conflict

Duneier’s work shows one of the ways in which interview results can disconfirm survey results. Duneier roughly conforms to a Popperian logic: he identifies a consequence of Klinenberg’s explanation (that many or most of the deceased in North Lawndale were socially isolated), and shows it to be false. Characterizing the logic of Duneier’s argument in this way, however, effaces much of its texture.

Duneier describes his method as “ethnographic,” but the word is used differently her than in the nursing literature (or the anthropological literature, for that matter). What he means is that his method involved “shoe leather.” He and a photographer walked the neighborhood, knocked on doors, and struck up conversations with passers-by. Had he not done this, e.g. had he made a random selection from the telephone book, it would have been very unlikely that he would have gathered the kind of information he did. Moreover, Duneier did not obtain his result by simple counting. His open, unstructured interviews were a crucial part of unfolding the stories that surrounded each unique individual. Sifting through these, he identified the themes that were key to understanding the ways in which the deceased may or may not have been made vulnerable by the social ecology of their neighborhoods.

Qualitative and quantitative methods in conflict

One of the reasons why the triangulation issue continues to be discussed is that qualitative researchers often feel threatened by quantitative work. Statistical methods have the status associated with big science, and the politics of qualitative and quantitative research continues to plague the discipline of nursing. Duneier’s work shows how there is less distance among methods than is commonly assumed. Methods associated with the qualitative research do not require a methodology that is opposed to quantitative research. Because the methods are consistent, a careful and imaginative researcher like Duneier can bring them into productive conflict, providing a richer and more empirically adequate picture.

Advertisements

People as Measurement Devices

November 7, 2009

Measurement and qualitative research

In his presentation at the 2009 International Philosophy of Nursing Conference in Bristol, John Paley argued that qualitative research could be understood as a kind of measurement. I was reminded of this provocative thesis when I read a new essay by Gualtiero Piccinini “First-Person Data, Publicity & Self-Measurement” (Philosophers’ Imprint, Vol 9, 2009). Piccinini engages the literature that tries to defend a “science of consciousness.” He criticizes the idea that first-person reports create a special kind of science, but concludes by accepting first-person data (when rightly understood) as scientifically legitimate and valuable.

These lines of argument challenge two common nursing conceptions of qualitative research. First, they challenge the characterizations of qualitative research that use measurement as a defining feature of quantitative research. Second, they challenge the common idea that the qualitative investigator is the measuring instrument.

First-person data and measurement

Paley’s argument – and note, this is based on my memory of his presentation in Bristol, so it may not accurately represent his view – assimilated qualitative research to a larger conception of measurement. Measurement is a form of representation, and the use of numbers to scale the measurement is a special case. Paley’s presentation described some interesting techniques for measuring attitude and belief that are semantically sensitive in the sense that they locate the subjects’ belief (attitude) in a semantic network. There is no numeric scale, yet there is an apparently reliable representational scale.

Piccinini is concerned with more with the scientific value of direct, first-person reports. He is arguing against those who take first-person reports of, say, pain or mental imagery to be private, yet legitimate scientific data. Those who support such a view hold that because first-person reports are private, any science which relies on them must have a different character than the familiar sciences with their public, reproducible data.

Puccinini challenges the notion of private scientific data on several grounds, and he ultimately rejects the idea that “the validity of first-person data is untestable by public means” (p. 10, cf. p. 3). His positive view is that first-person data are legitimate scientific data because they are public. Pain reports are a kind of first-person data that is essential to nursing research. One of the mistakes behind supposing that such reports are private is to suppose that the person who is in pain is an observer of the pain. Puccinini suggests that the subjects who are in pain are not the observers: the investigators are. The investigator measures (and thereby observes) the pain through the first-person reports of the subject. Thus:

A subject generating first-person behaviors to fulfill the purposes of a scientific observer is a self-measuring instrument. When a subject generates first-person behaviors, she embodies not only (part of) the experimental materials but also (part of) the measuring apparatus. (p. 11)

A person pointing at a degree of a pain scale thus provides an observation of pain in much the same way as a Geiger counter provides an observation of radiation.

Consequences for qualitative research

Paley and Puccinini are exploring an interesting and new way of assimilating qualitative and quantitative research. If they are right, then “measurement” does not discriminate between qualitative and quantitative research. Interviews (even if conceptualized as phenomenological researchers do) are measurements in much the same sense as glucose assays.

Even more interesting is Puccinini’s argument that the scientist is the observer, not the subject who reports the pain (etc.). Qualitative researchers have often said that the investigator is the measuring instrument of qualitative research. This is supposed to mean that investigator is to record her own responses to the interview, along with what is said (or done). The interpreter’s impressions are the data for the qualitative research. This is a mistake, if Puccinini is right. The researcher is not studying herself, she is studying a population (albeit a small one). They are the ones with the experiences, and their first-person reports of those experiences are the basis of any qualitative research. Puccinini’s essay is a nice reminder that qualitative research is more than autobiography.