In the second part of her interview, Laura Patsko asks teacher-researchers Christine Coombe and JD Brown more questions about research, and their book, The Cambridge Guide to Research in Language Teaching and Learning.
I notice that the volume is divided into 4 broad sections, of which “Getting ready” is by far the longest! Is it fair to deduce that this aspect of the research process is often underestimated?
[JD] We didn’t do that on purpose, but subconsciously we must have realized the importance of getting ready and planning research. I can’t tell you how many times people have come to me with their data and asked, “Now what can I do with all this?” That sort of question puts me in a very bad position: having to save a hopeless mess of a study. The problem was put succinctly by Carol Taylor at Educational Testing Service when she said, you cannot fix by analysis what is broken by design. Without truly getting ready and doing good planning, research will most often turn out to be a mess. Put another way, just having a bunch of data does not research make.
[Christine] This is an interesting observation but, as JD has said, we didn’t do it consciously. Of course, when you think about doing a study the most important steps are the ones that occur early on in the process.
Well said! Clearly, it’s very important to start on the right foot with good research design. But of course, real life never quite goes according to plan and research is no exception. How can researchers cope with the inevitable surprises that will pop up in the process?
[JD] When things start to come unravelled, it is always very useful to know your other options. Our book provides a good deal of information in this regard. But it may also be necessary to have a plan B (or even a plan C, or D) ready just in case. The important thing from my point of view is to remain flexible. This means avoiding the novice mistake of thinking that research is a rigid, set-in-cement series of steps. It is not the case that: first, you read the literature; second, you formulate research questions; third, you gather data; fourth, you analyse the results, etc. Indeed, I have always found research to instead be a series of recurring cycles of those sorts of elements, which means that you might find yourself going back to the literature, or revising your research questions. Thus it is important to remain flexible, so that you can accommodate the fact that everything may change a great deal from the beginning to the end of a research project.
You mentioned earlier that this new book will help to demystify the research process for teachers. Have there been any ‘lightbulb moments’ in your own experience, where something mysterious about research suddenly seemed to make sense?
[JD] Because my mother was a librarian and started me doing research from the time I could read, I have never found research to be mysterious. However, I have had ‘lightbulb moments’, many of them. Sometimes, they have occurred because of mistakes that I have made, of which there have been many. For example, in my very first published study (Brown, 1980), I made the mistake of overlooking the descriptive statistics when interpreting my inferential statistics. Had I closely examined the descriptive statistics and thought about what they were trying to tell me, I would have realized that small changes in the difficulty of my cloze passage would have created completely different results and interpretations. Needless to say, I never overlooked my descriptive statistics again. Later I learned from that mistake and included passage difficulty as a variable in other studies of cloze tests. It turned out to be an important variable. In fact, in Brown (2013), I outline how a whole series of mistakes and unexpected anomalies in my cloze testing research led to ‘lightbulb’ moments, each of which led in turn to the next study in the series.
[Christine] For me the ‘lightbulb moments’ have come through learning about different types of research. I finished my PhD a while back and at the time my program was heavy on quantitative research. Through reading the research conducted by others, I have come to see the value in qualitative and mixed methods of research and data collection and now regularly engage in it.
References
Brown, J. D. (1980). Relative merits of four methods for scoring cloze tests. Modern Language Journal, 64(3), 311-317.
Brown, J. D. (2013). My twenty-five years of cloze testing research: So what? International Journal of Language Studies, 7(1), 1-32.
You can watch a recording of Christine and JD’s webinar here; to find out more about their new book, click here.