In the last of our mini-series of research blogs, Victoria Murphy asks what counts as research and whether everything that teachers are presented with should be given equal weight.
Research comes in many forms, from reading reports of previous studies to carrying out randomised control trials (RCTs), and everything in between. There’s often an implicit hierarchy at work, and we are told that only large scale studies are more reliable, for example, or that only action research can capture the truth of teachers’ everyday experiences. The truth, though, is that the quality of the research cannot be determined simply by identifying the nature of it. For example, a systematic review (which takes a well-defined, systematic approach to reviewing the research literature to address a particular research question) is very different from, for example, a study where teachers are interviewed to determine their thoughts, opinions, and beliefs about a specific issue. Both can be important and rigorous, and both can be unimportant and critically flawed. For me, one of the most important variables that determines whether something can be characterised as ‘research’ is whether it goes beyond opinion.
Ideally, the evidence base researchers develop is as objective as possible, so that others can see exactly how the research was carried out, how the data was analysed and as much as is possible, can replicate the findings.
Most people have opinions on things, and most research starts with opinions. A case in point: At one point in human history, everyone shared the opinion that the earth was flat. It was easy to buy into this opinion because all we had to do was look outside and see that we were on a flat plane. A piece of research, however, needs to go beyond opinion and systematically investigate an issue to develop a credible evidence base which may (or may not) substantiate that original opinion. It took years of careful observation and hypothesis-testing to realise that we were not on a flat plane, after all, but on a sphere; a sphere that was not in the centre of the universe. In other words, a credible evidence base emerged from patient observation and from allowing others to examine those observations, leading eventually to new understanding about the shape of our world. Ideally, the evidence base researchers develop is as objective as possible, so that others can see exactly how the research was carried out, how the data was analysed and as much as is possible, can replicate the findings. This kind of research is maximally useful to teachers (and policy-makers) and I believe this is the kind to which we need pay closest attention.
For many years now, I have attended teacher-events which purport to help teachers engage with cutting-edge research. Some of these events include speakers who know how to go about investigating complex educational issues in an informative manner. Unfortunately, however, some of these events do not, and it’s worth thinking about how to distinguish the two. Let me give you some examples.
First, some good examples. I’ve been fortunate to be a part of different organisations (NALDIC is an example) that genuinely engage with relevant research and offer teachers a conduit to recent and important findings in relevant areas. Universities do the same thing regularly, bringing together researchers and teachers to talk about particular issues. Importantly, these are spaces where each can learn from the other and are usually either free, or the price of admission is set to only cover the expenses of running the event rather than to enable the organisers to make a profit.
Contrast that kind of event to something I encountered last year. I was invited to give a workshop on some of my recent EAL research by a big educational organisation that regularly offers such events to teachers. I am always naturally favourably predisposed to such meetings (i.e., talking to teachers) so we explored what I would do and how I might go about doing it. At some point in our discussions I was sent a preliminary flyer that the organisers were going to send out to recruit participants. Reader, I was shocked when I read it. It claimed that if people were to attend my workshop, then they would be able to increase their pupils’ progress by as much as 2 – 3 national curriculum months!! I told the organisers that I strongly disagreed with this statement, that I felt this was false advertising, and that I would not do it if they continued to advertise the event in this way. They were also charging the participants many hundreds of pounds for the privilege of learning how to help their pupils advance so efficiently. Ultimately, I refused to participate because I could not convince this particular organisation to pull that claim out of their flyer. Their response to me was that they regularly use this kind of language to attract an audience. The fact that there was absolutely NO evidence whatsoever that participating in the workshop would actually lead to that level of progress in students was irrelevant to them.
Research examines whether specific claims are true through a process of rigorous data-collection, openness to scrutiny, and a readiness to change our minds if contrasting data emerges. Opinion doesn’t have to do any of those things.
An event like this one is easily avoided – if it sounds too good to be true it most likely is. However, other events can be more tricky. There are genuine organisations out there with laudable ambitions to help teachers engage with research which unfortunately include speakers who can be, in my opinion, condescending to teachers and who present opinion as research. At these events, it becomes important to be able to wade through the schedule and identify what will be worthwhile and what will be a waste of time (assuming it’s the research you want). Even then, though, it can be hard to tell the difference just from the abstract. I was at an event once where the speaker claimed in the abstract that we were going to learn about the cognitive processing underpinning children’s literacy development. What we were actually exposed to in this presentation was a series of platitudes (such as, ‘if you can just get your students to read, then everything comes after’) masquerading as research. What was not discussed in this presentation was research (going beyond opinion) on how to help children learn to read, how to support comprehension processes, how to help students find the pleasure in reading and hence motivation to keep doing it outside of school and so on. It was a polished and professional presentation, to be sure, but it contained no research evidence at all … even though this is how it was advertised.
I feel this is important: we rely on our informed, expert opinions every day, and listening to other people sharing their experiences is often interesting and valuable, but it’s not research. Research examines whether specific claims are true through a process of rigorous data-collection, openness to scrutiny, and a readiness to change our minds if contrasting data emerges. Opinion doesn’t have to do any of those things. If we let people advertise one as the other (either fixing the research to fit personal opinions, or presenting opinions as having the rigour of research evidence) then we let other people take away the professionalism of our sector.
I think research is for everyone and I firmly believe that teachers in particular can be more effective if they are critically engaged with research. What concerns me is that with limited funding and resources, schools and teachers are regularly drawn in to the snakeoil salesperson side of the continuum, thinking they are engaging in good research-based CPD when really they aren’t. Asking ‘what is the evidence base?’ is a key question, I think. Is what you’re learning about an opinion (which is not bad, but should be recognised as such), or is what is being presented the product of an investigation which attempts to go beyond opinion? As long as it is clear which is which then we’re all fine.
I’d like to also offer another suggestion as to how teachers can engage in cutting-edge research: by collaborating with researchers. For example, I am currently carrying out research in a number of primary schools in Oxford. When my research group works with schools and teachers we genuinely wish to work with them, not just use them as mini research laboratories. I recently gave a presentation along with a student of mine at one school’s staff meeting, and one of the teachers came up afterwards to say that this type of quality CPD would have cost the school an arm and a leg. By working with us, she helped us produce high-quality and relevant research, and her school gained access to the broader research on current thinking on issues of key importance to them (and us).
So, if you’re a teacher reading this and receive requests from people like me to do research in your schools, I would recommend you engage with that researcher (assuming they are a reputable researcher from a trustworthy institution of course). Be demanding: make sure that your shool, colleagues, parents and children get something out of it too. Good researchers will be delighted to have interested people with whom to discuss their work.
Victoria Murphy is a professor of Applied Linguistics and convenor of the Research in EAL (REAL) group at the Departent of Education, University of Oxford.
The EAL Journal is published termly by NALDIC, the subject association for EAL. Visit www.naldic.org.uk to become a member.