PR4101–Research and the Professional.doc

PR4101 Research and the ProfessionalEmma KirbyHanham High School, BristolJune 2006 Introduction3What is the nature of the research you have chosen?3What are the aims and purposes of the research?3In what context was the research conducted?3Is the research question clear? How valid or worthwhile do you think it is?4What criteria could be used to evaluate the research?4Assumptions5What explicit assumptions has the researcher made about his/her epistemological and ontological paradigm?5What assumptions are implicit?5How do these affect the nature of the research?6Methodology6What methodological decisions have been made? Is a rationale offered for these decisions?6Data Collection8How was the data collected?8What issues are related to this?8How does the data collection relate to the aims of the research?9Are there any gaps in the data collection?10Data analysis10How was the data analysed?10How was it presented?10Are there any alternative analyses or explanations of the data?10Other issues10What ethical questions were addressed by the researcher (if any)?10What researcher effects were discussed and were potentially of concern?11Conclusions12How would you evaluate the research overall?12How would you evaluate the way the research has been written up?12Has your critique and evaluation of your chosen article informed your understanding of research and / or your own approach to research in the future discuss what you have learnt from engaging in this process.12References14IntroductionWhat is the nature of the research you have chosen?A small-scale research project which took place in Bristol, England between April 2002 and May 2003 although it purports to be of national and international interest. The project focused on the transition of Modern Foreign Language (MFL) learners from Key Stage 2 (KS2) to Key Stage 3 (KS3) from the perspectives of pupils, teachers, Heads of MFL departments and head teachers. It was carried out by 3 colleagues at the Graduate School of Education at the University of Bristol. What are the aims and purposes of the research?The aim of the research is to investigate some of the problems that arise during the transition of pupils studying languages from KS2 to KS3 in 6 schools in Bristol. It also aims to explore and suggest some strategies to overcome such problems. The purposes or objectives of the research are:v To identify the issues that learners, teachers and managers in the focus schools consider important in relation to transition. v To research the impact the identified issues on childrens enjoyment and achievement of language learning at KS3.v To establish the strategies that schools have in place to minimise the possible negative impacts that the identified issues. In what context was the research conducted?On the 25th March 1999, the Department for Education and Employment launched an initiative to promote and develop the provision and quality of MFL teaching and learning in primary schools. This was followed by the publication of the National Languages Strategy on 18th December 2002, which states that by September 2009 there should be “an entitlement to language learning for pupils at KS2” (DfEE, 2002). This research was carried out at a time when primary and secondary schools were beginning to look at formalising the teaching that was happening in many primary schools. Many secondary schools, including my own, were beginning to become more concerned about the issues that surround the transition of learners to KS3. Is the research question clear? How valid or worthwhile do you think it is?Cohen, Manion and Morrison (2000) explain that internal validity “seeks to demonstrate that the explanation of a particular event, issue or set of data which a piece of research provides can actually be sustained by the data” (page 107). Cohen, Manion and Morrison (2000) go on to describe external validity as “the degree to which the results can be generalised to the wider population, cases or situations”.Ensuring validity is imperative at every stage of a piece of research if “the researcher is to be able to have confidence in the elements of the research plan, data acquisition, data processing, analysis, interpretation and its ensuing judgement” (Cohen, Manion and Morrison, 2000, page 115) What criteria could be used to evaluate the research?As a teacher and a linguist I would use the following criteria to evaluate the research:v Is the research relevant to me and my situation?v Do the recommendations come from the data collected?v Are the recommendations realistic?As a researcher with several years of experience:v Does the research draw on recent data?v Has the research been validated?v Have the researchers used an appropriate methodology and appropriate methods of data collection?This is difficult as many of the findings of the study for example, enjoyment of language learning, openness to other cultures and confidence are very hard to evidence.Needs moving Assumptions What explicit assumptions has the researcher made about his/her epistemological and ontological paradigm?Cohen and Manion (1994, page 6) state that ontological assumptions “concern the very nature or essence of the social phenomena being investigated”. In this case the researchers ontological assumptions are based on their values and opinions of the problems faced by teachers and learners as they move from KS2 to KS3 i.e. the problems of continuity and progression. As explained by Cohen and Manion (1994, page 6) epistemological assumptions are concerned with the “nature, form, acquisition and communication” of knowledge. In this case the assumptions are based on soft knowledge i.e. more subjective, spiritual knowledge based on their personal experiences of being teachers/researchers. What assumptions are implicit?The research uses only one primary school yet makes suggestions for the teaching of MFL at primary schools in general. In my opinion the scale of the research is too narrow to make the wide-ranging assertions posited by the authors, particularly as four of the five secondary schools studied were selective independent schools. Any recommendations and changes in policy adopted by the government would not necessarily be implemented by the secondary schools in this case. As a case study the research is credible but as a basis for national and international claims it is unreliable.The researchers make a massive generalized claim when they state on page 36 that “one of the major disappointments in our (their) findings was the virtually total lack of liaison between primary and secondary phases”. If they are referring to the one primary and five secondary schools involved in the research then they need to be more explicit as the report reads that they are referring generally to schools, which is a hugely unsupported claim. In my opinion, the researchers, who are university lecturers involved in ITE, had pre-conceived ideas of the situation in schools before they began the research. They then applied some things they had learnt during their research and some issues raised by Burstall (1970), to schools in general. On page 37, the report states that “neither the primary school coordinator nor the secondary teacher had sufficient influence to be able to coordinate the curriculum between their two institutions”. The assumption here is that neither teacher held an influential post within the school for example as part of the senior management team, but the researchers again make assumptions without backing up their claim sufficiently and explaining their comments.Another implicit assumption is that there is a successful formula that can be applied to the transition of learners between phases. The researchers on page 37 state that no successful formula had yet been found. Given the fact that language learning deals with young people and teachers who live and work in many different contexts it would be narrow-minded to think that one solution could be found to suit all situations.On page 37 in the section titled Organisational Issues another huge assumption is made. The report says that “some of the uncertainties about the curriculum and its focus stem from a need for training” but there is nothing in the report write-up to support this claim, neither from the research project itself or from other sources. In my opinion the internal and external validity of the project is questionable, as the hypothesis does not seem to have been validated by any other source.The authors of this report make more generalized, unsubstantiated claims on page 37 when they state that “such schemes do exist” and that “limitations on secondary teachers availability were obvious”. They do not suggest where such schemes might be found nor why the limitations on teacher availability are obvious and to whom.How do these affect the nature of the research?These failings draw into question the credibility of the research project. It becomes a project that makes hugely impacting, generalized claims based on a very small-scale project. MethodologyWhat methodological decisions have been made? Is a rationale offered for these decisions?The researchers involved in this project have used decided to use a case study to explore possible problems with the transition of MFL learners from KS2 to KS3 however, at no point during the write-up do they offer a rationale for doing so. Stake (1994) says that case studies can be qualitative or quantitative; in this case it is qualitative.The research is what Stake (1994) calls “an instrumental case study” as it looks at one case, in this case a small number of schools closely linked geographically, in order to “provide insight into an issue or refinement of a theory”. Bassey (1999) states that there are at least three categories of educational case study:v Theory-seeking and theory testing case studyv Storytelling and picture-drawing case studyv Evaluative case studyThis project could be seen to fit into two of the above categories. It could be a theory seeking case study, as it seems to look to establish a theory on how transition from KS2 to KS3 should be managed. However, it contains elements of an evaluative case study as it evaluates the current management of transition between the phases. In my opinion, it sets out to be the former but due to lack of rigor at times it ends up being the second.Bassey (1999) quoting Stenhouse stays that there are four types of case study:v Ethnographicv Evaluativev Educationalv Action ResearchStenhouse, in Bassey (1999) says that “in evaluative case studies a single case or collection of cases is studied in depth with the purpose of providing educational actors or decision makers (administrators, teachers, parents, pupils etc) with information that will help them to judge the merit and worth of policies, programmes or institutions”. This backs up my claim that this piece of research is evaluative which is valid in its own right but tries to be more than it is in the way it makes generalised claims.Cohen and Manion (1989) explain that case studies involve the “observation of the characteristics of an individual unit a child, a clique, a class, a school or a community” (page 124). They then continue by saying that after such observations and investigation of the subject, case studies aim to establish generalisations that can be applied to a wider audience. If this is to be believed then it is acceptable that the researchers in this project draw generalisations form their work. However, my main concerns are that the sample is extremely small and not particularly representative. Nor did the recommendations made in the concluding part of the report seem to come from the body of research described in the report.There are advantages and disadvantages to using a case study method of enquiry. As described by Nisbet and Watt (1984), an advantage of a case study approach is that the results are accessible to a wider readership providing that the report is well written. In my opinion, this is one of the failings of the piece of research I have chosen to critique in that at times the language is poor and ambiguous. They also suggest that case studies are useful in identifying patterns of behaviour that may not be identifiable by other methods. I would agree with this point providing that the sample is wide and representative.Nisbet and Watt (1984) continue by explaining that the weaknesses of case studies include that the results are not necessarily easily generalizable, something which the researchers in this case do frequently. There are examples of this on page 37 as I have explained above.Data CollectionHow was the data collected?The data was collected using semi-structured interviews and discussion groups in a school-based context. These were secondary schools in the UKWhat issues are related to this?Cohen, Manion and Morrison (2000) suggest some steps that can be taken during the design and data gathering, analysis and reporting stages of a research project to minimise the risk of compromising the validity. The ones that could have been used in this case and could have therefore improved the internal and external validity of this piece of research are:v Using an appropriate sample e.g. one which is representative, not too small or too largev Selecting appropriate foci to answer the research questionsv Reducing the Hawthorn Effectv Avoiding making inferences and generalisations beyond the capability of the data to support such statements(Cohen, Manion and Morrison (2000)Cohen, Manion and Morrison (2000) define triangulation as the “use of two or more methods of data collection in the study of some aspect of human behaviour” (page 112). On page 36 of the report, in the section titled Methodology, the authors wrongly use the tool that is triangulation. The article states “the research involved interviews and small focus group discussions, designed to achieve triangulation by ensuring that each interview covered the same range of questions, regardless of the age or status of the interviewee”. Although the researchers used two different methods for collecting data they used only one method with each group. One could question as to whether they would have collected the same data if they had used a second method with each group.In the same section the researchers seem to have misunderstood the concept of semi-structured interviews. On page 36, they say that “each interview covered the same range of questions” but go on to say that “a semi-structured approach was adopted for the interviews so as to allow interviewees the maximum opportunity to express their views”. This is contradictory and raises the question of internal validity of the report. Either the interviews asked the same questions to everyone or they were semi-structured which allows for diversion of the questions depending on the responses of the interviewee. It is not possible to be both.How does the data collection relate to the aims of the research?It is important here to look at whether the researchers have taken steps when choosing a method of data collection to select one that will allow then to meet the aims and objectives of the project. For examples one of the aims of the research was to measure childrens achievements, yet it is unclear how semi-structured interviews and discussion groups can achieve that. Are there any gaps in the data collection?I am not sure to what extent semi-structured interviews with one cohort (staff) and focus group discussion with another (students) constitute a comprehensive data collection and maybe using both methods with both groups would have been fairer to the different communication styles of individuals.Data analysisHow was the data analysed?There seems to be no explanation of the analysis of the data except that the researchers mention that all interviews and discussions were taped and later transcribed. I would have expected to see an explanation of how data was coded and synthesised into evidence.How was it presented?The findings of the research are described under the following headings Transition-links between primary and secondary schools, Organisational Issues and Effects on Pupils motivation and achievement between KS2 and KS3. Are there any alternative analyses or explanations of the data?No, there are none offered.Other issuesWhat ethical questions were addressed by the researcher (if any)?There are many ethical issues raised in the research however none of them seem to be addressed by the researchers. I have concerns about the following ethical issues in relation to this piece of research:v “Each interview covered the same range of questions regardless of age or status of the interviewee” (Bolster, Balandier-Brown and Rea-Dickens, 2004, page 36). How can we be sure that the language of the interview was accessible to the pupils?v The report does not make clear that permission was sought to record and transcribe the interviews.v It is not clear whether the findings were taken back to the interviewees and checked that they were an accurate account of their thoughts and opinions.v This small-scale project leads the reader to make generalisations about KS2 to KS3 transition issues in English school. However, the recommendations are based on the experiences at 1 primary and 5 secondary schools. It may have been more representative to take the experiences of pupils at 5 primary schools with varying amounts of exposure t