Research Methods In Applied Linguistics Farhady.pdf
Vicki Patolot
Following this trend in general education, the concept of AL/AK gained significance in the language assessment field, too. Allocating the special issues of Language Testing Journal (2013) and Papers in Language Testing and Assessment (2017), as well as the 39th Language Testing Research Colloquium (LTRC) in Colombia-Bogot (2017) to the topic of language assessment literacy (LAL), is a witness of the significance of the issue in the language assessment field.
Because of the importance of the concept of LAK, scholars have suggested different models of LAK in the language assessment literature. Stabler-Havener (2018) summarized some of these models mentioned below. One of the early models of LAK, introduced by Brindley (2001), includes components such as the social context of assessment, defining proficiency, constructing and evaluating language tests, assessment in the language curriculum, and putting the assessment into practice. Later, Inbar-Lourie (2008) introduced another model of LAK which dealt with the why, what, and how of assessment. The why component referred to the reason for the assessment, the what component dealt with the trait being assessed, and the how component referred to the method of assessment. Davies (2008) introduced another popular LAK model focusing on three components of skills, knowledge, and principles. The skills component dealt with item writing, test analysis, using statistics, providing reports, and the ability to use software programs. The knowledge component referred to the background in measurement, description of the language, and contextualization of language assessment, and the principles component dealt with language test use, impact, ethics, and professionalism. Further, Fulcher (2012) provided a practices, principles, and contexts model of LAK. The practices dimension was concerned with the practice of language testing including the knowledge, skills, and abilities. The principles dimension was related to the guidance for practice including processes, principles, and concepts; and the contexts dimension was about the origins, reasons, and impacts of historical, social, political, and philosophical frameworks. Pill and Harding (2013) introduced a different model of LAK, taking into account five different components of illiteracy (not knowing the language assessment concepts and methods), nominal literacy (understanding that a specific term relates to assessment), functional literacy (having a sound understanding of basic terms and concepts), procedural and conceptual literacy (understanding the central concepts and using knowledge and practice appropriately), and multidimensional literacy (knowing ordinary concepts). Finally, a LAK model was introduced by Taylor (2013), which included eight dimensions of knowledge of theory, technical skills, principles and concepts, language pedagogy, sociocultural values, local practices, personal beliefs/attitudes, and scores and decision-making. The variety of the models of LAK within a decade shows how important this concept is in the language assessment field.
In line with the qualitative approaches mentioned above, this study was an attempt to add two more sources of data to enrich previous research. That is, in addition to direct face-to-face interviews with the teachers, we collected the actual tests that they prepared and their final evaluation of their students to see if there is any meaningful relationship between these variables. More specifically, we planned to answer the following research questions.
The interviews lasted between 20-40 min depending on the amount of information provided by the participants. The interviews were conducted individually in person, audio-recorded, and later transcribed for content analysis using QSR NVIVO version 10.
Further, teachers in both groups explained that they were willing to improve their LAK by attending workshops, in-service training classes, and studying recommended books. One of the teachers in the LG mentioned:
HG: I think language assessment knowledge means to assess your students throughout the semester. This can be done by bringing different examples to class, by bringing examples that are liked by the students, and by teaching in a way that whenever you ask the students, they can provide the information that you want.
LG: Teachers have to develop a test that has, you know, all parts in it. I think the vocabulary part, the grammar part, etc. A complete test of all parts. You know, creativity is always a matter of importance. You know, the indirect way of testing or assessing students is much preferred.
Also, similar themes were found in how they scored speaking and writing skills tests. The common points for scoring speaking were paying attention to grammar, vocabulary, accuracy, fluency, and pronunciation, and the common points for scoring writing were considering grammar, vocabulary, organization, punctuation, and spelling. A sample response from each group is provided below.
LG: For speaking, the first one is their fluency. The second one is using correct phrases. For writing, I actually pay attention to punctuation, the kind of grammar, the different kinds of words, and the synonyms they use.
Further, both groups provided similar responses for preparing students for high-stakes tests. They claimed that they used available sample tests, used preparation books, and taught the necessary tips and techniques to students. However, those who taught at lower-level ability classes said that they never prepared their students for standardized tests.
LG: You know, one thing that I do it all the time after each exam is I write down the most frequent mistakes in my notebook. After that, I will teach that point again on the board and I will design a very short, very small task for them based on that problem, and distribute it between them, and I ask them to complete it in a friendly group without any stress.
LG: A comprehensive test, I mean making a comprehensive test is more important. And I try to use the standard tests. I mean all parts must be included, and especially the vocabulary part and grammar part are more important than other parts. I have to focus on them.
Further, the types of tasks used in these teacher-made tests were compared. The most commonly used task types by the teachers in the HG were editing, fill in the blanks, matching, multiple-choice, question and answer, sentence completion, and true/false. However, the most commonly used task types by the teachers in the LG were only fill in the blanks and sentence completion. The comparison of the tests developed by the two groups of teachers showed that the teachers in the HG were more concerned about their tests and wrote longer tests which included different sections and a wider variety of tasks in each section, while the teachers in the LG mostly limited their tests to checking vocabulary and grammar. This implies that teachers with higher LAK scores are apt to apply their knowledge in practice, especially when they get involved in developing their tests.
This study was limited by the data collection techniques used and the small sample size. Further useful information can be collected from classroom observation on how teachers do classroom assessments and from focus-group interviews with teachers having different LAK levels on different language assessment issues. Using larger sample sizes may also provide more insights into the LAK realities of EFL teachers and how assessment issues are handled in various classroom contexts. Another limitation was related to the nature of the sample since it was not possible to differentiate between subgroups of teachers in terms of their gender, age, academic degree, major, teaching experience, and teaching context. Further research in which different subgroups of teachers are compared may provide illuminating results about multiple dimensions of LAK. This study was also limited to just the EFL context of Iran. Before making generalizations, it is desirable to do similar studies in various EFL and ESL contexts with diverse sub-groups of teachers.
Hossein Farhady is an associate professor of applied linguistics at Yeditepe University in Istanbul, Turkey. He received his MA in TESL and PhD in Applied Linguistics from University of California at Los Angeles in 1978 and 1980, respectively. His major interest is research on various aspects of language assessment. He has taught courses on language testing and research methods at the MA and PhD levels for the last four decades in several countries and presented papers in professional conferences. He has widely published on language testing, directed national and international research projects, and supervised over 50 MA theses and PhD dissertations.
Kobra Tavassoli is an assistant professor of TEFL at Islamic Azad University, Karaj, Iran. She is teaching language assessment, research methodology, and language teaching methodology courses at BA, MA, and PhD levels. Her areas of interest are language assessment and teacher education. She has published in different journals and presented in national and international conferences on these topics.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit
Hossein Farhady is a former associate professor of applied linguistics at Iran University of Science and Technology (IUST) in Tehran. He received his MA in TESL and Ph.D. in Applied Linguistics from UCLA in 1978 and 1980, respectively. His major area of interest is research on various aspects of language assessment.
b37509886e