reygnai jonetta vanetah

[Liliane Hubright]

English language educators create and adapt tests continuously throughout their careers, but often assessment can be an afterthought in course design. Bachman and Palmer write that assessment developers must be held accountable for the tests they create, because those tests affect stakeholders in significant ways (2010). For example, tests can affect whether or not students enter university or pass a class. They also determine pass/fail rates in classrooms, which can influence educator pay or promotion. For these reasons, it is important that educators understand best practices in test design, especially aligning classroom exams with student learning outcomes (i.e., the overarching goals of a course) and objectives (steps to achieve the learning goals), vetting assessments, creating strong test questions and evaluating performance results.

Furthermore, students need to clearly understand how tests support learning goals and objectives. Hughes (2003) writes that objectives based exams clearly show the degree to which students have reached learning goals, and they promote strong teaching practices and course design, as educators and students are keenly aware of course goals outlined in syllabi. This is why educators should focus on assessing goals and objectives within their exams.

Once a review of test specifications is complete, educators can begin item (question) selection and test design with their objectives in mind. Although many educators have access to tests either created by textbooks or by colleagues, it is crucial that these exams be thoroughly reviewed. One cannot assume that externally provided exams include well-written items (even exams published by textbooks), or that exams written by educators connect to learning outcomes or objectives. Whether analyzing or creating test questions, it is important to read and apply best practices in question design. Carnegie Mellon University provides a great resource that offers information for aligning test questions with objectives. They argue that test objectives should be clear and obvious to the student, and that specific test items should unmistakably reflect objectives.

Coombe, Folse and Hubley also discriminate between objective and subjective test questions. Objective test questions (i.e., multiple choice or true/false) are difficult to write, but most of the work is done prior to test delivery, and reliability is enhanced by standardized answer keys. Subjective test questions (i.e., essay or short answer) require students to produce a longer response and are easier to create, but are less reliable and demand more time and attention when scoring. In general, it is best to balance the test question types, ensuring that exams include both selection and supply items, and that exams have objective and subjective questions. This is the first point of focus when either analyzing or creating an exam, and the next area of focus is on the exam items themselves.

There are many types of objective test questions, like multiple choice (MCQs), true/false (T/F), cloze and matching. In general, when writing these questions, Coombe, Folse and Hubley recommend that educators do the following:

Another addition to this list is to include test question point values that differ by difficulty. For instance, vocabulary cloze questions may be worth one point for lower level students, while vocabulary production questions may be worth two points for the same group.

Since subjective test items usually demand longer responses from students, this testing format subsequently requires more attention and focus from educators when grading. However, they provide educators with opportunities to assess higher order thinking skills like critical thinking, reflection, justification, and interpretation (Coker, Kolstad & Sosa, 1988). Although assessing these skills is important, the authors point out that subjective questions also have diminished reliability and equitability. Therefore, it is important to consider these factors when scoring and when choosing test items (See Appendices E & F).

When creating an answer key for an exam, it is essential to be as clear and explicit as possible for the grader. If there are multiple points available for a question, describe how students will achieve those points differently. Hughes writes that educators must expect to receive as many disparate answers as possible, especially for items that are worth multiple points.

If educators are using a rubric to evaluate writing or speaking, make sure that it too aligns with learning outcomes and objectives. The rubric should not assess a skill or knowledge that has not been learned in the course, unless it was taught in a previous course or unless students were expected to have mastered that skill prior to instruction. For more information on creating strong rubrics, Yale University has an excellent guide with examples of holistic and analytic rubrics for writing.

Designing valid, reliable, and appropriately difficult tests can be challenging, but sharpening this task should be a fundamental professional development process for all educators. As this article outlined, the learning goals and objectives for a course should offer a blueprint for test design, and all decisions made concerning test creation or analysis should stem from them. Also, reviewing best practices for item design and critically analyzing existing tests is an important step in the test creation process. Finally, educators should be piloting, editing and revising tests based on pre-test feedback and post-test results. Taking these steps when creating tests will ensure that they accurately evaluate student performance and that educators develop important professional development skills that they will use throughout their careers.

For MCQs, there are many credible websites that provide useful tips. For a thorough explanation of MCQs, Vanderbilt University offers comprehensive guidelines. For a shorter synapsis, see The University of Texas at Austin. The existence of these online university websites prove that MCQs are frequently used in university classes, and that even professors need support when creating them.

A few important factors to pay attention to when writing MCQs, as Brown, Race and Smith (1996) highlight, is to make sure that they have a clear stem [question or statement], that they have distractors [incorrect answers] that are actually in the text or audio, and that the answer key is correct. Coombe, Folse and Hubley add to this list by stating that MCQs should all have the same number of answers that are similar in length. Paying attention to these areas, and ones described in the websites above, will support the creation of reliable test questions.

When writing cloze or gap-fill items, Hughes points out that putting questions into context can help examinees make stronger choices. For example when testing vocabulary, writing a story or paragraph rather than disconnected sentences provides an element of authenticity. Also, making sure that there is no more than one option that fits into the blank is very important for these questions, especially when testing vocabulary and grammar. Finally, directions must be very specific for cloze questions. For example, will students lose points for incorrect part of speech, word form or other errors that stray from the original task? Educators should use learning outcomes and objectives to support these decisions. If the objectives state that students demonstrate understanding of new vocabulary and different word forms, then both skills should be counted and these questions should be weighted higher than those only testing one skill.

Matching questions can be useful when testing students on ordering information, selecting vocabulary definitions, or classifying material. Coombe, Folse and Hubley write that educators should provide more answers than premises [questions or statements], should number the premises and letter the options, and should ask students to write the correct letter in a blank rather than draw lines, which can be confusing for graders. Finally, they suggest that all items in a matching activity be thematically related, which adds coherence. Many of the resources listed above offer tips for writing matching items, but two very useful ones come from the University of Waterloo and Kansas State.

Short answer items are often used on tests to evaluate specific information either read in a text or heard via an audio or video clip. Coombe, Folse and Hubley state that these questions can be useful when testing productive skills like describing the main idea of a text, demonstrating understanding of certain sections, or responding with opinions in a short order format. However they can be difficult to score, especially when grammatical or lexical skills interfere with communication. Intra-rater reliability (or inconsistent scoring from one grader) can also be a problem. For these reasons, it makes sense to use short answer questions on exams for high intermediate to advanced students. To reduce grader subjectivity, educators should mark exams without looking at the student name, and/or have a colleague teaching a similar level check 2-3 of their exams, to make sure grades are not inflated or deflated.

The purpose of Placement Testing is to determine accurate course placement based on a student's skills, abilities, and language needs in reading, writing, and mathematics. There are different kinds of placement tests for different purposes. Each type has specific testing criteria. Placement tests are Not Pass or Fail but are instead designed to place a student into the most appropriate course that matches their current skill level in that area.

The Broward College ACCUPLACER ESL (LOEP) is a multiple-choice format (except for WritePlacer ESL essay). Students taking this test are placed into classes according to their skill level. There's no time limit (except for WritePlacer ESL essay). Questions are presented based on individual's skill level. The response to each question drives the difficulty level of the next question. It's important to give each question as much thought as possible before selecting an answer. The ESL Reading Skills, ESL Language Use, ESL Listening, and NG QAS (Qualitative, Reasoning, and Statistics) math subtests of the ACCUPLACER ESL (LOEP) are adaptive. This means that the questions are chosen for you on the basis of your answers to previous questions. This technique selects just the right questions for your ability level. Because the test works this way, you must answer every question in the order it is given. You can change your answer to a particular question before moving on to the next question, but you cannot leave a question out or come back to it later to change your answer. If you do not know the answer to a question, try to eliminate one or more of the choices. Then pick from the remaining choices.

c01484d022