There are many steps you can take to ensure that your test is more effective and that test writing becomes a learning experience.
- The elements of a good test
- Validity of a test
- Reliability of a test
- The affect of tests
- Other features of a good test
- Assessing difficulty
- Conclusion
The elements of a good test
A good test will give us a more reliable indication of our students' skills and it ensures that they don't suffer unfairly because of a poor question. How can we be sure that we have produced a good test?
- One way is very simply to think about how we feel about it afterwards. Do the results reflect what we had previously thought about the skills of the students? Another simple way is to ask the students for some feedback. They will soon tell you if they felt a question was unfair or if a task type was unfamiliar.
Validity of a test
A good test also needs to be valid. It must test what it is meant to test. A listening test that has very complicated questions afterwards can be as much of a test of reading as listening. Also a test that relies on cultural knowledge cannot measure a student's ability to read and comprehend a passage.
Reliability of a test
A test should also be reliable. This means that it should produce consistent results at different times. If the test conditions stay the same, different groups of students at a particular level of ability should get the same result each time.
- A writing test may not be reliable as the marking may be inconsistent and extremely subjective, especially if there are a number of different markers. Thus to try and ensure the test is more reliable it is essential to have clear descriptors of what constitutes each grade.
- In an oral interview it is important to ensure that the examiner maintains the same attitude with all the candidates. The test will be less reliable if he is friendly with some candidates but stern with others. You should try to ensure that the test conditions are as consistent as possible.
The affect of tests
We must also bear in mind the affect of our tests. Has the test caused too much anxiety in the students? Are the students familiar with the test types in the exam?
- If a student has never seen a cloze passage before she may not be able to write a test that reflects her true ability. The solution to this is to try and reduce the negative effects by using familiar test types and making the test as non-threatening as possible.
Other features of a good test
Other features of a good test are that there is a variety of test types and that it is as interesting as possible.
- A variety of test types will ensure that the students have to stay focused and minimise the tiredness and boredom you can feel during a repetitive test.
- Finding reading passages that are actually interesting to read can also help to maintain motivation during a test. A test should also be as objective as possible, providing a marking key and descriptors can help with this.
Assessing difficulty
Another important feature of a good test is that it is set at an appropriate level. You can only really find this out by giving the test and studying the results. Basically if everyone gets above 90% you know it is too easy or if everyone gets less than 10% it is obviously too difficult. For tests that aren't so extreme you will need to do some analysis of your test. You can do this by analysing the individual items for difficulty.
- In order to do this mark all of the tests and divide them into three equal groups, high, middle and low.
- Make a note for each item of how many candidates got the answer correct from the high and the low group (leave aside the middle group). To find the level of difficulty you need to do a quick calculation.
- Take one question and add the number of students from the high group who have the correct answer to the number from the low group
- Then divide this by the total number of people from both groups (high and low). It is thought that if over 90% of candidates get the answer right it is too easy. If fewer than 30% get it right it is too difficult.
- Also bear in mind that if most of the answers are in the 30's and 40's it would be best to rewrite the test. It's the same if most of the answers are in the 80's and 90's.
- The final step is to reject the items that are too easy or difficult.
Conclusion
Always bear in mind though that the difficulty of an item may relate to whether it has been covered in class or it may give an indication of how well it was understood. Such test analysis can give us information about how effective our teaching has been as well as actually evaluating the test. Evaluating tests carefully can ensure that the test improves after it is taken and can give us feedback on improving our test writing.
Below is a suggested procedure for writing a test.
- Decide what kind of test it is going to be (achievement, proficiency)
- Write a list of what the test is going to cover
- Think about the length, layout and the format
- Find appropriate texts
- Weight the sections according to importance/time spent etc.
- Write the questions
- Write the instructions and examples
- Decide on the marks
- Make a key
- Write a marking scheme for less objective questions
- Pilot the test
- Review and revise the test and key
- After the test has been taken, analyse the results and decide what can be kept / rejected.
Richard Frost, British Council, Turkey