Analyzing And Improving The Appropriateness Of The Exams - Term Paper Example

Analyzing And Improving The Appropriateness Of The Exams Introduction One of the most difficult tasks that face examiners and teachers is the creation of tests that satisfy their course objectives. It is evident that the determination of the quality of an examination is done by assessing student performance, which can be determined by the appropriateness of the exam being taken. To determine whether an exam is of good quality, an examiner must consider each item in the examination, since the appropriateness of each individual question determines the quality of the whole examination. The examination being handed to the students has to be assessed to determine whether each question was fair, too difficult, or too easy for the students, where each scenario would show the quality of the teaching and examining procedure. To determine these factors, a myriad of assessment methods are used on the exam questions, including basic statistics to determine the performance of the students, qualitative and quantitative item analysis. In this case, basic statistics include the mean of each question in the exam and the entire test (Black, 2003). Qualitative item analysis refers to careful reading of the contents of the exam to determine the accuracy of the exam, find proofreading errors, and determine whether any components of the exam could tip the students towards the correct answer (Triola, 2003). This paper will analyze the hypothetical psychology test presented using the methods discussed above, after which a suggested method for improving the exam is given. The first part of this analysis will focus on basic statistics, followed by the qualitative item analysis and the quantitative item analysis. The quantitative analysis will be focused on determining a difficulty-rating index for each question in the test. Basic Statistics: Mean for Each Question Question 1 Question 2 Question 3 Question 4 Question 5 Student 1 2 2 0 2 0 Student 2 2 2 0 2 0 Student 3 2 0 0 2 0 Student 4 2 2 0 2 0 Student 5 2 2 2 2 2 Student 6 2 0 0 2 0 Student 7 2 2 0 2 0 Student 8 0 0 0 2 0 Student 9 2 2 0 2 0 Student 10 2 2 0 2 0 Mean 1.8 1.4 0.2 2 0.2 The table presented above shows the mean score for each question, as shown in the last row of the table. The mean for each question is found by adding the total scores for each question and dividing this by the number of students taking the test. Basic Statistics: Mean for Entire Test Question 1 Question 2 Question 3 Question 4 Question 5 Total Student 1 2 2 0 2 0 6 Student 2 2 2 0 2 0 6 Student 3 2 0 0 2 0 4 Student 4 2 2 0 2 0 6 Student 5 2 2 2 2 2 10 Student 6 2 0 0 2 0 4 Student 7 2 2 0 2 0 6 Student 8 0 0 0 2 0 2 Student 9 2 2 0 2 0 6 Student 10 2 2 0 2 0 6 Mean 5.6 The mean for the entire test is found by adding the total scores by each student, as represented by the totals column. The mean is then found by adding all the total scores and dividing the sum by the number of students taking the test (Black, 2003). Graphs for Each Question Analysis As previously mentioned, the analysis of the test will include an analysis of the basic statistics, which in this case is the mean for each question and the entire test. The mean score for the first question is 1.8, which indicates that the pass rate is high among all the students. Question 2 has a mean score of 1.4, while questions 3 and 5 have mean scores of 0.2. Question 4 has a mean score of 2. The mean scores indicated above show the average pass rate for each question for each question, and can be sued as an indicator for the difficulty rating for each question in the test. This indicates that the easiest question is question 4, and the most difficult questions are questions 3 and 5. The mean for the entire test is 5.6, and indicates that the pas rate was evenly distributed among the students. This pass rate is depicted by the graphs shown above, which indicate the scores for each question. To determine the difficulty and appropriateness of each question, we have to conduct both qualitative and qualitative analyses. Qualitative Analysis As previously stated, the qualitative analysis of a test is done by reading through the questions to determine whether any of them had any grammatical errors or whether the wording of the questions leads the students towards the correct answers (Triola, 2003). A qualitative analysis of each question indicates that question 4 should be revised, since it tips off the students towards the correct answer. The question is multiple choices, and from the answers provided, it is evident that the first choice, the correct choice, is leading. Quantitative Analysis Quantitative analysis is usually done to determine the item difficulty rating for each question (Triola, 2003), which indicates whether the exam is fair for the students. Difficulty-Rating Index The item difficulty index is an index that can be used by an examiner to determine whether the questions in the test are fair for all the students, and in this case, it is determined by finding the number of correct responses per question and dividing this figure by the number of students taking the test (Black, 2003). The results of the item difficulty index can be rated using the following scale. Item Difficulty Index Rating Less than 0.2 Too hard 0.2-0.8 Optimal More than 0.8 Too easy From the difficulty index shown above, each question can then be evaluated for fairness, and the following table shows the results of the rating. Question 1 Question 2 Question 3 Question 4 Question 5 Student 1 2 2 0 2 0 Student 2 2 2 0 2 0 Student 3 2 0 0 2 0 Student 4 2 2 0 2 0 Student 5 2 2 2 2 2 Student 6 2 0 0 2 0 Student 7 2 2 0 2 0 Student 8 0 0 0 2 0 Student 9 2 2 0 2 0 Student 10 2 2 0 2 0 Correct Answers 9 7 1 10 1 Index 0.9 0.7 0.1 1 0.1 Rating Too Easy Optimal Too Hard Too Easy Too Hard The rating shown above can then be used to evaluate the question, since it indicates that not all the questions are appropriate for the test. For example, question 2 is deemed too easy for the students, which indicates that the question should be revised to match the objectivity of the exam. In this case, only one case is deemed appropriate, or optimal, for the students. Question 2 is deemed optimal since, according to the item difficulty rating calculated, it is appropriate for all the students. The question 3 and 5 are too hard, while questions 1 and 4 are too easy for the students. Recommendations The aim of performing an analysis of a test is to identify the flaws in the test and improve them to match the objectives of the course. In the case of the test identified above, it has been identified that some of the questions are not appropriate for the students. The item difficulty rating calculated above reflects the results of the basic statistics identified earlier, since the too easy questions were passed by most students, and the too hard questions were failed. To improve the exam, the examiner should provide different questions, for example, question 4 could be improved by removing the multiple choices so that all the questions have the same difficulty rating. References Black, K. (2003). Business Statistics for Contemporary Decision-Making. New York: Wiley. Triola, M. (2003) Elementary Statistics. Boston: Addison Wesley. Read More