The Present Questionnaire - Assignment Example

? SXU4002 Assignment 3 Designing and critiquing questionnaires Questionnaires are useful tools when one wishes to gather data from a comparatively large number of individuals (Bowling, 2002), and when the subject matter in question is simple and pertinent to the respondent (McBurney & White, 2009; Oppenheim, 1992). Often, questionnaires are designed such that they may be filled in by the respondent without much supervision (McBurney & White, 2009; Kerlinger, 1986); and are thus quite simple to understand. While questionnaires are very useful when dealing with opinions, attitudes or topics that may be broken down into particular components; they are not very useful when attempting to elicit information on topics that require a lot of introspection and description (Oppenheim, 1992). Questionnaires typically consist of a number of simple questions (Sheatsley, 1983), each of which attempts to draw on the individual’s opinion or attitude on a single element (McBurney & White, 2009). Since these tools are typically used with a large sample of respondents (Kerlinger, 1986); the responses are often designed such that they may be numerically coded to aid comparison and statistical analysis (Bowling, 2002). Student evaluation of teaching modules is a practice that helps in refining the teaching material and teaching process and in identifying the concerns associated with student learning (CSEE, 2003). Since students are the recipients of the teaching process, it is important to understand whether the techniques used are working effectively, and whether students are having difficulty with any particular area (Greenberg, 2000). This is as important in undergraduate courses as in graduate courses, as ensuring strong learning outcomes at the undergraduate level will provide students with the interest and motivation to apply to and continue with graduate courses. The present questionnaire (in appendix) was designed to identify whether a particular module was effective in reaching its established goals from the perspective of the undergraduate student taking the course (CSEE, 2003). The questionnaire is designed to verify if the over-all goals of the module are reached; as against particular learning objectives that are subject specific. It attempts to ascertain the overall student experience with the module and its component parts. Since the questions used are generic, the questionnaire may be used across modules and with a few amendments (as required) across subjects. The questionnaire opened with some basic questions that identified the student and the course in order to ensure that the responses in the questionnaire were legitimate and useful (McBurney & White, 2009). The students were also asked to mark the percentage on attendance for the module; as this would indicate if the responses of the student were based on the entire module or on only a small part. Irregular students would be more likely to find the module difficult or to have a negative opinion about the way the module was conducted (Reid, 2002). On the other hand, a student may have missed a significant part of the module due to illness or other extenuating circumstances. It is important to remember that such students need to be given special consideration, and their data would be treated differently from the data of students who did not attend out of choice (Oppenheim, 1992). Thus, students with less than 75% attendance were asked to note down any particular reason for absence. The questions chosen were largely scaled multiple choice, objective questions (Kerlinger, 1986; Sheatsley, 1983). Objective questions make it possible to compare data across the responses of the same person (McBurney & White, 2009); as well as to compare responses across different individuals (Kerlinger, 1986). The responses to scaled questions may be combined to find a total or aggregate score (Martin, 2006; Sheatsley, 1983), and the data may be used for any statistical analysis deemed necessary (Anderson, Basilevsky, and Hum, 1983). In the case of Student Evaluations, the collected data is useful in identifying the percentage of students who were happy with the module versus the percentage that was not. It is also possible to find out which aspects of the module need to be worked on, and which are well received. At the end of the questionnaire, an open-ended, subjective question was added which required a descriptive response (Oppenheim, 1992). This ensured that if the student had an opinion or comment about an aspect not covered by the questionnaire, they have a chance to speak about it (McBurney & White, 2009; Bowling, 2002), and these opinions are taken on record. The questions were kept simple, and each question was worded such that it probed for only one aspect of the student’s experience at one time. The questionnaire was in three parts. The student was initially required to fill in demographic details as mentioned. This is not really a part of the questionnaire; but these questions ensure that the documentation is legitimate and valid (Trochin, & Donnelly, 2008). The first section of the questionnaire had 4 questions that were all answered using the same scale. All these questions were worded in the positive (Graeff, 2006) and it was believed that consistent framing would reduce confusing and mistaken marking. Positive statements are easier to understand and respond to, thus it was decided that the questions would be worded in the positive (Schwarz, 1999). This also gave a cue to the expectations that the administration had from the module and its teacher(s) (Graeff, 2006). The second part of the questionnaire had two questions that were framed such that the correct response from the scaled responses needed to be chosen to complete the statement (Bowling, 2002). This was done as justice could not be done to the questions using a generic scale; and it is important to emphasise the fact that the responses to these questions were slightly different from the previous questions (Schwarz, 1999). The third part of the questionnaire consisted of one subjective, open-ended question that probed for any thoughts or comments that the student may have about the module that were not covered by the previous questions. This questionnaire was given to a group of Psychology undergraduate students to mark as part of a pilot study; and the responses were analysed. Of the 25 students approached; 23 filled in the questionnaire keeping in mind a module they had just completed. These students were asked to refrain from filling in their Id; in order to ensure anonymity (Kerlinger, 1986), and since this was only a pilot study and the data would not be used to actually evaluate the module. The participants were also asked to write any comments they had about the questionnaire in the space provided after question 7. From the responses of the 23 participants, it was evident that they found the questions easy to understand. They also thought that the questionnaire was concise and easy to use, as the instructions were provided at each step. One comment made by several participants (10) was that question 4 could be problematic as it spoke about clarity of assessment criteria as well as their being provided well in advance (Martin, 2006). It is possible that one of these may occur in the absence of the other. Thus, perhaps it would be easier if these were made into two different questions such as: The assessment criteria for this module were clearly outlined and explained. The assessment criteria for this module were and shared beforehand/well in advance. Another observation from the pilot study was that 15 of the 23 participants mentioned in the response for question 7 that the teaching styles of the two teachers who taught the course were quite different. There was no indication if this was simply an observation or a problem; and it is possible that where two persons teach parts of one module, it may be difficult for students to integrate the information. This should be tested with an appropriate question like It was possible to integrate the information from the topics taught by the different teachers. Appendix Please fill in the questionnaire honestly and to the best of your knowledge. This information will be used for administration purposes only, and this data will not be published anywhere. Demographic details Student Id: ___________________________________ Course Name: _____________________________________ Approximate Percentage of Attendance: Less than 25% 26 – 50% 51 – 75% 76-90% More than 90% (Please mention any particular reasons for attendance lesser then 75%) ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ I Please respond to the following questions by marking the appropriate alternative from those provided. Please do not mark more than one response per question. 1. The lectures were useful in receiving and understanding information about the subject matter. 1 2 3 4 5 Not at all very slightly to some extent to a large extent all the time 2. The different parts of this module (lecture, tutorials, practical’s, assignments) worked together to provide a better understanding of the subject. 1 2 3 4 5 Not at all very slightly to some extent to a large extent all the time 3. The content of this module was presented in an interesting and involving manner. 1 2 3 4 5 Not at all very slightly to some extent to a large extent all the time 4. The assessment criteria for this module were clearly outlined and shared beforehand. 1 2 3 4 5 Not at all very slightly to some extent to a large extent all the time II Please choose the appropriate response from those provided to complete the sentence. Please choose only one response per question. 5. The relationship between this module and the rest of the course was_________ 1 2 3 4 5 Very Obscure Somewhat Obscure Identifiable Somewhat Clear Quite Clear 6. the workload of the course was ___________ 1 2 3 4 5 Negligible Quite Light Adequate Challenging but Doable Very Exhausting III Please respond to the following question honestly. 7. Please provide any comments you may have about your experience with this module that may not have been covered by the previous questions: __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ References Anderson, A., Basilevsky, A., and Hum, D. 1983. Scaling Techniques. In Rossi, Wright and Anderson (Editors), Handbook of Survey Research. pp. 246-269. New York: Academic Press. Bowling, A. 2002. Research methods in health. 2nd ed. Philadelphia: Open University Press. Committee on Standards for Educational Evaluation. 2003. The Student Evaluation Standards: How to Improve Evaluations of Students. Newbury Park, CA: Corwin Press. Graeff, T.R. 2006. Response Bias. In Kimberly Kempf-Leonard (Editor-in-Chief), Encyclopedia of Social Measurement. Pp. 411-418. Amsterdam: Elsevier Academic Press.  Greenberg, D. 2000. 21st Century Schools, edited transcript of a talk delivered at the April 2000 International Conference on Learning in the 21st Century. Kerlinger, F. 1986. Foundations of Behavioral Research 3rd Edn. New York: Holt, Rinehart & Winston. Martin, E. 2006. Survey Questionnaire Construction. In Kimberly Kempf-Leonard (Editor-in-Chief), Encyclopedia of Social Measurement. Vol 3. Pp. 723-732 Amsterdam: Elsevier Academic Press. McBurney, D.H.; White, T.L. 2009. Research methods, 8th edn. Belmont: Wadsworth Cengage. Oppenheim A.N 1992. Questionnaire design, interviewing and attitude measurement, 2nd edn. London: Pinter. Reid K. 2002. Truancy and Schools. Illinois: Routledge, Schwarz, N. 1999. Self-Reports: How the Questions Shape the Answers. American Psychologist, 54.2, Pp. 93-105. Sheatsley, P.B. 1983. Questionnaire Construction and Item Writing. In Rossi, Wright, and Anderson, Handbook of Survey Research. pp. 195-230. New York: Academic Press. Trochin, W.K. & Donnelly, J.P. 2008. Research Methods knowledge base 3rd. edn. Mason, OH: Cengage Learning. Read More