A pilot study of the effects of background characteristics of interviewers on the inter-rater reliability of the oral testing procedure for the senior high school French program in the province of Newfoundland
Although numerous studies have shown a high degree of validity and reliability in oral proficiency testing, not all areas have been investigated. This paper is, essentially, the report of a pilot study investigating one such area: the relationship between background characteristics of teacher interv...
Main Author: | |
---|---|
Format: | Thesis |
Language: | English |
Published: |
Memorial University of Newfoundland
1991
|
Subjects: | |
Online Access: | https://research.library.mun.ca/5220/ https://research.library.mun.ca/5220/1/Flynn_KevinFrancis.pdf https://research.library.mun.ca/5220/3/Flynn_KevinFrancis.pdf |
Summary: | Although numerous studies have shown a high degree of validity and reliability in oral proficiency testing, not all areas have been investigated. This paper is, essentially, the report of a pilot study investigating one such area: the relationship between background characteristics of teacher interviewers and the reliability of the rating of an oral proficiency test. It was hypothesized that, despite the formal training sessions offered to the interviewers, differences in the ratings of the oral proficiency of interviewers would occur, and that these differences could be related to the diverse characteristics of the teacher interviewer. The questions asked were: -- (1) Are there significant differences in the way interviewers -- (a) rank the five factors of speaking proficiency? -- (b) rate the acceptability of errors? -- (c) rate proficiency levels? -- (2) If significant variations do occur, could these differences be associated with the language proficiency or the background characteristics of the interviewers? -- A questionnaire was distributed to one-half of the trained interview population on the island portion of the province. A response rate of 84 percent was achieved. The background characteristics investigated were: -- (1) demographic information, -- (2) number of years of teaching experience, -- (3) number of university French courses taken, -- (4) time spent in a French milieu, -- (5) extent of speaking French in the milieu with friends and acquaintances and at home, -- (6) a self-rating of aural and oral proficiency levels. The dependent variables employed were: -- (1) a ranking of the importance of five factors (vocabulary, grammar, pronunciation, fluency and comprehension) of oral proficiency; -- (2) a rating of the acceptability of examples of various types of errors, and -- (3) a rating of described proficiency levels as employed in the oral testing manual for the French 3200 Senior High School program. -- The data were tabulated and a profile of the average respondent constructed; different ... |
---|