Loading...

The Effect of Raters' Training on Their Evaluation of EFL Learners' Speaking Abilities

Pashmforoosh, Roya | 2011

629 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: English
  3. Document No: 42534 (31)
  4. University: Sharif University of Technology
  5. Department: Languages and Linguistics Center
  6. Advisor(s): Tajeddin, Zia; Alemi, Minoo
  7. Abstract:
  8. Inconsistent rating seems to largely emanate from the application of different rating criteria (Bachman, 1990). It follows that rater training programs may bring about higher rating consistency. The objectives of this study were to discover (1) EFL teachers’ criteria in rating L2 learners’ speaking ability, (2) the impact of a rater training program for speaking on the raters’ rating criteria, (3) EFL teachers’ accuracy in rating L2 learners’ speaking ability as measured against the professional raters’ rating, (4) the effect of teachers’ background variables on their speaking ratings, and (5) EFL teachers’ attitudes toward speaking assessment in general and rater training program in particular. To this end, 60 EFL teachers rated 10 monologs. The control group (N=32) rated them once, but the experimental group (N=28) rated them both before and subsequent to the rater training program. Change in teachers’ assessment practice took place to varying degrees after the program. The findings show that the EFL teachers specified 10 rating criteria, ranging from fluency to communicative effectiveness. However, the teachers in the experimental group reconsidered the significance of many rating criteria after the program. In effect, raters’ subsequent rating accuracy after training indicated a wide range of performance approaching those of expert’s rating. All raters who took part in the rater training program appeared to make use of a wide range of scoring scale available to them in their ratings. In practical terms, the tendency among the EFL teachers to see differences between students’ performances more starkly then showed a pronounced effect of training on the EFL teachers’ ratings. The results suggest that with rigorous training, raters with a variety of backgrounds were typically engaged in overall judgment of students’ performance with more accuracy. Regardless of teachers’ backgrounds, it is necessary to run training programs regularly to enhance rating ability of EFL teachers to assess students’ speaking ability. Given the prominent status of the rater training program, the observed tendency for positive reactions among EFL teachers attending the program sets the ground for believing that sustaining development in assessment practices may produce encouraging results. Future extensions of the teacher training program maybe expected to achieve even more marked results, and thus the current study appears to provide some support for the impact of a rater training program for EFL teachers’ speaking ratings
  9. Keywords:
  10. Speaking Assessment ; Raters Training Program ; Raters Consistency ; Raters Rating Criteria

 Digital Object List

 Bookmark

No TOC