Loading...

An Investigation into the Use of Different Rating Scales: Effect of Rater Variables on Rating Iranian EFL Learners’ Speaking Performance

Chorakchi, Sanaz | 2014

610 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 46369 (31)
  4. University: Sharif University of Technology
  5. Department: Languages and Linguistics Center
  6. Advisor(s): Alemi, Minoo; Tajeddin, Zia
  7. Abstract:
  8. Since speaking seems to be the foremost skill in language learning, the evaluation of speaking ability has become increasingly important in recent years. Moreover, research on rater effect in speaking performance assessment has provided evidence for a considerable degree of variability among raters. This study investigated the use of holistic vs. nalytic rating scales in rating Iranian EFL learners‟ speaking performance. Moreover, the current study explored the effects of gender and experience on raters‟ rating in rating Iranian EFL learners‟ speaking performance applying holistic vs. analytic rating scales. The effects of ater variables (gender and experience) on weighing components of analytic rating scale were also investigated in the study. Finally,attitudes of Iranian EFL teachers toward the use of holistic and analytic rating scales in rating EFL learners‟ speaking performance were examined. To this end, 44 Iranian EFL teachers, holding MA degrees in TEFL and English Language and Literature, rated 10 speech samples of two descriptive tasks performed by 5 freshman university students majoring in English Language and Literature twice with a 7-day interval: once using a holistic rating scale, and the other time using an analytic rating scale. Finally, individual interviews with the raters were conducted to find out their preferences toward these two methods of rating. Results suggested significant differences between the holistic and analytic rating scales, and in raters‟ assessment using the scoring rubrics. Reliability measurement of the analytic rating scale was significantly higher than the holistic rating scale. Additionally, it was found that male and female raters performed reliably in rating speech samples holistically and analytically; however, the reliability of the participants‟ analytic rating was significantly higher than that of their holistic rating. Reliability of high teaching experience raters, in addition, by comparison with low teaching experience raters,iv was significantly higher. Although the participants of this study assigned different weights to components of the analytic rating scale, raters‟ bias toward pronunciation (stress and intonation) was evident. Regarding attitudes of the teachers toward rating methods, high teaching experience raters preferred the holistic rating scale, whereas low teaching experience ones favored the analytic method. These findings have some implications for language testing, teaching, and learning
  9. Keywords:
  10. Analytical Method ; Rating Scales ; Speaking Performance ; Holistic and Analytic Rating Scales ; Rater Variable ; Descriptive Task

 Digital Object List

 Bookmark

No TOC