Loading...

Criteria in the Holistic and Analytic Rating of EFL Learners' Production of Agreement and Disagreement Speech Acts among Non-native English Teachers

Motamedi, Mohammad | 2017

457 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: English
  3. Document No: 50068 (31)
  4. University: Sharif University of Technology
  5. Department: Languages and Linguistics Center
  6. Advisor(s): Alemi, Minoo
  7. Abstract:
  8. Conveying a successful message within a language profoundly stems from not only a linguistically appropriate utterance, but also a pragmatically appropriate discourse too. Broadly considering various facets of pragmatics, pragmatic assessment has not been potentially brought into perspective, and, as a result, the literature deserves more comprehensive attention. This study, guided by the principles of mixed methods design, pursued three purposes: to inspect the matches and mismatches, to explore rating variations, and to assess the rater consistency between the holistic and analytic rating methods of agreement and disagreement speech acts in L2 by non-native English teachers. To address this issue, 12 different pragmatic situations for agreement and disagreement DCTs accompanied by EFL learners’ response to each situation were rated by 50 non-native English teachers. Initially, they were asked to rate it holistically, incorporating both rating and providing comments. The content analysis of raters’ comments revealed twelve agreement and sixteen disagreement criteria. The descriptive statistics for agreement and disagreement rating indicated variations across different situations. In particular, the frequency and wide range of criteria adopted in both speech acts revealed that the raters were inconsistent in their use of the mentioned criteria. Further, they were asked to rate it analytically based on the assessment rubrics adopted from Ishihara and Cohen (2011). The findings of intra-class correlations implied that respondents were more consistent in analytic rating regarding both speech acts. Moreover, the results indicated that there was a convergence between the two rating methods suggesting that the raters adopted the same level of leniency and severity in rating. Overall, the results accentuated the significance of pragmalinguistic and sociopragmatic aspects of language for EFL raters. Finally, the results of the present study stress the necessity of rater training and informed, rubric-based pragmatic rating. This study offers crucial pedagogical implications, implications for syllabus designers, materials developers, language testers, and some suggestions for further research studies
  9. Keywords:
  10. Agreement ; Speech Acts ; Interlanguage Pragmatics ; Inter Language Pragmatic (ILP) Rating Criteria ; Disagreement ; Nonnative Raters

 Digital Object List

 Bookmark

No TOC