Analysis of pragmatic items in an ESL online adaptive placement test
Abstract
The work at hand is part of a wider study the aim of which was to determine what kind of factors influence pragmatic failure in an online adaptive placement test taken by Spanish students in their first year at university. A preceding analysis (Carrió and Martín, 2016) showed the type of personal factors that caused the exam takers pragmatic failure according to their own perception. In this paper, we go deeper and search about what specific pragmatic categories the items more frequently failed in the test belong to, so that we can reach some conclusions about its content validity and state whether the pragmatics construct is underrepresented or not in this test.
Downloads
References
Alderson, J. C. (2007). The challenge of (diagnostic) testing: Do we know what we are measuring? In J. Fox, M. Wesche, D. Bayliss, L. Cheng, C. Turner & C. Doe (Eds.), Language testing reconsidered (pp. 21-39). Ottawa: University of Ottawa Press.
Atlas, J. D. & Levinson, S. C. (1981). It-clefts, informativeness and logical form: Radical pragmatics (revised standard version). In P. Cole (Ed.), Radical pragmatics (pp. 1-62). New York, NY: Academic Press.
Austin, J. L. (1962). How to Do Things with Words. Oxford: Calderon Press. Retrieved February 12, 2017 from http://www.dwrl.utexas.edu/~davis/crs/rhe321/Austin-How-To-Do-Things.pdf
Bardovi-Harlig K. (2008). Recognition and production of formulas in L2 pragmatics. In Z.-H. Han (Ed.), Understanding second language process (pp. 205–222). Clevedon: Multilingual Matters.
Bardovi-Harlig, K. & Félix-Brasdefer, J. C. (Eds.). (2016). Pragmatics & Language Learning, Vol 14. Honololu: Natl Foreign Lg Resource Ctr. University of Hawi´i Press.
Barron A., Gu, Y. & Steen, G (Eds). (2017). The Routledge Handbook of Pragmatics. London: Routledge.
Bezuidenhout, A. L. & Morris, E. K. (2004). Implicature, Relevance and Default Pragmatic Inference. In D. Sperber & I. Noveck (Eds.), Experimental Pragmatics (pp. 257-282). London: Palgrave Macmillan.
Biesenbach-Lucas, S. (2007). Students writing emails to faculty: An examination of e-politeness among native and non-native speakers of English. Language Learning & Technology, 11(2), 59-81.
Blum-Kulka, S. (1982). Learning to Say What You Mean in a Second Language: A Study of the Speech Act Performance of Learners of Hebrew as a Second Language. Applied Linguistics, III(1), 29-59. doi: https://doi.org/10.1093/applin/III.1.29
Blum-Kulka, S., House, J. & Kasper, G. (Eds.) (1989). Cross-cultural pragmatics: Requests and apologies (Advances in Discourse Processes, XXXI). Norwood, NJ: Ablex.
Bouton, L. F. (1988). A cross-cultural study of ability to interpret implicatures in English. World Englishes, 17, 183-196.
Bouton, L. F. (1994). Conversational implicature in a second language: Learned slowly when not deliberately taught. Journal of Pragmatics, 22(2), 157-167.
Bouton L. F. (1999). Developing non-native speaker skills in interpreting conversational implicatures in English: Explicit teaching can ease the process. In E. Hinkel (Ed.), Culture in second language teaching and learning (pp. 47–70). Cambridge: Cambridge University Press.
Bromberek-Dyzman, K. & Ewert, A. (2010). Figurative competence is better developed in L1 than in L2, or is it? In M. Pütz & L. Sicola (Eds.), Cognitive processing in second language acquisition: Inside the learner’s mind (pp. 317–334). Amsterdam: John Benjamins.
Brown, J. D. (2008). Raters, functions, item types and the dependability of L2 pragmatics tests. In E. Alcón Soler & A. Martínez-Flor (Eds.), Investigating pragmatics in foreign language learning, teaching and testing (pp. 224-248). Bristol: Multilingual Matters.
Brown, P. & Levinson, S. D. (1987). Politeness: some universals in language usage. Cambridge: Cambridge University Press.
Carrió Pastor, M. L. & Martín Marchante, B. (2016). Identificación de errores pragmáticos en un test adaptativo de inglés como lengua extranjera. ELIA, 16, 79-102. doi: http://dx.doi.org/10.12795/elia.2016.i16.04
Cohen, A. D. & Shively, R. L. (2007). Acquisition of requests and apologies in Spanish and French: Impact of study abroad and strategy‐building intervention. The Modern Language Journal, 91(2), 189-212.
Cohen. D. (1996). Law, social policy, and violence: The impact of regional cultures. Journal of Personality and Social Psychology, 70, 961-978.
Contreras, E. (1990). El profesor universitario y la evaluación de los alumnos. Madrid: Universidad Politécnica de Madrid.
Coulmas, F. (Ed.) (1981). Conversational routine. The Hague: Mouton.
Dulay, H., Burt, M. & Krashen, S. (1982). Language Two. Oxford. Oxford University Press.
Félix-Brasdefer, J. C. (2007). Natural speech vs. elicited data: A comparison of natural and role play requests in Mexican Spanish. Spanish in Context, 4(2), 159-185.
Gallardo Pauls, B. (1995). El sobreentendido. Pragmalingüística, 3(4), 351-381.
Gallardo Pauls, B. (2005). El lenguaje y la mente. In A. López & B. Gallardo (Eds.). Conocimiento y lenguaje. Valencia: Universitat de Valencia.
Grice, H. P. (1975). Logic and Conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and Semantics, Speech Acts (pp. 41–58). New York, NY: Academic Press. Retrieved June 10, 2017 from http://www.sil.org/linguistics/bibliographylinguistics/ColePAnd Morgan 1975.htm.
Grice, H. P. (1989). Studies in the Way of Words. Harvard, MA: Harvard University Press. Retrieved February 10, 2017 from http://courses.media.mit.edu/2005spring/mas962/Grice.pdf
Gumperz, J. (1982). Discourse strategies. Cambridge: Cambridge University Press.
Halliday, M. A. K. (1979). El lenguaje como semiótica social. México: Fondo de Cultura Económica.
House, J. (1996). Developing pragmatic fluency in English as a foreign language: Routines and metapragmatic awareness. Studies in second language acquisition, 18(2), 225-252.
Kasper, G. (2006). Speech acts in interaction: Towards discursive pragmatics. In K. Bardovi-Harlig, J. C. Félix-Brasdefer & A. S. Omar (Eds.), Pragmatics & Language Learning, Vol. 11 (pp. 281–314). Honolulu: University of Hawai’i at Manoa.
Kasper, G. & Schmidt, R. (1996). Developmental issues in interlanguage Pragmatics. Studies in Second Language Acquisition, 18, 149–69.
Keenan, E. O. (1976). The universality of conversational postulates. Language in Society, 5(1), 67-79.
Kerbrat-Orecchioni, C. (1996). La conversation. Paris: Seuil.
Levinson, S. C. (2000). Presumptive meanings: The theory of generalized conversational implicature. Cambridge, MA: MIT press.
Lewis, M. (1993). The lexical approach: the state of ELT and the way forward. Hove: Language Teaching Publications.
Lewis, M. (2000) Language in the lexical approach. In M. Lewis (Ed.), Teaching collocation: further developments in the Lexical Approach (pp. 126-154). Hove: Language teaching publications.
Liu, J. (2006). Measuring interlanguage pragmatic knowledge of EFL learners. Frankfurt: Peter Lang.
Madrid, D. (1997). La evaluación del área curricular de la lengua extranjera. In H. Salmerón (Ed.), Evaluación Educativa: Teoría, metodología y aplicaciones en áreas de conocimiento (pp. 252-290). Granada: Grupo Editorial Universitario.
Martín Marchante, B. (2015). La evaluación de la competencia pragmática a través de una prueba adaptativa. Unpublished Doctoral Dissertation, Universitat Politècnica de València, Spain.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (pp. 13-103). New York, NY: Macmillan.
Murray, J. C. (2011). Do Bears Fly? Revisiting Conversational Implicature in Instructional Pragmatics. The Electronic Journal for English as a Second Language, 15(2), 1-10.
Murray, N. (2010). Pragmatics, awareness raising and the co-operative principle. ELT Journal, 64(3), 293-301.
Nattinger, J. & DeCarrico, J. (1992). Lexical phrases and language teaching. Oxford: Oxford University Press.
Payrató, LL. (2010). Pragmàtica, discurs i llenguatge oral. Introducció a l’anàlisi funcional dels textos. Barcelona: Universitat Oberta de Catalunya.
Purpura, J. (2004). Assessing Grammar. Cambridge: Cambridge University Press.
Purpura, J. (2009). The Oxford Online Placement Test: What does it measure and how? Retrieved February 6, 2017, from https://www.oxfordenglishtesting.com/uploadedfiles/6_ NewLook_and_Feel/Content/oopt_measure.pdf
Read, J. & Nation, P. (2004). Measurement of formulaic sequences. In N. Schmitt (Ed.), Formulaic sequences: Acquisition, processing and use (pp. 23-35). New York, NY: John Benjamins.
Reyes, G. (1990). La pragmática lingüística. Barcelona: Editorial Montesinos.
Roever, C. (2001). Web-based language testing: Opportunities and challenges. Language Learning and Technology, 5, 84–94.
Roever, C. (2005). Validation of a web- based test of ESL pragmalinguistics. Retrieved February 22, 2016, from http://ltj.sagepub.com/content/23/2/229.full.pdf
Roever, C. (2006). Validation of a web-based test of ESL pragmalinguistics. Language Testing, 23(2), 229-256.
Roever, C. (2011). Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463-481.
Schegloff, E. A. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge: Cambridge University Press.
Searle, J. (1969). Speech acts: An essay in the philosophy of language. Cambridge: Cambridge University Press.
Sperber, D. (1986). Relevance: Communication and Cognition. Cambridge, MA: Harvard University Press.
Taguchi, N. (2006) Analysis of appropriateness in a speech act of request in L2 English. Pragmatics 16(4): 513-533.doi: 10.1075/prag.16.4.05ta
Taguchi, N. (2008). Pragmatic comprehension in Japanese as a foreign language. The Modern Language Journal, 92(4): 558-576.
Taguchi, N. (2011). Rater variation in the assessment of speech acts. Pragmatics, 21(3), 453-471.
Taguchi, N. (2012). Context, individual differences, and pragmatic competence. New York, NY/Bristol: Multilingual Matters.
Taguchi, N. & Roever, C. (2017). Second Language Pragmatics. Oxford: Oxford University Press.
Tanaka, S. & Kawade, S. (1982). Politeness strategies and second language acquisition. Studies in second language acquisition, 5(1), 18-33.
Tsutagawa, F. (2013). Pragmatic Knowledge and Ability in the Applied Linguistics and second Language Assessment Literature: A review. Teachers College Columbia University Working Papers in TESOL & Applied Linguistics, 13(2), 1-20.
Turnbull, W. (2001). An appraisal of pragmatic elicitation techniques for the social psycho- logical study of talk: The case of request refusals. Pragmatics, 11, 31-61.
Walters, F. S. (2004). An application of conversation analysis to the development of a test of second language pragmatic competence. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, IL. Retrieved January 10, 2018 from https://www.ideals.illinois.edu/ handle/2142/79792
Walters, F. S. (2007). A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. Language Testing, 24(2), 155–183.
Wilson, D. & Carston, R. (2007). Relevance, Inference and Ad Hoc Concepts. A Unitary Approach to Lexical Pragmatics. Retrieved October 24, 2014 from http://www.phon.ucl.ac.uk /home/robyn/pdf/ Wilson-Carston-Unitary-Approach-2007.pdf
Yamashita, S. (2008). Investigating interlanguage pragmatic ability. In E. Alcón Soler & A. Martinez-Flor (Eds.), Investigating pragmatics in foreign language learning, teaching and testing (pp. 201–223). Bristol: Multilingual Matters.
Yule, G. (1996). Pragmatics. Oxford: Oxford University Press.
The works published in this journal are subject to the following terms:
1. The Publications Services at the University of Murcia (the publisher) retains the property rights (copyright) of published works, and encourages and enables the reuse of the same under the license specified in item 2.
2. The works are published in the electronic edition of the magazine under a Creative Commons Attribution Non-commercial Share Alike 4.0.
3.Conditions of self-archiving. Authors are encouraged to disseminate pre-print (draft papers prior to being assessed) and/or post-print versions (those reviewed and accepted for publication) of their papers before publication, because it encourages distribution earlier and thus leads to a possible increase in citations and circulation among the academic community.
RoMEO color: green
