ESP Students Processing Multimodal Websites Through the Eye-tracking Technique
Supporting Agencies
- Ministry of Science, Innovation, and Universities
Abstract
With the increasing importance of the Internet for teaching and learning, websites have become an interesting pedagogic resource as they entail interconnected modes of communication to convey meaning. When combined with multimodal website analysis, eye-tracking can provide valuable insights into how users engage with different modes of communication on a website. In this pilot experimental study, we analyze and compare how twenty-six Computer Science and Business and Law double degree students process two entrepreneurial websites to understand their meaning through eye-tracking (RealEyeTM online tool, https://www.realeye.io/). On the one hand, we analyze to what extent the eye-tracking technique contributes to the ESP students’ perception of the multimodality of websites. On the other hand, we tested the students’ reactions to using an activity with an eye-tracker in ESP courses, and we compared the results between two academic backgrounds. Our results show more fixations on titles and body text than photos or graphics, and the overall reading pattern entails fast scanning with no significant differences between the two groups. This research proves that eye-tracking can be a valuable tool for understanding how people process multimodal texts. It can be used to improve the effectiveness of such texts for communication and learning.
Downloads
References
Anliker, J. (1976), Eye Movements: On-line Measurement, Analysis, and Control. In: R.S. Monty/ J.W. Senders (Eds.), Eye Movements and Psychological Processes. Hillsdale, 185–199.
Barneva, R.P., Valentin E. Brimkov, V. E., Gelsomini, F., Kanev, K., & Walters, L. (2019) Integrating Open Educational Resources into Undergraduate Business Courses. Journal of Educational Technology Systems, 47(3) 337–358.
Bateman, John (2008). Multimodality and Genre. A Foundation for the Systematic Analysis of Multimodal Documents. New York, NY: Palgrave-Macmillan.
Bernad-Mechó, E. (2021). Combining Multimodal Techniques to Approach the Study of Academic Lectures: A Methodological Reflection. Atlantis, 43,(1), 178-198. https://doi.org/10.28914/Atlantis-2021-43.1.10
Bisson, M. J., Van Heuven, W. J., Conklin, K., & Tunney, R. J. (2015). The role of verbal and pictorial information in multimodal incidental acquisition of foreign language vocabulary. Quarterly Journal of Experimental Psychology, 68(7), 1306-1326.
Bucher, H-J. & Niemann, P. (2012). Visualising science: the reception of PowerPoint presentations. In Holsanova, J. (Ed.) Methodologies for multimodal research, Visual Communication, 11 (3), 283-306.
Cao, Y., Ding, Y., Proctor, R. W., Duffy, V. G., Liu, Y., & Zhang, X. (2021). Detecting users’ usage intentions for websites employing deep learning on eye-tracking data. Information Technology and Management, 22(4), 281-292.
Capellini, M. & Hsu, Y.-Y. (2022). Multimodality in webconference-based language tutoring: An ecological approach integrating eye-tracking. ReCALL, 34 (3), 255-273.
Conklin, K., & Pellicer-Sánchez, A. (2016). Using eye-tracking in applied linguistics and second language research. Second Language Research, 32(3), 453-467.
Conklin, K., Pellicer-Sanchez, A., & Carroll, G. (2018). Eye-tracking: a guide for applied linguistics research. Cambridge University Press.
Cutrell, E., & Guan, Z. (2007, April). What are you looking for? An eye-tracking study of information usage in web search. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 407-416).
Delabarre, E.B. (1898). A method of recording eye movements. American Journal of Psychology, 9, 572-574.
Darvin, R. (2023). Moving across a genre continuum: Pedagogical strategies for integrating online genres in the language classroom. English for Specific Purposes 70, 101-115.
Ekman, P., & Friesen, W. V. (1972). Hand movements. Journal of Communication, 22(4), 353-374.
Fox, W. (2016). Integrated titles: An improved viewing experience? In S. Hansen-Schirra & S. Grucza (Eds.), Eyetracking and Applied Linguistics, 5–30. Berlin: Language Science Press. DOI:10.17169/langsci.b108.233
Girón-García, C. & Fortanet-Gómez, I. (2023) Science dissemination videos as multimodal supporting resources for ESP teaching in higher education. English for Specific Purposes 70, 164-176.
Godfroid, A., Winke, P. & Conklin, K. (Eds.) (2020). Special Issue: Eye Tracking, Second Language Research. 36 (3).
Hansen-Schirra, S. & Grucza, S. (Eds.). 2016. Eyetracking and Applied Linguistics. (Translation and Multilingual Natural Language Processing 2). Berlin: Language Science Press. DOI: 10.17169/langsci.b108.230
Hjortsjö, C. H. (1970). Man's face and mimic language. Lund, Sweden: Studentlitteratur.
Holsanova, J. (2014). Reception of multimodality: Applying eye-tracking methodology in multimodal research. In C. Hewitt (Ed.) Routledge Handbook of Multimodal Analysis (pp. 285-296). Routledge.
Hsu, T. C., Chang, S. C., & Liu, N. C. (2018). Peer assessment of webpage design: behavioral sequential analysis based on eye tracking evidence. Journal of Educational Technology & Society, 21(2), 305-321.
iMotions (2018). Eye-tracking: The Complete Pocket Guide. Retrieved 05/09/2022 from https://imotions.com/products/imotions-online/#Webcam-Eye-Tracking
Just, M. A., & Carpenter, P. A. (1976). The role of eye-fixation research in cognitive psychology. Behavior Research Methods & Instrumentation, 8(2), 139-143.
Kong, K. (2009). A comparison of the linguistic and interactional features of language learning websites and textbooks. Computer Assisted Language Learning 22 (1), https://doi.org/10.1080/09588220802613799
Krejtz, K., Duchowski, A., Krejtz, I., Szarkowska, A., & Kopacz, A. (2016). Discerning ambient/focal attention with coefficient K. ACM Transactions on Applied Perception (TAP), 13(3), 1-20.
Kress, G. & van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. Arnold Publishers.
Kress, G. &Van Leeuwen, T. (2006[1996]) Reading Images: The Grammar of Visual Design. Routledge.
Lamare, M. (1892). Des mouvements des yeux dans la lecture. Bulletins et Mémoires de la Société Française d’Ophthalmologie, 10, 354-364.
Land, M., & Tatler, B. (2009). Looking and acting: Vision and eye movements in natural behaviour. Oxford University Press.
Lim, V. F. (2011). A systemic functional multimodal discourse analysis approach to pedagogic discourse. Doctoral Dissertation, National University of Singapore.
Low, A. R. L., & Aryadoust, V. (2023). Investigating test-taking strategies in listening assessment: A comparative study of eye-tracking and self-report questionnaires. International Journal of Listening, 37(2), 93-112.
Montero Perez, M., Peters, E., & Desmet, P. (2015). Enhancing vocabulary learning through captioned Video: An eye‐tracking study. The Modern Language Journal, 99(2), 308-328.
O’Hagan, M. & Sasamoto, R. (2016). Crazy Japanese subtitles? Shedding light on the impact of impact captions with a focus on research methodology. In S. Hansen-Schirra & S. Grucza (Eds.), Eyetracking and Applied Linguistics, (31–58). Language Science Press. DOI:10.17169/langsci.b108.234
O'Halloran, K. L. (2011). Multimodal Discourse Analysis. In K. Hyland and B. Paltridge (Eds.), Companion to Discourse Analysis. London: Continuum, 120-137.
O’Rourke, B. (2012). Using eye-tracking to investigate gaze behavior in synchronous computer-mediated communication for language learning. In Dooly, M. & O’Dowd, R. (Eds.), Researching online foreign language interaction and exchange: theories, methods, and challenges. (pp. 305-342), Peter Lang.
Palmer-Silveira, J. C. & Ruiz-Garrido, M. (forthcoming) Multimodal Discourse Analysis and Electronic Devices: A Pilot Study on Emotionally Supported Evidence. Atlantis.
Paltridge, B. & Starfield, S. (Eds.) (2013). The Handbook of English for Specific Purposes. Wiley-Blackwell.
Pauwels, L. (2012). A multimodal framework for analyzing websites as cultural expressions. Journal of Computer-Mediated Communication, 17(3), 247-265.
Prior, P. (2013). Multimodality and ESP research. In Paltridge, B. & Starfield, S. (Eds.), The Handbook of English for Specific Purposes (pp. 519-534), Wiley-Blackwell.
Rayner, K. (2009). Eye movements and attention in reading, Scene perception and visual search. Quarterly Journal of Experimental Psychology, 62, 1457-1506. http://dx.doi.org/10.1080/17470210902816461
Rayner, K., McConkie, G. W., & Zola, D. (1980). Integrating information across eye movements. Cognitive psychology, 12(2), 206-226.
Roper-Hall, G. (2007). Louis Émile Javal (1839–1907): The father of orthoptics. American Orthoptic Journal, 57(1), 131-136.
Ruiz-Madrid, N. (2021). A multimodal discourse approach to research pitches. Journal of English for Academic Purposes 52, 101003.
Rus, D. (2020). Using adequate materials in teaching English for Specific Purposes for the practice of language skills. Studia Universitatis Petru Maior. Philologia 2, 102-106.
Schriver, K. A. (1997). Dynamics in document design: creating texts for readers. New York: John Wiley and Sons.
Shank, J. D. (2014). Interactive Open Educational Resources: A Guide to Finding, Choosing, and Using What's Out There to Transform College Teaching. John Wiley & Sons, Incorporated.
Scinto, L. F., & Barnette, B. D. (1986). An algorithm for determining clusters, pairs or singletons in eye-movement scan-path records. Behavior Research Methods, Instruments & Computers, 18, 41-44.
Valeiras-Jurado, J. (2020). Genre-specific persuasion in oral presentations: Adaptation to the audience through multimodal persuasive strategies. International Journal of Applied Linguistics, 30 (2), 293-312.
Wedel, M., & Pieters, R. (2008). Eye tracking for visual marketing. Foundations and Trends® in Marketing, 1(4), 231-320.
Wolfer, S. (2016). The impact of nominalisations on the reading process: A case-study using the Freiburg Legalese Reading Corpus. In S. Hansen-Schirra & S. Grucza (Eds.), Eyetracking and Applied Linguistics, (163– 186). Language Science Press. DOI:10.17169/langsci.b108.298
Yamada, E. (1969). Some structural features of the fovea centralis in the human retina. Archives of ophthalmology, 82(2), 151-159.
Zamani, H., Abas, A., & Amin, M. K. M. (2016). Eye tracking application on emotion analysis for marketing strategy. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 8(11), 87-91.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The works published in this journal are subject to the following terms:
1. The Publications Services at the University of Murcia (the publisher) retains the property rights (copyright) of published works, and encourages and enables the reuse of the same under the license specified in item 2.
2. The works are published in the electronic edition of the magazine under a Creative Commons Attribution Non-commercial Share Alike 4.0.
3.Conditions of self-archiving. Authors are encouraged to disseminate pre-print (draft papers prior to being assessed) and/or post-print versions (those reviewed and accepted for publication) of their papers before publication, because it encourages distribution earlier and thus leads to a possible increase in citations and circulation among the academic community.
RoMEO color: green
