Regulation of cognition: Validation of a short scale for Portuguese first-year university students

Título: Regulación de la cognición: Validación de una escala corta para estudiantes universitarios portugueses de primer año. Resumen: La calidad del aprendizaje en la Educación Superior depende, especialmente, de las habilidades de los estudiantes para regular su cognición. Esta regulación requiere habilidades cognitivas y metacognitivas, así como dimensiones motivacionales. Dada su relevancia en el rendimiento académico y el desarrollo de habilidades para el aprendizaje a lo largo de la vida, es importante aumentar la investigación en el campo. Este estudio pretende adaptar y validar una versión abreviada de la dimensión Regulación de la Cognición del Metacognitive Awareness Inventory para estudiantes universitarios portugueses de primer año. Se empleó una muestra de 360 estudiantes y se identificó una estructura tridimensional (Planificación, 4 ítems; Estrategias, 7 ítems; y Monitoreo y evaluación, 7 ítems) con un factor de segundo orden (Regulación de la cognición). Los valores de consistencia interna de la escala reducida son aceptables para una escala de autoinforme y las correlaciones con el logro académico al final del primer año de la universidad garantizan su validez predictiva. Esta versión abreviada para medir la regulación de la cognición puede usarse en investigación junto con otros instrumentos en estudios más amplios y puede funcionar como una herramienta de diagnóstico para ayudar a los estudiantes en los desafíos del aprendizaje en la enseñanza superior. Palabras clave: Regulación de la cognición; Enseñanza superior; Validación de instrumentos; Aprendizaje. Abstract: The quality of learning in Higher Education is particularly dependent on students' skills in regulating their cognition. This regulation requires cognitive and metacognitive skills as well as motivational dimensions. Due to its relevance in explaining students' academic achievement and developing lifelong learning skills, it ́s important to increase research in the area. This study aims to adapt and validate a short version of the Regulation of Cognition of Metacognitive Awareness Inventory to firstyear Portuguese university students. A sample of 360 students was considered and was identified a three-dimensional structure (Planning, 4 items; Strategies, 7 items; and Monitoring and evaluation, 7 items) with a secondorder factor (Regulation of Cognition). The internal consistency values of the reduced scale are within the acceptable parameters for a self-report scale and the correlations with academic achievement at the end of the first year of the university guarantee the predictive validity of the scale. This short version of regulation of cognition measure allows its use in research with other instruments in larger studies and can function as a diagnostic / screening tool to help students in higher education learning challenges.


Introducción
Knowledge and regulation of cognition are two classic components of metacognition (Flavell, 1987;Schraw & Moshman, 1995), which refers to the ability to reflect upon, understand, and control one's learning (Schraw & Dennison, 1994;Swanson, 1990).Regulation of cognition is referred to the awareness of metacognitive activities that help student to control one's learning, like planning, monitoring and evaluation (Schraw & Moshman, 1995).This self-awareness is, most of the cases, not enough to attain the one's goals when lacks fundamental skills, but it can produce a readiness that is essential for personal change.For example, the awareness of the limitations in the learning process by the student can be compensated by the development of key processes (e.g., goal setting, learning strategies, self-evaluation, intrinsic task interest) (Zimmerman, 2002).So, metacognitive selfawareness contributes directly or indirectly for academic performance and success (Stoten, 2019;Teng, 2017).
Several component skills of regulation of cognition have been discussed extensively, including planning, information management strategies, comprehension monitoring, debugging strategies and evaluation (De Backer, Van Keer, & Valcke, 2015;Veenman, 2015).Those components are usually presented in self-report scales (Schraw & Dennison, 1994;Schraw & Moshman, 1995).The self-regulatory model proposed by Zimmerman (2002) is more comprehensive, considering not only the cognitive and metacognitive processes but also the affective-motivational component (Efklides, Schwartz, & Brown, 2018).Combining cognitive and motivational components (Zimmerman & Schunk, 2011), the model of self-regulation is composed by three cyclical phases: (1) forethought phase refers to processes and beliefs that occur before efforts to learn; (2) the performance phase refers to processes that occur during behavioral implementation; and (3) the self-reflection phase, which refers to processes that occur after each learning effort.In each phase, there are several self-regulation processes.In forethought phase are considered two categories of processes: task analysis (includes goal setting and strategic planning processes), and self-motivation beliefs (includes self-efficacy, outcome expectations, intrinsic interest/value, and learning goal orientation).Performance phase takes into consideration the categories of self-control (includes the processes of imagery, selfinstruction, attention focusing, and task strategies), selfobservation (includes self-recording and selfexperimentation processes).The third phase, self-reflection anales de psicología / annals of psychology, 2020, vol.36, nº 2 (may) englobes self-judgment (includes self-evaluation and causal attribution processes), and self-reaction (includes selfsatisfaction/affect regarding one's performance, and adaptive/defensive reactions) (Zimmerman, 2002).So, selfregulation is not a mental ability or an academic performance skill, but it refers to a sequence of self-generated thoughts, feelings, and behaviors that are oriented to attaining goals (Bernacki, 2018;Soto et al., 2019;Zimmerman, 2002).In this sense, motivational dimensions play an important role on cognitive learning strategies (De La Fuente, Martínez-Vicente, Salmerón, Vera, & Cardelle-Elawar, 2016;Fernández et al., 2013).
The quality and quantity of students' use of selfregulatory processes are correlated to academic achievement (Stoten, 2019;Teng, 2017).According Zimmerman (2002), self-regulated learners monitor their learning behaviors in terms of their goals and effectiveness.Because of their superior motivation and adaptive learning methods, self-regulated students are not only more likely to succeed academically but to view their futures optimistically (self-satisfaction).The regulatory processes of cognition contribute for academic achievement thru a complex combination of students' learning strategies, self-awareness and motivational beliefs (Karlen, 2016;Soto et al., 2019;Zimmerman, 2002).Swanson (1990) showed that high levels of metacognitive knowledge about problem-solving could compensate for low overall aptitude.In contrast, as claimed by Cornoldi, Carretti, Drusi, and Tencati (2015), learners with low regulatory skills may not have explicit learning goals and present low performance.Usually in this differential analysis of proficiency is considered both general and domain-specific metacognitive competencies (Shamir, Mevarech, & Gida, 2009;Veenman, 2015).
Unfortunately, the traditional instruction encourages passive rather than active learning strategies (Schraw & Moshman, 1995), even though more and more it's expected that higher education contributes to the development of student competencies for lifelong learning (Adabaş & Kaygin, 2016).That means an incidence on self-regulation strategies, for example, students' skills to plan, implement strategies, monitor, correct misunderstandings and evaluate their performance.Students' awareness of their own strategies and effectiveness is a fundamental requirement for success and perseverance in higher education learning challenges (Ferradás, Freire, & Piñeiro, 2018;Ning & Downing, 2015;Öz, 2016).After graduation, self-regulation is required and some research illustrates positive effects in general and domain-specific metacognition if classes and curricular tasks are oriented for metacognitive instruction and peer cooperative learning (Donker, De Boer, Kostons, van Ewijk, & Van der Werf, 2014;Shamir et., 2009).By observing proficient of others and by practicing and self-refining their skills, individuals learn, achieve, and develop life-long learning skills (Zimmerman, 2002).
The assessment of metacognition, self-regulation or regulation of cognition processes can be done by question-naires, structured interviews, think-aloud techniques, error detection tasks or systematic observations of performance (Greene, Deekens, Copeland, & Yu, 2018;Ozturk, 2017).Some questionnaires can be presented, for example the Metacognitive Assessment Inventory (MAI; Schraw & Dennison, 1994), to assess adult metacognition, in different components like knowledge of cognition and regulation of cognition; the Metacognitive Awareness of Reading Strategies Inventory (MARSI; Mokhtari & Reichard, 2002), to assess specific aspects of metacognition in adults and adolescents, such as use of strategies, global strategies, problem-solving and practical strategies of support; the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich, Smith, Garcia, & McKeachie, 1991), with 12 items for metacognitive monitoring and selfregulation.Also popular, is the interview protocol Self-Regulated Learning Interview Schedule (SRLIS; Zimmerman & Martinez-Pons, 1986) which identifies eight different selfregulated strategies: rehearsing and memorizing, organizing and transforming, seeking information, self-evaluation, goalsetting, and planning, keeping records and monitoring, selfconsequence and environmental structuring.Unfortunately, for more extensive research where it´s important to consider a large number of variables, the length of those questionnaires could limit their use.
In this study, our aim was to achieve a short scale to assess the regulation of cognition in Portuguese first-year college students.The Metacognitive Awareness Inventory (MAI; Schraw & Dennison, 1994), adapted for Portuguese population by Bártolo-Ribeiro, Simões, and Almeida (2016) was considered.The original version of MAI from Schraw and Dennison (1994) defends two general metacognitive dimensions: knowledge of cognition (17 items), which includes three sub-processes, and regulation of cognition (35 items), with five sub-processes, for a total of 52 items.Posterior studies have not found a stable factorial structure (e.g., Akin, Abaci, & Cetin, 2007;Kleitman & Stankov, 2007;Teo & Lee, 2012), suggesting being necessary to introduces changes in its theoretical and empirical foundations.The study of adaptation to the Portuguese population, Bártolo-Ribeiro et al. (2016), through Principal Component Analysis, also found a non-stable factorial structure as stated by Schraw and Dennison (1994).However, except for three items, the remaining 32 items of the five subcomponents of cognition regulation (planning, information management strategies, monitoring, debugging strategies, and evaluation) were aggregated into three factors.The present study considers the possibility to reduce the number of dimensions of regulation of cognition and the number of items without compromising its reliability and validity of the assessment of the regulation of cognition.

Method Participants
A sample of 391 first-year students in two Portuguese higher education institutions (a public university in the north anales de psicología / annals of psychology, 2020, vol.36, nº 2 (may) and a private university institute at south) was considered.Considering the age range from 17 till 53 years old were considered for statistical analysis only students with less than 26 years old, resulting on a sample of 360 participants.Most of the students were female (79.8%) and a mean age of 18.07, (SD = 1.56),The students from public institution attended courses of social sciences and humanities degrees, while in the private institution students were from biology and psychology.Most students reported that they were in the course of their first choice (70.0%) and also their first-choice of institution (76.7%), 60.4% reported that had made vocational guidance by a psychologist before joining the university, 92.7% were not employed, and 63.9% reported that they did not have or had brothers/sisters in higher education.

Instrument
Regulation of cognition were measured by Metacognitive Awareness Inventory (MAI) developed by Schraw and Dennison (1994), previously adapted to Portuguese population by Bártolo-Ribeiro et al. (2016).Three items from original version were removed because in the adaptation process revealed ambiguity in factorial structure (see Bártolo-Ribeiro et al., 2016, for details).Responses were done on a 5-point rating scale which ranges from "1-Nothing applies to me [0%]" to "5 -Applies entirely to me [100%]" to report respondents' level of appliance.The five dimensions of Regulation of Cognition are defined as follow (examples of items for each dimension are given): (1) Planning -Includes goal setting, and allocating resources prior to learning (e.g., "I set specific goals before I begin a task"); (2) Information Management Strategiescorresponds to skills and strategy sequences used on-line to process information more efficiently (e.g., "I slow down when I encounter important information"); (3) Monitoring -Corresponds to assessment of one's learning or strategy use (e.g., "I ask myself periodically if I am meeting my goals"); (4) Debugging Strategies -used to correct comprehension and performance errors (e.g., "I change strategies when I fail to understand"); and (5) Evaluation -Includes analysis of performance and strategy effectiveness after a learning episode (e.g., "I know how well I did once I finish a test").Internal consistency for the five dimensions of the inventory range from .62 thru .73(Bártolo-Ribeiro et al., 2016).

Procedure
The same procedure of data collection was used in both institutions.Two months after entrance into university, were applied a questionnaire with the regulation of cognition scale among others.This questionnaire corresponds to a protocol to evaluate the quality of adaptation and adjustment firstyear student entrants, including sociodemographic data, past academic background, academic expectations and expectation of difficulties that students anticipated in their academic adjustment.Student participation was voluntary, and informed consent was given by all participants.The question-naire was administrated in the class context, after teacher's permission and collaboration.

Statistical Analysis
Factorial structure of the Regulation of Cognition was evaluated through the use of Confirmatory Factor Analysis (CFA) on the polychoric correlation matrix of the items.Due to the ordinal nature of the data, was used the WLSMV (Weighted Least Squares Means and Variance Adjusted) estimator available in MPlus (v.7.4;Muthén & Muthén, 2015), which is better than Maximum Likelihood with orderedcategorical Likert-type items (Bandalos, 2014).Goodness-offit was evaluated using the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), the Root Mean Square Error of Approximation (RMSEA) and the Standardized Root Mean Square Residual (SRMR).CFI and TLI values higher than .90and RMSEA and SRMR values lower than .08 were considered indicative of a good structural fit (Kline, 2011).Cronbach's alpha and McDonald's omega were used to assess reliability.Predictive validity was assessed using Pearson's moment-product correlations.

Factorial Structure
To confirm the factorial structure of the scale, a CFA was performed for a solution of five intercorrelated dimensions considering the 32 items.The results suggested a poor adjustment of the model to the data (Table 1).Furthermore, the results evidenced strong relations between some of the dimensions, namely Monitoring with Evaluation and Information Management Strategies (IMS) with Debugging Strategies (DS).Moreover, the analysis of the modification indices also suggests relating items from the Monitoring dimension with Evaluation items as well as items from IMS with DS items.Based on these results, we decided to test a model of 3 factors which can be supported by Zimmerman's theoretical model of self-regulation (Zimmerman, 2002), in which different processes and sub-processes of selfregulation are organized in the three phases described before: Forethought, Performance, and Self-Reflection.The reduction of the structure to a solution of three factors did not improve the fit of the model to the data and we decided to eliminate some of the items based on conceptual, practical and statistical grounds.Thus, we removed the items presenting the lowest factor loadings and, at the same time guaranteeing, that the content of the removed item is assessed by the remaining items.Supported by successive analyses we achieved a three-factor solution with 18 items, which showed an adequate fit to the data (Table 1).In this solution, based on the modification indices suggestions and based on the content analysis of the items, 4 pairs of items were correlated (Figure 1).Besides the three-factor solution, we also tested a unidimensional and a hierarchical model.The threeanales de psicología / annals of psychology, 2020, vol.36, nº 2 (may) factor solution showed a better fit than the unidimensional model (Table 1).The hierarchical model, with a secondorder latent factor, provided also a good fit to the data taking into account that a higher-order structure "can never be better than the corresponding first-order model" (Marsh & Hocevar, 1985, p. 570).From these results, we can assume the hierarchical model as representing adequately the regulation of cognition construct measured by the Metacognitive Awareness Inventory.Considering the averages obtained in the three indicators, all are above 3.5 (see table 2), showing the tendency for students to answer positively to the questions posed.This tendency could be justified by the fact that they are selfreporting a behavior regarding the socially desirable learning process in the context of the assessment.While in the plan-ning and evaluation/monitoring dimensions a very similar result was obtained with an oscillation between 3.68 and 3.71, a slightly higher mean value was found in the strategies dimension (M = 3.97).
Internal consistency (Cronbach's α / McDonald's ω) for each of three dimensions is adequate for the type of questionnaire and for the number of items: .69/.70 for Planning; .71/.72 for Strategies; .73/.73 for Monitoring/Evaluation (see table 2), and .86/.86 for the global score.

Validity against a criterion
The criterion validity study was carried out with two types of criteria leading to two types of analysis: a postdictive and a predictive analysis, according to the criterion variable were determined before or after the predictor variables.Grade Point Average (GPA), Regulation of Cognition and university performance were assessed at different moments.The GPA was determined before admission to university from the results obtained in high school, the Regulation of Cognition was assessed through a questionnaire two months after the beginning of classes at university and the university performance at the end of the first year and is the average of the curricular units made with success during the first year.
In the postdictive analysis was used as criterion, the GPA that is an average obtained at the end of high school, that gives access to the public university education.The correlations are not statistically significant (Table 3) and quite low (r ≤ .10),which allows us to identify a non-association between GPA and the self-perception of planning, strategies use, and monitoring/evaluation.
In order to solve discrepancies in the grades of curricular units from the two universities, the variable of academic success for the study of predictive validity were standardized for each university.All the correlation coefficients were statistical significant for the three metacognitive dimensions (Planning: r = .24,p < .001;Strategies: r = .22,p < .001;Monitoring/Evaluation: r = .25,p < .001),as well as for global (r = .28,p < .001).The predictive capacity of the three dimensions is quite similar.The percentage of explained variance is less than 10%.However, the fact that the dimensions of planning, strategies, and monitoring/evaluation presented significant correlations, evidences the importance of these metacognitive dimensions on university academic performance.When the regulation of cognition questionnaire was applied, the participants were also asked about the number of hours dedicated to study, as well as the number of absences in classroom.The same question was asked relative to number of hours of study estimated at high school and concerning the two months of classes at University.All the correlations between regulation of cognition and the number of hours students dedicate to study at University are significant.On the other hand, we didn't find significant correlations be-tween Regulation of Cognition dimensions and number of absences to classes.The results of students' self-perception of their planning in relation to learning and the use of strategies in learning are concordant with the investment made in the learning process given by the number of hours of study.The average number of hours of study reported by respondents increased from 8.78h/week to 9.16h/week, from their estimate for high school to the first two months in university education, respectively.The involvement of students in the anales de psicología / annals of psychology, 2020, vol.36, nº 2 (may) University assessed through the hours of study increased both quantitatively and qualitatively, since the variable hours of weekly study after two months of entering the University is correlated with the three metacognitive indicators, and the same is not true of the variable hours of weekly study pertaining to secondary education.This alteration of the students' involvement in hours of study may be a consequence of the need for self-regulation face to an increasing of Higher Education demands.

Discussion and Conclusions
The self-regulation learning refers to the degree to which students are metacognitively, motivationally, and behaviorally active participants in their own learning process (Zimmerman & Schunk, 2011).Three phases are considered to describe self-regulation process: (i) planning before to carry out the tasks, (ii) the choice of strategies in learning process or tasks execution, and (iii) assessment of learning process and outcomes.This awareness of these processes could motivate students, as well as helping them to self-regulate and to introduce corrective actions in subsequent processes in order to achieve better performance in the learning process and to feel self-efficacy in learning control (Efklides, et al., 2018;Veenman, 2015;Zimmerman, 2002).Those competencies are particularly relevant in Higher Education where it´s assumed a more active students' participation in teaching-learning process and related activities (De Backer et al., 2015).
In this paper, the main objective was to adapt and validate a short version of the Metacognitive Awareness Inventory (Schraw & Dennison, 1994) for Portuguese first-year university students.Several challenges are present in the students' transition and adjustment to University, namely in terms of learning and academic achievement.In order to support students in this transition, it would be important to know their study habits and learning strategies.A short version the inventory facilitates its inclusion in an assessment protocol concerning students´ college adaptation because other personal and contextual variables must be included.
In our study, only the items for regulation of cognition assumed as more relevant for academic purposes have been considered.After testing several models of its internal dimensionality, a reduction to three dimensions and 18 items was assumed.Confirmatory factor analysis suggests best-fit indices for a multidimensional model with three primary factors like in other studies (Teo & Lee, 2012).These three factors replicate the three phases of self-regulation described by Zimmerman and Schunk (2011): planning (4 items), management strategies (7 items), and monitoring or evaluation (7 items).Also, a hierarchical model comprising a second-order factor showed an acceptable fit to the data.The secondorder factor can be defined as cognitive regulation dimension.Internal consistency coefficients (Cronbach's α/McDonald's ω), ranging from .68/.70 to .70/.86, for the three subscales what can be adequate considering the number of items per dimension and the type of scale (Evers et al., 2013).Several indicators of external validity are also obtained, namely significant correlations (even low in magnitude) with academic achievement at the end of first academic year, suggesting the relevance of metacognition processes in learning and academic achievement (Öz, 2016;Schraw & Dennison, 1994).Reliability and validity coefficients allow to use this short version as a screening instrument to identify students failing to have good strategies to be more autonomous and active in their learning processes, as a condition to improve their achievement.
As academic failure and dropout in first-year university students are related (Casanova, Cervero, Núñez, Almeida, & Bernardo, 2018;Ferrão & Almeida, 2019), the institution and staff must pay attention to the study habits and learning strategies students present, in order to implement early intervention plans.This screening scale allows students to be aware of their own behaviors of planning before to carry out the tasks, the choice and monitoring of strategies during learning process, and assessment attitudes of their learning process and learning outcomes.This awareness surely it´s important to motivate and to help students to become more perseverant, self-regulated and successful students, as mentioned in recent literature (Ferradás et al., 2018;Soto et al., 2019;Stoten, 2019;Teng, 2017).
Even this short version can be used in larger projects simultaneously with other instruments to describe students´ adaptation to Higher Education; new studies must be conducted to reinforce its psychometric proprieties and utilization in academic decisions.Large and more representative samples are required, as well as it´s decisive to complement the inventory information with effective daily learning behaviors (participation in class, interaction with teachers, work in projects or group works, etc.) and comprehensive students' interviews.These complementary measures are important considering MAI is a self-report inventory, usually with some level of biases such as acquiescence and social desirability (Harrison & Vallin, 2017).Finally, it´s relevant to consider the possibility to increase the number of items in first dimension in order an equilibrium of the three subscales because the three phases in self-regulation or the cognitive and motivational processes to describe the three dimensions of cognitive regulation are similar in terms of importance.

Figure 1 .
Figure 1.Hierarchical model from CFA with modifications suggested (standardized parameters reported).
As a consequence, three indicators and a global score as an overall regulation of cognition measure were created.The three indicators corresponding to the three components anales de psicología / annals of psychology, 2020, vol.36, nº 2 (may) identified in the factorial structure are Planning, Strategies (Information Management Strategies and Debugging Strategies), and Monitoring/Evaluation.Each of the indicators, as well the global score, are the unweighted mean of the scores.

Table 1 .
Confirmatory Factor Analysis adjustment indexes for original the models tested.

Table 2 .
Descriptive statistics, reliability measures and correlation matrix between the dimensions of the Regulation of Cognition Scale.
Note. k -number of items; α = Cronbach's Alpha.a Mean of part-whole corrected item-total correlations.ω = McDonald's ω.All correlation values are statistically significant at p < .001.

Table 3 .
Correlations coefficients for criterion-related validation study.