COMPLEX LEARNING: THE ROLE OF KNOWLEDGE, INTELLIGENCE, MOTIVATION AND LEARNING STRATEGIES

Juan L. Castejón, Raquel Gilar and Antonio M. Pérez

University of Alicante

This work presents the main theories and models formulated with the purpose of offering a global overview on the acquisition of knowledge and skills involved in the initial development of expert competence. Setting from this background, we developed an empirical work whose main purpose is to define those factors in a complex learning situation such as chapter-sized in a knowledge-rich domain. The results obtained in a sample of Master students reveal that the several variables intervening, such as the qualitative organization of knowledge, intellectual ability, motivation, the deliberate use of strategies, and a rich learning environment, contribute in an independent way to provide an explanation for the acquired knowledge.

Aprendizaje complejo: el papel del conocimiento, la inteligencia, motivación y estrategias de aprendizaje. Este trabajo presenta las principales teorías y modelos formulados con el propósito de ofrecer una perspectiva global sobre los factores que influyen en la adquisición del conocimiento y las habilidades que forman parte del desarrollo inicial de la competencia experta. A partir de este contexto teórico se desarrolla un trabajo empírico cuyo principal objetivo es el de definir los factores implicados en el aprendizaje de amplia información significativa en un dominio de conocimiento. Los resultados, obtenidos en una muestra de estudiantes universitarios, ponen de manifiesto que la habilidad de organización del conocimiento, la capacidad intelectual, la motivación, el uso deliberado de estrategias y un ambiente rico de aprendizaje, contribuyen de forma independiente a la explicación del conocimiento adquirido.

From the perspective of the traditional theories of expert competence (Ericsson & Lehmann, 1996; Ericsson, 2005), the key elements that take part in the acquisition of competence are the deliberate practice of the task and the knowledge and memory skills that are developed in direct relation to the former.

According to Ericsson, Krampe & Tesch-Römer (1993), the acquisition of expert performance is a factor of the amount and organization of domain knowledge acquired as a result of experience and practice in that domain, as well as the acquired short-term working memory abilities that enable individuals to circumvent their general processing abilities. In any case, both the structure of knowledge and memory abilities are only, or at least almost exclusively, acquired as a result of experience and practice.

Sternberg (1998), however, disagrees with the hypothesis that deliberate practice is the exclusive aspect in the acquisition of expert performance, basing his criticism on both theoretical and methodological aspects. Sternberg (1998, 1999a) proposes a model of expertise development in which several factors take part and interact. The elements of the model formulated by Sternberg (1998, 1999a) are five: the metacognitive skills, the learning skills, the thinking skills, knowledge and motivation.

The role of knowledge in the explanation of expert competence, and its relation to other components such as cognitive and thinking skills, has been object of theoretical controversy in the theories and models of expertise. Sternberg (1994) disputes the point of view of some expertise theorists for whom the critical factor in the development of expert performance is the way in which knowledge is integrated and differentiated in the individual’s structure of knowledge. This perspective considers that greater organization of knowledge in memory influences or causes learning and reasoning, as is also maintained by many researchers of development and learning (Castejón, Prieto, Pérez, & Gilar, 2004; Glaser, 1984).

The underlying questions are: 1) Does there exist an ability to organize knowledge that depends on general cognitive abilities, including aspects of intelligence, 2) or is such an ability independent of general abilities, or 3) does it act in conjunction with them? This last case would best reflect the interactive hypothesis: Part of the ability to organize knowledge would depend on intelligence and part would make a unique contribution to the explanation of expert competence.

The few empirical works that have directly dealt with this question (Minnaert & Janssen, 1996; Thompson & Zamboanga, 2004; Veenman, Elshout, & Meijer, 1997; Veenman & Beishuizen, 2004) have shown that the degree of previous knowledge and work methods used contribute to explaining complex learning that is independent of general intellectual abilities.

Therefore, it is necessary to carry out empirical studies that have as an objective to check the different theoretical hypotheses on the development of expert competence. These works should have a number of characteristics:

a) To include all or most of the factors implied in the development of expert competence, from knowledge organization and practice (Castejón & Gilar, 2006; Ericsson, 1999; Ericsson, 2005; Ericsson et al., 1993), to the components of Sternberg’s model (1999b).

b) To be focused in the initial stages of expert competence development, which normally begins with specialization at the academic level of university or professional training (Sternberg, 1998; Jackson & Ward, 2004), since most of these works, has been performed in superior or inferior levels of specialization.

c) To collect the new conceptualizations and operative definitions of general intellectual abilities (Sternberg, Castejón, Prieto, Hautamäki, & Grigorenko, 2001).

d) To increase the number of participants, and to apply them to areas of content different from those requiring logical-mathematical reasoning.

e) To extend the empirical studies on acquisition of knowledge involved in the development of expertise to environments much more significant and realistic than the solution of problems in the laboratory.

f) To define the characteristics of deliberate practice in real learning environments and its effects on the acquisition of knowledge, such as the learning of larger pieces of significant information.

Method

Participants

The participants in this study were 70 first-year students of Master on School Psychology at Alicante University, Spain; approximately 60% males and 40% females. Participants comprised nearly all students regularly attending classes. All students have received a university diploma and they possessed previous general knowledge of psychology and/or education. Furthermore, they underwent a selection process for this program of study.

Instruments and variables

This work involved the use of a variety of materials and instruments, some of which were developed during the course of the study.

Instructional material. This material consists of a didactic unit whose content refers to the psychology of learning. The unit’s content is part of a manual titled Introduction to Instructional Psychology (Castejón, 2001).

Tests for the evaluation of psychological characteristics. The STAT (Sternberg Triarchic Abilities Test), Level H, was utilized to evaluate the three aspects of triarchic intelligence—analytical, creative and practical intelligence—in three domains of content: verbal, numerical, and figurative (Sternberg, 1991, 1993). Level H is appropriate for higher secondary education and university students. The test consists of 36 items divided into nine scales of four items each, which are in turn grouped into the three categories: analytical, practical, and creative intelligence. Preliminary validation of the Level-H STAT (Sternberg, 2003; Sternberg et al., 2001; Sternberg, Prieto, & Castejón 2000) has shown that it is suitable for the purpose for which it was designed.

The instrument used for the evaluation of motivation was the Motivation and Anxiety Performance Questionnaire (MAE) (Pelechano, 1973), based on research in motivating performance within the neo-behaviorist paradigm. The MAE contains 72 items, grouped into six factors, of which we used four: Tendency toward Work Overload (M1); Indifference toward Work and the Capacity to Separate Private Life from Work (M2); Self-Demand at Work/Study, the degree to which the subject expects high standards from him or herself with regard to his or her job (M3); and Positive Motivation toward the Execution of the Action (M4).

The evaluation of learning strategies using an inventory was carried out using the Study Process Questionnaire (SPQ), originally devised by Biggs (1987) with samples of university students. The version of the questionnaire used is the adaptation to Spanish population made by Hernández (1996).

Evaluation of teaching and learning styles was done by means of an inventory designed during the research. The fundamentals of the Questionnaire of Teaching–Learning Styles (ESTIEA) lie in the theories on the development of expert performance (Ericsson, 1998; Ericsson et al., 1993; Ericsson & Lehmann, 1996; Goldman, Petrosino, & CTGV, 1999) in instructional and professional environments. The original questionnaire consists of 25 items encompassing five theoretically different aspects of the teaching and learning process (five items each): independent work, group discussion, teacher explanation, cooperative work, and practicals. Participants reply to each statement on a Likert-type scale with 5 reply gradations, ranging from not at all in agreement to totally in agreement. Contrary to expectations, the results of the questionnaire’s structural validation showed the existence of a single factor that explains the 80% variance, known as «global perception of the learning environment,» it reflects participants’ preference for the use of a different combination of methodological approaches in the teaching–learning process. The scale’s reliability, established by means of Carmines’s  (theta) coefficient (Carmines & Zeller, 1979), was .95.

Instruments for the evaluation of practice, study strategies, and independent learning. To evaluate practice, study strategies, and independent learning, students kept a specially designed diary. The construction of the diary, its analysis, and valuation follow the theoretical bases for the attainment of valid verbal reports (Ericsson & Simon, 1998; Ericsson et al., 1993). There has been considerable use of verbal protocols to analyse the strategies employed throughout the learning processes (Fleck & Weisberg, 2004). The diary consisted of 15 pages, one for each day of the week from the time the explanation of the subject was completed up until the day prior to examining the students. Each page determining three major types of activities: general «every day activities», such as leisure, sport; «activities related to the task of learning», such as studying in the library or group work, and «activities aimed specifically at the learning of the task» such as going over notes, making diagrams. For each activity, the place where it was carried out and the time spent on it were recorded.

The systematization of the data collected in the diary was done through a successive process of inductive categorization (Miles & Huberman, 1994). Based on these categories, three measurements were derived related to the activities of study and learning: the use of strategies, (UESTRATE) defined as the total number of different strategies used, with regard to the 24 possible previously identified strategies; the frequency of the use of strategies (FESTRATE), established as the sum of the number of times any of the strategies was used with regard to the 24 strategies used; and the time of utilization of strategies (ESTRAVA), calculated by multiplying the time used for study, converted to a nine-point scale, by the frequency of the use of the strategies. A measurement of time was also considered, that of total study time (TIEMPOES).

Instrument for the evaluation of cognitive structures. The evaluation of the structure of knowledge was done via Pathfinder (Kraiger, Salas, & Cannon-Bowers, 1995; Schvaneveldt, 1990).

In our study, students were presented on two occasions with a matrix of relationships between 20 central concepts concerning to the psychology of learning, before and after the instruction and the study phase. The participants’ task was to indicate the degree of relationship between each pair of concepts, on a 5-point scale. Two indices were calculated for each participant—coherence and similarity with the expert— before and after the learning task. The index of similarity required a referential structure with which to compare the students’ rankings. This expert structure was provided by two members of the research team (Van de Wiel, Boshuizen, & Schmidt, 2000). We used the computer program PCKNOT (Knowledge Network Organizing Tool) for PCs, Version 4.3, published in 1999 by Interlink, Inc. to calculate both indices.

The change in the index of coherence from the beginning to the end of teaching was not statistically significant (t₍₆₉₎= 1.84, p= .46). In contrast, for the index of similarity with the instructor from the beginning to the end of instruction, a highly significant change was recorded (t₍₆₉₎= -9.64, p= .000) in the expected direction. At the same time, neither the index of coherence nor that of similarity showed significant correlations with performance before the teaching–learning process (r= -.00 and r= .12), whereas significant relationships with performance after (r= .38 and r= .56 respectively) were recorded. These results indicate the validity of these indices, especially the index of similarity, as a measurement of conceptual organization.

Evaluation of final performance. The evaluation of the learning of each participant was done by an objective performance test. The items consisted of 20 statements with four alternative replies. The reliability of internal consistency was 0.70.

Procedure

All stages were carried out during the students’ practical lessons, from October to December. We began by administering the intellectual abilities test, the STAT, and the general motivation test, MAE, having provided participants with a general reason for the research. The administration of both tests took place before the development of the instructional phase. The instructional program was developed in November. Prior to the start of teacher lectures, the concept evaluation task was administered. Then, in four sessions that took place during the next two weeks, the teacher presented the material to be learned. The instructional strategy that followed was a blend of lecture, class discussion, and independent learning work.

In the session prior to the start of the instructional stage, detailed verbal and written instructions were given to the students for filling in the activities diary. During the development of the instruction process, and having completed 50 percent thereof, the Study Processes Questionnaire was administered. Coinciding with the end of the instructional phase, the questionnaire on teaching-learning styles (ESTIEA) was administered.

At the end of the instructional phase, study participants completed the knowledge evaluation test, which assessed students’ comprehension of the acquired knowledge, followed by the task of evaluating concepts, this time at post-test stage.

Design and data analysis

The proposed goals and procedure require the use of a correlational and predictive design in which different multiple correlation and regression analysis techniques are used, such as stepwise multiple regression and hierarchic regression. In the data analysis SPSS-Version 12 is used.

Results

Correlational analysis

The results of the Pearson linear correlation coefficients between the variables are presented in table 1. Our analyses indicated that the three measurements of intelligence show relatively low correlations with each other. Practical intelligence is the only one out of the three aspects of intelligence evaluated which shows significant correlations with the rest of the variables. The correlation between practical intelligence and the measurement of conceptual coherence (r= .32) and between practical intelligence and the final performance (r= .40).

It is worth highlighting that none of the six strategic variables, evaluated through inventory, shows any correlation with performance.

Likewise, neither is there any significant correlation between any of the variables with regard to the use of strategies and the strategies obtained through the analysis of the diaries. As for the motivational variables, a significant correlation takes place between the tendency to work overload and the positive motivation towards action (r= . 37). The only motivational variable with a significant relationship with performance is the degree of expectation of high professional standards within persons (r= .34).

The variable regarding students’ preferences for teaching styles has significant relationships with the final performance achieved (r= .35).

Of the main variables derived from the diary data, only study time shows significant correlations with the frequency of use of strategies (r= .55) and with the time strategies are employed (r= .84). The total time dedicated to study does not, however, appear to be related to performance. It is the use of strategies variable that shows a significant correlation (r= .31) with learning at the end of the process. It is therefore a matter of the variety of strategies used rather than the frequency or time dedicated to them that appears to be related to final performance.

The variables which keep significant correlations with the performance measures of knowledge acquisition at the end of the instructional process are practical intelligence, coherence and similarity measures taken at the end of the learning process, the motivational variable of self-demand at work/study, and the preference for a rich and broad global environment of learning.

Multiple regression analysis using the stepwise method

Table 2 presents the results of the stepwise method used to predict the participants’ final performance. The variable making the greatest contribution to the explanation of the criteria’s variance is conceptual similarity (β= .47, p= .0000); the deliberate use of strategies (β= .18, p= .0451), the participants’ perception of the teaching-learning process (β= .27, p= .0040); the motivational variable of work/study self-demand (β= .21, p= .0279); and practical intelligence (β= .19, p= .0390) also contribute significantly to the explanation of knowledge acquisition.

Taken as a whole, the effect of the variables was statistically significant to the explanation of the final performance and they explains sixty per cent variance in the criterion. The powerfulness of the statistic sample is 99%, for a value of R²= .60 and a number of participants of N= 70, taking into account a level alpha of significance of 0.01.

Furthermore, our data satisfy the requirements of normality, linearity and homogeneity of the variance, and independence of errors; the assumptions of the multiple regression analysis. Therefore, those variables that acquire the greatest predictive relevance are related to the organization of knowledge, the deliberate use of strategies, the perception of a varied and rich learning environment, the subject’s work/learning self-demand, and, to a lesser extent, practical intelligence.

Hierarchical regression analysis

In this type of analysis, the variables which are entered first act as covariant of those which are subsequently included. In this way, the effect of the first on the second may be partialized. Thus, an overestimation of the predictive value of the variables entering the equation first is obtained, which is why the decision to initially introduce some variables or others, was taken on a theoretical basis (Castejón & Navas, 1992; Cohen & Cohen, 1983) .

Table 3 shows the results of the hierarchic regression analysis into which there have been successively introduced the variables which were selected with the step by step method.

Firstly, practical intelligence is entered into the equation. The contribution of this variable is highly significant, (F= 11.08, p= .0015), and it’s obtained a multiple correlation coefficient of R= .40. The inclusion of the second variable, motivation, supposes a significant change in the explained variance (F= 7.41, p= .01). When the third step includes variables relating to deliberate use of strategies, does not presuppose a significant increase in the explained variance (F= 2.66, p= .10). With the inclusion of the conceptual similarity variable, in a forth step, a high significant change in the explained variance is produced (F= 28.17, p= .001). When the last step includes variable relating to perception of global learning environment, a significant increase of the explained variance is in fact obtained (F= 9.07, p= .004).

In short, the results of the hierarchical regression analysis show that the variables which contribute most significantly to the explanation of learning processes, are the intellectual abilities, motivation, conceptual organization and the student’s perception of teaching-learning process. However, the variable related to the deliberate use of strategies does not make a significant contribution.

Discussion

The variables that show a significant relationship with or contribute in a significant way to explain the knowledge and skills acquired during the teaching–learning process of a significant, complex material, undertaken in a real educational situation, revolve around the aspects of intellectual abilities –practical intelligence in this case– the organization of knowledge, motivation, the deliberated use of learning strategies, and the perception of a rich teaching context.

The results concerning intellectual abilities demonstrate that the different dimensions of intelligence evaluated show low inter-correlations as well as different relationships with the other variables, something that is consistent with other results obtained (Sternberg, 2003; Sternberg et al., 2001) regarding the independence of the three aspects of triarchic intelligence.

The quality of the organization of knowledge has the greatest influence upon the acquisition of knowledge and abilities. The results of the experimental validation of the evaluation procedure of cognitive structures show the consistency of these measurements.

Of special interest is the relation between knowledge organization and the general intellectual ability understood as intelligence in the traditional psychometric sense. The results of our work clearly show that both intelligence and knowledge organization make a contribution to the explanation of acquired knowledge and skills. These results are coincident with the results obtained in some studies that have considered the question of the independence of knowledge and general intellectual ability. Minnaert & Janssen (1996) found out that the specific prior knowledge of a domain had more effect to explain the acquired final knowledge in a subject of a university course, than general intellectual abilities, which also exerted a significant effect on learning skills.

The deliberate use of strategies during the study is another variable that is related with the acquisition of knowledge and the final performance. First, none of the factors in the Study Process Questionnaire show a significant correlation with performance. However, the use of strategies, identified through the analysis of the diary, was directly related to acquired knowledge in both the correlational and regression analyses. This fact does not appear to be circumstantial as suggested by Veenman, Elshout & Meijer (1997) because the measurements and strategies obtained by inventories and verbal protocols are not convergent (Núñez et al., 2006).

Motivation is another factor that influences the results of knowledge and skills acquisition. Motivation is the motor that drives the commitment to deliberate practice (Ericsson et al., 1993; Ericsson & Lehmann, 1996) and the necessary element to initially activate those factors intervening in the acquisition of knowledge and abilities (Sternberg, 1998; 1999a; Valle, Cabanach, Rodríguez, Núñez, & González-Pienda, 2006). Motivation is a complex mechanism in which biological and cognitive factors take part (Covington, 2000), determining the general drive toward activity, achievement, or a feeling of self-efficacy. Perhaps because of this it has been suggested that different types of motivation take part in the acquisition of expert performance (Sternberg, 1999a).

Our results indicate that the motivational aspect systematically related to performance is factor of self-demand in work and study; a motivational aspect linked more to the general drive toward activity and execution than with cognitive aspects of motivation. In fact, this motivational aspect has been regarded as the most important in the acquisition of expert performance (Ericsson & Lehmann, 1996).

An outstanding element is the context in which competence develops. In this case, the instructional context, defined by the learning environment, appears in our study to be systematically related with the learning results, regardless of other elements like intellectual ability or motivation. The preference for a rich and varied learning environment is related positively to the acquisition of knowledge and skills. The instructional implication for the development of expert performance is clear, rich and varied learning environments stimulate competence (Beier & Ackerman, 2005; De Corte, 2000; Ericsson, 1998; Goldman et al., 1999).

All in all, in our work it has been identified a set of variables which are directly implied in the acquisition of knowledge and cognitive skills which are part of the initial development of expert competence, in agreement with the synthetic theory of developing expertise (Sternberg, 1999b). The extension of this work to other participants, other contents, and other instructional context can be of use to consolidate the results obtained and to circumvent its limitations.

Beier, M.E. & Ackerman, P.L. (2005). Age, ability and the role of prior knowledge on the acquisition of new domain knowledge: promising results in a real-world learning environment. Psychology and Aging, 20(2), 341-355.

Biggs, J.B. (1987). Study Process Questionnaire (SPQ). Hawthorn, Victoria: Australian Council for Educational Research.

Carmines, E.G. & Zeller, R.A. (1979). Reliability and validity assessment. London: Sage Publ.

Castejón, J.L. (2001). Introducción a la psicología de la instrucción [Introduction to Instructional Psychology]. Alicante: Ediciones Club Universitario.

Castejón, J.L. & Gilar, R. (2006). Factors involved in complex learning: A model of expert competence acquisition. In S.N. Hogan (ed.): New Developments in Learning Research (pp. 1-34). Hauppauge, NY: Nova Science Publishers.

Castejón, J.L. & Navas, L. (1992). Determinantes del rendimiento académico en la enseñanza secundaria. Un modelo causal [Determinants of academic achievement in Compulsory education. A Causal Model]. Análisis y Modificación de Conducta, 18(61), 697-730.

Castejón, J.L., Prieto, M.D., Pérez, A.M., & Gilar, R. (2004). El rol del conocimiento y de las habilidades intelectuales generales en la adquisición del aprendizaje complejo [The role of knowledge and general intellectual abilities in complex learning acquisition]. Psicothema, 16(4), 600-605.

Cohen, J. & Cohen, P. (1983). Applied multiple regression correlation analysis for the behavior sciences. Hillsdale, NJ: LEA.

Covington, M.V. (2000). Goal theory, motivation, and school achievement: An integrative review. Annual Review of Psychology, 51, 171-200.

De Corte, E. (2000). Marrying theory building and the improvement of school practice: a permanent challenge for instructional psychology. Learning and Instruction, 10(3), 249-266.

Ericsson, K.A. (1998). The scientific study of expert performance: General implications for optimal learning and creativity. High Ability Studies, 9, 75-100.

Ericsson, K.A. (1999). Creative expertise as superior reproductible performance: Innovative and flexible aspects of expert performance. Psychological Inquiry, 10(4), 329-361.

Ericsson, K.A. (2005). Superior decision making as an integral quality of expert performance: Insights into the mediating mechanisms and their acquisition through deliberate practice. In R. Lipshitz & H. Montgomery (eds.): How professionals make decisions (pp. 135-167). Nahwah: Laurence Erlbaum Associates.

Ericsson, K.A., Krampe, R.T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363-406.

Ericsson, K.A. & Lehmann, A.C. (1996). Expert and exceptional performance: Evidence on maximal adaptations on task constraints. Annual Review of Psychology, 47, 273-305.

Ericsson, K.A. & Simon, H.A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture and Activity, 5(3), 178-186.

Fleck, J. & Weisberg, R. (2004). The use of verbal protocols as data: An analysis of insight in the candle problem. Memory and Cognition, 32(6), 990-1006.

Glaser, R. (1984). Education and thinking. The role of knowledge. American Psychologist, 39(2), 93-104.

Goldman, S.R., Petrosino, A.J., & Cognition and Technology Group at Vanderbilt (1999). Design principles for instruction in content domains: Lessons from research on expertise and learning. In F.T. Durso, R.S. Nickerson, R.W. Schvaneveldt, S.T. Dumais, D.S. Lindsay & M.T.H. Chi (eds.): Handbook of Applied Cognition (pp. 595-627). New York: John Wiley & Sons.

Jackson & Ward (2004). A fresh perspective on progress files: A way of representing complex learning and achievement in higher education. Assessment and Evaluation in Higher Education, 29(4), 423-449.

Hernández, F. (1996). Cuestionario de Procesos de Estudio [Study Process Questionnaire]. Murcia: Department of Methods of Investigation and Diagnostic in Education, University of Murcia, Spain.

Kraiger, K., Salas, E., & Cannon-Bowers, J.A. (1995). Measuring knowledge organization as a method for assesssing learning during training. Human Factors, 37, 804-816.

Miles, M. & Huberman, M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage Publications.

Minnaert, A. & Janssen, P.J. (1996). How general are the effects of domain-specific prior knowledge on study expertise as compared to general thinking skills? In M. Birenbaum & F. Dochy (eds.): Alternatives in assessment of achievements, learning processes and prior knowledge (pp. 265-282). London: Kluwer Academic Publishers.

Núñez, J.C., Solano, P., González-Pienda, J.A., & Rosario, P. (2006). Self-regulation processes measurement through self-report methodology. Psicothema, 18(3), 353-358.

Pelechano, V. (1973). Manual del cuestionario MAE [Manual of MAE Questionnaire]. Madrid: Fraser.

Schvaneveldt, R.W. (1990). Pathfinder associative networks. Studies in knowledge organization. Norwood, NJ: Ablex Publishing Co.

Schvaneveldt, R.W. (1999). PCKNOT 4.3 User’s manual. Mesa, AZ: Interlink Inc.

Sternberg, R.J. (1991). Theory-based testing of intellectual abilities: Rationale for the triarchic abilities test. In H.A. Rowe (ed.): Intelligence: reconceptualization and measurement (pp. 183-202). Hillsdale, NJ: LEA.

Sternberg, R.J. (1993). Sternberg Triarchic Abilities Test (Level H). Not published Test Manual.

Sternberg, R.J. (1994). Cognitive conceptions of expertise. International Journal of Expert Systems, 7(1), 1-12.

Sternberg, R.J. (1998). Metacognition, abilities, and developing expertise: What makes an expert student? Instructional Science, 26(1-2), 127-140.

Sternberg, R.J. (1999,a). Intelligence as developing expertise. Contemporary Educational Psychology, 24, 359-375.

Sternberg, R.J. (1999,b). Ability and expertise. It’s time to replace the current model of intelligence. American Educator, Spring, 10-13 & 50-51.

Sternberg, R.J. (2003). Construct validity of the theory of special intelligence. In R.J. Sternberg, J. Lautrey, & T. Lubart (eds.): Models of intelligence: International perspectives (pp. 55-77). Washington, DC: American Psychological Association.

Sternberg, R.J., Castejón, J.L., Prieto, M.D., Hautamäki, J., & Grigorenko, E. (2001). Confirmatory factor analysis of the Sternberg Triarchic Abilities Test (Multiple Choice Items) in Three International Samples: An empirical test of the Triarchic Theory. European Journal of Psychological Assessment, 17, 1-16.

Sternberg, R.J., Prieto, M.D., & Castejón, J.L. (2000). Análisis factorial confirmatorio del Sternberg Triarchic Abilities Test (nivel H) en una muestra española: resultados preliminares [Confirmatory factorial analysis of Sternberg Triarchic Abilities Test (level H) in a Spanish sample: preliminary results]. Psicothema, 12(4), 642-647.

Thompson, R. & Zamboanga, B. (2004).Academic aptitude and prior knowledge as predictors of student achievement in introduction to Psychology. Journal of Educational Psychology, 96(4), 778-784.

Valle, A., Cabanach, R.G., Rodríguez, S., Núñez, J.C., & González-Pienda, J.A. (2006). Academia goals, cognitive and self-regulatory strategies. Psicothema, 18(2), 165-170.

Van de Wiel, M.W.J., Boshuizen, H.P.A., & Schmidt, H.G. (2000). Knowledge restructuring in expertise development: Evidence from pathophysiological representations of clinical cases by students and physicians. European Journal of Cognitive Psychology, 12(3), 323-355.

Veeman, M., Elshout, J.J., & Meijer, J. (1997). The generality vs domain-specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7(2), 187-209.

Veenman, M. & Beishuizen, J. (2004). Intellectual and metacognitive skills of novices while studying text under conditions of text difficulty and time constraint. Learning and Instruction, 14(6), 621-640.