ABILITY-VERSUS SKILL-BASED ASSESSMENT OF EMOTIONAL INTELLIGENCE

Travis R. Bradberry and Lac D. Su

TalentSmart®

Emotional intelligence has received an intense amount of attention in leadership circles during the last decade and continuing debate exists concerning the best method for measuring this construct. This study analyzed leader emotional intelligence scores, measured via skill and ability methodologies, against leader job performance. Two hundred twelve employees from three organizations participated in this study. Scores on the Emotional Intelligence Appraisal^™, a skill-based assessment, were positively, though not significantly, correlated with scores on the MSCEIT, an ability-based assessment of emotional intelligence. Scores on the MSCEIT did not have a significant relationship with job performance in this study, whereas, scores on the Emotional Intelligence Appraisal^™ had a strong link to leader job performance. The four subcomponents of the Emotional Intelligence Appraisal^™ were examined against job performance. Relationship management was a stronger predictor of leader job performance than the other three subcomponents. Social awareness was the single emotional intelligence skill that did not have a significant link to leader job performance. Factor analyses yielded a two-component model of emotional intelligence encompassing personal and social competence, rather than confirmation of a four-part taxonomy.

Habilidad frente a destreza en la medición de la inteligencia emocional. Durante la última década, la inteligencia emocional ha recibido gran atención en los círculos de liderazgo, y este debate continúa acerca del mejor método para medir este constructo. Este estudio analizó las puntuaciones en inteligencia emocional de líderes, medidas a través de metodologías basadas en la destreza y en la habilidad, en relación con el rendimiento laboral de estos líderes. Doscientos doce empleados procedentes de tres organizaciones participaron en el estudio. Las puntuaciones en la Emotional Intelligence Appraisal™, una medida basada en la destreza, correlacionaron positiva, aunque no significativamente, con las puntuaciones en el MSCEIT, una medida de la inteligencia emocional basada en la habilidad. En este estudio, las puntuaciones en el MSCEIT no correlacionaron de forma significativa con el rendimiento en el trabajo de los líderes, mientras que las puntuaciones en Emotional Intelligence Appraisal™ sí correlacionaron. Los cuatro componentes de la Emotional Intelligence Appraisal™ fueron analizados respecto al rendimiento laboral. El manejo de las relaciones fue el predictor más fuerte de los cuatro componentes del rendimiento laboral del líder. La conciencia social fue la única destreza de inteligencia emocional que no mostró relación con el rendimiento laboral de los líderes. Más que confirmar la taxonomía de cuatro componentes, el análisis factorial reveló un modelo de inteligencia emocional de dos componentes: competencia personal y competencia social.

A matter of perception

As organizations institute leadership development programs, they rely upon feedback from co-workers to understand how their leaders are behaving in the workplace. As might be expected, employees’ self-ratings tend to differ greatly from ratings given to them by their co-workers, supervisors, and subordinates (Conway & Huffcutt, 1997; Harris & Schaubroeck, 1988). Sala (2001a) measured emotional intelligence with a 360-degree feedback instrument, the Emotional Competence Inventory (ECI) and found a discrepancy in self/other ratings of emotional intelligence the higher a leader is ranked in an organization. Emotional intelligence scores for leaders in the study were based upon aggregate scores from the ECI and suggest a division between individuals in positions of power in organizations (leaders) and those they work with. The study did not compare rating discrepancies to performance but discovered that the higher a leader’s position, the more the leader tended to rate his or her behavior inaccurately.

Management development frequently focuses on key skills, knowledge, and abilities that are considered to be fundamental to effective leadership behavior or high-performing individuals. The definition and acknowledgement of important leader behaviors allows these behaviors to be assessed and fed back to the individual. This assessment of key traits allows individuals to increase their understanding of areas of weakness and strength, which in turn can drive the development of improved leadership behavior (Atwater & Yammarino, 1992).

Multiple studies have shown that ratings received from co-workers, whether supervisors, peers, or subordinates, are much better predictors of performance than self-ratings (Brutus, Fleenor, & Taylor, 1996; Edwards, 1993; Salam, Cox, & Sims, 1997). A 360-degree feedback received from peers, subordinates, and supervisors tend to correlate higher than self-report ratings (Atwater et al., 1998; Furnham & Stringfield, 1998; Harris & Schaurbroeck, 1988). The daily contact a leader has with individuals they work with allows them to provide the perspective that is lacking from a person rating him/herself.

Most organizations invest considerable resources into skill development for individuals at the management level and above. This process is aimed at improving both the individual’s and the company’s performance. The return on this investment is notable and has been observed at more than $8,000 per employee for progressive employee development strategies (Huselid, 1994). Engaging in this process of management skill development is only one piece of the puzzle. Perceiving and rating work behavior is an important aspect of skill development. The other piece of the puzzle to consider when undertaking a management-development initiative is what set of skills an organization find important for improvement. Investing in the right skills is likely to result in better performing leaders with a higher return on investment.

Emotional intelligence

The roots of emotional intelligence follow the lines of the intelligence testing movement. Thorndike (1920) acknowledged there are multiple intelligences and social intelligence, or the ability to «act wisely in human relations», is one of them (p. 229). Social intelligence was problematic from its inception because it is inherently difficult to measure. Examining humans in interactions is a much more difficult task than measuring the cognitive abilities of an individual solving a math problem.

Despite the challenges, researchers still made efforts to measure social intelligence. Thorndike and Stern (1937) reviewed these attempts and concluded social intelligence was composed of three components: attitude toward society, social knowledge, and degree of social adjustment. They also determined social intelligence was too complex to be measured and the difficulties inherent in measuring interactions with people were too large an obstacle to overcome.

The field of psychology’s emphasis on behaviorism and IQ testing resulted in social intelligence essentially being ignored during the coming decades, despite Weschler’s (1952) acknowledgment of the «affective capacities» of individuals when he developed his first intelligence test. The recognition of social intelligence received a major boost by Gardner (1983) when his highly regarded theory of intelligence was published referencing two types of personal intelligence: interpersonal and intrapersonal.

The term, emotional intelligence, was first mentioned in a doctoral dissertation nearly 20 years ago (Payne, 1985). This qualitative study proposed one can overcome deficiencies in emotional functioning and regulation by showing strength in the face of fear or desire. Three years later, another dissertation referred to the «emotional quotient,» which is the term commonly used today to refer to an individual’s emotional intelligence score (Bar-On, 1988).

Researchers John Mayer and Peter Salovey conducted research a few years later attempted to answer why some individuals were better at reading emotions than others (Mayer, DiPaolo, & Salovey, 1990). It was in this study where they first published the term «emotional intelligence». Mayer and Salovey followed with a second study shortly thereafter that proposed the first model of emotional intelligence and brought its attention to the research community (Salovey & Mayer, 1990). The article provided an overview of research in a number of previously unrelated areas and suggested findings from those areas were indicative of the presence of a new cognitive ability: emotional intelligence. In this influential article, emotional intelligence was defined as «the ability to monitor one’s own and other’s feelings and emotions, to discriminate among them, and to use this information to guide one’s thinking and action» (Salovey & Mayer, 1990, p. 189). Mayer and Salovey described emotional intelligence as a unique cognitive ability based upon emotion that is operationalized in an individual’s social environment. Subsequent editorials and studies by the authors in the early 1990s implicated the importance of emotional intelligence as a variant of standard intelligence, a component of coping mechanisms, and a key component of self-regulation (Mayer & Salovey, 1993; Mayer & Stevens, 1994; Salovey, Hsee, & Mayer 1993).

Goleman (1995) brought emotional intelligence to the mainstream public. He reviewed the work of Mayer and Salovey, presented his own similarly construed model of emotional intelligence, and forever changed the landscape of public awareness of the term. One of the areas that quickly latched onto emotional intelligence was the leadership development community. Emotional intelligence is attractive to organizations because it provides a framework from which emotionally-based soft skills can be designed and measured.

The awareness of emotional intelligence brought about by Daniel Goleman’s book also fueled much research that was subsequently published during the second half of the 1990s. One of the biggest complaints surrounding emotional intelligence upon its inception was the lack of research to support its validity. Indeed, this skepticism was well founded at the time because emotional intelligence was thrust into the public eye by Goleman’s (1995) book that was largely theoretical. Some skepticism continues today, criticizing the methodology of the flood of research during the last seven years (Barrett, 2001).

Taxonomy of emotional intelligence

While there are three predominant emotional intelligence taxonomies in widespread use today, the Goleman (2002) taxonomy offers a four-part structure which focuses on an individual’s ability to understand his or her own emotions and emotional state, to manage and regulate responses to these emotions, to recognize the emotional state of others, and to respond to the emotions present in others to interact effectively.

Although it follows the same theoretical foundation and structure of the other models, Goleman et al’s (2002) model is designed for application in organizational theory, research and practice. This framework operates under the assumption that it can be used to develop the effectiveness of individuals in the workplace and in leadership positions (Goleman, 2001).

Personal competence encompasses an individual’s capacity to manage him or herself. Therefore, it includes both self-awareness and self-management (Goleman et al., 2002). Self-awareness includes emotional self-awareness, accurate self-assessment, and self-confidence. Self-management consists of emotional self-control, transparency, adaptability, achievement, initiative and optimism.

Social competence is a factor that includes an individual’s capability to manage relationships. Social competence is composed of both social awareness and relationship management (Goleman et al., 2002). Social awareness includes empathy, organizational awareness, and service. Relationship management comprises inspirational leadership, influence, developing others, catalyzing change, managing conflict, and teamwork and collaboration.

Most models of emotional intelligence support both personal and social competencies (Bar-On, 2000; Bar-On, 2006; Goleman, et al., 2002; Mayer & Salovey, 1997; Brackett & Salovey, 2006). Thus far, only one large-scale study has sought to confirm the structure of Goleman’s model of emotional intelligence. This study looked at 596 respondents to the Emotional Competence Inventory (ECI), a multi-rater measure of emotionally intelligent behavior, in multiple industries. Findings suggested strong support for a model of emotional intelligence based upon personal and social competence (Boyatzis, Goleman, & Rhee, 2000), but the study did not support splitting the taxonomy into additional skills.

Emotional intelligence and performance

Emotional intelligence has been linked to numerous important organizational outcomes and is frequently studied as a correlate with performance (Boyatzis, 2006; Goleman, 1995, 2001, 2002). The findings linking emotional intelligence to leadership performance are highly important. However, the significance of the emotional intelligence construct is truly felt only when one considers that, unlike other predictors of success such as general intelligence, emotional intelligence can be learned. Studies conducting emotional intelligence training with university students show a marked increase in emotional intelligence measured by a pre- and post-test measure (Ashkanasy, 2001; Clark, Callister, & Wallace, 2002). This finding has also been observed with leaders in corporate settings (Sala, 2001b; Young & Dixon, 1996). A follow-up study of increases in emotional intelligence as a result of direct learning efforts has revealed the maintenance of changes as long as seven years after the intervention (Wheeler, 1999).

Emotional intelligence is an excellent correlate of job success for leaders. Sosik and Megerian (1999) found leaders high in emotional intelligence outperformed their low emotional intelligence counterparts when measured by organizational performance data. Other carefully conducted studies have correlated emotional intelligence with performance on job-related cognitive ability tasks (Graves, 1999; Lam & Kirby, 2002). Perhaps the strongest evidence to date for the utility of emotional intelligence for predicting on-the-job success for leaders comes from a study by Cavallo and Brienza (2002). This study assessed the leadership behavior of 358 leaders at Johnson and Johnson Corporation, at locations across the globe. The study found the best performers were those high in emotional intelligence as rated by their supervisors, peers, and subordinates in the Emotional Competency Inventory (ECI), a 360-degree feedback instrument based upon Goleman’s (2001) model. Emotional intelligence competencies that were the best predictors of success in this study were self-confidence, achievement orientation, initiative, leadership, influence and catalyzing change.

Hypothesis 1

Multi-rater feedback ratings of leader emotional intelligence skills will be more closely correlated with job performance than the ability-based scores. Studies analyzing 360-degree or multi-rater feedback find employees with accurate self-ratings tend to be the best performers (Atwater, Ostroff, Yammarino, & Fleenor, 1998; Atwater & Yammarino, 1992). It is an interesting finding that, given the general lack of congruence between self/other ratings, employees who provide ratings similar to those of their coworkers outperform those with incongruent ratings. It is likely that these individuals perform better because they are more cognizant of their behavior and use this self-awareness to improve their skills.

Hypothesis 2

Each dimension of emotional intelligence (self-awareness, self-management, social awareness, and relationship management), as measured by the Emotional Intelligence Appraisal^™ will explain unique variance in predicting management performance.

Hypothesis 3

Leader utilization of relationship management, as measured by the Emotional Intelligence Appraisal^™, will be more positively related to job performance than the other three emotional intelligence skills.

Relationship management is considered to be the ultimate outcome of emotional intelligence; a skill whose foundation is built upon the other three skills in the model (Goleman et al., 2002). A leader’s utilization of the relationship management skill is far more important to his or her job performance than the other three skills in the emotional intelligence model. Although relationship management is widely considered to be the most important sub component of emotional intelligence, little research has been conducted to confirm or deny this belief. Therefore, this study is an important contribution to the growing body of research in emotional intelligence.

A statistical analysis was conducted to assess the underlying factor structure of the instruments used in the study, which pertains to the unproven statistical validity of the Emotional Intelligence Appraisal^™ instrument as a measure of emotional intelligence. A confirmatory factor analysis was conducted to assess the reliability of the underlying factor structure of the Emotional Intelligence Appraisal^™ instrument.

Internal-consistency reliability coefficients were calculated for the three measures and their sub-components. To interpret the factor solution suggested from the principal component analysis, the factors were rotated using Varimax technique.

Method

Participants

Two hundred and twelve subjects participated in this study. One hundred and twenty-three (58%) were male and eighty-nine (42%) were female. Participants were recruited from three organizations: a homebuilder, a telecommunications company and an irrigation systems manufacturer. At the time of data collection, these organizations employed a total of 600 individuals in the locations that participated in the study. To meet the criteria for inclusion in the research study, employees receiving the 360º ratings had to directly supervise a minimum of three individuals and have one year of experience with the company in the same position, at the current location. To maximize continuity and ensure consistency, individuals delivering ratings to a peer, supervisor, co-worker or subordinate had to have worked directly with that person in that position for a minimum of six months.

To test the hypotheses in this study, leaders from the three organizations were rated by their co-workers on workplace behavior and performance. Leaders who participated in the sample ranged from mid-managers to senior executives. Co-workers who provided ratings included supervisors, peers, and subordinates. Participation in the study was voluntary. The organizations were offered the incentive of a free 360º feedback coaching session for the leaders in the study.

A commencement meeting reviewed the 360º process for potential participants in the study. More than 60% of subjects invited to participate in the study attended the face-to-face meetings. Subjects were informed in writing that their individual responses about a co-worker would only be shown to that person in aggregate form. Also, it was stated the survey was to be used only for skill development in the leader and research purposes. Each leader received his or her feedback in a summary report without identification of individual responses. Also, actual responses on individual leaders or individual-employee forms were not shared with the organization. However, the researcher verbally shared trends in the data with the organization, such as which skill stood out for development in the management team as a whole.

The data received from the leaders and employees participating in the survey were not anonymous to the researcher. The researcher knew who was invited to take the survey and aggregated the employee results for each leader. This process was necessary to collect data and to create the feedback reports for each leader. This prevented anonymity in the collection of the research data. However, identifying information was not considered in the statistical analysis, as the data was coded prior to being analyzed.

Co-workers rated the leader’s behavior by responding to a 28-item measure of emotional intelligence via the Emotional Intelligence Appraisal^™. In the same online session, co-workers also answered nine-items targeting the job performance of the leader to which they were providing feedback.

The leaders in the study took an ability-based test of emotional intelligence, the MSCEIT, in addition to rating themselves through the multi-rater Emotional Intelligence Appraisal^™. The ability-based emotional intelligence instrument provided comparison scores to the performance-based Emotional Intelligence Appraisal^™ scores.

Two hundred and seventy-four individuals were sent email-internet-links inviting them to take the survey. Two hundred and thirty surveys (84%) were returned to the researcher. Twelve surveys could not be used because the leader did not complete the emotional intelligence instrument. An additional six surveys from different leaders could not be used because they did not include performance data on the subject or were otherwise incomplete.

Of the 360º feedback surveys completed and used in the study, 66 (31%) were from employees rating a peer, 42 (20%) were from leaders rating a subordinate and 104 (49%) were from employees rating their leader. Each leader was rated by an average of 8.5 other individuals.

Measures

Emotional Intelligence Appraisal^™

The Emotional Intelligence Appraisal^™ is a 28-item performance-based assessment of emotional intelligence in Daniel Goleman’s four-factor taxonomy. The assessment provides an overall EQ score as well as a score in each of the four emotional intelligence factors. Research with the instrument has yielded Cronbach alpha reliability ratings ranging from .85 to .91 (Bradberry & Greaves, 2003).

The Emotional Intelligence Appraisal^™ is designed to assess behavior demonstrative of emotional intelligence skills. The four factors of the Emotional Intelligence Appraisal^™ are self-awareness, self-management, social awareness and relationship management. The items in the Emotional Intelligence Appraisal^™ were written with «behavioral impact statements» designed to measure the impact specific behaviors have on an individual’s environment and those around them.

Job Performance Measure

For the purpose of this study, job performance was assessed using a 9-item measure constructed to provide an overall index of job performance through in-role behavior (Miller & Cardy, 2000). Questions are based upon a five-point Likert-type scale measuring level of agreement with statements describing on-the-job behavior. Items in the instrument are designed to measure initiative, productivity, quality, commitment, and involvement. This measure was adapted by Miller and Cardy (2000) based upon validated scales from recent research in a large and diverse group of organizations (Becker, Billings, Eveleth, & Gilbert, 1996; Duarte, Goodson, & Klich, 1994; Freese, Kring, Soose, & Zempel, 1996). Research with the instrument has yielded Cronbach alpha reliability ratings ranging from .67 to .87 (Miller & Cardy, 2000). Co-workers were instructed to rate the leader’s performance based upon their level of agreement with each statement.

MSCEIT - Emotional Intelligence Ability Test

For the purpose of this study, the emotional intelligence abilities of leaders were measured using the only published ability-based measure of emotional intelligence, the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT). As an ability-based measure, the MSCEIT requires participants to look at pictures and rate facial expressions, respond to the emotional tone of stories, and perform other skill tasks that are based on emotion. Research on the MSCEIT has confirmed the four factor structure with branch score Cronbach alphas ranging from .59 to .87.

Results

Hypothesis one states multi-rater feedback ratings of leader emotional intelligence through the Emotional Intelligence Appraisal^™ will be more closely correlated with job performance than ability-based scores via the MSCEIT. The correlation between the overall emotional intelligence scores on the Emotional Intelligence Appraisal^™ and the MSCEIT yields a non-significant positive correlation, r (211)= .149, p<.1. A hierarchical multiple regression was performed using these two dimensions as predictors and job performance as the criterion variable. The first table below displays the model summary while the second table indicates each variable’s unique contribution.

In the overall model, the Emotional Intelligence Appraisal^™ and the MSCEIT emotional intelligence scores account for a significant amount of variance, R= .359, p£.007. The standardized regression weights for the Emotional Intelligence Appraisal^™ and MSCEIT emotional intelligence factors are, ß= 0.585, p<.003 and ß= 0.061, p= .748, respectively. The standardized regression weight for the Emotional Intelligence Appraisal^™ is significant while the MSCEIT is not.

The difference between the correlations for the Emotional Intelligence Appraisal^™ and the MSCEIT emotional intelligence scores provides a measurement of the actual difference between the scores when compared to job performance. The standardized comparison of the correlations yields a highly significant z score of 5.3, indicating a large and statistically significant difference between the values. This finding was also present in the aggregate data, with a highly significant z score above 5.0.

Hypothesis 2

Hypothesis two states each dimension of emotional intelligence (self-awareness, self-management, social awareness, and relationship management), as measured by the Emotional Intelligence Appraisal^™, will explain unique variance in predicting management performance. A hierarchical multiple regression was performed using these four dimensions as predictors and job performance as the criterion variable. The table below displays the model summary.

In the overall model, the four components of emotional intelligence account for a significant amount of variance, R2= .403, p£.001. Self-awareness, self management and relationship management are significantly correlated with job performance and explain unique variance. Social Awareness is not significantly correlated with job performance and does not explain unique variance. The standardized regression weights for the self-awareness, self management and relationship management are significant, ß= 0.197, p<.046, ß= 0.291, p<.01 and ß= 0.419, p£.001, respectively. The standardized regression weight for social awareness is not significant; ß= 0.149, p<.20.

It is important to note that the emotional intelligence model is not supported in the present study as four unique constructs. A principal component analysis suggested a two-factor solution with the two factors accounting for 62.57% of the variance in the correlation matrix. The resulting eigenvalues and associated percent of variance accounted for are shown in the table below. Catell’s scree test (Catell, 1966) provided further confirmation of the proposed two-factor structure with the plot of the eigenvalues breaking and becoming horizontal at or near two.

Hypothesis 3

Hypothesis three stated leader utilization of relationship management, as measured by the Emotional Intelligence Appraisal^™, would be more positively related to performance than the other three factors of emotional intelligence. A hierarchical multiple regression was performed using these two dimensions as predictors and job performance as the criterion variable. The following table displays each variable’s unique contribution.

The standardized regression weight for relationship management is significant, ß= 0.419, p£.001, and greater than the regression weights for the other 3 emotional intelligence subscales. However, the test for a statistically significant difference between the components and job performance is based upon their correlation to the criterion variable. The correlation between relationship management and job performance r(211)= .612, p£.001, the largest difference of the four emotional intelligence variables and job performance, was compared to the correlation between self-awareness and job performance r(211)= .519, p£.001, the smallest correlation with job performance. The resulting difference of the comparison between the standardized correlations yields a z- score of 1.4, indicating a non-significant difference between the scores. Therefore, Hypothesis 3 is not fully supported by this sample. The same correlational comparison in the aggregate sample also yielded a non-significant difference.

Discussion

The non-significant, positive correlation between the Emotional Intelligence Appraisal^™ and the MSCEIT suggests a distinction between the construct measured by these assessments. Emotional intelligence has been linked to leader performance in a variety of corporate settings (Cavallo & Brienza 2002; Graves, 1999; Lam & Kirby, 2002; Sosik & Megerian, 1999). However, few studies have considered the incremental variance of each component of emotional intelligence across companies and industries. Hypothesis two investigated the relationship between each dimension of emotional intelligence and job performance. Three dimensions of emotional intelligence (self-awareness, self-management, and relationship management) had a significant and unique impact upon the job performance of leaders in the study. One dimension, social awareness, did not.

Relationship between leader emotional intelligence and job performance

It appears for leaders in this sample, an awareness of the emotional state of others did not have a significant impact upon how they performed in their jobs. Previous research suggests self and relationship management are components of the emotional intelligence model that demonstrate an individual’s ability to take action (Goleman et al., 2002). Therefore, it is possible awareness of others does not impact job performance because it is the actual action of managing relationships that impacts those you work with. However, the awareness of emotional state, management of emotions, and management of interactions with others did have an impact upon overall job performance for leaders in this sample.

A study by Cavallo and Brienza (2002) found relationship management to be the best correlate of leader performance compared to the four factors of emotional intelligence. Hypothesis three suggested the same and this unique finding was replicated by the present study. Managing the emotional side of relationships had greater impact on job performance than awareness of emotions or management of the self.

Relationship management is considered to be the ultimate outcome of emotional intelligence; a skill whose foundation is built upon the other three skills in the model (Goleman et al., 2002). In this study, a leader’s utilization of the relationship management skill was far more important to his or her job performance than the other three skills in the emotional intelligence model. Although relationship management is widely considered to be the most important sub-component of emotional intelligence, diminutive research has been conducted to confirm or deny this belief. Therefore, this finding is an important contribution to the growing body of research in emotional intelligence.

Limitations of the present study

Generalizability of the findings is limited to organizations in the telecommunications, manufacturing and homebuilding industries. While this is a fairly diverse sample of organizations, it cannot be assumed that what is true for leaders in telecommunications is true for all leaders who work in high technology. Three organizations are not sufficient to generalize to the entire U.S. working population.

This study investigated employee perceptions of job performance using a previously validated job performance assessment. This instrument did not provide a measure of outcomes specific to the position of the leaders tested. In the study, all co-workers who worked closely with the leader provided job performance ratings. Human observation is rarely objective and organizations often use metrics to measure the performance of their leaders. A more objective measure of leader job performance may have been preferred in lieu of the manner in which this study was conducted.

Consistency of the performance appraisal process in the three organizations in this study was a barrier to the use of more objective measures. Metrics were not available for all leaders who participated. The performance appraisal process for each organization was unique, using a variety of rating scales, rating methods, and job performance outcomes. The use of different assessments of performance in each organization could have been a potential confound had it been used as the measure of job performance.

Another limitation in this study is the instruments were only administered once. It is important to retest participants on multiple occasions to assess the stability of findings across time. Retesting respondents with positive results highlights the stability and merit of findings in any scientific study. Test-retest reliabilities were not ascertained in this study and there is no data to suggest whether or not these findings will remain stable across time.

Suggestions for future research

The ability based model of emotional intelligence, as measured by the MSCEIT in this study, did not have a link to job performance, while emotional intelligence ratings from co-workers did have a strong link to job performance. The «best» way to measure emotional intelligence is often debated and may depend upon the purpose for which emotional intelligence is being measured. As such, different instruments may have more utility depending upon how they are being used. Further research should provide additional comparisons between ability-based tests and other assessments that measure emotional intelligence through other means.

Conclusions and practical implications

Companies large and small are budgeting significant amounts of capital for leadership development (Delahoussaye, 2002). The training and development industry is roughly $40 billion annually in the United States alone. Much of the efforts directed at improving leadership skills are not based upon sound empirical data. It is important to make interventions based upon good research findings if they are to produce real fiscal results for the organization.

Emotional intelligence is very appealing to most, yet it can be daunting for some to explore emotions and feelings in a work setting. Providing a leadership assessment that is not too cumbersome or abstract is important. This study suggests a valid measure of emotional intelligence need not be more than 100 items long nor take an hour to administer. Scores on the Emotional Intelligence Appraisal^™ were more tightly linked to job performance than those on the MSCEIT, yet the entire Emotional Intelligence Appraisal^™ assessment takes one-fifth of the time to administer. Some participants in the sample complained about the length and focus of the MSCEIT as being inappropriate for a work setting, and that it was too long and did not seem to «make sense.» It is important to balance the statistical and theoretical validity of an instrument with usability for those who will have to take it.

Intuitively, emotional intelligence is important because it provides an excellent framework to look at how people understand and manage emotions. Emotional intelligence considers how people interact in a way cognitive ability theories can not fully account for. Books and leadership development programs on emotional intelligence are extremely popular for this reason. A strong research link between emotional intelligence and important «real world» outcomes, such as job performance, suggests that this intuition is accurate. Leaders who use emotional intelligence to build solid relationships are likely to perform well in their jobs.

Ashkanasy, N.M. (2001). The case for emotional intelligence in workgroups. Symposium presentation at the annual conference of the Society for Industrial and Organizational Psychology, April, San Diego, CA.

Atwater, L.E., Ostroff, C., Yammarino, F.J., & Fleenor, J.W. (1998). Self-other agreement: does it really matter? Personnel Psychology, 51, 577-598.

Atwater, L.E. & Yammarino, F.J. (1992). Does self- other agreement on leadership perceptions moderate the validity of leadership and performance predictors? Personnel Psychology, 45, 141-155.

Bar-On, R. (1988). The development of an operational concept of psychological well being. Unpublished doctoral dissertation, Rhodes University, South Africa.

Bar-On, R. (2000). Emotional and social intelligence: insights from the emotional quotient inventory. In R. Bar-On & J.D.A. Parker (eds.): Handbook of Emotional Intelligence (pp. 363-388). San Francisco: Jossey-Bass.

Bar-On, R. (2006). The Bar-On Model of Emotional-Social Intelligence (ESI). Psicothema, 18, supl., 13-25.

Barrett, G.V. (2001). Emotional intelligence: the Madison avenue approach to science and professional practice. Symposium presented at the annual meeting of the Society for Industrial Organizational Psychology, April, 2001, San Diego, California.

Becker, T.E., Billings, R.S., Eveleth, D.M., & Gilbert, N.L. (1996). Foci and bases of employee commiTMent: implications for job performance. Academy of Management Journal, 39, 464-482.

Boyatzis, R.E., Goleman, D., & Rhee, K. (1999). Clustering competence in emotional intelligence: insights from the Emotional Competence Inventory (ECI). In R. Bar-On & J.D.A. Parker (eds.): Handbook of emotional intelligence (pp. 343-362). San Francisco: Jossey-Bass.

Boyatzis, R.E. (2006). Using tipping points of emotional intelligence and cognitive competencies to predict financial performance of leaders. Psicothema, 18, supl., 124-131.

Brackett, M.A. & Salovey, P. (2006). Measuring emotional intelligence with the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT). Psicothema, 18, supl., 34-41.

Bradberry, T.R. & Greaves, J.M. (2003). The Emotional Intelligence Appraisal™ Technical Manual. San Diego: TalentSmart, Inc.

Brutus S., Fleenor J., & Taylor S. (1996). Methodological issues in 360-degree feedback research. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, San Diego.

Catell, R.B. (1966). The scree test for number of factors. Multivariate Behavioral Research, 1, 245-276.

Cavallo, K. & Brienza, D. (2002). Emotional competence and leadership excellence at Johnson and Johnson: the emotional intelligence leadership study. Website: http://www.eiconsortium.org.

Clark, R. & Callister, Wallace (2002). Undergraduate management skills courses and student’s emotional intelligence. Manuscript submitted for publication.

Conway, J.M. & Huffcutt, A.I. (1997). Psychometric properties of multisource performance ratings: a meta-analysis of subordinate, supervisor, peer and self-ratings. Human Performance, 10, 331-360.

Delahoussaye, M.E., Ellis, K., & Bolch, M. (2002). Measuring corporate smarts. Training, 39, 20-35.

Duarte, N.T., Goodson, J.R., & Klich, N.R. (1994). Effects of the dyadic quality and duration on performance appraisalTM. Academy of Management Journal, 37, 499-521.

Edwards, J.R. (1993). Problems with the use of profile similarity indices in the study of congruence in organizational research. Personnel Psychology, 46, 641-665.

Freese, M., Kring, W., Soose, A., & Zempel, J. (1996). Personal initiative at work: differences between East and West Germany. Academy of Management Journal, 39, 37-63.

Furnham, A. & Stringfield, P. (1998). Congruence in job-performance ratings: a study of 360-degree feedback examining self, manager, peer and consultant ratings. Human Relations, 51, 517-530.

Gardner, H. (1983). Frames of mind: the theory of multiple intelligences. New York: Basis Books.

Goleman, D. (1995). Emotional intelligence: Why it can matter more than IQ. Bantam: New York.

Goleman, D. (1998a). Working with emotional intelligence. Bantam: New York.

Goleman, D. (2001). Emotional intelligence: issues in paradigm building. In Cherniss, C. & Goleman, D. (eds): The emotionally intelligent workplace. How to select for, measure and improve emotional intelligence in individuals, groups and organizations (pp. 13-26). San Francisco: Jossey-Bass.

Goleman, D., Boyatzis, R., & McKee, A. (2002). Primal leadership: realizing the power of emotional intelligence. Harvard Business School Press: Boston.

Graves, J.G. (1999). Emotional intelligence and cognitive ability: predicting performance in job-simulated activities. Unpublished doctoral dissertation, California School of Professional Psychology, San Diego, CA.

Harris, M.M. & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer and peer-supervisor ratings. Personnel Psychology, 41, 43-62.

Huselid, M.A. (1994). Documenting HR’s effect on company performance. HR Magazine, 39, 79-87.

Lam, L.T. & Kirby, S.L. (2002). Is emotional intelligence an advantage? An exploration of the impact of emotional and general intelligence upon individual performance. The Journal of Social Psychology, 142, 133-143.

Mayer, J., Caruso, D., & Salovey, P. (2000). Selecting a measure of emotional intelligence: the case for ability scales. In R. Baron-On & J.D.A. Parker (eds.): The handbook of emotional intelligence (pp. 320-342). New York: Jossey-Bass.

Mayer, J.D., DiPaolo, M.T., & Salovey, P. (1990). Perceiving affective content in ambiguous visual stimuli: a component of emotional intelligence. Journal of Personality Assessment, 54, 772-781.

Mayer, J.D. & Salovey, P. (1993). The intelligence of emotional intelligence. Intelligence, 17, 433-442.

Mayer, J.D. & Salovey, P. (1997). «What is emotional intelligence?» In P. Salovey & D. Sluyter (eds.): Emotional development and emotional intelligence: implications for educators (pp. 3-31). New York: Basic.

Mayer, J.D. & Stevens, A. (1994). An emerging understanding of the reflective (meta) experience of mood. Journal of Research in Personality, 28, 351-373.

Miller, J.S. & Cardy, R.L. (2000). Self-monitoring and performance appraisalTM: rating outcomes in project teams. Journal of Organizational Behavior, 21, 609-626.

Payne, W.L. (1985). A study of emotion: developing emotional intelligence: self integration; relating to fear, pain and desire (theory, structure of reality, problem-solving, contraction/expansion, tuning in/coming out/letting go). Unpublished doctoral thesis, The Union Institute, Cincinnati, Ohio, USA.

Sala, F. (2001a). It’s lonely at the top: executives’ emotional intelligence self [mis] perceptions. Website: http://eiconsortium.org.

Sala, F. (2001b). Do programs designed to increase emotional intelligence at work work? Website: http://eiconsortium.org.

Salam, S., Cox, J.F., & Sims, H.P. (1997). In the eye of the beholder: how leadership relates to 360-degree performance ratings. Group and Organizational Management, 22, 185-209.

Salovey, P. & Mayer, J.D. (1990). Emotional intelligence. Imagination, Cognition and Personality, 9, 185-211.

Salovey, P., Hsee, C., & Mayer, J.D. (1993). Emotional intelligence and the self-regulation of affect. In D.M. Wegner and J.W. Pennebaker (eds.): Handbook of mental control (pp. 258-277). Englewood Cliffs, NJ: Prentice-Hall.

Sosik, J.J. & Megerian, L.E. (1999). Understanding leader emotional intelligence and performance: the role of self-other agreement on transformational leadership perceptions. Group and Organization Management, 24, 367-390.

Thorndike, E.L. (1920). Intelligence and its uses. Harper’s Magazine, 140, 227-335.

Thorndike, R.L. & Stern S. (1937). An evaluation of the attempts to measure social intelligence. Psychological Bulletin, 34, 275-284.

Weschler, D. (1952). The range of human capacities. New York: Hafner.

Wheeler, J.V. (1999). The impact of social environments on self-directed change and Learning. Unpublished doctoral dissertation, Case Western Reserve University.

Young, D.P. & Dixon, N.M. (1996). Helping leaders take effective action: a program evaluation. Center for Creative Leadership: Greensboro.