Advances in the Assessment of Social, Emotional, and Self-Regulatory Skills

A special issue of Journal of Intelligence (ISSN 2079-3200).

Deadline for manuscript submissions: closed (15 July 2022) | Viewed by 24357

Special Issue Editors


E-Mail Website
Guest Editor
GESIS—Leibniz Institute for the Social Sciences, Survey Design and Methodology, P.O. Box 12 21 55, 68072 Mannheim, Germany
Interests: individual differences; personality; cognitive ability; assessment; psychometrics

E-Mail Website
Guest Editor
Department of Educational Psychology, Goethe-University Frankfurt, 60629 Frankfurt am Main, Germany
Interests: educational psychology; assessment; problem solving; multivariate statistics
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Academic to Career Research Center, Research & Development, Educational Testing Service, Princeton, NJ 08541, USA
Interests: psychometrics; measurement; personality; cognitive psychology; response time; noncognitive skills
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Social, emotional, and self-regulatory skills (SESS) – also known as non-cognitive skills, 21st-century skills, or soft skills – have gained currency among both researchers and policymakers. SESS cover a wide variety of skills outside those measured by traditional intelligence tests and usually emphasize emotional and motivational aspects. Examples comprise communication skills, leadership, collaboration, work ethic, self-control, and emotional resilience. SESS are considered important to select and pursue goals, build and maintain relationships, regulate one’s emotions, and related everyday tasks. Recent contributions suggest that SESS predict various measures of success (e.g., academic achievement, career success) above and beyond measures of intelligence, also in cases where intelligence is broadly conceptualized and measured. Moreover, evidence suggests that measures of SESS are largely independent of traditional measures of intelligence, somewhat reminiscent of “multiple intelligences”.

Despite recent advances, the assessment of SESS has long lagged behind that of cognitive abilities. There are a number of reasons for this. SESS are typically assessed through multi-item self-reports inventories using rating scales, mostly using agree–disagree response formats. Such inventories often follow the Big Five model of personality as an organizing framework. Many existing studies on SESS, in fact, use Big Five inventories as measures of SESS. Such inventories allow for a highly economical assessment of multiple SESS and often show at least moderate correlations with a range of relevant outcomes (e.g., for achievement, attainment, social participation, or health).

However, rating-scale based assessments have several downsides. For example, they are prone to bias from response styles (e.g., social desirability, acquiescence). Moreover, they almost invariably measure SESS as typical performance, rather than maximum performance. As a consequence, the objectivity and validity of current SESS assessments, the distinction between SESS and personality traits, and the theoretical status of SESS continue to be hotly debated.

In order to advance research on SESS, there is a need for novel assessment approaches as well as for further conceptual groundwork. The aim of this special issue is to promote such advances, both on a rather fundamental and on an applied level. We are looking for psychometric, theoretical, and empirical contributions including on (but not limited to) the following issues:

  • Fundamental conceptual questions regarding the definition of SESS, their relation to intelligence and personality, and their nature as psychological constructs;
  • The structure of SESS, their relation to intelligence and personality, and their incremental validity over intelligence and personality
  • Relations between rating-scale based measures of SESS and other types of measures (e.g., behavioral tasks, psychometric games);
  • Ways to improve traditional rating-scale based assessment approaches, such as multiple-informant approaches, situational judgement tests, or anchoring vignettes;
  • Innovative assessment approaches, such as technology-based assessments, psychometric games and behavioral tasks, or machine learning approaches using text, audio, or video data or even using multi-channel data;
  • Case studies and empirical applications of innovative assessment approaches for SESS in educational, vocational, and other settings

We explicitly welcome innovative and “risky” approaches, including reports of failed attempts to assess SESS or studies providing critical perspectives on the (often weak) relations between questionnaire-based and other SESS measures. Attempts to measure SESS from a maximum-performance perspective will be highly welcome.

Dr. Clemens Lechner
Prof. Dr. Samuel Greiff
Dr. Patrick C. Kyllonen
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a double-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Journal of Intelligence is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • non-cognitive skills
  • socio-emotional skills
  • assessment
  • psychometric testing

Published Papers (5 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

18 pages, 371 KiB  
Article
A Big Five-Based Multimethod Social and Emotional Skills Assessment: The Mosaic™ by ACT® Social Emotional Learning Assessment
by Kate E. Walton, Jeremy Burrus, Dana Murano, Cristina Anguiano-Carrasco, Jason Way and Richard D. Roberts
J. Intell. 2022, 10(4), 72; https://0-doi-org.brum.beds.ac.uk/10.3390/jintelligence10040072 - 20 Sep 2022
Cited by 6 | Viewed by 2553
Abstract
A focus on implementing social and emotional (SE) learning into curricula continues to gain popularity in K-12 educational contexts at the policy and practitioner levels. As it continues to be elevated in educational discourse, it becomes increasingly clear that it is important to [...] Read more.
A focus on implementing social and emotional (SE) learning into curricula continues to gain popularity in K-12 educational contexts at the policy and practitioner levels. As it continues to be elevated in educational discourse, it becomes increasingly clear that it is important to have reliable, validated measures of students’ SE skills. Here we argue that framework and design are additional important considerations for the development and selection of SE skill assessments. We report the reliability and validity evidence for The Mosaic™ by ACT® Social Emotional Learning Assessment, an assessment designed to measure SE skills in middle and high school students that makes use of a research-based framework (the Big Five) and a multi-method approach (three item types including Likert, forced choice, and situational judgment tests). Here, we provide the results from data collected from more than 33,000 students who completed the assessment and for whom we have data on various outcome measures. We examined the validity evidence for the individual item types and the aggregate scores based on those three. Our findings support the contribution of multi-method assessment and an aggregate score. We discuss the ways the field can benefit from this or similarly designed assessments and discuss how the assessment results can be used by practitioners to promote programs aimed at stimulating students’ personal growth. Full article
39 pages, 2334 KiB  
Article
The Behavioral, Emotional, and Social Skills Inventory (BESSI): Psychometric Properties of a German-Language Adaptation, Temporal Stabilities of the Skills, and Associations with Personality and Intelligence
by Clemens M. Lechner, Thomas Knopf, Christopher M. Napolitano, Beatrice Rammstedt, Brent W. Roberts, Christopher J. Soto and Marion Spengler
J. Intell. 2022, 10(3), 63; https://0-doi-org.brum.beds.ac.uk/10.3390/jintelligence10030063 - 05 Sep 2022
Cited by 5 | Viewed by 5808
Abstract
Social, emotional, and behavioral (SEB) skills comprise a broad set of abilities that are essential for building and maintaining relationships, regulating emotions, selecting and pursuing goals, or exploring novel stimuli. Toward an improved SEB skill assessment, Soto and colleagues recently introduced the Behavioral, [...] Read more.
Social, emotional, and behavioral (SEB) skills comprise a broad set of abilities that are essential for building and maintaining relationships, regulating emotions, selecting and pursuing goals, or exploring novel stimuli. Toward an improved SEB skill assessment, Soto and colleagues recently introduced the Behavioral, Emotional, and Social Skills Inventory (BESSI). Measuring 32 facets from 5 domains with 192 items (assessment duration: ~15 min), BESSI constitutes the most extensive SEB inventory to date. However, so far, BESSI exists only in English. In three studies, we comprehensively validated a novel German-language adaptation, BESSI-G. Moreover, we expanded evidence on BESSI in three ways by (1) assessing the psychometric properties of the 32 individual skill facets, in addition to their domain-level structure; (2) providing first insights into the temporal stabilities of the 32 facets over 1.5 and 8 months; and (3) investigating the domains’ and facets’ associations with intelligence, in addition to personality traits. Results show that BESSI-G exhibits good psychometric properties (unidimensionality, reliability, factorial validity). Its domain-level structure is highly similar to that of the English-language source version. The facets show high temporal stabilities, convergent validity with personality traits, and discriminant validity with fluid and crystallized intelligence. We discuss implications for research on SEB skills. Full article
Show Figures

Figure 1

16 pages, 399 KiB  
Article
Predicting Actual Social Skill Expression from Personality and Skill Self-Concepts
by Simon M. Breil, Ina Mielke, Helmut Ahrens, Thomas Geldmacher, Janina Sensmeier, Bernhard Marschall and Mitja D. Back
J. Intell. 2022, 10(3), 48; https://0-doi-org.brum.beds.ac.uk/10.3390/jintelligence10030048 - 29 Jul 2022
Cited by 5 | Viewed by 2878
Abstract
Social skills are of key importance in everyday and work life. However, the way in which they are typically assessed via self-report questionnaires has one potential downside; self-reports assess individuals’ global self-concepts, which do not necessarily reflect individuals’ actual social behaviors. In this [...] Read more.
Social skills are of key importance in everyday and work life. However, the way in which they are typically assessed via self-report questionnaires has one potential downside; self-reports assess individuals’ global self-concepts, which do not necessarily reflect individuals’ actual social behaviors. In this research, we aimed to investigate how self-concepts assessed via questionnaires relate to skill expression assessed via behavioral observations after short interpersonal simulations. For this, we used an alternative behavior-based skill assessment approach designed to capture expressions of predefined social skills. Self- and observer ratings were collected to assess three different social skills: agency (i.e., getting ahead in social situations), communion (i.e., getting along in social situations), and interpersonal resilience (i.e., staying calm in social situations). We explored how these skills were related to self-concepts by differentiating between a classic personality measure (i.e., Big Five Inventory 2; BFI-2) and a novel skill questionnaire (i.e., Behavioral, Emotional, and Social Skills Inventory; BESSI). The results (N = 137) showed that both personality and skill self-concepts predicted self-rated skill expression, with the BESSI showing incremental validity. For both personality and skills self-concepts, the relationships with observer-rated skill expression were significant for agency but not for communion or interpersonal resilience. We discuss these results and highlight the theoretical and practical importance of differentiating between skill self-concepts and actual skill expression. Full article
24 pages, 1012 KiB  
Article
Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills
by Jessica Andrews-Todd, Jonathan Steinberg, Michael Flor and Carolyn M. Forsyth
J. Intell. 2022, 10(3), 39; https://doi.org/10.3390/jintelligence10030039 - 04 Jul 2022
Cited by 4 | Viewed by 2705
Abstract
Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals’ CPS competency are necessary, as traditional assessment types such as multiple -choice items are not well suited [...] Read more.
Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals’ CPS competency are necessary, as traditional assessment types such as multiple -choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals’ CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students’ display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts. Full article
Show Figures

Figure 1

19 pages, 1211 KiB  
Article
Measuring Personality through Images: Validating a Forced-Choice Image-Based Assessment of the Big Five Personality Traits
by Airlie Hilliard, Emre Kazim, Theodoros Bitsakis and Franziska Leutner
J. Intell. 2022, 10(1), 12; https://0-doi-org.brum.beds.ac.uk/10.3390/jintelligence10010012 - 07 Feb 2022
Cited by 10 | Viewed by 8643
Abstract
Selection methods are commonly used in talent acquisition to predict future job performance and to find the best candidates, but questionnaire-based assessments can be lengthy and lead to candidate fatigue and poor engagement, affecting completion rates and producing poor data. Gamification can mitigate [...] Read more.
Selection methods are commonly used in talent acquisition to predict future job performance and to find the best candidates, but questionnaire-based assessments can be lengthy and lead to candidate fatigue and poor engagement, affecting completion rates and producing poor data. Gamification can mitigate some of these issues through greater engagement and shorter testing times. One avenue of gamification is image-based tests. Although such assessments are starting to gain traction in personnel selection, few studies describing their validity and psychometric properties exist. The current study explores the potential of a five-minute, forced-choice, image-based assessment of the Big Five personality traits to be used in selection. Study 1 describes the creation of the image pairs and the selection of the 150 best-performing items based on a sample of 300 respondents. Study 2 describes the creation of machine-learning-based scoring algorithms and tests of their convergent and discriminate validity and adverse impact based on a sample of 431 respondents. All models showed good levels of convergent validity with the IPIP-NEO-120 (openness r = 0.71, conscientiousness r = 0.70, extraversion r = 0.78, agreeableness r = 0.60, and emotional stability r = 0.70) and were largely free from potential adverse impact. The implications for recruitment policy and practice and the need for further validation are discussed. Full article
Show Figures

Figure 1

Back to TopTop