Assessing the Structure of the Five Factor Model of Personality (IPIP-NEO-120) in the Public Domain

The Five Trait Domains of Personality

Personality traits are important for most, if not all, life outcomes, and have demonstrated predictive validity in subjective outcomes such as relationships and well-being (Roberts, Kuncel, Shiner, Caspi, & Goldberg, 2007). Traits also relate to a variety of objective life-outcomes, such as annual income and educational attainment, as shown in studies with a Swedish nation-wide sample (e.g., Kajonius & Carlander, 2017). Personality traits furthermore seem to be growing in importance with the contexts of individualism in modern society (Skirbekk & Blekesaune, 2014). There is also emerging evidence that personality traits tend to cluster geographically (Jokela, Bleidorn, Lamb, Gosling, & Rentfrow, 2015; Kajonius & Mac Giolla, 2017), providing nations, regions, or neighborhoods with certain characteristics. Personality traits are principally known to be fairly stable and develop predictably throughout life (Briley & Tucker-Drob, 2014). A study based on almost 15 million twin pairs (Polderman et al., 2015) reported that the heritability for personality traits generally is 50% (h² = .49). The majority of reliable (error-corrected) trait variance over a life-time (83%) has been shown to be due to stable influences such as genetics and social maturation (Anusic & Schimmack, 2016).

Personality is most frequently measured with the five factor model (FFM; McCrae, 2010). This represents regularities of thoughts, feelings, and behaviors in individuals expressed in five broad trait domains: N = Neuroticism, E = Extraversion, O = Openness, A = Agreeableness, C = Conscientiousness. Recurring correlations among these have reignited the importance of studying the structure of the FFM (Strus, Cieciuch, & Rowiński, 2014). This has particularly come to the forefront with the advent of the use of personality traits in the DSM-5 (American Psychiatric Association, 2013). Many psychologists today agree that the FFM framework can be used as a foundation for integrating common and abnormal personality traits (Markon, Krueger, & Watson, 2005). The usefulness of FFM has received much empirical support (see a review by Miller, 2012).

The Lower Order Facet Structure of the FFM

Organizing personality into five trait domains is often too general for certain purposes, such as when differentiating job candidates for a specific task or individualizing clinical diagnoses. For instance, recognizing that someone is high on trait factor Extraversion could indicate that the person is sociable, happy, energetic, or dominant, or all of these. In other words, what is the scope and meaning when someone scores high on Extraversion? Another example might be illustrated by the trait factor Openness. There are several specific lower order facet traits which could be more informative, such as Adventurousness in predicting tendency to travel, or Intellect in predicting choice of education. Research and practice might therefore be better served by using narrower and more specific traits. Facets should enable higher precision of analysis (see Ziegler & Bäckström, 2016). The IPIP-NEO instrument makes use of this by including a number of facet traits, consisting of dispositions towards certain behaviors, affects, and cognitions within each factor domain (see Zillig, Hemenover, & Dienstbier, 2002). The 30 facet traits of IPIP-NEO are presented in Table 1, with 6 facet traits for each one of the 5 trait domains, arranged in columns, followed by example items. For instance, the first facet trait Friendliness in trait domain Extraversion (E1) attempts to capture the propensity of being friendly and experienced as socially warm person. One item example of the Friendliness facet trait is the extent one “makes friends easily”. Personality models are often constructed by items making up facet traits which in turn make up the broad trait domain, which aims at generating both content scope and precision (Johnson, 2014; McCrae, 2010). The extent to which this hierarchical structure is valid can be tested by assessing how items, facets, and trait factor are interlinked. Optimally, both trait and facet levels should be saturated by the variance provided by items.

The Present Study

The overall endeavor of the present study was to test the structure of one public domain version of the Five Factor Model (FFM) of personality, IPIP-NEO-120 (Johnson, 2014). The specific objective was to investigate how facet traits organize according to its proposed structure (Table 1) in respective FFM trait domain. The general hypothesis was that facet traits would contribute to each trait domain according to their positions in the established FFM literature (Costa & McCrae, 1995). Confirming the structure of personality assessment in the public domain would give important information on scope and meaning of this publically available FFM to professionals who need a simple and accessible instrument in their research and practice.

Sample and Procedure

We were able to make use of one of the largest US samples to date (N = 320,128). The sample consisted of 40% males (N = 127,695) and 60% females (N = 192,433), with an average age of 28.1 years (SD = 10.1). The respondents were 19–69 year old. The data was collected through an online survey, in the form of a website offering visitors feedback on their FFM personality after filling out 120 items. The visitors were at large volunteers from all walks of life, nationwide in the US, who could have reached the site by word-of-mouth, Internet search engines, and other informal channels. Every participant had to actively acknowledge that the survey would be time-consuming, used for educational and research purposes, and that careless responding would invalidate the usefulness. No login was required, and no personal, traceable data (e.g., IP-addresses) was collected, neither data on ethnicity, socio-economic status, or other demographic characteristics. The data-set can be accessed at an open psychology research depository (see Supplementary Materials).

Data Limitations

Despite the generous number of respondents this likely is not a representative nationwide sample. Due to the active volunteering of respondents, a reasonable assumption is that several of the trait facet levels such as in Openness to Emotionality, Openness to Intellect, and Altruism were likely unrepresentatively high (see Appendix), since these traits are known to characterize people interested in psychology (Vedel, 2016).

Another concern was that online surveys are often known for inconsistent responding or intentional misrepresentations. However, the quality of the present online survey data was analyzed according to guidelines in Johnson (2005). For example, duplicates (showing long strings of equivalent data) were removed. Also, repetitive patterns, which may indicate curiosity to see the items with no purpose of doing going through with the test, were removed. The missing data (1%) was finally corrected by imputing item means. Due to the present size of sample (N = 320,128), the item-correlations with and without the missing data imputed showed near perfect convergence (r = .998). The conclusion is that these commonly raised problems may not interfere with the purpose of the present study.

Instrument

IPIP-NEO-120 is a publically available representation of the FFM (Johnson, 2014), drawing 120 items from the International Personality Item Pool (IPIP; Goldberg et al., 2006). IPIP-NEO was built on open-source items correlating with the original NEO-PI-R (Costa & McCrae, 1995). IPIP-NEO was created seeking to optimize length, reliability, and validity in FFM measurement, and even surpassed the original in mean facet reliability (α > .80) (Johnson, 2014). The trait scales are built by four correlated items within each facet trait, while keeping bandwidth in meaning (i.e., removing repetitive items), as well as tending to balancing items (+ and – keyed items). Each of the items are measured on a 1 (almost never) – 5 (almost always) scale which then are summarized into facet traits (Min = 4, Max = 20). Six facet traits in turn are averaged into one trait factor. Overall, the mean reliability for facet scales in the present study was α = .78. Only four facets (13%) were below α = .70. See Table 2 for a summary of alphas.

Statistical Method

The analyses were based on the 30 facet traits in the FFM (IPIP-NEO-120). We first explored the unconstrained factor structure with facet correlations and exploratory factor analysis to see if facets organized into five proposed factors. The main objective was then finalized with testing hierarchical structural CFA models, one for each trait domain. The models were based on established FFM literature (Costa & McCrae, 1995), and tested the loadings between manifest items, latent trait facets, and respective latent trait domain. We utilized fit indices for two ways of modeling each FFM trait domain. The first way was the original and most used hierarchical second-order structure in two levels (Costa & McCrae, 1995). This is characterized by the general latent trait domain at the top, loaded by six facet traits, which in turn are loaded by 24 item measurement items. The second way of modeling was a bi-factor model, where both the general latent trait factor and specific latent facets are loaded onto directly by measurement items. The bi-factor approach is also helpful in revealing if any facet traits could be viewed as trait domains on their own—If items would not load on the general trait domain factor, but only on the facet trait, this could imply independence from the FFM domains. The CFA model was conducted in a SEM-framework, extracted with Maximum Likelihood, using AMOS v.23. The models were attempted as largely unconstrained, without covarying error terms. Due to the large sample size, all results were significant and standard errors were less than 0.01.

Exploring Five Factors in the IPIP-NEO

Before investigating the main objective of testing the five-factor structure as measured by IPIP-NEO, we checked whether Exploratory Factor Analysis (EFA) would confirm five factors. Utilizing Maximum Likelihood (ML) for extraction, and oblique rotation, assuming correlations between facets, the summary of loadings (structural pattern) is reported in Table 2. Only five factors had Eigenvalues above 1, which was an initial confirmation of the FFM IPIP-NEO structure. The goodness of fit was χ² (295) = 38,877, RMSEA = 0.07, and TLI = 0.84. The first factor extracted 21%, the second 11%, the third 8%, the fourth 6%, and the fifth 5% of total variance, making the total 51%. Overall, the facet traits revealed five clear trait structures (Table 2). However, there were also 4 facet discrepancies, as marked in italics in Table 2. Facet Trust (A1) loaded slightly more on Extraversion; Assertiveness (E3) and Activity (E4) organized mostly under Conscientiousness; and Emotionality (O3) under Neuroticism.

The Structure of the Five Factors in the IPIP-NEO

The main objective was to test the structures of the FFM trait domains in the public IPIP-NEO-120 instrument. All five CFA trait structure models, one for each domain, were conducted in two ways: Second-order models; See Figure 2a, and bi-factor models; See Figure 2b). All parameters were unconstrained, and conducted without modifications (e.g., covariances between errors).

Click to enlarge

Figure 2

a) The structure of the FFM second-order models. b) The structure of the FFM bi-factor models.

Note. FFM = Five Factor Model.

Table 3 accounts for the results of the second-order models (based on Figure 2a), one for each FFM domain. The table includes fit indices (χ²(df), RMSEA, TLI, and CFI), reported in the trait domain rows, and facet and item loadings, reported in the facet rows. The first observation was that the second-order structures overall showed at best tolerable model fits. Most were above recommended stricter limits (RMSEA > 0.05 and CFI, TLI < 0.90; Cheung & Rensvold, 2002). Fit indices, however, could be improved by allowing one or two selected error variances to covary within each trait facet, which would result in Neuroticism, Openness, and Conscientiousness structures fitting within recommended limits (RMSEA < 0.05 and CFI, TLI > 0.90). A second observation was that loadings both on facet and trait factor level were overall high. The average facet loading in the second-order models was 0.65, and the average item loading 0.70. However, a few weaker loadings (defined as β < .50) were seen, such as facet traits Trust and notably Modesty in the Agreeableness domain, or Assertiveness and Activity in the Extraversion domain. Openness showed the overall weakest construct coherence, with three of its facet traits, that is, Imagination, Emotionality, and Liberalism, showing weak loadings. Such facet traits may not be contributing robustly to their respective general trait factors. Also, on the contrary, several facet traits showed notably strong loadings (β > .90)—Friendliness on Extraversion, Self-discipline on Conscientiousness, and Anxiety and Vulnerability on Neuroticism almost showed convergent identity.

Table 3

IPIP-NEO Second-order FFM Trait Structures (Based on Figure 2a)

Trait domain / facet trait	ECV	Facet (β)	Item a	Item b	Item c	Item d
Neuroticism χ2(246) = 23,295; RMSEA = 0.06; TLI = 0.90, CFI = 0.91
N1_Anxiety	.49	1.0a	.62	.67	.68	.79
N2_Anger	.62	.57	.85	.82	.82	.65
N3_Depression	.59	.72	.82	.73	.87	.65
N4_Self-conscious	.40	.54	.74	.61	.63	.53
N5_Immoderation	.40	.41	.61	.62	.61	.68
N6_Vulnerability	.47	.98	.77	.70	.64	.62
Extraversion χ2(246) = 30,751; RMSEA = 0.07; TLI = 0.87, CFI = 0.88
E1_Friendliness	.53	.96	.72	.79	.75	.63
E2_ Gregarious	.51	.89	.72	.76	.60	.75
E3_Assertive	.61	.47	.85	.76	.81	.70
E4_Activity	.44	.43	.77	.83	.60	.29
E5_Excitement	.42	.54	.73	.73	.49	.60
E6_Cheerfulness	.51	.75	.66	.74	.73	.72
Openness χ2(246) = 17,301; RMSEA = 0.05; TLI = 0.88, CFI = 0.89
O1_Imagination	.45	.45	.52	.63	.79	.71
O2_Artistic	.44	.88	.77	.55	.60	.70
O3_Emotionality	.39	.41	.54	.59	.50	.66
O4_Adventurous	.33	.46	.52	.68	.69	.59
O5_Intellect	.44	.67	.54	.73	.59	.77
O6_Liberalism	.43	.35	.88	.34	.83	.35
Agreeableness χ2(246) = 26,313; RMSEA = 0.06; TLI = 0.86, CFI = 0.87
A1_Trust	.61	.42	.81	.69	.81	.81
A2_Morality	.43	.63	.68	.59	.83	.49
A3_Altruism	.41	.91	.63	.74	.62	.57
A4_Cooperation	.37	.64	.49	.58	.72	.63
A5_Modesty	.47	.13	.46	.87	.92	.24
A6_Sympathy	.42	.78	.66	.71	.57	.64
Conscientiousness χ2(247)* = 17,513; RMSEA = 0.053; TLI = 0.92, CFI = 0.93
C1_Self-efficacy	.46	.89	.73	.65	.67	.72
C2_Orderliness	.58	.55	.69	.70	.84	.80
C3_Dutifulness	.42	.62	.80	.57	.35	.79
C4_Achievement	.49	.81	.72	.69	.75	.63
C5_Self-discipline	.40	.99	.99	.67	.66	.45
C6_Cautiousness	.66	.49	.82	.77	.83	.82

Note. Each FFM facet loads (β) on the top trait domain factor, and is made up by four items (a, b, c, and d). See Figure 2a. ECV = Explained Common Variance (based on respective items); FFM = Five Factor Model; RMSEA = Root Mean Square Error of Approximation; TLI = Tucker Lewis Index; CFI = Comparative Fit Index. 95% CI were less than ±0.01.

^aA standardized beta equal to or above 1.0 can legitimately occur, and indicates high multicollinearity (Deegan, 1978).

Table 4 accounts for the bi-factor models (based on Figure 2), one for each FFM domain. The first observation here was that the bi-factor models, which control for a general trait factor, overall showed better model fits than second-order models. This is however expected since fit indices such as RMSEA reward parsimony (i.e., adding constraints). Nevertheless, CFI were also superior, and the difference was substantial for two of the trait structures: Extraversion (.90 compared to .87) and Agreeableness (.91 compared to .86). A second result was that ECV average was .47 on facet level and .50 on trait factor level. These results can overall be interpreted as unidimensionality in the components of the IPIP-NEO structure, which support a more nuanced facet structures as hypothesized. Along this line of reasoning, however, some trait structures had facets that seemed disconnected from the trait domain. For instance, the trait factor Agreeableness and facet Modesty showed a disconnection. Items “I think I am better than others” and “I think highly of myself” did not load on the general trait domain Agreeableness (β = .05 and β = .09), but highly on the facet trait (β = .88 and β = .90). (Recall that facet trait Modesty did not load satisfactorily on Agreeableness in the second-order model). Also, Self-discipline (in trait domain Conscientiousness) and Friendliness (in Extraversion) items even loaded negatively on respective facets, while loading strongly on respective general trait factors. This could be interpreted as these facets being identical with the general trait factor.ⁱ These mentioned examples may represent the core of respective general factors but may at the same time be too similar to optimally function as independent facet traits.

Table 4

IPIP-NEO Bi-factor FFM Trait Structures (Based on Figure 2b)

Trait domain / facet trait	Item a (G)	Item b (G)	Item c (G)	Item d (G)
Neuroticism χ2(230)a = 20,704; RMSEA = 0.06; TLI = 0.90, CFI = 0.92, ECV = 0.52
N1_Anxiety	.16 (.60)	.57 (.65)	.13 (.66)	.55 (.80)
N2_Anger	.75 (.44)	.61 (.52)	.71 (.44)	.49 (.41)
N3_Depression	.58 (.58)	.46 (.54)	.66 (.60)	.37 (.51)
N4_Self-conscious	.66 (.38)	.50 (.34)	.53 (.34)	.38 (.35)
N5_Immoderation	.54 (.28)	.63 (.18)	.56 (.24)	.58 (.33)
N6_Vulnerability	.56 (.76)	.31 (.69)	.10 (.63)	.04 (.62)
Extraversion χ2(228) = 22,111; RMSEA = 0.06; TLI = 0.90, CFI = 0.91, ECV = 0.55
E1_Friendliness	-.01 (.73)	.09 (.76)	.64 (.71)	.12 (.62)
E2_Gregarious	.70 (.58)	.28 (.69)	.10 (.58)	.28 (.66)
E3_Assertive	.77 (.38)	.65 (.37)	.73 (.34)	.59 (.37)
E4_Activity	.79 (.25)	.70 (.39)	.49 (.35)	.31 (.03)
E5_Excitement	.45 (.46)	.48 (.42)	.71 (.06)	.63 (.32)
E6_Cheerfulness	.32 (.55)	.31 (.64)	.60 (.50)	.62 (.49)
Openness χ2(228) = 12,405; RMSEA = 0.05; TLI = 0.91, CFI = 0.92, ECV = 0.43
O1_Imagination	.31 (.46)	.49 (.35)	.84 (.27)	.59 (.35)
O2_Artistic	.39 (.64)	.11 (.56)	.29 (.51)	.59 (.53)
O3_Emotionality	.47 (.25)	.51 (.29)	.42 (.25)	.69 (.18)
O4_Adventurous	.44 (.27)	.57 (.35)	.70 (.26)	.45 (.36)
O5_Intellect	.25 (.48)	.60 (.46)	.33 (.47)	.63 (.48)
O6_Liberalism	.80 (.31)	.30 (.16)	.82 (.25)	.31 (.17)
Agreeableness χ2(228) = 15,003; RMSEA = 0.05; TLI = 0.91, CFI = 0.93, EVC = 0.48
A1_Trust	.76 (.30)	.57 (.39)	.77 (.28)	.71 (.38)
A2_Morality	.49 (.48)	.37 (.46)	.59 (.60)	.17 (.48)
A3_Altruism	.40 (.49)	.62 (.55)	.22 (.56)	.24 (.50)
A4_Cooperation	.39 (.33)	.58 (.35)	.40 (.52)	.33 (.52)
A5_Modesty	.41 (.45)	.88 (.05)	.90 (.09)	.20 (.35)
A6_Sympathy	.68 (.37)	.54 (.46)	.19 (.52)	.41 (.47)
Conscientious χ2(228) = 13,700; RMSEA = 0.05; TLI = 0.94, CFI = 0.95, ECV = 0.53
C1_Self-efficacy	.23 (.66)	.38 (.54)	.37 (.57)	.42 (.56)
C2_Orderliness	.57 (.40)	.54 (.43)	.72 (.45)	.66 (.45)
C3_Dutifulness	.66 (.47)	.38 (.40)	.13 (.34)	.64 (.47)
C4_Achievement	.35 (.60)	.38 (.56)	.52 (.59)	.40 (.50)
C5_Self-discipline	-.13 (.64)	-.05 (.66)	.39 (.66)	.29 (.56)
C6_Cautiousness	.72 (.39)	.66 (.39)	.73 (.40)	.69 (.45)

[i]) Note. FFM = Five Factor Model; RMSEA = Root Mean Square Error of Approximation; TLI = Tucker Lewis Index ; CFI = Comparative Fit Index. Each FFM facet is made up by four items (a, b, c, and d), and is controlled by G (General trait domain factor). See Figure 2b. Very weak loadings (below .20) are marked in italics. 95% CI were less than ±0.01. Fit indices (χ², RMSEA, TLI, CFI) are reported based on Figure 2b, without improvement modifications (e.g., covariations between errors or between facets).

^aDid not converge in a first run with all parameters free, returning negative error variance or Heywood cases, which was amended by relaxing two constraints.

A last and final complexity was that all the trait domain models may be confounded by common variance bias, such as social desirability or acquiescence responding. This could potentially confound fit indices, as well as loadings (See the extended discussion in Anusic, Schimmack, Pinkus, & Lockwood, 2009). Consequently, we also attempted to estimate to what extent a common method factor may be present. We created a so called bogus marker variable, which was made up by four uncorrelated fake items, which we added to the bi-factor model of the trait structure Agreeableness, chosen for its face appeal and social desirability. The average loadings from this fictive variable on the common factor, even though small, were not trivial (β = .18). With no common method variance present, this bogus variable should have been closer to zero. This result indicated that almost 4% of the variance in Agreeableness may be attributed to common method factors due to the respondent source.