Originally published by: Elsevier
Written by: Lucas De Francisco Carvalho PhD, A.R.L. Costa, F. Otoni, P. Junqueira

Background and objectives

This study aimed to establish a clinical cut-off for the self-report Conscientiousness dimension of the Dimensional Clinical Personality Inventory 2 (IDCP-2) for people with Obsessive–compulsive personality disorder (OCPD) diagnosis. The following hypotheses were raised: (1) scale should be more reliable for assessing more pathological levels compared to milder levels of conscientiousness, and (2) items should be able to discriminate OCPD patients from other groups (i.e., non-patients and mental health patients).

Methods

Participants were 2449 adults, among outpatients diagnosed with OCPD, outpatients diagnosed with other PDs, and adults from the community.

Results

According to the Wright map, outpatients were located at the very high level on the latent continuum of the Conscientiousness dimension, with a larger effect size for the mean difference between OCPD patients and non-patients. The ROC curve indicated a cutoff at −.33 score in theta standardization, with 85% of sensitivity and 58% of specificity.

Conclusion

Findings from this investigation suggest the IDCP-2 Conscientiousness dimension as useful for screening purposes of the core traits of the OCPD. Potential clinical applications for the scale and limitations from the present study are discussed.

Introduction

People diagnosed with personality disorders (PDs) express psychological distress, interpersonal relationship impairment, as well as deviant behavior. Officially, PD diagnosis is given under the categorical model assumptions, although several criticisms were presented. Dimensional models were proposed (e.g., HiTOP) to replace the current official model. Under the dimensional perspective, pathological traits composing PDs must be evaluated through assessment measures, generally self-report scales (e.g., Minnesota Multiphasic Personality Inventory-2; MMPI-2 Personality Inventory for DSM-5; PID-5).

Our focus in this study was a Brazilian self-report measure for pathological traits assessment, the Dimensional Clinical Personality Inventory 2 (IDCP-2), aligned with new proposals in mental health, composed by 12 core dimensions: Dependency, Aggressiveness, Mood Instability, Eccentricity, Attention Seeking, Distrust, Grandiosity, Isolation, Criticism Avoidance, Self-sacrifice, Conscientiousness, and Inconsequence. We perform a narrowed and specific investigation of the Conscientiousness dimension. We relied on Item Response Theory (IRT) procedures and the Receiver Operating Characteristic (ROC) curve to analyze the IDCP-2 Conscientiousness dimension and establish an empirical cut-off for the identification of patients diagnosed with an obsessive–compulsive personality disorder (OCPD).

OCPD prevalence in clinical samples was from 2.1 to 7.9% and more frequent in women. OCPD is characterized by excessive preoccupation with order, perfectionism, and interpersonal inflexibility. In an epidemiological study in the Metropolitan Region of São Paulo, Brazil, PDs from Cluster C, including OCPD, presented prevalence equal to 4.6%.

According to the fifth edition of the Diagnostic Manual of Mental Disorders (DSM-5), OCPD diagnosis is composed of eight diagnosis criteria, with at least four of them being required to the diagnosis. Criteria include excessive preoccupation with details and organization that hamper finishing activities, devotion to work and productivity giving up leisure activities and interpersonal ties, inflexibility regarding what is morally correct, inability to get rid of useless objects, reluctance to designate tasks and work in groups, adoption of a cheap lifestyle to excessively save money, and stubbornness and emotional rigidity. Although the DSM-5 considers OCPD as a diagnostic category, taxometric studies indicate evidence of a latent dimensional structure, consistent with the proposal of the assessment tool focus of this study, the IDCP-2.

The IDCP-2 Conscientiousness dimension is closely related to OCPD traits. The items of this dimension are grouped into six factors, referring to the need to perform tasks in an exaggerated organized and systematic way, interpersonal and cognitive inflexibility, excessive focus on obligations, excessive preoccupation with details, perfectionism, rigid rules in personal and professional relationships, and emotional and financial restraint. IDCP-2 Conscientiousness dimension, since its previous version (IDCP), has been related to OCPD pathological traits, capable of discriminating people with this diagnosis from people without this condition.

This study aimed to replicate previous procedures to establish a clinical cut-off for people with OCPD diagnosis. Therefore, analyses based on the Item Response Theory (IRT) mathematical model and ROC curve procedure were applied. The following hypotheses were raised: (1) people with more pathological levels of conscientiousness will present higher scores in comparison to healthy people, (2) scale should be more reliable for assessing more pathological levels compared to milder levels of conscientiousness, and (3) items should be able to discriminate OCPD patients from other groups (i.e., non-patients and mental health patients).

Methods

Sample

Participated 2449 subjects, aged between 18 and 90 years (M=28.76; SD=10.62), being 50.6% female. 48.3% were college students and 4.30% psychiatric patients. They were assessed by convenience. We divided the sample into four groups, a posteriori, according to external criteria: who declared never had been under psychological, psychiatric and/or psychotropic medicine treatment (non-patients; n=2288), who declared had done or being under psychological, psychiatric and psychotropic medicine treatment and without a known psychiatric diagnosis (mental health patients; n=273), psychiatric outpatients diagnosed with other personality disorder (non-OCPD patients; n=79), psychiatric outpatients diagnosed with OCPD (OCPD patients; n=26). Psychiatric outpatients were diagnosed by experienced psychiatrists using the Structured Interview for DSM-IV axis II (SCID-II) (Table 1).

Instrument

The Conscientiousness dimension from IDCP-2. The IDCP-2 is based on a dimensional approach, in which extreme responses suggest pathological personality pattern. About its successor, IDCP-2 has been updated, according to several models (e.g., criteria B of the Alternative Model for Personality Disorders; dimensions from the Schedule for Nonadaptive Personality; SNAP; and factors of the Shedler-Westen Assessment Procedure; SWAP. The Conscientiousness dimension contains 22 items divided into six factors named Need for routine (3 items), Concern with details (3 items), Thoroughness (3 items), Work compulsion (4 items), Self-directed perfectionism (4 items) and Emotional constriction (5 items), together they form a total score to the dimension. Items are answered in a Likert-type scale of four points (1=“it has nothing to do with me”; 4=“it has a lot to do with me”). Psychometrics properties, i.e., validity evidence (internal structure and relation with external criteria) and reliability coefficients, have shown adequacy in previous studies.

Procedures

The Ethical Review Board of São Francisco University approved the study protocol. All participants signed the consent form, containing information about the purpose of the research. Participants were assessed in Brazilian university centers in São Paulo city (paper and pencil administration), online social networks (online administration), and a public psychiatric hospital in São Paulo (paper and pencil administration). The administration was collective for college students, lasting about 30min, and individual for patients, lasting about 60min.

We used a dataset composed by 4830 participants: 1240 responded IDCP-2, 1004 responded IDCP-2 and its previous version (IDCP), and 1586 responded only IDCP. To provide a score for all participants, we applied the equating procedure. The IDCP and IDCP-2 Conscientiousness dimension shared a total of 7 items, also used as anchors to generate a score (i.e., theta).

Data analysis

All analyses performed considered only items from Conscientiousness dimension from IDCP-2. We used the rating scale model, which deemed relatively similar to the scalar intervals between points for all items. The difficulty parameter bi represents the location of item i or the average intensity of the thresholds of an item, so an item representing extremes of latent dimension have high average threshold since they are located on the most intense level. Item and subject model parameters were calibrated by the Joint Maximum Likelihood Estimation method, implemented in the Winsteps software. This calibration procedure considered original and revised items, so the parameters were estimated by all the participants, which is the purpose of the equating method.

The adequacy of the calibration was evaluated by the indexes fit, infit and outfit, calculated for all items and participants. These values are directly proportional to the residuals that reflect differences between the observed and expected responses from the hypothesized knowledge of the model parameters, thus providing evidence of how well the model fits the data. Values greater than 1.7 indicate misfit. Complementarily, we verified item-theta correlation, general reliability (α and Rasch) and test curve information.

We proceeded to Wright map, group comparisons (ANOVA) and ROC curve, aiming better understanding of latent construct and establishing the dimension's cutoff. Regarding the ROC curve, the psychiatric sample accounted for approximately 1.12% of the groups participating on this analysis (i.e., non-patients and OCPD patients), which is in line with the range of prevalence of PDs in the general population. Moreover, we highlight that the OCPD patients represent 1.06% of the total sample, close of DSM-5 report,1 especially the subsample from Part II of the National Comorbidity Survey Replication. Qualitative item map analyses were based on Elliot et al. recommendations.

Results

We first conducted analyses to verify psychometrics assumptions of Conscientiousness dimension of IDCP-2. The dimensionality test indicated that the variance explained by the items is 18.1%, and the unexplained variance is 2.5 eigenvalues. To test the stability of measures, we performed simulated analyses with five simulated datasets, and the pattern has changed (between 1.2 and 1.5), suggesting that the previous value (2.5) occurred by chance, and indicating the unidimensionality of the items set.

The rating scales show a broader use of categories 1 and 4. Fig. 1 demonstrates category probabilities to be chosen (axis Y) given the person's latent trait (axis X). The level of thresholds (represented by * in the figure) required to choose 2 over 1 is −.76, to choose 3 over 2 is .05 and, to choose 4 over 3 is .71. This demonstrates the adequacy of categories to discriminate subjects with a different level of OPD traits.

Table 2 presents the difficulty level for endorsement to items (δ) value for each item of Conscientiousness dimension, standard error, adjustment indices (infit and outfit), correlation theta-item, and internal consistency reliability indices (based on the Rasch model and Cronbach's alpha).

The difficulty level of each item is present in ascending order in Table 2, wherein items with higher difficulty indicate a lower likelihood of endorsement, once they require higher levels in the latent trait. These items represent the most severe traits of Conscientiousness dimension, therefore are better to discriminate subjects with severe pathological characteristics. The extremes (i.e., pole positive in the scale, with the less endorsed pathological items) are represented by item I321, related to social issues due to devotion to work, and by item A001, related to responsibility at work, (i.e., pole negative in the scale, with the less pathological items). All item presented a good fit for the model, with infit and outfit ranged from 0.5 to 1.5. Furthermore, items showed a moderate to high correlation with the total score (between .37 and .63), and the coefficient alpha was also high.

The relationship between subjects and items distribution were investigated using an item information curve, that considers the mean location of items and subjects (see Fig. 2). The test demonstrates that the peak of information for Conscientiousness dimension is approximately between −1.0 and 1.0. This range encompasses people's mean (−0.42), 73.8% of group 4 (other PDs patients), and 86% of group 3 (OCPD patients).

Previous analyses had the purpose of investigating the adequacy of psychometric properties of Conscientiousness dimension of IDCP-2, as an assumption to the focus analyses of the study. We now present these analyses. The Wright map represented in Fig. 2 shows the relationship between item difficulty and person trait level. For illustrative purpose, the Wright map presents the distribution of 330 subjects selected at random, but item difficulty was fixed equally as total sample. On the map, 1 represent non-patients (n=125), 2 represent mental health patients (n=100), 3 represents patients with other PD (n=79), and 4 represents patients with OCPD (n=26).

Fig. 3 shows that items concentrate in the middle of the continuum. Also presents that 61.54% of OCPD patients locate above the items mean, and 76.92% locate above person mean. Regarding patients with other PD, 31.64% of are located above the items mean, and 44.30% locate above person mean. Most of the non-patients locate below item mean.

To provide further evidence of the discriminative capacity of the Conscientiousness dimension we performed an analysis of variance among groups (ANOVA) and a ROC curve analysis. ANOVA demonstrated difference among groups [F=10.59, df=4, p<.001]. Age was not significant, but sex showed significance (p<.001). We also calculated the effects size for ANOVA, by Cohen's d for peer-to-peer comparisons,37 being the larger effect between non-patients and patients with OCPD. Turkey's post hoc analysis differentiated groups 1 (non-patient), 2 (mental health patient) and 3 (other PDs patients) from group 4 (OCDP patients) (p<.05). The ROC curve comprised an area under the curve of .77 (95% IC 0.68–0.87; p<.001), with high sensitivity (85%) and moderate specificity (58%), considering as a cut-off δ=−.33 (Table 3).

The Predictive Positive Power and Negative Predictive Power were calculated using Equation 15 and 17 of Streiner. These values refer to the capacity of the test to diagnose someone with high levels of Conscientiousness traits when the person indeed has OCPD diagnostic (PPP), and the ability to diagnose someone that has low levels of Conscientiousness traits when the person indeed has not an OCPD diagnostic (NPP). Considering the prevalence of OCPD equal to 2.1%, the PPP was 4.2%, and the NPP was 99.5%. Positive and negative incremental (PPPinc and NPPinc) values were calculated using equations 25 and 26 of Streiner. The incremental PPPinc value was equal to 2.1%, and NPPinc was equal to 73.73%. The incremental values indicate the improvement of the diagnostic of OCPD by using Conscientiousness dimension of IDCP-2. The efficiency test was equal to 56.48%, showing the percentage that the test was right in diagnosing people with OCPD.

Discussion

Considering that assessment tools for clinical use should assist the professional in decision making, the objective of this research was to investigate the clinical functioning of the Conscientiousness dimension of IDCP-2. Then, three hypotheses were proposed, aiming to understand in a more specific way the capacity of this dimension to discriminate sample groups according to the pathological levels related to OCPD traits. Findings corroborated the second hypotheses, while the first hypothesis was partially supported. As an assumption for verifying the clinical functioning of the dimension, we first investigated its psychometric properties, in which we observe the adequacy of the parameters of the item. Specifically, the items presented fit averages close to 1.00, indicating an optimal relationship between the identified model and the true model. Also, the items have a reliability value of 1.00, suggesting that the recovered parameters are highly likely to be replicated in several samples.

We observed that the raw scores have a positive relationship with the latent score, i.e., the higher the raw score higher the latent score. These findings were verified in the following analyses: response categories, which presented adequate functioning according to the mathematical model criteria; item map and information curve, indicating that OCPD patients have more extreme scores and the scale is more reliable in the extreme positive range. OCPD patients showing extreme scores in the Conscientiousness dimension was previously observed when administering IDCP.

The first hypothesis was partially corroborated, i.e., the instrument is more informative for people who have pathological levels in the traits evaluated by the Conscientiousness dimension. The items that presented the highest level of difficulty (i.e., unlikely to be endorsed by healthy people), described more severe behaviors and were mostly from the Work compulsion and Self-directed perfectionism factors, whose contents evaluate the excessive focus on work, interpersonal rigidity and need for perfection. On the other hand, easier items (i.e., likely to be endorsed by healthy people) were mostly from the Concern with details and Thoroughness factors, whose contents refer to excessive concern with details that are often overlooked by other people, and need of perfection at work. These results are in agreement with the findings reported by Harford et al., where they identified that OCPD criteria have similar difficulty levels, except for criteria 6 and 8 that presented lower values, being related to perfection in activities, similar to the factors 2 (Concern with details) and 3 (Thoroughness) presently investigated.

The second hypothesis of this study was also corroborated, indicating that the scale can be used for clinical screening to differentiate non-patients from OCPD patients. This was evidenced by ANOVA, in which the OCPD patients group differed from other groups, indicating the usefulness of the instrument to identify pathological levels of Conscientiousness related to OCPD symptoms. Also, effect sizes were higher between the non-patient versus OCPD patients and the mental health patient versus OCPD patients, where the OCPD patients presented higher mean values. The smallest difference was observed for mental health patients versus other PDs patients, which was expected since the former group was composed by people experiencing some mental health treatment, and although they were not patients with a diagnosis of PD, they are more likely to present pathological traits of personality.

The cut-off established for the Conscientiousness dimension was −.33 on the theta scale, where we explicitly favor the sensitivity and penalize the specificity of the test. We based this choice on guidelines that suggest that screening scales should favor false positives over false negatives. When considering only the PPP value, we observed that the instrument was not accurate to confirm the diagnosis. However, the NPP result indicates that the instrument has a high capacity (99.5%) to differentiate groups at the pathological level of the disorder. In this perspective, we can state that the Conscientiousness dimension of IDCP-2 is a useful scale to be used as clinical screening in cases where the clinician suspects of OCPD diagnosis. Screening tools are of great importance for the help of professionals who are fulfilling an initial assessment with the patient, allowing decisions that can refute or accept diagnostic hypotheses, supporting the professional to continue or not in the process of psychological assessment of specific demands.

Some limitations of this study must be observed. The relationship between conscientiousness and OCPD is complex due to the heterogeneity of people profiles with this disorder since only four criteria are necessary for the diagnosis. Thus, one limitation was that the criteria for each patient were not identified to allow analysis by different OCPD profiles. Also, the number of participants diagnosed with OCPD was small, and not all of them responded to all items in the revised dimension. We recommend that future research focus on the different OCPD profiles and the relationship with the factors of the Conscientiousness dimension.

References available in original article on Elsevier.