Permalink/APA Citation Due

Description

PLEASE USE THIS TO COMPLETE THE ASSIGNMENT

After reading through your article, you will submit:

1. The “Permalink” (as described/demonstrated in the library video link). This provides access to the full citation/abstract “Detailed Record” view of the selected article. 10 points

2. Include the complete APA citation for your article. The best way to submit this is from the “Cite” link (as demonstrated in link above). Copy and paste a clean APA style reference, or you may type it out. 10 points

3. In two paragraphs (minimum 5-7 sentences per paragraph) provide a detailed summary of your selected article specifically addressing: 30 points

what the study was about (hypothesis)
methodology used – must explain whether descriptive, correlation, or experimental design was used (Santrock, 2022, pp. 31-32)
detailed/comprehensive participant information
the overall results/findings.

**Synthesize and explain in your own words – Recreations of significant portions of the abstract are graded as zero and may be subject to the college/syllabus plagiarism policy**

4. One paragraph essay (5-7 sentences) explaining which one specific developmental domain/processes (Santrock, 2022, pp. 12-13) best fits your article and why. Develop your response about only one domain – not all three. This is NOT a repeated summary of your article, but an articulation of your supported critical thoughts across sources. This section requires textbook use along with citing and referencing as modeled. 20 points

This information literacy assignment assesses your ability to locate, and to some degree evaluate research, and apply critical thinking. Click on the assignment title at the top to submit. Submit directly in the text box, do not submit in comments box, or you may lose points on format.

The above criteria must be met for full credit. Cite and reference as necessary. Contact me immediately with any questions or confusion.

References:

Santrock, J.W. (2022). Essentials of Life-Span Development, 7th ed. New York, NY: McGraw Hill

Unformatted Attachment Preview

This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Special Issue: Advancing the Reproducibility of Psychological Assessment
Across Borders and Populations
Registered Report
Actively Open-Minded Thinking,
Bullshit Receptivity, and
Susceptibility to Framing
Evaluating the Dual-Process Account in North America
and Bulgaria
Nikolay R. Rachev1 , Sandra J. Geiger2 , Jáchym Vintr3 , Desislava Kirilova1,
Anna Nabutovsky4,5 , and Joakim Nelsson6
1
Department of General, Experimental, Developmental, and Health Psychology, Faculty of Philosophy,
Sofia University St. Kliment Ohridski, Sofia, Bulgaria
2
Department of Psychology, Faculty of Social and Behavioural Sciences, University of Amsterdam, The Netherlands
3
Department of Psychology, Faculty of Arts, Charles University, Prague, Czechia
4
Department of Psychology, Department of Statistical Science, University of Toronto, ON, Canada
5
School of Law, University of California, Los Angeles, CA, USA
6
Department of Psychology, Faculty of Social Sciences, Uppsala University, Sweden
Abstract: The framing effect occurs when different presentations of the same problem lead to predictably different preferences. The dualprocess framework of higher cognition assumes that the effect violates rational principles, but alternative accounts and recent evidence have
contested this interpretation. Contributing to this debate, we tested the dual-process assumption by investigating associations between
susceptibility to framing and the willingness and ability to think in line with rational norms, conceptualized as actively open-minded thinking
and pseudo-profound bullshit receptivity. We conducted two online studies among North American (N = 259) and Bulgarian (N = 248)
university students and administered several framing problems within-subjects, presumably necessary for the associations to appear.
Confirmatory factor analyses showed that susceptibility to framing was associated with decreased actively open-minded thinking and
increased bullshit receptivity in both sites. Exploratory multi-group analyses demonstrated partial strong invariance and showed that the
findings generalize across both sites in terms of direction and partially in terms of magnitude. These results broadly support the dual-process
account of the framing effect. Our study further contributes to adapting existing measures to a novel setting and expanding the findings across
borders and populations.
Keywords: framing effect, actively open-minded thinking, bullshit receptivity, dual-process account, confirmatory factor analysis
Imagine you are asked to evaluate a vaccine described as having an 80% success rate and, sometime later, a vaccine
described as having a 20% failure rate. If you are like many
people, chances are you like the former description more than
the latter, thus showing a framing effect, whereby different presentations of the same options lead to predictably different
responses (Levin et al., 1998). Does the discrepancy between
your two evaluations result from shallow thinking that could
European Journal of Psychological Assessment (2022), 38(6), 440–451
https://doi.org/10.1027/1015-5759/a000685
be overcome, had you given it more thought? According to
the dual-process framework of higher cognition (Evans &
Stanovich, 2013; Kahneman, 2003), the answer is yes, but
alternative accounts and recent evidence question this
assumption (e.g., Mandel, 2014; Sher & McKenzie, 2006).
Contributing to this debate, the present study aimed to test
the dual-process assumption by investigating whether actively
open-minded thinkers and those who are more successful in
detecting pseudo-profound bullshit are less susceptible to
framing manipulations. Crucially, we tested these propositions
in two sites, North America and Bulgaria, which served to
adapt existing measures to a novel setting and to test whether
our findings generalize across borders and populations.
! 2022 Hogrefe Publishing
N. R. Rachev et al., Actively Open-Minded Thinking, Bullshit Receptivity, and Framing
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
The Framing Effect: A Thinking Bias?
The dual-process framework of higher cognition (Evans
& Stanovich, 2013; Kahneman, 2003) distinguishes between
two types of processes. Intuitive Type 1 processes (System 1)
require minimal resources of the limited working memory
capacity and are typically fast and effortless, while deliberate Type 2 processes (System 2) rely on working memory
and tend to be slower and effortful. Often, a conflict arises
between heuristic Type 1 outputs and some normative
considerations. Failure to detect this conflict or inefficient
double-checking with Type 2 processes leads to systematic
deviations from norms of rational thought, so-called biases
(De Neys, 2018; Pennycook, Fugelsang, et al., 2015a). One
particular bias, the framing effect, violates the rational
principle of descriptive invariance, stating that preferences
should not be affected by how options are presented
(Tversky & Kahneman, 1981, 1986). Presumably, the effect
results from System 2 passively accepting the intuitive decision delivered by System 1, which is heavily influenced by
salient but irrelevant contextual features, namely the presented frame (Kahneman, 2003). Hence, individuals willing to revise their first hunches would be less susceptible
to framing, much like they are less susceptible to other
thinking biases (Pennycook, Fugelsang, et al., 2015b).
Alternative accounts, however, question the dual-process
account and its assumption that the framing effect violates
norms of rational thought. For instance, the two versions of
a framing problem might not necessarily be interpreted as
equivalent (Kühberger & Tanner, 2010; Mandel, 2014; Sher
& McKenzie, 2006). Regarding our opening problem, one
version might be interpreted as “at least 80% success rate,”
while the other one as “at least 20% failure rate.” Careful
thinkers would realize that the former is superior, and
further thinking is unlikely to equal ratings.
441
be suboptimal for testing the predictions of the dual-process
account. A single encounter with a framing task lacks a cue
to a potential conflict and thus provides no advantage for
more analytical thinkers (Kahneman, 2003). In contrast,
within-subjects administrations allow decision-makers to
encounter both versions of the task and establish their
logical equivalence. In this context, the dual-process
account predicts that more analytic thinkers will be more
likely to follow the invariance principle and thus be less susceptible to framing manipulations than less analytic individuals. The specific superiority of analytic thinkers in resisting
framing in within- but not in between-subjects contexts has
received some empirical support (LeBoeuf & Shafir, 2003).
Accordingly, we chose a within-subjects setting where
participants were presented with both versions of several
framing problems (Bruine de Bruin et al., 2007).
Actively Open-Minded Thinking (AOT)
as a Measure of Rational Thought
A salient feature of rational thinkers is that they are actively
open-minded (Baron, 2008, 2019). That is, before deciding
on a problem, they actively search for reasons why their
favored conclusions might be wrong and are willing to
update their beliefs – and their confidence therein– in the
face of new evidence. Baron (1991) has argued that individuals’ own standards of good thinking predict how individuals think. Hence, individuals whose beliefs about thinking
match better the prescriptive AOT standards would be less
prone to thinking biases. Indeed, AOT is associated with
utilitarian moral judgment, normative problem-solving
(Baron et al., 2015), and reduced overconfidence (Haran
et al., 2013). Based on the dual-process view, AOT would
be negatively related to susceptibility to framing for North
Americans (Hypothesis 1a) and Bulgarians (Hypothesis 1b).
Susceptibility to Framing and Thinking
Dispositions: A Way to Contribute to the
Debate
Pseudo-Profound Bullshit Receptivity
(BSR) and Failure to Detect Conflicts
It is thus possible to test the dual-process account of framing
by investigating whether susceptibility to framing is related
to one’s willingness and ability to think by rational norms.
Using this approach, Mandel and Kapler (2018) failed to find
a reliable association between susceptibility to framing and
various measures of analytic cognitive style or the propensity to rely on Type 2 rather than Type 1 processes. Pointing
to similarly weak relations in previous studies (e.g., West
et al., 2008), they concluded that “any theory positing that
framing effects are largely due to reliance on heuristic
‘System 1’ reasoning processes is wrong” (p. 10).
However, Mandel and Kapler (2018) used a single
framing task administered between subjects, which might
Susceptibility to framing might also be associated with the
tendency to uncritically accept meaningless statements as
profound. Known as pseudo-profound bullshit receptivity
(BSR), this tendency has been related to a host of cognitive
biases (Pennycook, Cheyne, et al., 2015). BSR consists of
two factors (Pennycook, Cheyne, et al., 2015): a general
proneness to uncritically accept claims of variable credibility
and a specific failure to detect bullshit. If susceptibility to
framing is a thinking bias, as suggested by the dual-process
framework, it might be related to both factors. A generally
unskeptical attitude toward novel information is likely
to favor passive acceptance of the presented frame
(Kahneman, 2003). Furthermore, both failed to detect
! 2022 Hogrefe Publishing
European Journal of Psychological Assessment (2022), 38(6), 440–451
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
442
N. R. Rachev et al., Actively Open-Minded Thinking, Bullshit Receptivity, and Framing
Figure 1. Confirmatory factor analysis (CFA) model including expected parameter estimates for the power analysis. The CFA model has 524
degrees of freedom with variances of the latent variables fixed to 1. The average factor loadings have been estimated using the Spearman-Brown
prediction formula based on Cronbach’s αFR = .62 (Bruine de Bruin et al., 2007), Cronbach’s αAOT = .89 (Baron & High II, 2019), and Cronbach’s αBSR
= .82 (Pennycook, Cheyne, et al., 2015). The residual variances have been set to 1 ! λ2. The correlations represent disattenuated estimates; the
values in parentheses are the uncorrected estimates obtained in previous studies with identical or similar constructs (Pennycook, Cheyne, et al.,
2015; Rachev et al., 2021; Toplak et al., 2011; Turpin et al., 2019). AOT = Actively Open-Minded Thinking; BSR = Pseudo-Profound Bullshit
Receptivity; FR = Susceptibility to Framing.
bullshit and framing susceptibility (in within-subjects contexts) might represent instances of a more general failure
to detect a conflict. Hence, the dual-process account would
predict that BSR is positively associated with susceptibility
to framing for North Americans (Hypothesis 2a) and Bulgarians (Hypothesis 2b).
We used confirmatory factor analyses (CFA) to test the
proposed hypotheses and correct for measurement unreliability (Westfall & Yarkoni, 2016). As displayed in Figure 1,
we expected a three-factor model with AOT, BSR, and susceptibility to framing as latent variables because every scale
measures a theoretically separate construct. We predicted
correlations between latent variables, as proposed in the
hypotheses and shown in previous research (Turpin et al.,
2019). Overall, we expected that the proposed model would
fit the data from (a) North America and (b) Bulgaria at least
adequately (Hypothesis 3).
Method
We conducted a dual-site online survey assessing AOT,
BSR, and susceptibility to framing. The survey was administered among North American and Bulgarian participants,
following the same protocol.
Participants
Sampling Plan
A priori power analyses were used to determine the sample
size required to detect (a) the target effect sizes (Wang &
Rhemtulla, 2020) and (b) model misspecifications based
on the test of close and not-close fit of the root mean
European Journal of Psychological Assessment (2022), 38(6), 440–451
square error of approximation (RMSEA; MacCallum
et al., 1996). Following previous comparable research, we
expected moderate disattenuated correlations for BSR
(rdisattenuated = .28 and robserved = .20; Pennycook, Cheyne,
et al., 2015, Rachev et al., 2021) and AOT (rdisattenuated =
!.29 and robserved = !.18; Toplak et al., 2011) with susceptibility to framing. The power analysis with the pwrSEM app
(Wang & Rhemtulla, 2020) was based on the CFA model in
Figure 1 and yielded 240 participants per site to detect the
expected effects with 93% (AOT-Framing) and 88% (BSRFraming) power. The power analysis with the semTools R
package (Version 0.5-2; Jorgensen et al., 2019) suggested
at most 229 participants per site to detect model misspecifications, with 90% power and a cut-off value of RMSEA ”
.08 for acceptable fit. Details to replicate both power analyses can be found in Supplement 1 at https://osf.io/hx8qt/
(Geiger et al., 2020).
Following the power analyses and accounting for potential outlier exclusions, we aimed to collect 250 complete
responses per site after excluding responses classified as
duplicate, inattentive, or too fast. As there were more
outliers than expected, we deviated from this plan and
collected 250 complete responses per site after applying
the three preregistered criteria and excluding outliers.
For inclusion, participants needed to be at least 18 years
old and university students with a major other than psychology to minimize familiarity with the framing effect. They
were further required to be either (1) citizens and residents
of North America (i.e., Canada or the US) and native
English speakers, or (2) citizens and residents of Bulgaria
and native Bulgarian speakers. Participants in both sites
were recruited via social media and emails and compensated via a random draw of 90 (85 # $5 and 5 # $30) gift
! 2022 Hogrefe Publishing
N. R. Rachev et al., Actively Open-Minded Thinking, Bullshit Receptivity, and Framing
cards for North American participants and 165 (160 # $3
and 5 # $20) gift cards for Bulgarian participants. The final
samples are presented in Table 1.
443
Table 1. Description of the final samples
North America
(N = 259)
Bulgaria
(N = 248)
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Gender
Data Exclusion
Participants were excluded if they failed to answer all items,
revoked their consent, or completed the survey multiple
times, as indicated by identical IP addresses, withdrawal
codes, and demographic information (i.e., age, gender,
major, and year of studies; Koo & Skinner, 2005). Participants were also excluded if they failed the attention check.
A substantial number of participants (n = 51 of 341 in North
America and n = 7 of 282 in Bulgaria) completed the survey
faster than 10 min. As these responses were no outliers
according to our preregistered criterion (i.e., 3.5 median
absolute deviations from the overall time), we deviated
from our plan and included only responses with a minimum
completion time of 10 min.
Multivariate outliers were detected using generalized
Cook’s distance from the R package influence. SEM (Version
2.2; Pastore & Altoe, 2018), using the default boxplot criterion Q3 + (Q3 ! Q1) # 1.5 (with Q1: first quartile and Q3:
third quartile; Koran & Jaffari, 2020). As a result, 31 influential multivariate outliers from the North American sample
and 23 from the Bulgarian sample were excluded listwise.
For robustness checks, the analyses were additionally performed, including outliers, and reported in Supplement 2
(Geiger et al., 2020). Overall, the results were relatively
robust toward outlier exclusions.
Materials
The survey assessed participants’ AOT, BSR, and susceptibility to framing. The Bulgarian version was created from
the original English-language materials using forward and
backward translation with four independent, experienced
translators.
The Actively Open-Minded Thinking Scale assessed selfreported willingness to thoroughly consider the evidence
and update one’s opinion accordingly, on a 5-point scale
from 1 (= strongly disagree) to 5 (= strongly agree). After
the fifth item, the following attention check was included
with the same response scale: “Please select the option
‘neutral’ and proceed to the following question.”
The Bullshit Receptivity Scale (Pennycook, Cheyne, et al.,
2015; Bulgarian translation by Rachev et al., 2021) consisted of 10 statements with a seemingly deep meaning
and correct syntactic structure, created by 2 generators
randomly mashing together words pertaining to New Age
vocabulary. To obscure their purpose, they were intermixed
with 10 prototypically profound motivational statements,
which were solely used for exploratory purposes. Participants rated the profoundness of both types of statements
! 2022 Hogrefe Publishing
Males
131
53
Females
113
192
Other
15
3
21.9 (4.2) [18, 49]
23.2 (6.2) [18, 52]
Age (in years)
M (SD) [Min, Max]
Education
No high school
1
–
High school
184
202
Bachelor’s degree
68
34
Master’s degree
–
12
Doctorate
1
–
Computer science (40)
Engineering (35)
Biology (22)
Philology (49)
Computer science (28)
Economics (27)
Most common
degrees
on a 5-point scale from 1 (= not at all profound) to 5 (= very
profound).
The Resistance to Framing Scale is a 14-item subscale
from the Adult Decision-Making Competence Index
(Bruine de Bruin et al., 2007). It assessed the susceptibility
to risky-choice and attribute framing (7 problems each) on a
6-point scale, with two formally equivalent versions per
problem, one framed positively and one framed negatively.
Susceptibility to framing was represented by the absolute
deviation between the ratings in the positive and the corresponding negative frame for every problem (Bruine de
Bruin et al., 2007), with higher scores indicating stronger
susceptibility.
Procedure
Participants completed the online survey administered in
their native language via Qualtrics. The study had been
approved by the Research Ethics Committees at both sites.
After providing informed consent and demographics,
participants were presented with the positive and negative
frame-block, in a counterbalanced order, whereby the
risky-choice subblock was always presented first within
each frame-block (Bruine de Bruin et al., 2007). Between
the two frame-blocks, participants completed the AOT
and BSR in randomized order, separating the administration of the positive and the negative frame by approximately 5 min (Aczel et al., 2018). Items of every scale
and framing problems within a subblock were randomized
(excluding the attention check) and forced-response. After
being debriefed, participants were redirected to a separate
survey recording their email addresses for compensatory
purposes.
European Journal of Psychological Assessment (2022), 38(6), 440–451
444
N. R. Rachev et al., Actively Open-Minded Thinking, Bullshit Receptivity, and Framing
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Statistical Analyses
All analyses were performed using R (Version 4.0.2) and a
threshold of α = .05 for statistical significance. We fit a
three-factor CFA model for each site (Figure 1) in lavaan
(Version 0.6-6; Rosseel, 2012) to test the proposed hypotheses. We used Maximum-Likelihood (ML) estimation with
robust standard errors, and Satorra-Bentler scaled test
statistics for all models, as the Henze-Zirkler test (R package
MVN, Version 5.8, Korkmaz et al., 2014) showed that
multivariate normality was always violated. Based on
these models, we assessed each scale’s reliability using
McDonald’s ω1 in semTools (Version 0.5-3; Jorgensen
et al., 2019), with ω1 $ .60 representing acceptable reliability. Model fit was evaluated based on the w2-test of exact fit
combined with several fit indices, as shown in Table 2. If the
model did not fit (i.e., most indicators did not show at least
acceptable fit), residual covariances were added based on
the highest modification indices. The modified model tested
the hypotheses if the indices suggested acceptable fit and if
the modified model fits better than the original model (i.e.,
two of three criteria, including significant likelihood-ratio
test, lower AIC, and BIC).
We further fit a multi-group CFA model to exploratively
test differences of the hypothesized associations between
the two sites (Fischer & Karl, 2019). This included testing
configural (i.e., equal models), weak factorial (i.e., equal
factor loadings), strong factorial (i.e., equal intercepts),
and strict (i.e., equal residual variances and covariances)
measurement invariance.
The full details of the preregistered contingency plan are
available in our Stage 1 report at https://osf.io/hx8qt/.
Results
Study 1: North America
Descriptive Statistics and Reliability
Descriptive statistics for each item are reported in Table 3.
Generally speaking, susceptibility to framing and BSR were
low, while AOT was high. Based on the selected model (see
Confirmatory Analyses), the reliability for the susceptibility
to framing (McDonald’s ω1 = .55) and AOT (McDonald’s
ω1 = .54) scale did not meet the pre-specified cut-off value
of ω1 $ .60 and was therefore judged as questionable. The
BSR scale was highly reliable (McDonald’s ω1 = .88).
Confirmatory Analyses
Contrary to Hypothesis 3a, the hypothesized model did not
fit acceptably (Table 4). As preregistered, we modified the
model based on the highest modifications index (mi =
18.66) and added a residual covariance between BSR6
European Journal of Psychological Assessment (2022), 38(6), 440–451
Table 2. Cut-off values for the fit indices
Excellent
RMSEA
Incremental fit indices
(CFI, TLI/NNFI, IFI)
SRMR (only with a
significant w2-test)
< .06 > 0.95
< .06 Good < .07 > 0.92
< .07 Acceptable < .08 > 0.89
< .08 Questionable < .09 > 0.86
< .09 Poor < .10 > 0.83
< .10 Unacceptable > .10
< 0.83 > .10
Note. Adapted from Greiff and Allen (2018). CFI = Comparative Fit Index; IFI
= Incremental Fit Index; SRMR = Standardized Root Mean Square Residual; RMSEA = Root Mean Square Error of Approximation; TLI/NNFI =
Tucker-Lewis Index/Non-Normed Fit Index.
(“We are in the midst of a self-aware blossoming of being
that will align us with the nexus itself.”) and BSR9 (“We
are in the midst of a high-frequency blossoming of interconnectedness that will give us access to the quantum soup
itself.”). This modification can also be theoretically justified, as the wording of the two items is similar. The results
of fitting the modified model can be found in Figure 2. We
had preregistered that we would continue to modify the
model until the BIC could no longer be improved, which
resulted in 18 model modifications (Supplement 2,
Table C1; Geiger et al., 2019). As this seemed infeasible,
we deviated from our plan and preferred the more parsimonious model with one modification. This deviation did not
change any of the conclusions about the hypotheses (Supplement 2, Table C1; Geiger et al., 2019).
The modified model fits the data better than the hypothesized model, as shown by a significant likelihood-ratio test
as well as a lower AIC and BIC for the modified model
(Table 4). Despite no exact fit of the modified model,
w2(523) = 669.28, p < .001 (scaling correction factor: 1.07), most robust fit indices now indicated acceptable, good, or excellent fit, CFI = 0.89, IFI = 0.89, RMSEA = .03, 90% CI [.03, .04], and SRMR = .06. Only the robust TLI/NNFI still indicated questionable fit, TLI/NNFI = 0.88. The model with the residual covariance between BSR6 and BSR9 thus fit the data acceptably and was used to test Hypotheses 1a and 2a. As predicted in Hypothesis 1a, AOT and susceptibility to framing were negatively correlated, r = !.20, 95% CI [!.37, !.02], SE = .09, p = .029. As predicted in Hypothesis 2a, BSR and susceptibility to framing were positively correlated, r = .28, 95% CI [.12, .44], SE = .08, p < .001. The associations are illustrated on Figure 3 (See Supplement 3 for correlations based on means; Geiger et al., 2019). Study 2: Bulgaria Descriptive Statistics and Reliability Susceptibility to framing and BSR were low, while AOT was high (Table 3). Susceptibility to framing and AOT ! 2022 Hogrefe Publishing N. R. Rachev et al., Actively Open-Minded Thinking, Bullshit Receptivity, and Framing 445 Table 3. Descriptive statistics for each item North America This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Item Potential Minimum–Maximum M SD Bulgaria Minimum–Maximum M SD Minimum–Maximum RC_pesticide 0–5 1.09 1.25 0–5 1.41 1.45 0–5 RC_taxes 0–5 0.90 1.07 0–5 0.86 1.15 0–5 RC_dropout 0–5 0.88 1.18 0–5 1.00 1.23 0–5 RC_disease 0–5 1.14 1.33 0–5 0.99 1.13 0–5 RC_cancer 0–5 0.89 1.32 0–5 0.96 1.30 0–5 RC_stocks 0–5 0.98 1.18 0–5 0.98 1.21 0–5 RC_soldiers 0–5 1.13 1.21 0–5 1.01 1.19 0–5 AF_condom 0–5 0.40 0.65 0–3 0.45 0.80 0–4 AF_beef 0–5 0.54 0.85 0–4 0.67 0.88 0–4 AF_cheating 0–5 0.46 0.70 0–3 0.62 1.03 0–5 AF_budget 0–5 0.51 0.71 0–3 0.54 0.75 0–3 0–3 AF_exams 0–5 0.28 0.48 0–2 0.41 0.65 AF_parking 0–5 0.57 0.78 0–4 0.54 0.83 0–4 AF_cancer 0–5 0.38 0.61 0–3 0.42 0.65 0–3 AOT1 1–5 4.20 0.86 1–5 4.12 0.96 1–5 AOT2 1–5 4.75 0.45 3–5 4.34 0.88 1–5 AOT3a 1–5 2.10 0.98 1–5 3.13 1.19 1–5 AOT4 1–5 4.76 0.43 3–5 4.43 0.74 1–5 AOT5a 1–5 1.19 0.40 1–3 1.94 1.05 1–5 AOT6 1–5 4.14 0.92 1–5 4.21 0.83 1–5 AOT7a 1–5 1.39 0.66 1–4 2.49 1.24 1–5 AOT8a 1–5 1.83 0.91 1–5 2.41 1.14 1–5 AOT9 1–5 3.86 1.09 1–5 3.10 1.24 1–5 AOT10 1–5 4.73 0.46 3–5 4.55 0.65 2–5 BSR1 1–5 2.27 1.09 1–5 2.92 1.24 1–5 BSR2 1–5 1.98 1.03 1–5 2.48 1.21 1–5 BSR3 1–5 1.93 1.03 1–5 2.60 1.17 1–5 BSR4 1–5 2.05 1.02 1–5 2.54 1.19 1–5 BSR5 1–5 1.95 1.09 1–5 2.73 1.22 1–5 BSR6 1–5 1.82 1.11 1–5 2.73 1.25 1–5 BSR7 1–5 1.97 1.14 1–5 2.65 1.29 1–5 BSR8 1–5 2.49 1.16 1–5 3.00 1.17 1–5 BSR9 1–5 1.81 1.18 1–5 2.32 1.33 1–5 1–5 1.99 1.11 1–5 2.28 1.13 1–5 BSR10 a Note. Reverse-coded items (i.e., lower values reflect higher AOT). SD = Standard Deviation; M = Mean; AOT = Actively Open-Minded Thinking; BSR = Pseudo-Profound Bullshit Receptivity; AF = Attribute Framing; RC = Risky-Choice Framing. Table 4. Fit and comparison of the hypothesized and modified model: North America w2(df) p RMSEA CFI TLI/NNFI IFI SRMR robust robust robust robust robust scaled Bentler AIC BIC Hypothesized model 684.77 (524) < .001 0.04 CI [0.03, 0.04] 0.88 0.87 0.88 0.06 21,496 21,749 Modified model (BSR6 %% BSR9) 669.28 (523) < .001 0.03 CI [0.03, 0.04] 0.89 0.88 0.89 0.06 21,480 21,737 Model Δw2(Δdf) p 8.43 (1) .004 Note. AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; CFI = Comparative Fit Index; CI = 90% Confidence Interval; IFI = Incremental Fit Index; SRMR = Standardized Root Mean Square Residual; RMSEA = Root Mean Square Error of Approximation; TLI/NNFI = Tucker-Lewis Index/NonNormed Fit Index. displayed questionable reliability (McDonald’s ω1 = .44 and .52, respectively); however, the BSR scale was reliable (McDonald’s ω1 = .84). These results mirror those from North America. Confirmatory Analyses The results of fitting the hypothesized model to the data are displayed in Figure 4. Supporting Hypothesis 3b, the model fit acceptably according to our pre-specified criteria. Even ! 2022 Hogrefe Publishing European Journal of Psychological Assessment (2022), 38(6), 440–451 This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 446 N. R. Rachev et al., Actively Open-Minded Thinking, Bullshit Receptivity, and Framing Figure 2. Modified model including standardized parameters: North America. AOT = Actively Open-Minded Thinking; BSR = Pseudo-Profound Bullshit Receptivity; FR = Susceptibility to Framing. *p < .05, **p < .01, ***p < .001. though the w2-test indicated no exact fit, w2(524) = 610.67, p = .005 (scaling correction factor: 1.07), the robust fit indices suggested acceptable to excellent fit, CFI = 0.89, TLI/NNFI = 0.887, IFI = 0.90, RMSEA = .03, 90% CI [.02, .04], and SRMR = .07. Supporting Hypothesis 1b, AOT and susceptibility to framing were negatively correlated, r = !.32, 95% CI [!.56, !.08], SE = .12, p = .008. As predicted by Hypothesis 2b, BSR and susceptibility to framing were positively correlated, r = .20, 95% CI [.01, .38], SE = .09, p = .036. The associations are illustrated in Figure 5. Discussion Exploratory multi-group CFA (N = 507) tested whether the hypothesized associations differed between sites. To achieve partial strong invariance, we needed to add a covariance between BSR6 and BSR9 (mi = 18.66), as we did earlier in the North American model, and free two factor loadings and 12 intercepts. Since at least two items per factor remained invariant (Cieciuch & Davidov, 2015), results could thus be meaningfully compared between North America and Bulgaria. To test differences between sites, we constrained all latent correlations to be equal between sites. This constrained model fit worse than the partial strong invariance model (Table 5), suggesting that the correlations differed between sites. Using the partial strong invariance model, a two-tailed test for independent correlations (R package psych, Version 1.9.12.31; Revelle, 2020) revealed that the AOT-framing correlation was larger in Bulgaria (r = !.41) than in North America (r = !.24, z = 2.23, p = .025). However, there was no evidence for different BSR-framing correlations between sites (North America: r = .30, Bulgaria: r = .21, z = 1.06, p = .291). Although the framing effect is a robust phenomenon, different conflicting explanations exist. In particular, the dual-process account conceives of the framing effect as a deviation from rational thought. However, this view has recently been contested (e.g., Mandel, 2014). To advance the debate, we tested two predictions derived from the dual-process framework regarding the association between susceptibility to thinking and two qualities of rational thought, actively-open minded thinking (AOT) and pseudoprofound bullshit receptivity (BSR). Using a within-subjects administration of framing tasks, we found a negative association between AOT and susceptibility to framing as well as a positive association between BSR and susceptibility to framing among North Americans and Bulgarians. When provided with a cue, individuals who are more willing and able to think analytically are less susceptible to framing, among other thinking biases (Pennycook, Fugelsang, et al., 2015b). However, the small to moderate associations point to additional factors other than thinking dispositions. However, one such factor is the alternative interpretation of framing tasks (Kühberger & Tanner, 2010; Mandel, 2014; Sher & McKenzie, 2006), which is unlikely in this study. If individuals interpreted the two versions of a framing task as different rather than equivalent, they would shift their preferences depending on the task version. Such instances of justified reconsideration would be misclassified as susceptibility to framing, leading to an underestimated association between thinking qualities and using a superior decision strategy given one’s interpretation. However, the small mean differences between the two versions of the framing tasks (Table 3) suggest that most participants interpreted the two versions as equivalent, and importantly, participants who were more willing and able to think analytically had an advantage in following the normative principle. Hence, the European Journal of Psychological Assessment (2022), 38(6), 440–451 ! 2022 Hogrefe Publishing Exploratory Measurement Invariance