The Impact of Encoding Information as a Fact or Opinion on the Illusory Truth Effect

• Main Author: Riesthuis, Paul
• Format: Text
• Language: unknown
• Published: Open Science Framework 2022
• Subjects: Social and Behavioral Sciences; Psychology; FOS Psychology; Clifford; Paine; North Atlantic
• Online Access: https://dx.doi.org/10.17605/osf.io/kxypn; https://osf.io/kxypn/

The Impact of Encoding Information as a Fact or Opinion on the Illusory Truth Effect Hypotheses: First, we expect to observe the illusory truth bias (Brashier & Marsh, 2020). That is, we expect that statements that are repeated will lead to higher truth ratings compared with statements that are presented for the first time. Second, we expect that statements that are repeated and are encoded as facts by the participants (vs. opinions) will lead to higher truth ratings. Methods Participants We will recruit participants from Amazon Mechanical Turk. An a priori power analysis conducted for a two-tailed paired sample t-test in G*Power (Faul et al., 2009) with a power of .80, Cohen’s dz = .21, and α = .05 indicated that we will need 180 participants. Our effect size was based on our smallest effect size of interest (SESOI; Lakens 2014). That is, because framing information in the form of a fact (vs. opinion) is such a low cost manipulation, we are interested in whether it can result in .2 raw mean difference in truth rating on a 7-point Likert scale. Based on previous research examining the illusory truth effect using similar Likert scale measures (Unkelbach & Speckmann, 2021), we calculated that a .2 raw mean difference in truth ratings is equal to a Cohen’s d = .21. Participants will be financially compensated for participation. Materials We will use a list of 99 statements used in previous research (Hassan & Barber, 2021). These statements have been judged to be relatively unknown but deemed plausible. For our experiment, we transformed 33 of these statements into opinions and another 33 into misinformation resulting into three types of statements: True information, misinformation, and opinion. For instance, to create an opinion statement we adapted statements like “The refrigerator was invented in 1748 by William Cullen” into “The refrigerator is the best invention since 1748” while to create a misinformation statement we changed it into “The refrigerator was invented in 1898 by Thomas Paine”. We decided a priori to have 27 critical items that will be repeated and will be used for the analyses which consisted of 9 true information, 9 misinformation, and 9 opinion statements (see Appendix A; see preregistration). For the initial phase wherein participants have to indicate whether a statement is a fact or an opinion, we will include 36 distractor statements of which 12 will be true information, 12 misinformation, and 12 opinions. In total there will be 63 statements for Part 1, of which 27 critical items and 36 distractor items. For the second phase wherein participants rate the truthfulness of statements, we will add 36 new statements of which 12 will be true information, 12 misinformation, and 12 opinion statements. To match the amount of critical items, we chose a priori 27 control statements out of those 36 new statements. The 27 control statements consisted of 9 statements for each type of statement (see Appendix A and preregistration). Hence, in total, there will be 63 statements for Part 2, of which 27 will be repeated (i.e., critical items) and 36 will be new. The truth ratings will be given on a 7-point Likert scale (1 = not truthful, 7 = very truthful). Procedure & Design For the present experiment we will use a within-subject design. Participants will first be instructed to read the informed consent. If they agree to participate, they will start Part 1 of the experiment wherein they will receive the following instructions: “Previous research has revealed that high school students are not always able to distinguish between how opinions and facts are constructed. In this study, we are interested whether you are able to indicate whether a statement is a fact or an opinion based on its syntax structure. Hence, in the next phase you will receive multiple statements and then we would like you to indicate whether it is an opinion or a fact.”. Afterwards, participants will engage in a 5-minute filler task (e.g., playing Tetris). Then, in Part 2 of the experiment, participants will be given a list of 63 statements (27 critical repeated items, 27 control/new items, 9 distractor/new items) and will be asked to indicate the truthfulness of each statement on a 7-point Likert scale. Afterwards, participants will be thanked and debriefed. Data Analysis Plan First, we will examine whether the participants were attentive during the experiment. To do so, we will include two attention checks in line with best practice recommendations (Abbey & Meloy, 2017; Thomas & Clifford, 2017). Our two attention check are: 1) The present city of Atlanta was originally named Terminus (To show that you have read this information, we would like you to answer “Opinion” on this page), and 2) 1 in 5,000 north Atlantic lobsters are born bright blue. (To show that you have read this information, we would like you to answer “1 (Not truthful)” on this page). If participants fail both attention checks, their data will be excluded from the data analyses. For our first hypothesis, we will compare the truth ratings of all repeated statements with the truth ratings of the corresponding control items using a paired sample t-test. Moreover, we will run individual analyses for each type of statement (i.e., true statement, misinformation, and opinion) comparing the truth ratings of the critical items to their corresponding control items using paired sample t-tests. For our second hypothesis, we will compare truth ratings for true information, misinformation, and opinions using paired sample t-tests. Data Collection No data have been collected for this study yet.