Limits...
A propaganda index for reviewing problem framing in articles and manuscripts: an exploratory study.

Gambrill E, Reiman A - PLoS ONE (2011)

Bottom Line: To determine the effectiveness of an index in increasing recognition of misleading problem framing in articles and manuscripts.However many instances remained undetected.This propaganda index warrants further exploration as a complement to reporting guidelines such as CONSORT and PRISMA.

View Article: PubMed Central - PubMed

Affiliation: School of Social Welfare, University of California, Berkeley, California, United States of America. gambrill@berkeley.edu

ABSTRACT

Objective: To determine the effectiveness of an index in increasing recognition of misleading problem framing in articles and manuscripts.

Design: A propaganda index consisting of 32 items was developed drawing on related literature. Seventeen subjects who review manuscripts for possible publication were requested to read five recent published reports of randomized controlled trials concerning social anxiety and to identify indicators of propaganda (defined as encouraging beliefs and actions with the least thought possible). They then re-read the same five articles using a propaganda index to note instances of propaganda.

Data source: Convenience sample of individuals who review manuscripts for possible publication and sample of recent published reports of randomized controlled trials regarding social anxiety in five different journals by different authors, blinded by author and journal.

Results: Data showed that there was a high rate of propagandistic problem framing in reports of RCTs regarding social anxiety such as hiding well argued alternative views and vagueness. This occurred in 117 out of 160 opportunities over five research reports. A convenience sample of 17 academics spotted only 4.5 percent of propaganda indicators. This increased to 64 percent with use of the 32 item propaganda index. Use of a propaganda index increased recognition of related indicators. However many instances remained undetected.

Conclusion: This propaganda index warrants further exploration as a complement to reporting guidelines such as CONSORT and PRISMA.

Show MeSH

Related in: MedlinePlus

Propaganda detection before/after using the P.I.
© Copyright Policy
Related In: Results  -  Collection


getmorefigures.php?uid=PMC3102054&req=5

pone-0019516-g003: Propaganda detection before/after using the P.I.

Mentions: The Master P.I. was used to determine the number of opportunities to spot propaganda across the five articles. All five RCT's reflected hiding of controversies regarding problem framing, failure to recognize that prevalence is in dispute and claims of significant distress and adverse effects (see Figure 2). The second author independently reviewed the five articles. Inter-rater reliability between the first and second author was .88. Then, the data from the articles submitted by each participant before using the index and after using the index were analyzed to determine the percentage of propaganda detected by participants before and after using the P.I. Results indicate that participants were able to detect propaganda at a higher rate after using the P.I. (see Figure 3). For example, out of a possible 38 propaganda indicators concerning the nature of the problem presented across five RCT's, participants detected an average of 1.5 indicators before using the Propaganda Index, and an average of 21.3 indicators after using the index. Similarly, participants identified an average of 2.4 out of 30 indicators concerning reported prevalence before using the Propaganda Index, and an average of 20 indicators after using the index. Furthermore, before and after using the propaganda index, the dimension of under-diagnosis was most commonly missed by participants. The dimension of under-treated saw the most improvement in detection after using the index, raising the rate of detection by 67% (average detection of 1.3 items out of 5 before the index, and 4.7 items out of 5 after the index). The mean percentage of indicators detected over all five articles before use of the index for the 17 subjects was 4.5 percent. This increased to 64.3 percent following use of the index. Test-retest reliability for subjects was .89 (range .82–.97).


A propaganda index for reviewing problem framing in articles and manuscripts: an exploratory study.

Gambrill E, Reiman A - PLoS ONE (2011)

Propaganda detection before/after using the P.I.
© Copyright Policy
Related In: Results  -  Collection

Show All Figures
getmorefigures.php?uid=PMC3102054&req=5

pone-0019516-g003: Propaganda detection before/after using the P.I.
Mentions: The Master P.I. was used to determine the number of opportunities to spot propaganda across the five articles. All five RCT's reflected hiding of controversies regarding problem framing, failure to recognize that prevalence is in dispute and claims of significant distress and adverse effects (see Figure 2). The second author independently reviewed the five articles. Inter-rater reliability between the first and second author was .88. Then, the data from the articles submitted by each participant before using the index and after using the index were analyzed to determine the percentage of propaganda detected by participants before and after using the P.I. Results indicate that participants were able to detect propaganda at a higher rate after using the P.I. (see Figure 3). For example, out of a possible 38 propaganda indicators concerning the nature of the problem presented across five RCT's, participants detected an average of 1.5 indicators before using the Propaganda Index, and an average of 21.3 indicators after using the index. Similarly, participants identified an average of 2.4 out of 30 indicators concerning reported prevalence before using the Propaganda Index, and an average of 20 indicators after using the index. Furthermore, before and after using the propaganda index, the dimension of under-diagnosis was most commonly missed by participants. The dimension of under-treated saw the most improvement in detection after using the index, raising the rate of detection by 67% (average detection of 1.3 items out of 5 before the index, and 4.7 items out of 5 after the index). The mean percentage of indicators detected over all five articles before use of the index for the 17 subjects was 4.5 percent. This increased to 64.3 percent following use of the index. Test-retest reliability for subjects was .89 (range .82–.97).

Bottom Line: To determine the effectiveness of an index in increasing recognition of misleading problem framing in articles and manuscripts.However many instances remained undetected.This propaganda index warrants further exploration as a complement to reporting guidelines such as CONSORT and PRISMA.

View Article: PubMed Central - PubMed

Affiliation: School of Social Welfare, University of California, Berkeley, California, United States of America. gambrill@berkeley.edu

ABSTRACT

Objective: To determine the effectiveness of an index in increasing recognition of misleading problem framing in articles and manuscripts.

Design: A propaganda index consisting of 32 items was developed drawing on related literature. Seventeen subjects who review manuscripts for possible publication were requested to read five recent published reports of randomized controlled trials concerning social anxiety and to identify indicators of propaganda (defined as encouraging beliefs and actions with the least thought possible). They then re-read the same five articles using a propaganda index to note instances of propaganda.

Data source: Convenience sample of individuals who review manuscripts for possible publication and sample of recent published reports of randomized controlled trials regarding social anxiety in five different journals by different authors, blinded by author and journal.

Results: Data showed that there was a high rate of propagandistic problem framing in reports of RCTs regarding social anxiety such as hiding well argued alternative views and vagueness. This occurred in 117 out of 160 opportunities over five research reports. A convenience sample of 17 academics spotted only 4.5 percent of propaganda indicators. This increased to 64 percent with use of the 32 item propaganda index. Use of a propaganda index increased recognition of related indicators. However many instances remained undetected.

Conclusion: This propaganda index warrants further exploration as a complement to reporting guidelines such as CONSORT and PRISMA.

Show MeSH
Related in: MedlinePlus