Limits...
Can Simple Transmission Chains Foster Collective Intelligence in Binary-Choice Tasks?

View Article: PubMed Central - PubMed

ABSTRACT

In many social systems, groups of individuals can find remarkably efficient solutions to complex cognitive problems, sometimes even outperforming a single expert. The success of the group, however, crucially depends on how the judgments of the group members are aggregated to produce the collective answer. A large variety of such aggregation methods have been described in the literature, such as averaging the independent judgments, relying on the majority or setting up a group discussion. In the present work, we introduce a novel approach for aggregating judgments—the transmission chain—which has not yet been consistently evaluated in the context of collective intelligence. In a transmission chain, all group members have access to a unique collective solution and can improve it sequentially. Over repeated improvements, the collective solution that emerges reflects the judgments of every group members. We address the question of whether such a transmission chain can foster collective intelligence for binary-choice problems. In a series of numerical simulations, we explore the impact of various factors on the performance of the transmission chain, such as the group size, the model parameters, and the structure of the population. The performance of this method is compared to those of the majority rule and the confidence-weighted majority. Finally, we rely on two existing datasets of individuals performing a series of binary decisions to evaluate the expected performances of the three methods empirically. We find that the parameter space where the transmission chain has the best performance rarely appears in real datasets. We conclude that the transmission chain is best suited for other types of problems, such as those that have cumulative properties.

No MeSH data available.


Related in: MedlinePlus

Impact of the contribution threshold τ.The red line indicates the probability that the chain generates a correct solution for different values of the contribution threshold τ, assuming a group size of N = 10, a proportion of correct answers q1 = 0.6 and the confidence distributions shown in Fig 1. In these conditions, the optimal value for the contribution threshold is τ = 0.83, for which the chain produces the correct solution 93% of the time. The grey and blue lines indicate the success chances of the majority and the weighted-majority rules, respectively.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC5120860&req=5

pone.0167223.g003: Impact of the contribution threshold τ.The red line indicates the probability that the chain generates a correct solution for different values of the contribution threshold τ, assuming a group size of N = 10, a proportion of correct answers q1 = 0.6 and the confidence distributions shown in Fig 1. In these conditions, the optimal value for the contribution threshold is τ = 0.83, for which the chain produces the correct solution 93% of the time. The grey and blue lines indicate the success chances of the majority and the weighted-majority rules, respectively.

Mentions: The second specificity of the chain is the presence of social influence. Unlike the two other methods, the chain does not aggregate the independent answers of every group member. Instead, it filters out the answers of those who are confident enough to change the collective solution [49]. The contribution threshold τ is thus an important parameter that determines which individuals are confident enough to contribute to the final solution. What is the impact of the contribution threshold τ? To address this question, we explored the performances of the transmission chain while varying the contribution threshold from τ = 0 to τ = 1. For each value of τ, we generated 1000 groups of size N = 10 with q1 = 0.6 and the confidence distributions shown in Fig 1, and measured the success chance C of the chain (i.e., how often it produced a correct answer). The result is shown in Fig 3. When the contribution threshold is small and approaches τ = 0, every group member is confident enough to contribute. In this case, everyone overrides the previous person’s solution and the outcome of the chain is simply the answer of the last individual of the chain. Because every individual has a probability q1 = 0.6 of being correct, the success chance of the chain is also C = 0.6. Likewise, when the activity threshold approaches τ = 1, none of the group members are confident enough to contribute. In this case, the answer of the first individual of the chain remains unchanged until the end of the chain. Because the first individual has a probability q1 = 0.6 of providing a correct answer, success chance of the chain is also C = 0.6. Between these two extreme values, the performance of the chain reaches a peak for τ = 0.83. At this point, the success chance of the chain is C = 0.93 (i.e. the chain yields a correct answer 93% of the time). In fact, the optimal threshold value maximizes the probability to pick a positive contributor, while at the same time minimizes the probability to pick a negative contributor. Note that this performance measure takes into account the order effect because groups are randomly ordered in each replication. For comparison, the success chance of the majority rule is M = 0.73 under these conditions, and the success chance of the weighted-majority is W = 0.90.


Can Simple Transmission Chains Foster Collective Intelligence in Binary-Choice Tasks?
Impact of the contribution threshold τ.The red line indicates the probability that the chain generates a correct solution for different values of the contribution threshold τ, assuming a group size of N = 10, a proportion of correct answers q1 = 0.6 and the confidence distributions shown in Fig 1. In these conditions, the optimal value for the contribution threshold is τ = 0.83, for which the chain produces the correct solution 93% of the time. The grey and blue lines indicate the success chances of the majority and the weighted-majority rules, respectively.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC5120860&req=5

pone.0167223.g003: Impact of the contribution threshold τ.The red line indicates the probability that the chain generates a correct solution for different values of the contribution threshold τ, assuming a group size of N = 10, a proportion of correct answers q1 = 0.6 and the confidence distributions shown in Fig 1. In these conditions, the optimal value for the contribution threshold is τ = 0.83, for which the chain produces the correct solution 93% of the time. The grey and blue lines indicate the success chances of the majority and the weighted-majority rules, respectively.
Mentions: The second specificity of the chain is the presence of social influence. Unlike the two other methods, the chain does not aggregate the independent answers of every group member. Instead, it filters out the answers of those who are confident enough to change the collective solution [49]. The contribution threshold τ is thus an important parameter that determines which individuals are confident enough to contribute to the final solution. What is the impact of the contribution threshold τ? To address this question, we explored the performances of the transmission chain while varying the contribution threshold from τ = 0 to τ = 1. For each value of τ, we generated 1000 groups of size N = 10 with q1 = 0.6 and the confidence distributions shown in Fig 1, and measured the success chance C of the chain (i.e., how often it produced a correct answer). The result is shown in Fig 3. When the contribution threshold is small and approaches τ = 0, every group member is confident enough to contribute. In this case, everyone overrides the previous person’s solution and the outcome of the chain is simply the answer of the last individual of the chain. Because every individual has a probability q1 = 0.6 of being correct, the success chance of the chain is also C = 0.6. Likewise, when the activity threshold approaches τ = 1, none of the group members are confident enough to contribute. In this case, the answer of the first individual of the chain remains unchanged until the end of the chain. Because the first individual has a probability q1 = 0.6 of providing a correct answer, success chance of the chain is also C = 0.6. Between these two extreme values, the performance of the chain reaches a peak for τ = 0.83. At this point, the success chance of the chain is C = 0.93 (i.e. the chain yields a correct answer 93% of the time). In fact, the optimal threshold value maximizes the probability to pick a positive contributor, while at the same time minimizes the probability to pick a negative contributor. Note that this performance measure takes into account the order effect because groups are randomly ordered in each replication. For comparison, the success chance of the majority rule is M = 0.73 under these conditions, and the success chance of the weighted-majority is W = 0.90.

View Article: PubMed Central - PubMed

ABSTRACT

In many social systems, groups of individuals can find remarkably efficient solutions to complex cognitive problems, sometimes even outperforming a single expert. The success of the group, however, crucially depends on how the judgments of the group members are aggregated to produce the collective answer. A large variety of such aggregation methods have been described in the literature, such as averaging the independent judgments, relying on the majority or setting up a group discussion. In the present work, we introduce a novel approach for aggregating judgments—the transmission chain—which has not yet been consistently evaluated in the context of collective intelligence. In a transmission chain, all group members have access to a unique collective solution and can improve it sequentially. Over repeated improvements, the collective solution that emerges reflects the judgments of every group members. We address the question of whether such a transmission chain can foster collective intelligence for binary-choice problems. In a series of numerical simulations, we explore the impact of various factors on the performance of the transmission chain, such as the group size, the model parameters, and the structure of the population. The performance of this method is compared to those of the majority rule and the confidence-weighted majority. Finally, we rely on two existing datasets of individuals performing a series of binary decisions to evaluate the expected performances of the three methods empirically. We find that the parameter space where the transmission chain has the best performance rarely appears in real datasets. We conclude that the transmission chain is best suited for other types of problems, such as those that have cumulative properties.

No MeSH data available.


Related in: MedlinePlus