Limits...
The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration.

Baranes AF, Oudeyer PY, Gottlieb J - Front Neurosci (2014)

Bottom Line: Adult subjects played a series of short computer games of variable difficulty, and freely choose which game they wished to sample without external guidance or physical rewards.Subjects performed the task in three distinct conditions where they sampled from a small or a large choice set (7 vs. 64 possible levels of difficulty), and where they did or did not have the possibility to sample new games at a constant level of difficulty.The results suggest that intrinsically motivated exploration is shaped by several factors including task difficulty, novelty and the size of the choice set, and these come into play to serve two internal goals-maximize the subjects' knowledge of the available tasks (exploring the limits of the task set), and maximize their competence (performance and skills) across the task set.

View Article: PubMed Central - PubMed

Affiliation: Department of Neuroscience, Columbia University Medical Center New York, NY, USA.

ABSTRACT
Devising efficient strategies for exploration in large open-ended spaces is one of the most difficult computational problems of intelligent organisms. Because the available rewards are ambiguous or unknown during the exploratory phase, subjects must act in intrinsically motivated fashion. However, a vast majority of behavioral and neural studies to date have focused on decision making in reward-based tasks, and the rules guiding intrinsically motivated exploration remain largely unknown. To examine this question we developed a paradigm for systematically testing the choices of human observers in a free play context. Adult subjects played a series of short computer games of variable difficulty, and freely choose which game they wished to sample without external guidance or physical rewards. Subjects performed the task in three distinct conditions where they sampled from a small or a large choice set (7 vs. 64 possible levels of difficulty), and where they did or did not have the possibility to sample new games at a constant level of difficulty. We show that despite the absence of external constraints, the subjects spontaneously adopted a structured exploration strategy whereby they (1) started with easier games and progressed to more difficult games, (2) sampled the entire choice set including extremely difficult games that could not be learnt, (3) repeated moderately and high difficulty games much more frequently than was predicted by chance, and (4) had higher repetition rates and chose higher speeds if they could generate new sequences at a constant level of difficulty. The results suggest that intrinsically motivated exploration is shaped by several factors including task difficulty, novelty and the size of the choice set, and these come into play to serve two internal goals-maximize the subjects' knowledge of the available tasks (exploring the limits of the task set), and maximize their competence (performance and skills) across the task set.

No MeSH data available.


Related in: MedlinePlus

General performance. (A) Performance as a function of speed in the 3 versions. Each bin represents the average and standard error (s.e.m.) of the fraction correct for the corresponding dot speed across all the subjects tested. (B) The distribution of performance levels in the 7-game condition. The points show the average and s.e.m. (across subjects) of the number of games in each of 6 performance bins.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4196545&req=5

Figure 2: General performance. (A) Performance as a function of speed in the 3 versions. Each bin represents the average and standard error (s.e.m.) of the fraction correct for the corresponding dot speed across all the subjects tested. (B) The distribution of performance levels in the 7-game condition. The points show the average and s.e.m. (across subjects) of the number of games in each of 6 performance bins.

Mentions: As shown in Figure 2A, the 7-game version afforded fewer options and a smaller range of dot speeds relative to the 64- and 64N- versions (6°/s–29°/s vs. 1°/s–75°/s). However, games of equivalent speed elicited equivalent performance in the three versions (p = 0.72 for effect of task version, p < 0.05 for effect of speed, Two-Way ANOVA for speeds of 6°/s–29°/s). In addition, the average fraction correct on the 7-game version did not fall below 0.4, but the games that the subjects played in this condition spanned the entire performance range (Figure 2B). Thus, the subjects' abilities and performance ranges were matched across the 3 task versions.


The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration.

Baranes AF, Oudeyer PY, Gottlieb J - Front Neurosci (2014)

General performance. (A) Performance as a function of speed in the 3 versions. Each bin represents the average and standard error (s.e.m.) of the fraction correct for the corresponding dot speed across all the subjects tested. (B) The distribution of performance levels in the 7-game condition. The points show the average and s.e.m. (across subjects) of the number of games in each of 6 performance bins.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4196545&req=5

Figure 2: General performance. (A) Performance as a function of speed in the 3 versions. Each bin represents the average and standard error (s.e.m.) of the fraction correct for the corresponding dot speed across all the subjects tested. (B) The distribution of performance levels in the 7-game condition. The points show the average and s.e.m. (across subjects) of the number of games in each of 6 performance bins.
Mentions: As shown in Figure 2A, the 7-game version afforded fewer options and a smaller range of dot speeds relative to the 64- and 64N- versions (6°/s–29°/s vs. 1°/s–75°/s). However, games of equivalent speed elicited equivalent performance in the three versions (p = 0.72 for effect of task version, p < 0.05 for effect of speed, Two-Way ANOVA for speeds of 6°/s–29°/s). In addition, the average fraction correct on the 7-game version did not fall below 0.4, but the games that the subjects played in this condition spanned the entire performance range (Figure 2B). Thus, the subjects' abilities and performance ranges were matched across the 3 task versions.

Bottom Line: Adult subjects played a series of short computer games of variable difficulty, and freely choose which game they wished to sample without external guidance or physical rewards.Subjects performed the task in three distinct conditions where they sampled from a small or a large choice set (7 vs. 64 possible levels of difficulty), and where they did or did not have the possibility to sample new games at a constant level of difficulty.The results suggest that intrinsically motivated exploration is shaped by several factors including task difficulty, novelty and the size of the choice set, and these come into play to serve two internal goals-maximize the subjects' knowledge of the available tasks (exploring the limits of the task set), and maximize their competence (performance and skills) across the task set.

View Article: PubMed Central - PubMed

Affiliation: Department of Neuroscience, Columbia University Medical Center New York, NY, USA.

ABSTRACT
Devising efficient strategies for exploration in large open-ended spaces is one of the most difficult computational problems of intelligent organisms. Because the available rewards are ambiguous or unknown during the exploratory phase, subjects must act in intrinsically motivated fashion. However, a vast majority of behavioral and neural studies to date have focused on decision making in reward-based tasks, and the rules guiding intrinsically motivated exploration remain largely unknown. To examine this question we developed a paradigm for systematically testing the choices of human observers in a free play context. Adult subjects played a series of short computer games of variable difficulty, and freely choose which game they wished to sample without external guidance or physical rewards. Subjects performed the task in three distinct conditions where they sampled from a small or a large choice set (7 vs. 64 possible levels of difficulty), and where they did or did not have the possibility to sample new games at a constant level of difficulty. We show that despite the absence of external constraints, the subjects spontaneously adopted a structured exploration strategy whereby they (1) started with easier games and progressed to more difficult games, (2) sampled the entire choice set including extremely difficult games that could not be learnt, (3) repeated moderately and high difficulty games much more frequently than was predicted by chance, and (4) had higher repetition rates and chose higher speeds if they could generate new sequences at a constant level of difficulty. The results suggest that intrinsically motivated exploration is shaped by several factors including task difficulty, novelty and the size of the choice set, and these come into play to serve two internal goals-maximize the subjects' knowledge of the available tasks (exploring the limits of the task set), and maximize their competence (performance and skills) across the task set.

No MeSH data available.


Related in: MedlinePlus