Limits...
Evaluating more naturalistic outcome measures: A 1-year smartphone study in multiple sclerosis.

Bove R, White CC, Giovannoni G, Glanz B, Golubchikov V, Hujol J, Jennings C, Langdon D, Lee M, Legedza A, Paskavitz J, Prasad S, Richert J, Robbins A, Roberts S, Weiner H, Ramachandran R, Botfield M, De Jager PL - Neurol Neuroimmunol Neuroinflamm (2015)

Bottom Line: Among patients with MS, low scores on PROs relating to mental and visual function were associated with dropout (p < 0.05).Finally, averaging repeated measures over the study yielded the most robust correlation matrix of the different outcome measures.A smartphone platform may therefore enable large-scale naturalistic studies of patients with MS or other neurologic diseases.

View Article: PubMed Central - PubMed

Affiliation: Program in Translational Neuropsychiatric Genomics (R.B., C.C.W., B.G., M.L., S.P., A.R., H.W., P.L.D.J.), Ann Romney Center for Neurologic Diseases, and the Partners Multiple Sclerosis Center, Department of Neurology, Brigham and Women's Hospital, Brookline, MA; Harvard Medical School (R.B., B.G., S.P., H.W., P.L.D.G.), Boston, MA; Blizard Institute (G.G.) and Royal Holloway (D.L.), University College London, London, UK; Vertex Pharmaceuticals Incorporated (V.G., A.L., S.R., R.R., M.B.), Boston MA; Woo Sports (J.H.), Boston, MA; McGovern Institute Neurotechnology Program (C.J.), MIT, Cambridge, MA; and Biogen-Idec (J.P., J.R.), Cambridge, MA.

ABSTRACT

Objective: In this cohort of individuals with and without multiple sclerosis (MS), we illustrate some of the novel approaches that smartphones provide to monitor patients with chronic neurologic disorders in their natural setting.

Methods: Thirty-eight participant pairs (MS and cohabitant) aged 18-55 years participated in the study. Each participant received an Android HTC Sensation 4G smartphone containing a custom application suite of 19 tests capturing participant performance and patient-reported outcomes (PROs). Over 1 year, participants were prompted daily to complete one assigned test.

Results: A total of 22 patients with MS and 17 cohabitants completed the entire study. Among patients with MS, low scores on PROs relating to mental and visual function were associated with dropout (p < 0.05). We illustrate several novel features of a smartphone platform. First, fluctuations in MS outcomes (e.g., fatigue) were assessed against an individual's ambient environment by linking responses to meteorological data. Second, both response accuracy and speed for the Ishihara color vision test were captured, highlighting the benefits of both active and passive data collection. Third, a new trait, a person-specific learning curve in neuropsychological testing, was identified using spline analysis. Finally, averaging repeated measures over the study yielded the most robust correlation matrix of the different outcome measures.

Conclusions: We report the feasibility of, and barriers to, deploying a smartphone platform to gather useful passive and active performance data at high frequency in an unstructured manner in the field. A smartphone platform may therefore enable large-scale naturalistic studies of patients with MS or other neurologic diseases.

No MeSH data available.


Related in: MedlinePlus

Illustration of a practice effect by examining longitudinal performance measures in patients with MS and cohabitants(A) Response time for each trial of the Trails A test performed by a participant pair (patient with multiple sclerosis [MS] and cohabitant) over the course of the study. Each point represents one trial. (B) Comparison of the first, last, and mean values for Trails A in study completers (n = 39). The last value appears to more closely match the mean than the first value does. (C, D) For each Trails A trial, a boxplot of the mean score for all MS (n = 22, C) and cohabitant completers (n = 17, D) is shown. These plots illustrate a gradual decrease in mean scores for all individuals over the duration of the study, with narrowing of the variance as the practice effect wanes. (E, F) Results of the spline analysis for Trails A reveal a longer practice effect, on average, in patients with MS (blue line) than in cohabitants (red line) (E). A similar effect is noted for responses on the 9-Hole Peg Test (F). (G) Illustration of the longitudinal performance of a patient with MS on the Trails A test. The individual scores (black dashes) appear to improve over time. The black solid line represents the inflection point analysis. After 7 trials, the practice effect appears to taper off. The location of the inflection point is determined by finding the maximal R-squared value (red line), which peaks around 7 trials for this individual.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4608760&req=5

Figure 4: Illustration of a practice effect by examining longitudinal performance measures in patients with MS and cohabitants(A) Response time for each trial of the Trails A test performed by a participant pair (patient with multiple sclerosis [MS] and cohabitant) over the course of the study. Each point represents one trial. (B) Comparison of the first, last, and mean values for Trails A in study completers (n = 39). The last value appears to more closely match the mean than the first value does. (C, D) For each Trails A trial, a boxplot of the mean score for all MS (n = 22, C) and cohabitant completers (n = 17, D) is shown. These plots illustrate a gradual decrease in mean scores for all individuals over the duration of the study, with narrowing of the variance as the practice effect wanes. (E, F) Results of the spline analysis for Trails A reveal a longer practice effect, on average, in patients with MS (blue line) than in cohabitants (red line) (E). A similar effect is noted for responses on the 9-Hole Peg Test (F). (G) Illustration of the longitudinal performance of a patient with MS on the Trails A test. The individual scores (black dashes) appear to improve over time. The black solid line represents the inflection point analysis. After 7 trials, the practice effect appears to taper off. The location of the inflection point is determined by finding the maximal R-squared value (red line), which peaks around 7 trials for this individual.

Mentions: As with traditional performance tests, we observed an apparent practice effect across all performance tests (Trails A, Trails B, Ishihara, n-back, 9-Hole Peg Test; appendix e-1 and figure e-3). As illustrated by Trails A (figure 4), the mean performance score across the 26 or more times that study completers took this test was closer to the last score recorded than to the first score (figure 4B). Although present in cohabitants, the practice effect was more pronounced in patients with MS (figure 4, C and D). Figure 4A illustrates this trend at the level of one pair. To more rigorously identify the point at which scores stop improving and variance narrows, we applied a spline analysis in the context of a repeated-measures regression and identified the optimal inflection point. In Trails A, we found that the model fit best (as determined by the Akaike information criterion) with an inflection point at 8 trials for patients with MS and 2 trials for cohabitants (figure 4E). Similar inflection points were identified in the 9-Hole Peg test, where optimal inflection points for patients with MS and cohabitants were 7 and 2, respectively (figure 4F). At the participant level, we propose the location of the inflection point as a new outcome measure for MS that relates to an individual's ability to learn a new task (figure 4G). In the current study, almost all study completers attained a plateau after 10 trials (appendix e-1 and figure e-4).


Evaluating more naturalistic outcome measures: A 1-year smartphone study in multiple sclerosis.

Bove R, White CC, Giovannoni G, Glanz B, Golubchikov V, Hujol J, Jennings C, Langdon D, Lee M, Legedza A, Paskavitz J, Prasad S, Richert J, Robbins A, Roberts S, Weiner H, Ramachandran R, Botfield M, De Jager PL - Neurol Neuroimmunol Neuroinflamm (2015)

Illustration of a practice effect by examining longitudinal performance measures in patients with MS and cohabitants(A) Response time for each trial of the Trails A test performed by a participant pair (patient with multiple sclerosis [MS] and cohabitant) over the course of the study. Each point represents one trial. (B) Comparison of the first, last, and mean values for Trails A in study completers (n = 39). The last value appears to more closely match the mean than the first value does. (C, D) For each Trails A trial, a boxplot of the mean score for all MS (n = 22, C) and cohabitant completers (n = 17, D) is shown. These plots illustrate a gradual decrease in mean scores for all individuals over the duration of the study, with narrowing of the variance as the practice effect wanes. (E, F) Results of the spline analysis for Trails A reveal a longer practice effect, on average, in patients with MS (blue line) than in cohabitants (red line) (E). A similar effect is noted for responses on the 9-Hole Peg Test (F). (G) Illustration of the longitudinal performance of a patient with MS on the Trails A test. The individual scores (black dashes) appear to improve over time. The black solid line represents the inflection point analysis. After 7 trials, the practice effect appears to taper off. The location of the inflection point is determined by finding the maximal R-squared value (red line), which peaks around 7 trials for this individual.
© Copyright Policy - open-access
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4608760&req=5

Figure 4: Illustration of a practice effect by examining longitudinal performance measures in patients with MS and cohabitants(A) Response time for each trial of the Trails A test performed by a participant pair (patient with multiple sclerosis [MS] and cohabitant) over the course of the study. Each point represents one trial. (B) Comparison of the first, last, and mean values for Trails A in study completers (n = 39). The last value appears to more closely match the mean than the first value does. (C, D) For each Trails A trial, a boxplot of the mean score for all MS (n = 22, C) and cohabitant completers (n = 17, D) is shown. These plots illustrate a gradual decrease in mean scores for all individuals over the duration of the study, with narrowing of the variance as the practice effect wanes. (E, F) Results of the spline analysis for Trails A reveal a longer practice effect, on average, in patients with MS (blue line) than in cohabitants (red line) (E). A similar effect is noted for responses on the 9-Hole Peg Test (F). (G) Illustration of the longitudinal performance of a patient with MS on the Trails A test. The individual scores (black dashes) appear to improve over time. The black solid line represents the inflection point analysis. After 7 trials, the practice effect appears to taper off. The location of the inflection point is determined by finding the maximal R-squared value (red line), which peaks around 7 trials for this individual.
Mentions: As with traditional performance tests, we observed an apparent practice effect across all performance tests (Trails A, Trails B, Ishihara, n-back, 9-Hole Peg Test; appendix e-1 and figure e-3). As illustrated by Trails A (figure 4), the mean performance score across the 26 or more times that study completers took this test was closer to the last score recorded than to the first score (figure 4B). Although present in cohabitants, the practice effect was more pronounced in patients with MS (figure 4, C and D). Figure 4A illustrates this trend at the level of one pair. To more rigorously identify the point at which scores stop improving and variance narrows, we applied a spline analysis in the context of a repeated-measures regression and identified the optimal inflection point. In Trails A, we found that the model fit best (as determined by the Akaike information criterion) with an inflection point at 8 trials for patients with MS and 2 trials for cohabitants (figure 4E). Similar inflection points were identified in the 9-Hole Peg test, where optimal inflection points for patients with MS and cohabitants were 7 and 2, respectively (figure 4F). At the participant level, we propose the location of the inflection point as a new outcome measure for MS that relates to an individual's ability to learn a new task (figure 4G). In the current study, almost all study completers attained a plateau after 10 trials (appendix e-1 and figure e-4).

Bottom Line: Among patients with MS, low scores on PROs relating to mental and visual function were associated with dropout (p < 0.05).Finally, averaging repeated measures over the study yielded the most robust correlation matrix of the different outcome measures.A smartphone platform may therefore enable large-scale naturalistic studies of patients with MS or other neurologic diseases.

View Article: PubMed Central - PubMed

Affiliation: Program in Translational Neuropsychiatric Genomics (R.B., C.C.W., B.G., M.L., S.P., A.R., H.W., P.L.D.J.), Ann Romney Center for Neurologic Diseases, and the Partners Multiple Sclerosis Center, Department of Neurology, Brigham and Women's Hospital, Brookline, MA; Harvard Medical School (R.B., B.G., S.P., H.W., P.L.D.G.), Boston, MA; Blizard Institute (G.G.) and Royal Holloway (D.L.), University College London, London, UK; Vertex Pharmaceuticals Incorporated (V.G., A.L., S.R., R.R., M.B.), Boston MA; Woo Sports (J.H.), Boston, MA; McGovern Institute Neurotechnology Program (C.J.), MIT, Cambridge, MA; and Biogen-Idec (J.P., J.R.), Cambridge, MA.

ABSTRACT

Objective: In this cohort of individuals with and without multiple sclerosis (MS), we illustrate some of the novel approaches that smartphones provide to monitor patients with chronic neurologic disorders in their natural setting.

Methods: Thirty-eight participant pairs (MS and cohabitant) aged 18-55 years participated in the study. Each participant received an Android HTC Sensation 4G smartphone containing a custom application suite of 19 tests capturing participant performance and patient-reported outcomes (PROs). Over 1 year, participants were prompted daily to complete one assigned test.

Results: A total of 22 patients with MS and 17 cohabitants completed the entire study. Among patients with MS, low scores on PROs relating to mental and visual function were associated with dropout (p < 0.05). We illustrate several novel features of a smartphone platform. First, fluctuations in MS outcomes (e.g., fatigue) were assessed against an individual's ambient environment by linking responses to meteorological data. Second, both response accuracy and speed for the Ishihara color vision test were captured, highlighting the benefits of both active and passive data collection. Third, a new trait, a person-specific learning curve in neuropsychological testing, was identified using spline analysis. Finally, averaging repeated measures over the study yielded the most robust correlation matrix of the different outcome measures.

Conclusions: We report the feasibility of, and barriers to, deploying a smartphone platform to gather useful passive and active performance data at high frequency in an unstructured manner in the field. A smartphone platform may therefore enable large-scale naturalistic studies of patients with MS or other neurologic diseases.

No MeSH data available.


Related in: MedlinePlus