Limits...
Multivariate and repeated measures (MRM): A new toolbox for dependent and multimodal group-level neuroimaging data.

McFarquhar M, McKie S, Emsley R, Suckling J, Elliott R, Williams S - Neuroimage (2016)

Bottom Line: This approach has a number of drawbacks as certain designs and comparisons of interest are either not possible or complex to implement.Comparisons with existing approaches and software packages for dependent group-level neuroimaging data are made.Follow-up of these multimodal models using linear discriminant functions (LDA) is also discussed, with applications to future studies wishing to integrate multiple scanning techniques into investigating populations of interest.

View Article: PubMed Central - PubMed

Affiliation: Neuroscience & Psychiatry Unit, Stopford Building, The University of Manchester, Oxford Road, Manchester M13 9PL, UK. Electronic address: martyn.mcfarquhar@manchester.ac.uk.

No MeSH data available.


Related in: MedlinePlus

Comparisons between the four different multivariate test statistics for a non-exact multivariate test (a) thresholded at p p-value approximations and (b) thresholded at pp-values derived from 5,000 permutation tests. Results are presented as p-values transformed using –log10. The numbers above each image indicate the number of voxels that survive thresholding.
© Copyright Policy - CC BY
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4862963&req=5

f0050: Comparisons between the four different multivariate test statistics for a non-exact multivariate test (a) thresholded at p p-value approximations and (b) thresholded at pp-values derived from 5,000 permutation tests. Results are presented as p-values transformed using –log10. The numbers above each image indicate the number of voxels that survive thresholding.

Mentions: As indicated earlier, when using the multivariate GLM, there is a choice of four potential test statistics. Such a choice provides greater complexity to the use of the multivariate GLM in neuroimaging when using contrasts that produce non-exact F values. Though these tests have been compared numerous times in the statistical literature (Ito, 1962, Lee, 1971, Mikhail, 1965, Olson, 1974, Pillai and Jayachandran, 1967), we sought to briefly investigate their behaviour when applied to real neuroimaging data. To do this, we used the C matrix from the main effect of condition contrast detailed earlier with A = Ik. We compared both the approximate p-values associated with the different test statistics as well as the p-values derived from 5000 permutations. Fig. 10a shows the results for the classical p-value approximations, with the test statistics displayed from most conservative to least conservative. Here, the nature of Roy's largest root as an upper-bound on the F-value is clear. Pillai's trace, Wilks' lambda, and the Hotelling–Lawley trace are all similar, with the Hotelling–Lawley trace the most liberal, and Pillai's trace the most conservative. These results agree with previous recommendations suggesting Pillai's trace is the safest test to use as it provides the best control over type I errors. These results also suggest that the F approximation to Roy's largest root should generally be avoided unless there is good reason to only consider the upper-bound. In Fig. 10b, we present the same comparisons thresholded using p-values derived from permutation testing. Because we only ran 5000 re-shuffles the largest possible value in the map is -log101/5000 = 3.70. What is noticeable is that for Roy's largest root, the pattern of results is much more in keeping with the activation maps found for the other test statistics. The permutation approach therefore appears to converge the behaviour of the test statistics as under permutation the p-values of Roy's largest root no longer represent an upper-bound, rather they more closely reflect the true F. In addition, it is interesting to note that in this example, Wilks' lambda appears the most consistent between the approximate and permutation-based p-values. This suggests that, although not necessarily generalisable to every dataset and contrast, when using permutation approaches, the differences between the test statistics may be less of a concern and the choice can be driven more by the computational considerations discussed earlier.


Multivariate and repeated measures (MRM): A new toolbox for dependent and multimodal group-level neuroimaging data.

McFarquhar M, McKie S, Emsley R, Suckling J, Elliott R, Williams S - Neuroimage (2016)

Comparisons between the four different multivariate test statistics for a non-exact multivariate test (a) thresholded at p p-value approximations and (b) thresholded at pp-values derived from 5,000 permutation tests. Results are presented as p-values transformed using –log10. The numbers above each image indicate the number of voxels that survive thresholding.
© Copyright Policy - CC BY
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4862963&req=5

f0050: Comparisons between the four different multivariate test statistics for a non-exact multivariate test (a) thresholded at p p-value approximations and (b) thresholded at pp-values derived from 5,000 permutation tests. Results are presented as p-values transformed using –log10. The numbers above each image indicate the number of voxels that survive thresholding.
Mentions: As indicated earlier, when using the multivariate GLM, there is a choice of four potential test statistics. Such a choice provides greater complexity to the use of the multivariate GLM in neuroimaging when using contrasts that produce non-exact F values. Though these tests have been compared numerous times in the statistical literature (Ito, 1962, Lee, 1971, Mikhail, 1965, Olson, 1974, Pillai and Jayachandran, 1967), we sought to briefly investigate their behaviour when applied to real neuroimaging data. To do this, we used the C matrix from the main effect of condition contrast detailed earlier with A = Ik. We compared both the approximate p-values associated with the different test statistics as well as the p-values derived from 5000 permutations. Fig. 10a shows the results for the classical p-value approximations, with the test statistics displayed from most conservative to least conservative. Here, the nature of Roy's largest root as an upper-bound on the F-value is clear. Pillai's trace, Wilks' lambda, and the Hotelling–Lawley trace are all similar, with the Hotelling–Lawley trace the most liberal, and Pillai's trace the most conservative. These results agree with previous recommendations suggesting Pillai's trace is the safest test to use as it provides the best control over type I errors. These results also suggest that the F approximation to Roy's largest root should generally be avoided unless there is good reason to only consider the upper-bound. In Fig. 10b, we present the same comparisons thresholded using p-values derived from permutation testing. Because we only ran 5000 re-shuffles the largest possible value in the map is -log101/5000 = 3.70. What is noticeable is that for Roy's largest root, the pattern of results is much more in keeping with the activation maps found for the other test statistics. The permutation approach therefore appears to converge the behaviour of the test statistics as under permutation the p-values of Roy's largest root no longer represent an upper-bound, rather they more closely reflect the true F. In addition, it is interesting to note that in this example, Wilks' lambda appears the most consistent between the approximate and permutation-based p-values. This suggests that, although not necessarily generalisable to every dataset and contrast, when using permutation approaches, the differences between the test statistics may be less of a concern and the choice can be driven more by the computational considerations discussed earlier.

Bottom Line: This approach has a number of drawbacks as certain designs and comparisons of interest are either not possible or complex to implement.Comparisons with existing approaches and software packages for dependent group-level neuroimaging data are made.Follow-up of these multimodal models using linear discriminant functions (LDA) is also discussed, with applications to future studies wishing to integrate multiple scanning techniques into investigating populations of interest.

View Article: PubMed Central - PubMed

Affiliation: Neuroscience & Psychiatry Unit, Stopford Building, The University of Manchester, Oxford Road, Manchester M13 9PL, UK. Electronic address: martyn.mcfarquhar@manchester.ac.uk.

No MeSH data available.


Related in: MedlinePlus