Thumbnail 935x561

Clustering Brain Activation to Study Social Cognition

We performed a meta-analysis and clustered neuroimaging data from 4207 participants. Results provide a roadmap for dissecting human social cognition into more elementary and neurobiologically grounded processes.

Over the last 20 years, thousands of studies have tried to capture social abilities, and came up with many ingenious ways to measure their basis in the human brain. In a recent study, we performed a meta-analysis of some of these results, including data from 4207 participants. The initial picture emerging from our review suggested quite some heterogeneity, and so we decided to explore a data-driven approach for sorting it out. Specifically, we wanted to see if findings can be clustered into different patterns of brain activation.

To implement the idea of a cluster analysis, we first had to decide how brain activation maps are compared and grouped together. We wanted to capture the pattern of activity across the whole brain (rather for a specific brain region of interest) when characterizing the similarity between maps. In many recent works, this is determined based on multi-voxel pattern similarity, which is commonly calculated as Pearson image correlation. However, several questions arose at this point for us. First, should image similarity be determined for brain activation maps that have been statistically thresholded, and thus only contain the most reliable information about brain activation? Second, should we use a parametric or a non-parametric measure for image correlation, that is, Pearson's or Spearman's correlation coefficients?

To find the answers to these questions, we determined how well different approaches can distinguish between meta-analysis maps derived from studies belonging to the same task group (and therefore sampling very similar cognitive states), versus maps from studies from different task groups. Good discrimination performance is evident in the case of high image correlation between the former maps (same task group), and low correlation between the latter. 

Concretely, we picked two different task groups from our meta-analysis, and divided each into two random halves. We then ran separate meta-analyses for these 4 sub-samples, and calculated pairwise correlations between sub-samples drawn from either the same or from different task groups. We repeated this procedure 100 times with new random-halves (i.e. running 400 new meta-analyses), and compared correlation results for different parameterizations and thresholdings. In particular, we compared three types of maps:

  • Unthresholded Hedges' g maps (representing the effect size of brain activation)
  • Unthresholded SDM z maps (weighed effect sizes accounting for inter-study heterogeneity)
  • Thresholded SDM z maps (where non-statistically significant voxels had null value) 

Moreover, we compared two correlation methods (i) pearson and (ii) spearman correlation. The results are shown in Figure 1, where correlation values are z-transformed. As you can see, the largest differences in correlation strength between same and different group data is found for unthresholded Hedges' g maps. We could also corroborate this observation by running a repeated measures ANOVA on the z-transformed correlation coefficients.


Bar chart showing Pearson correlation and Spearman rank correlation

Figure 1
Legend: z-transformed image correlation values (Mean, SD) found for the different methods we tested. The to be correlated images were either subsamples from the same task group or drawn from two different task groups. Bar colors indicate which type of meta-analysis map was used.


Clustering brain activation maps into common networks

Having found a well suited measure for whole brain image similarity (Hedges' g maps), we determined image dissimilarity (1-r) between a range of meta-analyses for different social tasks, as shown in Figure 2A. Based on this image dissimilarity matrix, it was easy to then carry out a cluster analysis. We decided to use hierarchical agglomerative clustering, and Figure 2B shows the initial dendrogram illustrating task-by-task and cluster-by-cluster relations.

Graph showing A) Dissimilarity between meta-analytics maps and B) Dendrogram from custering

Figure 2
Legend: (A) Degree of dissimilarity between meta-analytic result maps (given as 1-Pearson’s r). High dissimilarity reflects little correspondence or overlap between brain maps. (B) Dendrogram from hierarchical clustering. The height of the branches indicates cophenetic distances and thus the dissimilarity between subclusters at that model order.

Based on the information shown in the dendrogram, we selected an optimal clustering based on two features, which are shown in Figure 3. First, our desired model should ensure a good separation of brain activation patterns between clusters. The left Y-axis in Figure 3 (red) gives the relative difference in cophenetic distances when moving from one model to the next. A relatively high difference in cophenetic distances indicates that introducing new subclusters results in a better separation of brain activity patterns and thus good clustering. Second, the components of a good model should be sufficiently abstract to generalize across concrete instances, that is, multiple task groups, rather than picking up variance related to one outlier task group. The plot in Figure 3 shows on the right Y-axis and in blue the density of task separation, reflecting whether separating clusters into subclusters maintains a balanced number of task groups in each subcluster. A relatively low density of task separation indicates good clustering. Metrics in Figure 3 suggest that a good clustering shows up on two levels: On the one hand, the best overall clustering performance was reached when the data were divided into a three-cluster solution. However, further local peaks in performance were found when the data were split into eight and eleven clusters.


Graph showing clustering metrics

Figure 3
Legend: Evaluation and comparison of different clustering solutions. The plot shows changes in two metrics when moving from 2- to 3-, 3- to 4-, . . . , and 10- to 11-cluster solutions. Preferred clusterings in terms of both metrics are indicated on the X-axis: 3, 8, and 11. Metric changes are shown for the clustering of complete meta-analyses (main analysis) and clusterings based on leave-one-out jackknife sensitivity analysis with 5,000 repeats (jackknife mean, standard deviation).

The whole structure of our proposed multi-level model is shown below in Figure 4. We carried out pooled meta-analyses, i.e., one individual meta-analysis for each cluster, where all its task groups were joined together. The colors show how the 3-cluster solution relates to both higher- and lower-level clusterings (blue – cognitive cluster, green – intermediate cluster, red – affective cluster). At the lowest level of the dendrogram, we provide for each cluster some exemplary stimulus and task categorizations.

Graph showing brain activation for clusters at different model orders

Figure 4
Legend: Brain activation for clusters at different model orders. Analyses were thresholded at a voxel-wise threshold of p < .005 uncorrected and a cluster extent threshold of 10 voxels.


The higher-level, three-cluster solution provides a solid foundation of evidence for the assumption that different social cognition tasks share certain processes, and therefore brain activity. However, the concurrent existence of an additional lower level of clustering (essentially by task) highlights the question of the appropriate level of concreteness and detail in neurocognitive accounts of social cognition. We think these results speak in favor of modeling social cognition as a multilevel construct, similar to models from other psychological fields such as intelligence research. A well-known example for this is the concept of “general intelligence”. Usually, it is based upon a latent variable of a model, and reflects a broader, more abstract neurocognitive function. It is assumed that this function is played out differently when solving various problems. Similarly, our results found a high-level clustering that points out two overarching networks, which mediate more sensory-affective versus more abstract and decoupled representations of others’ mental states.

What we conclude from our study is that although previous theories of social abilities were good at explaining how specific sub-sets of skills are implemented in the brain, they did not consider how different components relate to each other. Our results present a way to make this possible, which is capturing social cognition in terms of a multilevel model.



Schurz, M., Radua, J., Tholen, M. G., Maliske, L., Margulies, D. S., Mars, R. B., ... & Kanske, P. (2021). Toward a hierarchical model of social cognition: A neuroimaging meta-analysis and integrative review of empathy and theory of mind. Psychological Bulletin, 147(3), 293-327.




This work is licensed under the Creative Commons Attribution 4.0 International License.

Creative Commons License

Portrait of Matthias Schurz 
Credit: Matthias Schurz

Written by Matthias Schurz in February 2022

Assistant Professor at DiSC & Department of Psychology

University of Innsbruck

About the author

Humans are a highly social species, and complex societies rely on mutual understanding, cooperation, and trust. In my work, I am studying how central social abilities are implemented in the human brain. In particular, I would like to understand how we can represent another's perspective while at the same time keeping track of reality. Comparative work from other species as well as patient studies show us that this high-level skill is not easy for the brain. Presumably, it requires the interplay of several large-scale networks. To deepen our understanding of these mechanisms, I study functional brain activity and connectivity with statistical and machine learning methods.

Research area

Computational Neuroscience, Brain Networks, and Human Cognition

Nach oben scrollen