Two weeks ago, I had — what my friends tell me — a somewhat weird idea to go over the entirety of my psychology curriculum with the goal to see what stuck, relearn stuff that I found interesting, and perhaps ignite some of the lost passion for the field. I started with the introductory course, located here (for an intro to psychology, I can recommend it), and powered through until I reached the chapter about thinking and intelligence. In it, I stumbled upon the word concept and I realized that I not only don't know what it means (beyond an intuitive understanding), but I also don't know how it differs from the word construct. It turns out, I'm not alone. Many psychologists use these terms interchangeably.
Other psychologists are quick to argue that the previous sort of psychologists (probably the majority, yours truly included) are exactly the reason the entire field has gone to shit through replication failures, crises of theory and generalization, heterogeneity of effects, and so on. Because concepts and constructs are to psychology (probably) what quarks and bosons are to quantum physics (i.e. the building blocks), if the scientists researching them don't even agree on what they are, we have a problem. While concepts and constructs are abstract, vague terms, and you might argue that distinguishing them isn’t easy — I agree with you on that — we seem to struggle even with superficially way easier concepts. Enter: a paper about penguins.
Okay, full disclosure, the paper discusses not only penguins. It'd be cool if it did, but papers are rarely that fun. So before we plunge into the deep waters of semantics the paper unravels, we need to be somewhat clearer on one thing: what are these concepts that I kept babbling about this entire time? Concepts can be understood as general notions stemming from linguistic information, images, ideas, or memories. They are — in my understanding — the big ideas we distill from the particular details we observe, generalizations of specific cases. Concepts help us understand the world in- and around us. For instance, we have the abstract concept of intelligence, which is some sort of quality we ascribe to people. People fall on a spectrum between smart and dumb, and the concept of intelligence somehow captures this reality. We then use the concept of intelligence to, say, understand ourselves, other people and adapt our behavior1.
But concepts can also be more concrete, such as the ones used in the paper. There, the authors chose animals and politicians2 as concepts and tested — in two experiments — whether when, for instance, people read “Obama” or “Ostrich”, they imagine the same entities behind each respective word or not. Let's see what they did and found.
The Paper
What the authors did was the following: they gave each participant one random word (politician or animal), together with 36 unique pairs of other objects from the same domain. For instance, participants got asked: “Which is more similar to a finch, a whale or a penguin?” A person could then choose either a whale or a penguin. Plus, they were asked to guess how many people out of 100 would agree with their assessment.
The authors then did some Bayesian magic, using arcane spells such as “non-parametric Bayesian clustering model”, “Beta-Binomial likelihood”, “Gibbs sampler”, and “ecological estimator” and came up with the following answers.
When two random people in the population talk about penguins, the probability they imagine the same concept in their heads is about a coin toss. In the scientific lingo: “the intersubject [between-people] reliability across all concepts was 50 % (ranging from 33% to 62% with no significant differences between animals or politicians).” This means that if you get the question “Which is more similar to a penguin, a whale or a finch?” and I get the same question, it's about 50/50 that you’d choose a whale and I’d choose a finch.3
Then we have Figure 44 which I found interesting.
Let's start with the left side. As you can see, the top graph pertains to animals, the bottom to politicians, and represents the probabilities that only “a single conceptual representation for each word exists.” The scale is logarithmic, meaning the probabilities are minuscule (each tick on the y-axis is 10 times smaller than the previous one) and all the words have a near-zero probability of having only one concept. Visually speaking, the smaller the bars, the higher the probability that only a “single [concept] for the word exists.” Interestingly, the probability that, for instance, when you and I would talk about salmons or Bernie Sanders, we are comparably much more likely to have similar concepts in mind than, say, if we were talking about finches or Trump. While I can't explain the high conceptual variance for finches, I kind of can understand why Trump has it; people are quite polarized on that particular subject, and where some imagine him as some sort of superbusinessman/wonderful president, others imagine him as a dried up peach (without the sweetness).
Moving to the middle of the graphic, the scale changes to linear, so we see the actual probabilities ranging from 0 to 15. And probability there refers to the likelihood of two people sharing the same concept.
As you can see, animals, in general, have a higher probability than politicians to be the same concept, with Obama and Clinton being somewhat comparable to seals and chickens, and penguins shamefully reaching the conceptual lows of Lincoln and Nixon (didn't know that Lincoln is so conceptually vague to be honest). Eagles and robins seem quite clear, conceptually speaking — there is a 3/4 probability that when we talk about them, we actually imagine the same animal.
Lastly, on the right, we have the correlation between predicted and actual answers.
That's also some juicy stuff because as you can see, people tend to overestimate other's rate of agreement. Visually, if people imagined the same concept, the points would fall on the diagonal line. But that's not the case — we have a cloud of points stretching from 50 to 75 % (y-axis), which means that:
a) half to two-thirds of the participants assumed the other participants would rate the concepts similarly to them and;
b) most participants believe their response is in the majority (all the answers were above ~ 50 %) even though sometimes, nobody agrees with them (there are quite many points around the 0 % on the x-axis).
This tendency is more pronounced with politicians (I assume the blurring means there are many points there, but showing them would make things look messy).
In the second experiment, the authors chose a slightly different approach:
they first asked 16 users about features (single-word adjectives) of the 10 animals or politicians. Basically, these people were asked what comes to their minds when they think about x (a politician or an animal). Think: smart, dumb, majestic, slimy, or fluffy.
they then asked 1000 other people to rate their agreement with the features the previous 16 participants had produced (they came up with 105 features in the end).
For instance, the participants in the second batch were asked “Is a finch smart?” (smart being a feature the first 16 users came up with) and could respond “yes” or “no”. The Bayesian clustering magic produced these results:
On the y-axis, you can see three examples of animals and politicians each (again, politicians on the bottom), and the x-axis plots the mean percentage of “yes” responses.
On the surface level, we can see that Obama (bottom red) is quite conceptually clear — most features are clustered either on the right or on the left, with few being in the middle. Bush, in contrast, is less conceptually clear.
And then we have some random observations and factual inconsistencies. Why do around 10 % of people think that whales are patriotic and around 80 % they are plump, chunky, fat, and hungry? Weird. 75 % of people indicated that salmons are flightless. Apparently, no bears have participated in this study.
Around 60 % of people indicated that penguins are majestic. I'd guess the percentage would be higher, considering:
I didn't find any fun stuff in the politician section but feel free to have a look yourself.
Implications
What does this all mean? First, when two people talk about a concept, they most certainly don't picture the same entity. For each person, the word on the page likely means something else, similar but not the same. People also have a tendency to assume that most others have the same image in their minds as they do, even when they do not. Your penguin is not my penguin6. If this isn't a recipe for disagreement, I don't know what is. Now bear in mind this study pertained to animals and politicians as proxies of all the concepts out there, so the results are only tentative and we can't really be sure how this generalizes to other concepts; I imagine people would be more conceptually-aligned when asked about natural concepts, say “snow” or “sun”, and the rates would plummet for artificial (highly socially contested) concepts like “gender”, “fairness”, “equality” (what is equality anyway?).
Second, how does all this relate to psychological research? As I mentioned in the beginning, concepts are the currency of psychology: finding correlations or differences between concepts, rank-ordering concepts, observing how people deal with concepts, interviewing them about concepts, conceptualizing new concepts7, and so on. Most commonly, psychology uses surveys or questionnaires to measure concepts. These instruments of psychological research (to me, this weirdly reads as instruments of mass destruction) allow the researcher to make claims about some latent, not directly observable qualities that people are assumed to possess (personality, attitude, belief, etc.), based on how they react to concepts (= words) presented to them. For that, the poor souls who partake in surveys participants are asked to indicate their (dis)agreement, usually on a scale ranging from low to high agreement (say, 1 to 7), with some specific items (examples below). Many people are asked to rate many items. And the assumption is that if I am asked the following items (taken from here):
I am always prepared.
I pay attention to details.
I get chores done right away.
I follow a schedule.
I am exacting in my work.
I do not like order. (Reversed)
I leave my belongings around. (Reversed)
I make a mess of things. (Reversed)
I often forget to put things back in their proper place. (Reversed)
I shirk my duties. (Reversed)
And you are asked the same items, we are activating more or less the same concept in our minds. Now usually, researchers find correlations in our answers, leading them to conclude that there is a construct called “conscientiousness”, and recommend people be tested on their personality profiles, finding all sorts of (spurious) correlations with other variables, and inflating the importance of what they do8.
Anyway, the results of this study suggest that what you imagine under those five items and what I imagine, is probably very different. Your penguin is not my penguin. The (hitherto still globally accepted) assumption which makes the gears in many fields of psychology spin at all — that people imagine more or less the same things under the same labels (words) — is if not wrong, then at least very wonky. Many people have mentioned this, including me, and there is no remedy for it, despite what all the validation and reliability-measurement efforts might lead researchers to believe. If you deal in words, you deal in a diversity of meanings. For this reason — and for the last time, I promise — your penguin is not my penguin. Keep that in mind when you have a heated discussion that is, at its core, about some conceptual issue. Even if we define a word precisely, its meaning will still differ among individuals. Recall this lesson also when you read any psychology paper which dealt with concepts9: despite some averaged effect being found, it’s likely that the conceptual label obscures a lot of variation among people as to what this label actually stands for10.
All right, all right, for those who were interested in the dichotomy between concepts and constructs, the simple answer is that concepts are broader, generalizable beyond “populations and contexts”, and constructs are somewhat tighter, logically derived from underlying theories. For instance, the construct of intelligence can be viewed through the lens of the Triarchic theory of intelligence by Sternberg, and operationalized (= made quantifiable) by some psychological instrument. In other words, if psychologists talk about constructs, they (ought to) have a very specific, tightly defined thing in mind.
One has to wonder how they came up with this particular pairing and if it was at all intentional that they would be - at least in my mind - compared to each other. I sincerely hope no animals were offended by this comparison.
If you think about it, this question quite interestingly captures the effect the authors have found. You can liken penguins to whales because both are associated (in my mind, at least) with cold water and aquatic life. But you can also liken penguins to finches because both are birds. The latent diversity between concepts the authors have found is related to these associations that any one individual unconsciously, intuitively makes: for me, penguins might be more whale-like than finch-like, and vice versa for you. Therefore, if we talk about penguins, it might be you're thinking about feathers and beaks (related concepts), and I’m thinking about blubber and slime (for some reason, those are the first two words that came to my mind while writing this sentence).
not that figure four, for the climbers out there reading this blog. Also notice how I made it the fourth footnote, which was totally planned.
0 being low and 1 being high probability; now the bars are reversed - the higher the bar, the higher the probability, and vice versa.
mine is more majestic.
Psychologists’ favorite! There’s nothing better than to “come up” with a new concept, name it, and try to defend it against other psychologists who - oftentimes rightfully - claim that it’s simply a new label for “this other thing we already have”.
To be fair, every field does this. Whether they succeed is up to people buying the snake oil; if they do, the thing works. If not, it falls into obscurity
rule of thumb: all of them
Add this to the “usual”, surface-level obscurity achieved by averaging individual responses into measures of central tendency — means, medians, modes, standard deviations, and so on.