Theory protects from “personal intuitions and culturally biased folk theories.”
Cathleen O’Grady – 2/16/2019, 11:00 AM
A jumbled jigsaw puzzle, AKA the state of theory in the behavioral sciences.
A replication crisis has called into question results from behavioral (and other) sciences. Complaints have focused on poor statistical methods, the burying of negative results, and other “questionable research practices” that undermine the quality of individual studies.
But methods are only part of the problem, as Michael Muthukrishna and Joseph Henrich argue in a paper in Nature Human Behaviour this week. It’s not just that individual puzzle pieces are low in quality; it’s also that there’s not enough effort to fit those pieces into a coherent picture. “Without an overarching theoretical framework,” write Muthukrishna and Henrich, “empirical programs spawn and grow from personal intuitions and culturally biased folk theories.”
Doing research in a way that emphasizes joining the dots constrains the questions you can ask in your research, says Muthukrishna. Without a theoretical framework, “the number of questions that you can ask is infinite.” This makes for a scattered, disconnected body of research. It also feeds into the statistical problems that are widely considered the source of the replication crisis. Having too many questions leads to a large number of small experiments—and the researchers doing them don’t always lay out a strong hypothesis and its predictions before they start gathering data.
This isn’t the first time someone’s argued that better theory makes for better science. It’s a conversation that’s been going on for some time among the people agitating for more robust research. But this is a particularly loud klaxon, in one of the biggest journals in the field, meaning that it might make people sit up and take notice—and possibly start some concrete initiatives to improve theory at the same time as current attempts being made to improve statistical rigor.
Theories about theory
Paul Smaldino, a cognitive scientist who has also been vocal about the need for better theory, points to an infamous psychology paper as a perfect example of what happens when experimental work is divorced from theoretical scientific frameworks. The paper, published in 2011, reported finding evidence of precognition. But that, says Smaldino, “is not a psychology finding. That’s a physics finding. That is everything we know about the laws of physics and causality and how time works, all being wrong.”
The problem with the paper wasn’t just that the methods were bad, he argues, but also that “theory [in psychology] is so weak that something that completely contradicts hundreds of years of science was evaluated without that context.”
Science, he explains, is about accumulating sets of observations that occur reliably—the Sun appears at different places in the sky depending on the season and time of day; finches have different shaped beaks depending on what they eat. “That’s the raw ingredients,” he says. “To make sense of it requires a framework to say, this is how all these different facts fit together, and this is why.” We explain these observations by developing theoretical models—of how the Earth rotates around the Sun on a tilted axis, of natural selection.
Having a good theoretical framework makes it possible to make sense of sets of disconnected facts, and to explain why things happen sometimes and not at other times. Perhaps most importantly, it allows for predictions of what will be found in the data: if our model of human evolution is true, we could predict that we should find huge similarities in the genomes of humans and other great apes—and that’s exactly what we do find. If we made a prediction like this and found it to be false—say, our genomes turned out to be more similar to birds than to other great apes—it would undermine the theoretical framework.
Doing scientific research within a theoretical framework can help to highlight results that are surprising—like finding precognition—and that might therefore need a closer look and plenty of replications to test whether the finding holds up. By drawing connections between behavioral science findings and findings across other fields, “overarching theoretical frameworks pave the way toward a more general theory of human behavior,” write Muthukrishna and Henrich.
More math for the behavioral sciences
Part of what Muthukrishna and Henrich are advocating is a greater use of formal models—a way of setting down ideas about something in cold, hard math. For instance, you might think that, when children are learning whether it’s “toh-may-toe” or “toh-mah-toe” (or many of the other myriad arbitrary cultural variants that humans swim in), they’ll copy the majority of speakers they hear around them.
Original Paper
Nature Human Behaviour, 2018. DOI: 10.1038/s41562-018-0522-1 (About DOIs).
HT/MangoChutney
Sometimes I think my head is going to explode reading all this. I need to take a week off, shut the lid on my Mac and go work on my boat.
Simple – “social sciences” are not sciences. The headline of this post (I don’t know where it comes from) says it all: “theory protects from … theories”. They have no idea what a theory is, or what a science is. A theory does not need a protection; it needs to be confirmed, or disproved.
The “Replication Crisis” pretty much applies to all “sciences.” We might take some solace in that only a fraction of irreducible results are fraudulent, but it’s disheartening that most are due to incompetence.
Especially so-called climate science.
Ottokorekt strikes again?
Irreproducible?
And even theories that are the current best model should always be challenged. If you will not allow people to ask questions or challenge then two things are obvious. 1) You are basing things off of belief and dogma, 2) you are not practicing science. People asking questions does not mean something is wrong. It is an opportunity to strengthen your model. When they treat it as heresy though it should be very clear immediately that they are not practicing science. They just use the label to try to shut down challenge to their dogma with an appeal to authority fallacy.
Likewise, reporting failures is just as important to science as successes. Yet there is this stigma about being wrong. Yet if we are never wrong, how can we learn?
Indeed, the real sciences use raw ingredients like observations (to be repeated) and data (to be re-checked). Social “sciences” start with interpretations meaning that the authors have to sell something.
If it ain’t falsifiable it ain’t science but non-science nonsense. Eschew ad-hockery.
So you need an over arching “theory” about beaks before you do any research ? No wonder these pseudo-sciences are in such a mess.
What this really is , is a plea for an orthodoxy to govern a priori what gets investigated. If you “know” it does not fit the orthodoxy , there is not point in doing the research. It can only produce wrong answers, so it is a waste of effort.
So the most important factor is to have a firm preconceived idea of what you expect to find. That way you can ensure you get the “right” results. This helps prevent objective analysis leading you astray.
Greg, the prediction helps prevent p-hacking and post hoc analysis, the leading causes of unrepeatable ‘science’
But that’s what science is: starting with a “preconceived notion”(hypothesis) of what you will find and then either finding it (thereby validating the hypothesis) or not finding it (thereby falsifying the hypothesis). Without the preconceived notion and objective analysis of the results you don’t have science.
The trick is getting the objective analysis when the so-called “scientists” are invested in the truth of their preconceived notion.
Greg, the scientific method that we all got taught in school has a very straightforward method. You have a theory and you test it. You have to know what you are looking for before you look for it. Even in a non-experimental observation study, you need to have some idea of what you want to observe. Then you have to fit it into a theory to understand it. Even if the theory is so trivial that it is pedantic to mention it (ie: plants use water to grow), you still are using one.
I spent all last summer restoring (to the best of my limited abilities) a 1964 Dorsett San Juan. Stripped and sanded all of the interior wood and fiberglass surfaces, tear out the controls, replace all the gauges, the headliner — you name it.
Got it ready for water just as the marina was putting away all the cradle-stored boats for the winter, so now it’s wrapped in plastic and sitting up on the hard. April can’t get here soon enough.
…And the sun is usually found to continue appearing as predicted for a very very long time, as compared to human timescales.
But a psychologist, say, may do impeccable ‘scientific’ research into a certain human behavior yet there is no guarantee that humans will reliably behave the same way tomorrow, next week, or a century from now.
Many experiments really are a waste of time and resources when a long view is taken.
…Thus some people currently appear to think they can get a Democrat elected President the same way Trump got elected. They may be right, but there is a very good chance they may be wrong.
What the psychologist needs is a theory of how people, say, learn to perceive time. The concept that some things are yesterday, some are today, and they often have a connection to what things are tomorrow.
A simple behavior such as- I bit my sister and got smacked on the bottom yesterday. Today by sister bit me and got smacked on the bottom. I think if I bite my sister tomorrow I will get smacked on the bottom. I won’t bite her tomorrow and see what happens.
People seem to learn a lot of things by associations of happenings. What psychological postulates might explain that and how do they interact with each other?
I’ll throw this out there as a postulate for psychology:
“The psyche is the pattern in the activity of the brain as it develops throughout life” (TM) Phil Cartier 02/17/2019
And the activity of the brain is the direct result of …… what your environment nurtured you to be.
What a person learns (nurtured) today, is directly dependent upon what they learned yesterday, …… and all the yesterdays back to the day of their birth.
…. then there are the things which we ( and animals ) do without having to be taught or learn by experience.
Yup, …….. Inherited Survival Traits (instincts) that are encoded in the DNA one inherits from their biological parent(s).
It’s easy to mistake the direction of causality. Progressives seem to be very good at doing it, too.
Which comes first — theory or evidence? Well, actually, each needs to buttress the other to make valid conclusions about cause and effect. And they need to develop more or less simultaneously as well. Observations will lead to a tentative theory that will stand or fall with more observations.
I think what they are saying is that while there are lots of observations, nobody is using those observations to create a theory. At this time.
I think what they are saying is that there are lots of individual theories and observations so anybody can pick and chose their favorite sets, but nobody is collating them into a whole assembly which would reinforce the replicated ones, and highlight the non-replicated ones.
AKA; There is no overarching guidance or control. No goal in mind, no direction. It’s just random stuff (some true, some not true) being thrown onto a huge pile of writings, all of which is becoming a spaghetti word salad.
The first and most important question is: Is it getting warmer? Until there is a conclusive, replicated answer to that question then there can be no viable theory that explains “why.”
I got something much more sinister. It sounds to me like they think scientists should pay more attention to theory so their experiments support the theory … just like they already do in climate science.
A lot of people conflate “hypothesis” with “theory”, too, and use the words interchangeably. They’re not.
An overarching theoretical framework is good when it is sound; the problem is that a false theory can cause huge problems.
At worst facts are adjusted to fit the theory, the way they are by climate alarmists.
Not sure how this applies to Climate science. You have some 4 surface records plus Nick Stokes reanalysis that are on top of one another plus space temp records that mirror them, to less accuracy, as would be expected. Despite “skeptics” saying you can’t measure the surface temp of the earth, the scientists do it over and over again, and each group gets the same result.
In Paleo-land, Mann et al’s 98 result has been replicated by at least 4 studies through the years, and hey, there’s no revision. It stands.
So climate science is not behavioral science. Not even close.
Repeating an error does not change that fact it’s an error in the fist place .
If it dumb to jump of a cliff the first time , the 100 time it is done, it still remains ‘dumb’
“Repeating an error does not change that fact it’s an error in the fist place .”
No, but repeating a lie over and over will make some people believe it
The “97%” has falsified at least three times and no retraction. It’s still a monumental lie.
How many times do you need to crop off inconvenient data to “hide the decline” until it becomes accepted fact ?
Trafamadore,
Thanks. It would be helpful if you could cite the four, post – Mann 98, paleo-land studies, to which you refer, please .
trafamadore, a few points.
1. Replication of a study by the same people using the same data is not replication. Mann and Jones replicating Jones and Mann is not “independent” replication of Mann, Briffa and Hughes.
2. You can calculate the global temps until the cows come home, but it means nothing. It’s a completely artificial value with no relationship to the real world. Example: Let us say that mid winter temps at the North and South pole rise by 10 degrees over a decade. (Note that even with this rise temps are way below freezing so there is no real physical difference in the region) So you take that 10 degree change and average it over the whole year, then you smear it over the whole planet and happily announce the “Global Average Temperature” has risen by .15 degrees in the decade.
What a load of rubbish. The Equatorial and Temperate zones haven’t changed at all and the Polar regions haven’t changed in any real fashion. Seriously, if you think there is a real difference on the ground between -60 degrees and -50 degrees, you have rocks in your head.
3. It doesn’t matter that a number of temp series agree with each other if they are all doing the same silly averaging. A “Global Average Temperature” will not ever tell you what is actually happening to the climate, what we really need is a triple score; What are the temps doing in the Tropics, the Temperate zones and at the Poles.
4. If your thermometers are accurate to .1 degree then having a hundred of them doesn’t increase accuracy to .001 degrees.
Also in behavioral studies college students aren’t anything like normal people. And they are usually the study subjects.
https://psmag.com/social-justice/joe-henrich-weird-ultimatum-game-shaking-up-psychology-economics-53135
The article nicely sums up what Anthropology (and by extension most of the Social Sciences in including cultural sociology) is in most of today’s Academia.
“Many anthropologists took to the navel gazing of postmodernism and swore off attempts at rationality and science, which were disparaged as weapons of cultural imperialism.”
Theory does not protect from conflation of logical domains for political, social, and financial benefits and “solutions”. In fact theory has been demonstrated vulnerable to the whims of democratic processes and political myths spread by press, party, and special and peculiar interests.
Science, as a practical matter, and self-evidently, is a near-space and time logical domain, philosophy, and practice. Theory belongs in the philosophy domain until its claims are observed, replicated, and its conclusions reached through deduction, not inference (i.e. created knowledge).
Deduction is inference.
https://www.thesaurus.com/browse/inference
Yeah! A posting I completely agree with. You cannot conduct “science” on a philosophical ideology, i.e. there may be a God (or Goddess) but you cannot use science to prove it, or disprove it. Its a belief system in place of facts, and not testable.
A good theory does not “belong” in the philosophical domain – after all who gets to decide when it moves from a “good theory” to a fact? That in itself is a philosophical decision.
A good theory proposes a mechanism that explains observed data, it is falsifiable through testing. If you do not allow a theory to be falsified through a set of preconceived conditions, then it isn’t science or a good theory – its just a belief. Take AGW as an example of a pseudo-theory that cannot be falsified. No matter what happens, the believers find a way to reinforce their belief. They are not interested in discovering the truth – they already KNOW.
A theory moves into the realm of accepted fact after a suitable amount of time and testing which fails to falsify it. Now that is truly just philosophical… When did Eisenstein’s theories become accepted as facts? People are still trying to break them – and that is good science. Keep up the testing!
Many so-called sciences (I call them soft-sciences to be polite) cannot ever attain the level of confidence that a true science requires because too much of the underlying mechanism(s) are just assumed. You can achieve a certain amount of statistical certainty if the populations are large enough, but you can’t apply any of it to the individual. It’s kind of like trying to predict with certainty the position of a specific electron… Since it isn’t truly a point, it doesn’t truly have a position, it just has probabilities. With the mechanisms (or framework) you can’t make any useful predictions of new findings.
Most social sciences are completely buried under bias – the bias swamps the signal. If you dare to discover something that is not acceptable to the culture, you better bury the test, the data, and burn any theory it is associated with.
i dont know finches but i can tell from someones mug if he eats lots of spam
AOC’s “Green New Deal” is one of those theories where the puzzle pieces don’t seem to fit together (to prove the theory correct). I thought that’s where this article was going, but it deviated…and it just made me more confused, like rbabcock’s first post here alluded to… I think my head is going to explode. . . I better get pack to my painting…
Oops no editing….I better get back to my painting…
Avoid trying too hard to turn this article into purely political fuel. It shouldn’t be seen that way. AOC is an example of a conclusion-making that so far detached from reality that this article and her Green New Deal are light years apart. This article is examining the very roots of science, which has potentially far reaching implications about contemporary theories, models, experiments, and conclusions in all branches science (including climate change).
The word “theory” is used with abandon, with not a chance for “hypothesis” to get a look in.
Ya got the slugger in the batters box, facing the pitcher that can throw 100 mph, I doubt either one of them is thinking about math during those fractions of a second that matter.
Sadly today their is a tendancy to use the word Science for everything, ie “Social Science” . So what is needed is a clear understanding of just what the word actually means.
Any suggestions readers.
MJE
Read Karl Popper’s *The Logic of Scientific Discovery*
Those interested in psychology as it applies to raising children may want to check out family psychologist John Rosemond at http://www.johnrosemond.com. His weekly column is called “Living with Children.”
He frequently comments on the lack of any scientific or experimental basis for many of the afflictions ascribed to children like ADHD, etc.
I would agree with the need for a more rigorous scientific approach to research but, in the full article, we find that this means more computer modelling. I’m not sure whether the two are fully compatible.
Numberwatch used to be a good site for querying flaky research and theories, but it went off-line a while ago, although if you type numberwatch into Google it comes up with a page that says ‘returning soon’.
One can only hope!
Stephen
The way back machine is your friend.
https://web.archive.org/web/20180629143443/http://www.numberwatch.co.uk/number%20watch.htm
Thank you Roy, that helps a lot.
I have used it frequently in the past to get ammo for discussions with AGWs, and will be looking forward to Numberwatch’s comments on current matters when/if it comes back on line.
Incidentally, does anyone know where to get a tutorial on how to use the way back machine?
Quoting article:
And if you thought that …… then you would have thought wrong.
The majority of speakers has nothing to do with it. The parent or guardian that is nurturing the child is the culprit.
What do you expect from a field whose classic literature includes The Interpretation of Dreams? Reading Freud is a challenge in suppressing one’s laughter. It has progressed since Freud. Now the lack of sound theories is supported by plenty of correlations and statistics of random events.
https://tambonthongchai.com/2019/02/03/hidden-hand/
“Having a good theoretical framework makes it possible to make sense of sets of disconnected facts, and to explain why things happen sometimes and not at other times.”
One might consider that when the IFCC was based on the Maurice Strong assumption that humans cause climate change and that the IPCC ignores the impact of a very important greenhouse gas (H2O) that neither UN organization has established a good theoretical framework to generate the conclusions which they make.
The theory of “climate change” is not “the world is warming due to co2”.
The theory of climate change is “the use of fossil fuels will destroy the world”.
The problem is that no one recognizes the true theory. Instead everyone beats about the bushes. Arguing in a vacuum without any agreement about what we are arguing about.
I would urge everyone in their discussions about climate change to first formally state the true theory. Get it out there so that everyone can agree what we are truly talking about.
More:
The theory of “climate change” is not “the world is warming due to co2”. That is the theory of global warming.
The theory of climate change is “the use of fossil fuels will destroy the world”.
The problem is that no one recognizes the true theory being discussed.
The problem is that there IS no “theory of climate change.”
A theory, in the scientific sense, should be a working model of the subject, with strong predictive power. “Theory” is often used as a synonym for “hypothesis,” which it’s not. Gravitation has a theory. Relativity has a theory. Music has a theory. But climate change does not have a theory. It’s a mashup of everything including the kitchen sink.
It’s not a theory when climate change can cause flood, drought, tornadoes, a lack of tornadoes, hurricanes, reduced snowpack, increased snowfall, you name it. If you can’t say that if this, this, this happen, this should happen. They can’t do that. Because there is no theory of climate change.
That’s what indoctrination into the standard model is all about. Quite an advocate.
And here’s the Diesel Skandal / Nitrogen oxides replication crisis:
http://m.spiegel.de/suche/?suchbegriff=Bezweifeln+Stickoxide+&Suchen=Absenden
“finches have different shaped beaks depending on what they eat”: That is a very misleading statement, basically Lamarkian. A finch’s beak doesn’t change significantly depending on its diet growing up; rather, the shape of their beaks depends on the shape of their ancestors’ beaks. Better to say that finches eat different things depending on the shape of their beaks. Only over evolutionary time would a population of finches who was dependent on a particular food supply develop beaks of a different shape.