Improving science

Science can be defined as the systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the world. Science is one of humanity's greatest inventions which has the potential of improving our lives and our societies. The core of the scientific process consists of scientists making observations, reading scientific literature, formulating questions, testing ideas through systematic studies, and sharing their findings. The system of science contains principles and sets of rules which help make science self-correcting and cumulative. Scientists are required to share not only their findings through publication but also provide detailed descriptions of their studies so that replication of their studies by other scientists becomes possible. A process of peer review functions as a filter to guarantee that only research that meets scientific standards is published in journals. Replication studies make it possible to test findings using the same methods but with different subjects and experimenters.


By building on robust findings and revising or discarding findings that fail to be replicated scientific theories are built over time. Here is a simplified description of the subsequent stages of knowledge development in science. In early stages, scientists observe associations between phenomena. Then, they are able to make predictions. Next, they gain insight in underlying causal mechanisms. Finally, they are able to control phenomena and to use scientific knowledge.

An example of science at its best is the theory of evolution. Roughly 150 years ago, Charles Darwin, after many years of careful observation and thought, produced a theory that still drives the contempory scientific agenda. Darwin's breakthrough insight was that in organisms whose environment changed nonrandomly and whose reproductive success in that environment depended on inherited traits, evolution became inevitable. Today, the theory of evolution has developed into an extremely well-tested macro theory consisting of a diverse set of micro theories which together yield invaluable technologies.

There is a difference between this idealized description of the scientific enterprise on the one hand and scientific practice on the other. While science as an idea is wonderful there are many aspects of scientific practice which require improvement. When we think about refining and improving the scientific process we usually focus on improving the core of the scientific process. For example, we may discuss whether or not the large scale use of significance testing in social science is justified or not. While important, this is not the only area in which improvement may be achieved. The effectiveness and efficiency of the scientific process is also importantly determined by several other factors, somewhat more in the periphery, which are easily overlooked. The scientific process can be seen as a chain of events. Strengthening the weakest links in this chain may yield the greatest improvements. The figure below is a simplified depiction of this 'chain'.

Here are some ideas about how each of these links in this chain affects the overall effectiveness and efficiency of scientific progress and some (sometimes implicit) suggestions for improvement:

1. Research funding: in order to be able to do research scientists (usually) have to write grants to find financial resources. Decisions about grants for research are not only often made on scientific grounds alone. Three factors threaten scientific impartiality in grant allocation: 1) commercialization of science: an example is how most medical research is now funded by industry; 2) political agendas: political convictions may have an important influence on whether certain scientific research is funded or not (for instance stem cell research); 3) vested interests of conventional scientists: decisions about grants are usually made by well-known scientists who may be wedded to conventional ideas and approaches and may have an interest in these conventional ideas and practices.
2. Original research: the quality of the methods and techniques used remains an essential determinant of scientific progress. Ongoing discussions about the proper design of studies and about the correct use of (statistical) analyses remain important.
3. Peer review: in order to be published papers have to pass the process of peer review. While this is intended to provide a guarantee of scientific quality there is certainly room for improvement in peer review practice. Many instances of peer review failure have been identified which may have to do with limited or no access of the reviewers to the data and/or the details of the method or with bias by vested interest of the anonymous reviewers.  Open peer review has been suggested as a way to improve the transparency and accuracy of the peer review process. 
4. Replication studies: replication studies play an essential role in the self-correcting nature of science. But in practice scientists are not eager to do replication studies, in particular when they think the results of the study were wrong. Also, journal editors typically prefer to publish groundbreaking new research and are often very reluctant to publish replication studies. Some even have an explicit policy of not publishing replication studies (source). Recently, Daryl Bem reported some very surprising results on so-called precognition. Here is an article about how replication studies were rejected by journals and about some solutions suggested by psychologist Richard Wiseman.
5. Follow up studies: follow up studies play an important role in the cumulative, progressive character of science. By building on previous research, theories may be tested and refined and, gradually, theories may be developed which are massively supported by evidence. This paper discusses, among other things the critical and perhaps underestimated role that anomalies play in building better theory. 
6. Further dissemination of scientific knowledge: Scientific knowledge may be spread through popularized science books, magazines, websites like Google scholar, Wikipedia and Youtube, television programs,  and education. An interesting and useful development is also how some universities have started to create chairs for the public understanding of science (for instance Oxford). There are now also peer reviewed scholar sites (scholarpedia.org). And there are more and more researchers uploading pdf files on their websites. Some universities are very generous in sharing their knowledge, too. Despite all these initiatives, there remains a substantial gap between scientific developments and the public's knowledge and understanding of it. Some worry that the gap growths. Even basic scientific knowlegde is unknown to large proportions of the population of many countries. Some people argue for a free open access to all publicly funded scientific knowledge for everyone. 
7. Science education: education is a specific way of disseminating scientific knowledge and to teach students about scientific methods and making students scientifically literate. Scientific illiteracy of politicians, parents and school managers may pose serious threats to this process. For example, politicians may argue that equal attention should be paid in class to creationism and evolution theory ("teach both sides"), which, wrongly, suggests that there is, among scientists, a controversy about the evidence for evolution (actually, the evidence is overwhelming). Politicians may even go so far as to make libel laws which may practically prevent scientists to criticize claims made by sellers of alternative medicine or other approaches which are unsupported by scientific evidence.  
8. Application of scientific knowledge: the availability and application of scientific knowledge would be facilitated by improving the practices described above. Scientific progress would happen faster and would benefit more people. 

The scientific enterprise is extremely important and we should continue to improve scientific practice.

Overdiagnosed: too much diagnosis is turning more and more of us into patients

The rationale for the increasing emphasis on diagnosis

In Overdiagnosed, H. Gilbert Welch (photo), with Lisa M. Schwartz, and Steven Woloshin, explains how the medical profession has an increasing tendency to make diagnosis which is not good for us. The rationale for more diagnosis seems good. When we diagnose more we are able to detect abnormalities, like cancer, high blood pressure, diabetes, high cholesterol, etc., earlier so that we can treat them earlier and prevent serious health problems. An example of this greater emphasis on diagnosis is the prevalence of disease awareness campaigns which encourage people to undergo medical screenings. Another example of increased tendency to diagnose is when doctors have patients tested for things about which they have no complaints.


When are abnormalities dangerous?
Although more diagnosis may sound good, according to the author it is actually, on balance, not when it leads to overdiagnosis. Overdiagnosis is the detection of abnormalities which are harmless. Overdiagnosis is likely to happen when diagnosis is done when there are no symptoms. What is important to know is that many asymptomatic people have abnormalities of some kind which are no threat for their health and which will never become a threat to their health. Some of these unharmful abnormalities may fall into innocent sounding categories like slight overweight, or pain in the knee, but some unharmful abnormalities involve terribly dangerous sounding words like stroke, cancer, high blood pressure, diabetes, high cholesterol, etc. While all these dangerous sounding words may involve abnormalities which indeed can kill you, in many cases abnormalities like these won't kill you or even harm you. For instance a study found that 10 percent of healthy participants had had strokes (silent strokes) without them being aware of it. Another example is that a large percentage of adults have some evidence of thyroid cancer. For most of them this will never become dangerous because cancers are often static or progress extremely slowly. Some abnormalities are dangerous, many abnormalities aren't. How does the medical profession decide when abnormalities should be considered pathological?

How lowering cutoffs creates new patients
Medical professionals consider an abnormality pathological when the value they find in a diagnosis is above a certain cutoff score. This cutoff score is not something which is purely scientifically established. The distinction between abnormal and normal can be rather arbitrary. The author explains that cutoffs are defined by panels of physicians and they involve not only scientific judgments but also personal values and opinions and financial interests. Cutoffs are also not fixed. They frequently change over time. The direction of this change has been to set cutoffs lower and lower. Due to this, the threshold to make a diagnosis has fallen and the number of individuals who are labeled sick has increased, often dramatically. The picture below (source) illustrates this with the -arbitrary- example of cholesterol level cutoffs.

As the picture shows, lowering the cutoff only slightly, creates a great number of new 'patients'.

What forces drive the lowering of cutoffs?
The author mentions at least three forces driving the lowering of cutoffs. First, there is the true belief by many that early detection is a good thing because it can prevent worse. Second, new technology enables medical professionals to screen more easily, cheaper and more refined. Third, finding more abnormalities and defining these abnormalities more and more as pathological enlarges the medical market which leads to financial benefits. The author explains that this third force is a dangerous one. He says the commercialization of medicine is a corrupting force. It may have been fine if medical care would have been a free market but it is nowhere close to being a free market. Sellers in the medical care market create demand for their wares by being in the position to decide whether or not you need to consume their products. Turning more people into patients is (like) expanding the market, something of which the whole medical-industrial complex financially benefits. Medical research is also negatively affected by commercialization. In order to do research researchers have to apply for grant money. Decisions about grants for research are not only often made by the commercial companies like the pharmaceutical industry (most medical research is now funded by industry) but also by other researchers who are wedded to conventional ideas and approaches. Sympathetic sounding disease awareness campaigns also increasingly involve paid advertising.

What is the problem with more diagnosis and lower diagnosis thresholds?
More diagnosis and lower thresholds lead to more treatments. A cycle of seeing more, finding more, and doing more emerges. People with small abnormalities will be treated more than was the case before. There are two problems with this. First, people with milder abnormalities stand to benefit less from treatment than those with severe abnormalities. A few may be helped but many will be overdiagnosed and some of them will be harmed and no one knows who. The following picture (source) shows the relationship between treatment benefit and level of abnormality.

Second, being diagnosed and being treated has some serious disadvantages. There are potential psychological (for example viewing oneself as a patient), practical (for example not being admitted to insurance), financial (for example spending more on healthcare) and medical harms (for example side effects, sometimes very serious, even life threatening) of being diagnosed and treated. When patients have seriously threatening abnormalities there is often little that can be done about these harms. In those cases treatment is necessary and its benefits outweigh the negative effects of being diagnosed and treated. But for patient with mild abnormalities chances are that negative effects of being diagnosed and treated outweigh its medical benefits.

If it ain't broken ...
This book is not a criticism against conventional health care. It does not advocate alternative healthcare. It is also not against diagnosis and treatment. On the contrary, the author is all for conventional healthcare and urges people with symptoms to talk to their doctor. It is diagnosis and treatment in the absense of symptoms the author warns against. Often, it is wise to remember: if it ain't broken, don't fix it.

Interview with Wally Gingerich

By Coert Visser

Wallace Gingerich is Professor Emeritus of Social Work at the Mandel School of Applied Social Sciences, Case Western Reserve University, Cleveland, Ohio. As a core member of the Brief Family Therapy Center in Milwaukee (BFTC), Wisconsin, in the 1980s, he has been an important contributor to the development of the solution-focused approach. In this interview, he looks back on how and why he joined BFCT and on how the solution-focused approach emerged in the next few years after he joined. Also, he talks about the BRIEFER project and about a soon to be published review of the research on the effectiveness of the solution-focused approach. Finally, he reflects on the ways the solution-focused approach may further develop.

Read the full article »

Whistling Vivaldi And Other Clues to How Stereotypes Affect Us (Book Review)

BOOK REVIEW: Steele, C.M. (2010). Whistling Vivaldi: And Other Clues to How Stereotypes Affect Us (Issues of Our Time). New York, W.W. Norton & Company.

By Coert Visser

This review was first published on Positive Psychology News Daily

This book by social psychologist and Columbia University provost, Claude Steele, is a splendid example of how psychologists can make valuable contributions to society. In the book, Steele writes about the work he and his colleagues have done on a phenomenon called stereotype threat, the tendency to expect, perceive, and be influenced by negative stereotypes about one’s social category, such as one’s age, sex, sexual orientation, ethnicity, profession, nationality, political affiliation, mental health status, and so on.

Experiments demonstrating the impact of stereotype threat

When trying to understand certain performance gaps between groups, Steele and his colleagues did not focus on internal psychological factors.. Instead, they tried to understand the possible causal role of identity contingencies, the things you have to deal with in a situation because you have a given social identity. Over the years they carried out a series of creative experiments* in which there was a control condition in which a task was given under normal life conditions. In the experimental condition, the identity contingency was either cleverly removed or it was deliberately induced. Here are three examples of experiments to clarify how they worked.

Experiment 1: Steele and Aronson (1995)

In this experiment the researchers had African American and white college students take a very challenging standardized test. In the control condition, the test was presented as these tests are always presented – as a measure of intellectual ability. This condition contained the stereotype that African Americans would be less intelligent. In the experimental condition the test was presented in a non-evaluative way. The test takers were told that the researchers were not interested in measuring their ability with the test but that they just wanted to use the test to examine the psychology of verbal problem solving. In the control condition, the African American test takers, on average, scored much lower than the white test takers. For the white test takers there was no difference in their scores between the control condition and the experimental condition. For the African American test takers there was a big difference between the control condition and the experimental condition. They solved about twice as many problems on the test in the experimental condition. Moreover, there was no difference between the performance of the black test takers and the white test takers.

Experiment 2: Aronson, Lustina, Good, Keough, Steele & Brown (1999)

In this experiment, the researchers asked highly competent white males to take a difficult math test. In the control condition the test was taken normally. In the experimental condition, the researchers told the test takers that one of their reasons for doing the research was to understand why Asians seemed to perform better on these tests. Thus, they artificially created a stereotype threat. In the experimental condition, the test takers solved significantly fewer of the problems on the test and felt less confident about their performance.

Experiment 3: Shih, Pittinsky & Ambady (1999)

In this experiment, a difficult math test was given to Asian women under three conditions. In condition one, they were subtly reminded of their Asian identity, in condition 2 they were subtly reminded of their female identity. In the control condition they were not reminded of their identity. The women reminded of their Asianness performed better than the control group, whereas those reminded of their female identity performed worse than the control group.

How does stereotype threat harm performance?

Today, research on stereotype threat effects is done throughout the world by many researchers. Much insight has been gained into what it is and how it works. Briefly, you know your group identity and you know how society views it. You are aware that you are doing a task for which that view is relevant. You know, at some level, that you are in a predicament: your performance could confirm a bad view of your group and of yourself as a member of that group. You may not consciously feel anxious but your blood pressure rises and you begin to sweat. Your thinking changes. Your mind starts to race: you become vigilant to all things relevant to the threat and to what your chances of avoiding it are. The book title comes from an observed behavior: an African American whistling Vivaldi to make clear that certain stereotypes attached to the group don’t apply. You get some self-doubts and start to worry about how warranted the stereotype may be. You start to constantly monitor how well you are doing. You try hard to suppress threatening thoughts about not doing well or about the negative consequences of possibly failing. While you are having all of these thoughts you are distracted from the task at hand and your concentration and working memory suffer.

Does it always happen? No. There is only one prerequisite for stereotype threat to happen: the person in question must care about the performance in question. The fear of confirming the negative stereotype then becomes upsetting enough to interfere with performance. It is now known that stereotype has the strongest negative impact when people are highly motivated and performing at the frontier of their skills.

Solutions: bridging performance gaps through small interventions

Can something be done about it? Yes. The promising news is that there are some rather small interventions which can help a lot. Experiments have shown that subtly removing or preventing stereotype threats can completely or largely eliminate performance gaps between stereotyped groups and non-stereotyped groups.

Examples of helpful interventions are:

1. Make it clear in the way you give critical feedback that you use high standards and let the person know that you expect him or her to be able to eventually succeed.
2. Improve the number of people from the social category in the setting so that a critical mass is reached.
3. Make it clear that you value diversity.
4. Foster inter-group conversations and frame these as a learning experience.
5. Allow the stereotyped individuals to use self-affirmations.
6. Help the stereotyped individuals to develop a narrative about the setting that explains their frustrations while projecting positive engagement and success in the setting.

Conclusion

The tone of the book is informal, friendly, and personal, and the content is profound. The topic is highly relevant both to the development of social psychology and to the development of our educational systems and societies at large. Of course it also can inspire positive psychology research: how have certain individuals managed to overcome stereotype threat, how do certain organizations manage to bridge performance gaps, how do societies manage to do the same?

References

• Steele, C.M. (2010). Whistling Vivaldi: And Other Clues to How Stereotypes Affect Us (Issues of Our Time). New York, W.W. Norton & Company.
• Steele, C.M. & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology 69, 797-811

* I am especially in awe of the work Steele has done in collaboration with Joshua Aronson, who is now an eminent professor at New York University.

Self-Determination Theory Meets Solution-Focused Change: Autonomy, Competence and Relatedness Support in Action

By Coert Visser

This article looks at the Solution-Focused approach (SF) through a Self-Determination Theory (SDT) lens. SDT is an influential macro theory of human motivation which has been applied to many life domains, including sports, education, psychotherapy and work. The theory focuses mainly on the benefits of self-determined behaviour and the conditions that promote it. Its relevance for helping professionals such as psychotherapists and counsellors has been recognized by previous authors. A counselling approach which has been associated with SDT is motivational interviewing (MI). This approach has some important similarities to SF but there are also some key differences. This article focuses on the relevance of SDT for SF and vice versa. Although the literature on SF makes only a few mentions of SDT, SF fits well with its main propositions and findings. The strategies, principles and interventions of SF have the effect of supporting the perception of autonomy, competence and relatedness of clients which, according to SDT, are keys to enhance self-determination. It is argued that the SDT framework and body of research are relevant for SF. They help to understand better how SF works and may be used to further refine and develop the approach. In the same way, SDT theorists and practitioners may benefit from learning about the specific and often subtle ways in which SF supports clients’ autonomy.

Full reference to the article: Visser, C.F. (2010). Self-Determination Theory Meets Solution-Focused Change: Autonomy, Competence and Relatedness Support In Action, InterAction - The Journal of Solution Focus in Organisations, Volume 2, Number 1, May 2010 , pp. 7-26(20)

Read the full article »

How Equality Drives Thriving

Coert Visser, 2010


The Relationship between Equality and Thriving

Richard Wilkinson and Kate Pickett, two English epidemiologists, have written The Spirit Level: Why Greater Equality Makes Societies Stronger, a provocative book on how high levels of inequality in societies is harmful for everyone within them. Their research shows that while economic policies in developed countries stress the importance of economic growth, economic growth is mainly an important determinant of the degree to which societies thrive. After a certain point the contribution of further economic growth begins to create only diminishing marginal returns: the relationship between economic growth and certain objectively measurable outcomes, like life expectancy, level off (see figure 1).

Source: Wilkinson & Pickett, The Spirit Level (2009)

In developed countries, national income per person and economic growth are not the most important predictors of societal thriving but the level of equality. Wilkinson and Pickett’s research shows that many health-related and social problems are associated with the level of inequality of society. Here is how they did their research. They gathered data from 23 of the richest countries in the world from the World Bank and gathered internationally comparable data on the following health and social problems: level of trust, mental illness (including drug and alcohol addiction), life expectancy and infant mortality, obesity, children’s educational performance, teenage births, homicides, imprisonment rates, and social mobility. Here is an example of a graph, showing how an index of these measures is related to income inequality.

Source: Wilkinson & Pickett, The Spirit Level (2009)

To test and cross-validate their findings, the researchers tested these findings in a new data sample which consisted of the 50 American states. This research confirmed their findings across nations which adds to the credibility of the claims. Below is a graph showing how the index of health and social problems is related to income inequality in US States.

Source: Wilkinson & Pickett, The Spirit Level (2009)

The book contains many more graphs showing specific relationships between income inequality and separate measures of societal functioning (see these slides).

Why is this relevant for positive psychology?

Positive psychologists have done much research into how money is associated with happiness and some of their findings, at first glance, seem to be at odds with Wilkinson and Pickett’s findings. Berg and Veenhoven (2010), for instance, found little relationship between income inequality and average happiness in nations. It seems paradoxical that income equality would be related to many objectively measurable problems but hardly at all with happiness. How can one be equally happy when objectively things are worse? What is going on here?

This question takes us back to the original formulation of positive psychology’s mission. Seligman and Csikszentmihalyi (2000) wrote: “We believe that a psychology of positive human functioning will arise that achieves a scientific understanding and effective interventions to build thriving in individuals, families, and communities.” This wonderful definition allows for research to identify any type of determinant of thriving to be found, like personal, motivational, habitual, or situational.

In practice, however, positive psychology is usually more narrowly operationalized. For instance, on the Wikipedia page of February 15, 2010, it is defined as follows: “Positive psychology is a recent branch of psychology that studies the strengths and virtues that enable individuals and communities to thrive”. This definition mentions only strengths and virtues as candidates for causal factors of thriving and it makes no mention at all of contextual, situational or structural factors affecting thriving.

A broader perspective on thriving and its determinants

As psychologists, we have known for a long time how important situational factors are in influencing our perceptions, beliefs, behaviors, feelings, and performance. Wilkinson and Pickett’s research is another example of this and it fits splendidly within the original definition of positive psychology in the sense that it contributes to the scientific understanding of how communities and its members thrive.

Also, it is an illustration of the limitations of using subjective criterion measures in research. Apparently, we can report well-being while objectively things are not going too well, both on an individual level and on a society level. We should not equate thriving or flourishing with subjective well-being. On this Barbara Fredrickson, author of Positivity, writes: “Flourishing goes beyond happiness, or satisfaction with life. True, people who flourish are happy. But that's not the half of it. Beyond feeling good, they're also doing good -adding value to the world. People who flourish are highly engaged with their families, work, and communities. They're driven by a sense of purpose: they know why they get up in the morning.’

For positive psychology to thrive, it needs to move beyond a somewhat narrow focus on happiness and strengths and take into account a broader perspective on thriving and its determinants.

References

• Berg, M. & Veenhoven, R. (2010). Income inequality and happiness in 119 nations. In search for an optimum that does not appear to exist. In: BentGreve (Ed.) ‘Social Policy and Happiness in Europe’, Edgar Elgar (in press)
• Fredrickson, B. L. (2009). Positivity. Groundbreaking Research Reveals How to Embrace the Hidden Strength of Positive Emotions, Overcome Negativity, and Thrive. New York: Crown Publishers.
• Pickett, K. (2010). Why Inequality is bad for your health. http://bigthink.com/ideas/18463
• Seligman, M.E.P., & Csikszentmihalyi, M. (2000). Positive Psychology. An Introduction. American Psychologist. Vol 55. No. 1. 5 14
• Wilkinson, R. & Pickett, K. (2009). The Spirit Level: Why Greater Equality Makes Societies Stronger. New York, Bloomsbury Press.

Supporting Clients’ Solution Building Process by Subtly Eliciting Positive Behaviour Descriptions and Expectations of Beneficial Change

Visser, C.F. & Schlundt Bodien, G. (2009). Supporting Clients’ Solution Building Process by Subtly Eliciting Positive Behaviour Descriptions and Expectations of Beneficial Change. InterAction I (2), 9-25

SF co-developer Steve de Shazer wrote, in his classic publications Keys to Solution in Brief Therapy (1985) and Clues: Investigating Solutions in Brief Therapy (1988), that SF practitioners should help their clients create an expectation of beneficial change by getting a description of what they would do differently once the problem was solved. Also, he claimed subtle and implicit interventions by the SF practitioner would work best. At the time, de Shazer did not support these claims with empirical evidence. This article provides evidence for each of the assertions made by de Shazer. Only part of the evidence presented here was already available at the time of de Shazer’s writing. Evidence is discussed from diverse lines of research like Rosenthal’s Pygmalion studies, Dweck’s research on self-theories, Fredrickson’s broaden-and-build theory, research on Winograd’s prospective memory, Jeannerod’s research on the perception-action link, Wilson’s research on brief attributional interventions, research on Brehm’s reactance theory, and Bargh’s research on priming. The article closes with some reflections on what these research findings imply for SF theory and practice.

Full article: Supporting Clients’ Solution Building Process by Subtly Eliciting Positive Behaviour Descriptions and Expectations of Beneficial Change