The Rules of Replication: Part II

May 10, 2017 in Replications

How should we deal with failed manipulation checks in replication studies?
Continue reading

Rich Lucas

Personality psychologist in the Department of Psychology at Michigan State University. Interested in personality and well-being, measurement, and replicability.

Professor

United States

no post found

Posts

Aug 8, 2019

Time For A Change At SPSP Journals

Aug 8, 2019

Disclaimer: This post reflects my own opinions and is not an official statement of policy at the Society for Personality and Social Psychology. I received permission from SPSP’s Executive Director to discuss the outcomes of the recent SPSP board meeting. Last week, I went to St. Louis to attend my final meeting as member at large for the Society for Personality and Social Psychology. Board members and elected officers use these meetings to discuss new policies and to make decisions about the society’s goals and priorities.

Measuring Happiness Is Harder (But Maybe Also Easier) Than You Think

Mar 3, 2019

tl;dr: Experiential measures of subjective well-being solve, in a clear and obvious way, one possible threat to validity that affects global self-reports. The obviousness of this advantage, however, sometimes leads people to overlook the fact that experiential measures may have their own unique threats to validity. Specifically, the fact that experiential measures require respondents to repeatedly answer the same questions over and over again may pull for responses that deviate from what the researcher expects.

How to Measure Happiness

Feb 2, 2019

Anyone who has ever taught Intro Psych knows that one of the most popular lectures is the one that covers visual illusions. It’s easy to understand why. Illusions like the famous Müller-Lyer illusion shown below provide a clear example of times when our perception of the world is clearly, and demonstrably wrong. Although it’s almost impossible not to see the three lines as differing in length, simply removing the arrows at the ends shows that they are all, in fact, the same.

Using R to Create Multiple Choice Exams

Jan 1, 2019

About ten years ago, I made one of the best work-related decisions I’ve ever made: Switching from Windows (and SPPS) to linux (and R). Not having to deal with the endless load times of SPSS or the inevitable Windows slowdowns that make you think, after six months of use, that you need a new computer—plus all the unfixable bugs, unexplainable crashes, and generally strange behavior—has done more for my blood pressure than any medication or change in diet ever could.

HARPing: Hedging After a Replication is Proposed

Jul 7, 2018

Remember the early days of the replication crisis? The first few high profile attempts to replicate famous psychological findings were not always embraced by the original authors (to put it mildly). Original authors (and their defenders) often resorted to name calling (even in blogs!) and other attacks that would prompt any self-respecting member of the tone police to write a scathing admonishment in a society-sponsored newsletter.1 Fortunately, those responses have softened over time.

Happiness Research During the Replication Crisis

Mar 3, 2018

In honor of International Day of Happiness, I’m posting my SPSP talk from a few weeks ago, where I discuss happiness research during the replication crisis. I talk about some good things about happiness research, and also some things that could be improved. But more importantly, I brag about my seventh grade awards, talk about the type of happiness research that I’m most skeptical about, praise data thugs, and tell the story of how we became replication bullies.

Yes, Your Field Does Need to Worry About Replicability

Sep 9, 2017

One of the most exciting things to happen during the years-long debate about the replicability of psychological research is the shift in focus from providing evidence that there is a problem to developing concrete plans for solving those problems. Whether it is journal badges that reward good practices, statistical software that can check for problems before papers are published, collaborative efforts to deal with limited resources and underpowered studies, proposals for new standards of evidence, or even entire societies dedicated to figuring out what we can do to make thing better, many people have devoted an incredible amount of thought, time, and energy to figuring out how we can fix any problems that exist and move the field forward.

W.W.P.M.D?

Jun 6, 2017

One of the most contentious issues in recent debates about replication studies concerns the importance of context in explaining failed replications. Those who question the value of direct replication often suggest that many psychological effects should be expected not to replicate because they depend so strongly on a multitude of seemingly inconsequential contextual factors. Thus, because you can’t step in the same river twice, direct replication attempts should often be expected to fail.

The Rules of Replication: Part II

May 5, 2017

Cece doesn’t understand the rules of the couch. Do replication studies need special rules? In my previous post I focused on the question of whether replicators need to work with original authors when conducting their replication studies. I argued that this rule is based on the problematic idea that original authors somehow own an effect and that their reputations will be harmed if that effect turns out to be fragile or to have been a false positive.