User:Katie.anderson.5/sandbox

Assignment 9

Katie and I updated our article in response to your comments you left. The finished work is still under the assignment 8 heading. Justin.a.arp (talk) 05:13, 8 December 2016 (UTC)

Article (Assignment 8)

Lead

The Reproducibility Project: Psychology was a collaboration of 270 contributing authors to repeat 100 published experimental and correlational psychological studies. This project was led by the Center for Open Science (COS){linked} and its co-founder, Brian Nosek, who started the project in November of 2011. The results of this collaboration were published in August 2015. Reproducibility is the ability to produce a copy or duplicate, in this case it is the ability to replicate the results of the original studies. The project has illustrated the growing problem of failed reproducibility in science.This project has started a movement that has spread through the science world with the expanded testing of the reproducibility of published works.

Reproducibility Project

Brian Nosek of University of Virginia and colleagues sought out to replicate 100 different studies that all were published in 2008. The project pulled these studies from three different journals, Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition, published in 2008 to see if they could get the same results as the initial findings. In their initial publications 97 of these 100 studies claimed to have significant results. To stay as true as they could the group went through extensive measures to remain true to the original studies, to the extent of consulting the original authors. Even with all the extra steps taken to ensure the same conditions of the original studies only 36.1% of the studies replicated, and if if they did replicate their effects were smaller than the initial studies effects. The authors emphasized that the findings reflect a problem that affects all of science not just psychology, and that there is room to improve reproducibility in psychology. Katie.anderson.5 (talk) 06:44, 24 November 2016 (UTC)

Statistical relevance

Failure to replicate can be caused by a few different reasons. The first is a type II error, which is when you accept the null hypothesis when it is false. This can be classified as a false positive. A type I error is the rejection of a null hypothesis even if it is true, so this is considered a false negative. Katie.anderson.5 (talk) 05:44, 24 November 2016 (UTC)

Center for Open Science

The Center for Open Science which was founded by Brian Nosek and Jeff Spies in 2013. They have built a team that today has about 50 individuals on it. The goal of the group is to help increase the openness, integrity and reproducibility of scientific research. The COS which is a rather small number of individuals oversee much larger groups that are helping with the COS's mission. The group is made up of multiple different kinds of scientists which include astronomers, biologists, chemists, computer scientists, education researchers, engineers, neuroscientists, and psychologists. Justin.a.arp (talk) 07:44, 24 November 2016 (UTC)

Outcome and importance

There have been multiple implications of the Reproducibility Project. People all over have started to question the legitimacy of scientific studies that have published in esteemed journals. Journals typically only publish articles with big effect sizes that fail to reject the null hypothesis. Leading into the huge issue of people re-doing studies that have already found to fail, but not knowing because there is no record of the failed studies, which will lead to more false positives to be published. It is unknown if any of the original study authors committed fraud in publishing their projects, but some of the authors of the original studies are part of the 270 contributors of this project.

Dr. Council's comments on Assignment 8

This is coming along, but not yet ready to publish. The main problems are:

1. The lead is too long and detailed. It should be a concise summary of the main article that follows. Many of the specific details in your lead should be moved to the main article.
2. Writing needs to be proofread and polished up. Needs to be properly formatted as a Wikipedia article. J.R. Council (talk) 03:58, 1 December 2016 (UTC)

Justin.a.arp (talk) 05:25, 24 November 2016 (UTC) Katie.anderson.5 (talk) 16:39, 23 November 2016 (UTC)

Note from Dr. Council on Group 3 combined lead

It is not clear to me from the next section which lead I should be reviewing. Since I believe it is the first one, "Combined Lead," I will comment on that one. Only Katie has signed this, so I need to know what Justin contributed.

• Also note, I have not seen any contributions from Tom. He should be aware that these are not difficult assignments, and every time he misses one his overall grade for the course will suffer. J.R. Council (talk) 21:46, 10 November 2016 (UTC)

Combined Lead

The Reproducibility Project: Psychology is a collaboration completed by 270 contributing authors led by Brian Nosek starting in 2011 to repeat 100 published experimental and correlational psychological studies from three different journals. [To meet the criteria for reproducibility, a study must.....] The group found that only 39 of the studies could be replicated with statistically significant results. This has made many start to question the strength of many other studies claims. It is unknown if any of the original study authors committed fraud in publishing their projects. While the authors emphasize that the findings reflect the reality of doing science and that there is room to improve reproducibility in psychology; they have been interpreted as part of a growing problem of "failed" reproducibility in science. The ideals of reproducibility mark the differences between what is considered science and a nonscience (Open Science Collaboration, 2012). The collaborators, 2012, state that reproducibility should be a main part of science to prove past thoughts, but the current model involves scientists and psychologists always trying to prove something new.

Dr. Council's comments: This looks good. There are just a couple of minor edits needed, and then you should proceed with developing your main article.

1. Be sure to include a concise but accurate definition of "reproducibility." I've indicated where you should do that above.
2. You need to include reference citations.
3. Since you are treating this study as a "done deal," you need to write in the past, not present tense. (However, as I stated in previous comments, reproducibility research in psychology is ongoing at Nosek's center.) J.R. Council (talk) 21:46, 10 November 2016 (UTC)

Katie.anderson.5 (talk) 03:26, 8 November 2016 (UTC)

Updated Lead

The Reproducibility Project: Psychology was a collaboration of 270 contributing authors to repeat 100 published experimental and correlational psychological studies. This project was led by the Center for Open Science and its co-founder, Brian Nosek, who started the project in November of 2011. The project pulled these studies from three different journals published in 2008 to see if they could get the same results as the initial findings. To stay as true as they could the group went through extensive measures to remain true to the original studies, to the extent of consulting the original authors. The group found that only 39% of the studies could be replicated with statistically significant results. This is down from the 97% that reported significant results in their initial publications. While the authors emphasized that the findings reflect a problem that affects all of science and that there is room to improve reproducibility in psychology. The project has illustrated the growing problem of failed reproducibility in science. It is unknown if any of the original study authors committed fraud in publishing their projects, but some of the authors of the original studies are part of the 270 contributors of this project.

Justin.a.arp (talk) 04:18, 8 November 2016 (UTC)

Lead Section Drafts Justin Arp

The Reproducibility Project is a collaboration lead by Brian Nosek in 2011. The project repeated 100 published psychological studies to test if they could reproduce them receiving the same results as the initial results. The group of 270 individuals found that only 39 of the projects could be replicated. This has made many start to question the strength of many other studies claims. It is unknown if any of the original study authors committed fraud in publishing their projects. In the scientific field, it is known that you can come across false positive results which is what could have happened with these original studies. Therefore, others are encouraged to attempt studies to test the reproducibility of projects. Justin.a.arp (talk) 02:37, 27 October 2016 (UTC)

• My impression of the Reproducibility Project is that it is ongoing, not just the single study described above. Am I mistaken about this? J.R. Council (talk) 19:45, 28 October 2016 (UTC)

Katie's Reply to Justin

I really enjoy the way you worded this, I feel like it is maybe an easier read, and since this is a good source for non-academic people, this is a great way to word the section to grab attention. I really like your last sentence about making sure others try this because it is really important to be able to test others to know for sure. Do you think we should talk about how there are projects that have no effect aren't published, but then there are the false positives so stuff like the reproducibility project have to happen? Katie.anderson.5 (talk) 04:28, 26 October 2016 (UTC)

Lead Section Drafts Katie Anderson

The Reproducibility Project: Psychology was a collaboration completed by 270 contributing authors to repeat 100 published experimental and correlational psychological studies from three different journals to see if they could get the same results a second time. The project was set up in 2011 by Brian Nosek and his collaborators. It showed that only 39 percent of replications obtained statistically significant results. While the authors emphasize that the findings reflect the reality of doing science and that there is room to improve reproducibility in psychology; they have been interpreted as part of a growing problem of "failed" reproducibility in science. The ideals of reproducibility mark the differences between what is considered science and a nonscience (Open Science Collaboration, 2012). The collaborators, 2012, state that reproducibility should be a main part of science to prove past thoughts, but the current model involves scientists and psychologists always trying to prove something new. Katie.anderson.5 (talk) 02:01, 26 October 2016 (UTC)

• My comment to Justin applies here as well. Isn't this an ongoing project? I know there is an institution set up to support reproducibility research. Maybe should mention that. J.R. Council (talk) 19:55, 28 October 2016 (UTC)

New heading in page STATISTICAL RELEVANCE

Failure to replicate can be caused by a few different reasons. The first reason is a type II error. A type II error is a false failure to reject the null hypothesis. This can be classified as a false positive. Katie.anderson.5 (talk) 02:01, 26 October 2016 (UTC)

Group Three's Assignment 4

1. I think something that should be added is the statistical reasons to why the researchers could be publishing unreplicable things, such as false positives.

2. Patil, P., Peng, R.D., Leek, J.T. (2016). What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Association for Psychological Science, 11, 539-544.

Nosek, B.A. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Association for Psychological Science, 7, 657-660.

3. Should there be a little info box about when the project was initiated and by who for easier access?

• I think an info box is a great idea! J.R. Council (talk) 14:15, 28 September 2016 (UTC)

Should there more people stated for not being able to reproduce studies.

• Not sure what you mean by this. You could certainly list some research projects that were not reproducible. J.R. Council (talk) 14:15, 28 September 2016 (UTC)

Katie.anderson.5 (talk) 02:48, 28 September 2016 (UTC)

1. I think that because the article is a stub we could improve it in many ways. A couple things we could look into are a few of the biggest studies it found could not be replicated and what were the implications of this.

2. Etz, A., & Vandekerckhove, J. (2016). A Bayesian perspective on the reproducibility project: Psychology. PloS one, 11(2), e0149794.

Patil, P., Peng, R. D., & Leek, J. T. (2016). What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Perspectives On Psychological Science, 11(4), 539-544. doi:10.1177/1745691616646366

3. Should we talk about the individuals that performed this project? Justin.a.arp (talk) 15:54, 28 September 2016 (UTC)

• Yes, that's a good idea. There is one person in particular who is a prime mover on this project. Don't recall his name right now. J.R. Council (talk) 14:15, 28 September 2016 (UTC)

Justin

• Justin - you had nowiki code that prevented your signature from taking. Just sign with the four tildes ~~~~ J.R. Council (talk) 14:15, 28 September 2016 (UTC)

To-Do

1. Add the research from the sources we found.

2. Add an info box. (I'm not really sure how to add this). There should also be a table of contents.

• just type WP:infobox into the Wikipedia search box. J.R. Council (talk) 22:20, 27 October 2016 (UTC)

3. Add information about false positives and the statistics behind that. So we need to find a source that details statistics.

• For statistics, see this article: http://science.sciencemag.org/content/349/6251/aac4716.full?ijkey=1xgFoCnpLswpk&keytype=ref&siteid=sci

J.R. Council (talk) 22:25, 27 October 2016 (UTC)

4. Find famous projects and studies that could not be replicated.
Katie.anderson.5 (talk) 22:46, 14 October 2016 (UTC)

1. go through each other's sources
2. table of contents that follow general outline format
3. find out more about individuals leading project
4. find most significant studies that could not be replicated Justin.a.arp (talk) 23:14, 14 October 2016 (UTC)

Outline

• Information box that quickly states what everything is Katie.anderson.5 (talk) 22:52, 14 October 2016 (UTC)
  
• HEADING - Reproducibility Project Katie.anderson.5 (talk) 22:52, 14 October 2016 (UTC)
  
• Explain what the project means, give its definition and who has been the driving force Katie.anderson.5 (talk) 22:52, 14 October 2016 (UTC) .
  • individuals leading project
  • why they decided to do this Justin.a.arp (talk) 23:33, 14 October 2016 (UTC)
• State specific studies that were not replicated Katie.anderson.5 (talk) 22:52, 14 October 2016 (UTC)
  • implications of findings
  • biggest studies comfirmed
    Justin.a.arp (talk) 23:33, 14 October 2016 (UTC)
• State why it's important Katie.anderson.5 (talk) 22:52, 14 October 2016 (UTC)
  • explain possible reason for non-replicable studies
  • why others need to be able to replicate studiesJustin.a.arp (talk) 23:33, 14 October 2016 (UTC)

Outline looks good: this will generate a nice lead and article. J.R. Council (talk) 22:29, 27 October 2016 (UTC)

New References

Nosek, B.A. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Association for Psychological Science, 7, 657-660.
Katie.anderson.5 (talk) 22:53, 14 October 2016 (UTC)

Open Science Collaboration. 2012. An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science,7, 657-660. Katie.anderson.5 (talk) 01:27, 27 October 2016 (UTC)

Patil, P., Peng, R.D., Leek, J.T. (2016). What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Association for Psychological Science, 11, 539-544.
Katie.anderson.5 (talk) 22:53, 14 October 2016 (UTC)

Wiki Information

Katie.anderson.5

— Wikipedian ♀ —
Name	Katie M Anderson
Current location	Fargo, ND North Dakota State University
Education and employment
Occupation	Psychology major Department of Psychology

My name is Katie and I'm a junior at North Dakota State University, I'm currently working on a project for my capstone experience class for my Psychology major.

My sandbox

This user is a student.

This user is a member of APS-Wikipedia Initiative.