Wikipedia talk:Ethically researching Wikipedia

Don't share off-wiki info on-wiki

I mentioned this on the mailing list as well. Something should be added along the lines of, "If a researcher learns something about a subject/Wikipedia editor off-wiki, don't disclose it to the community on-wiki unless unless the subject explicitly gives you permission. This applies regardless of how you learn the information off-wiki, including interviews, external websites, publiclly available information, etc."

Obviously, this is not meant to apply in any way to information published in reliable sources about notable individuals (who may also edit Wikipedia), which is appropriate for article content. Superm401 - Talk 02:15, 15 November 2008 (UTC)

Seems prudent.--_{Piotr Konieczny aka Prokonsul Piotrus| talk} 18:31, 15 November 2008 (UTC)

Ah, I misinterpreted. I didn't see the "external websites, public information..." section. Sure, I'll add this in. Any questions I have about off-wiki representations/slurs of editors would be best done in interviews instead of in public. Stu ^(aeiou) 03:33, 18 November 2008 (UTC)

surveys

I've just submitted a proposal to an IRB with a survey (not primarily of Wikipedians but of wiki users in general). It is important to note that surveys do require IRB approval (if, from my understanding, they are often expedited and easily accepted). I think we should discuss surveys in a separate section, and note various survey IRB-related issues that appear. Same for the interviews. Content research, which I've done before, DOES NOT, indeed, as far as I know, require IRB approval. --_{Piotr Konieczny aka Prokonsul Piotrus| talk} 18:36, 15 November 2008 (UTC)

Agreed that surveys get a separate section. Content analysis too. It falls under the "analysis of public data" exemption from IRB research. However, it is often recommended that one officially get an exemption from the IRB if doing a content analysis - I have done this before, and the turnaround was under 24 hours. Stu ^(aeiou) 03:07, 18 November 2008 (UTC)

children

My particular research avoids children since I survey social movement wiki activists, who are almost always 18+. But indeed, how do deal with the fact that many (we don't really know how many until the main survey is done) of anonymous Wikipedians are children. This should be addressed. "It is customary for Institutional Review Boards to waive the requirement for direct informed consent if the ethnographer works with the community to develop a commonly agreed-upon research protocol that establishes an alternative way of informing participants that they are the subject of research" - has this been verified with IRB experts? Can we really solve the children subject issue here? --_{Piotr Konieczny aka Prokonsul Piotrus| talk} 18:49, 15 November 2008 (UTC)

First, I wouldn't automatically assume that everyone is over 18. Second, there is no universal IRB - it is all local, and different boards have different interpretations that make them more or less strict than others. I don't think that the statement can be "verified" - I have been told that it is likely if it upholds the spirit of the requirement for IRB, but I can't speak about others.

With the children issue, I've also been told that the spirit of the requirement was to protect children from risky interventions that could cause them harm. I plan on showing that because I will be acting as an ordinary Wikipedian editor, my interventions will not pose any additional risk to children who are already participating in Wikipedia. Ethnographers of Wikipedia are not enrolling otherwise dormant children in the same way as, say, a psychologist who puts children in a lab and then makes them watch violent media. Stu ^(aeiou) 03:24, 18 November 2008 (UTC)

Bias

Probably all of us who will be involved in drafting this are what is known as complete members (Adler, Patricia and Adler, Peter. 1987. Membership Roles in Field Research. Sage Publications Inc.) - ones highly involved in group activities (and if we create content and discuss policies like this one, I am pretty sure we fit the bill). This should be noted, and we should add a statement along the following lines: "As members of Wikipedia community and Wikipedia editors, we realize that we may be biased in certain regards. We admit this, and we pledge to try to eliminated a possible bias of a complete member with a rigorous control of myself as suggested by Adler (1987). On the other hand, our familiarity with Wikipedia software, community, daily operations and types of data generated throughout the project allowes us a use a deeper level of understanding of those processes compared to a casual observer; thus our familiarity with the project should be more of a benefit than a hindrance." --_{Piotr Konieczny aka Prokonsul Piotrus| talk} 18:59, 15 November 2008 (UTC)

Cultural anthropologists will tell you that bias is quite a loaded term. :) Seriously though, it sounds good, and these are problems that any researcher has to juggle. There is always the danger of going native, although the native account is the intention of the ethnographer. Acknowledging specific ways in which we may be biased could help. Stu ^(aeiou) 04:01, 18 November 2008 (UTC)

Endorsements

Finally, I suggest we transform the "Staeiou's ethnographic research protocol" into a section that we can endorse by signing under it.--_{Piotr Konieczny aka Prokonsul Piotrus| talk} 19:00, 15 November 2008 (UTC)

I sure would appreciate this. I would be nice to show the IRB a list of Wikipedians who have endorsed the protocol. Stu ^(aeiou) 03:35, 18 November 2008 (UTC)

Still an active proposal?

Just trying to clean up Category:Wikipedia proposals so wondering if this is still an active proposal or if it can be tagged otherwise? Hiding T 09:45, 25 August 2009 (UTC)

American Association for Public Opinion Research

AAPOR and its international organization have ethical codes that should be listed, and followed by survey researchers. Similarly, the International Statistical Institute has an ethical code that governs statistical work. Researchers should abide by both.  Kiefer.Wolfowitz 04:33, 25 April 2011 (UTC)

@Kiefer.Wolfowitz: Good idea, I say be bold and add links to them to this page. --_{Piotr Konieczny aka Prokonsul Piotrus| reply here} 10:50, 28 March 2014 (UTC)

WMF's "rate this page"

WMF's "rate this page"

After answering a survey on an article, readers are invited to edit as part of WMF's strategy to recruit new editors.

I opened a discussion of the WMF's "rate this page" initiative to recruit editors from readers.

Such recruitment "surveys" are prohibited by the ethical code of public-opinion researchers.

Sincerely,  Kiefer.Wolfowitz 22:56, 27 February 2012 (UTC)

IRB

I am concerned about this fragment: "Nevertheless, researchers should notify their IRBs that they are conducting such research; IRBs have the responsibility to agree that a given activity is exempt".

IRB requirements differ between countries. While it is true that most social science research in the US would require an IRB approval, it is much less common outside US, where IRB are mostly restricted to biomedical sciences, and social science research into vulnerable populations (children, prisoners, etc.). Even within US not all researchers are affiliated with institutions that have IRB bodies (our article already notes in the lead that IRBs are required for federally-funded research only), and some are simply independent. As such, we should make it clear that IRB approval is needed only if the local bureaucracy needs it. From our perspective, it is not really necessary. I'll make some clarifying edits. --_{Piotr Konieczny aka Prokonsul Piotrus| reply here} 10:55, 28 March 2014 (UTC)

I feel that researchers should conform to the practices of their country or institution. In the United States, researchers should get IRB approval because all universities and research institutions require it. It is not about a federal requirement; it is about institutional guidelines and social pressure.

I agree that IRB oversight should only be necessary for research on Wikipedia if local bureaucracy requires it, but still that is a strong statement and I feel quite necessary because for whatever reason, many researchers who know they should get local IRB approval fail to do so. Somehow research on Wikipedia does not trigger their instincts even when other kinds of web-based research would. For those who do get IRB approval I would like for them to make it available to the public through a link on Wikipedia just as they would in non-Wikipedia contexts. Blue Rasberry (talk) 12:25, 4 April 2014 (UTC)

While I like BlueRasberry's idea for sharing IRB material among WP participants, I think we can probably try to go a few steps further, with regard to ethical oversight of research in/on WP. In particular, is there scope for a specifically-designated Wikipedia/Wikimedia-based IRB, that could provide some of these review services to the WP research community at large?

Here's why I suggest it. A key challenge derives from the ubiquitous distribution of web access devices and systems across varied, and often incompatible, legal regimes and arenas of social norms. Given the inherently trans-jurisdictional nature of any online study, there is a pressing need for harmonizing ethics observance regulations and guidelines for social science research in/on any online social network systems or other participatory cyber-environment like WP that spans across multiple dominions and administrative areas.

It turns out that newly-proposed changes in the U.S. policies and regulations governing human studies are likely to have a significant impact on the online research community. The latest notice of proposed rule-making was just issued several weeks ago, with comments due in December (See https://www.federalregister.gov/articles/2015/09/08/2015-21756/federal-policy-for-the-protection-of-human-subjects). While the new regulations pay particular attention to cyber-topics like personal identity, privacy, and anonymity, these policies also reach into professional fields that may not have previously been very familiar with human research ethics, norms, and practices. I think that it behooves us WP-ians to get out in front of this process, and find ways to conduct ethics oversight in a practical and coherent way.

I'm not sure how to coordinate a project like this, but yr comments would be appreciated. Apologies for not being involved with this Talk page earlier - it only recently came up on my radar. Tnx all! jxm (talk) 16:12, 16 October 2015 (UTC)

jxm In my opinion, the way to coordinate a project for better research protections would start with a web of one-on-one voice conversations among persons interested in the topic. When enough interested people are identified, and they come to know each other a little, then perhaps a small group could make a proposal for how things should be. That proposal could be taken to the Wikimedia community for general comment, then if there is consensus, the outcome could be adopted as project policy.

Speaking for myself, I love talking about IRBs and am ready to talk with anyone about research oversight in Wikimedia projects. You or any other Wikimedian may email me to arrange a video or phone chat, see what we have in common, then go from there.

Human subject research on Wikimedia contributors is already happening and developing its trends. Safety practices will need to be developed eventually. Blue Rasberry (talk) 16:32, 16 October 2015 (UTC)

Don't disrupt Wikipedia for research purposes

I wonder if we could use a section based on WP:POINT: don't disrupt Wikipedia through fake vandalism or such for research purposes. PS. This is already said at Wikipedia:Research#Advice_for_researchers. I wonder if we should merge this page with that? --_{Piotr Konieczny aka Prokonsul Piotrus| reply here} 11:16, 28 March 2014 (UTC)

Yes. All the information should be together until there is a enough to fork. Right now there is not much information to read at all. Blue Rasberry (talk) 12:26, 4 April 2014 (UTC)

Should be recommend that researchers of Wikipedia disclose themselves as editors in their papers?

In my experience it is an extremely rare practice for the researchers of Wikipedia to disclose even the fact that they have a Wikipedia account or have edited Wikipedia. Should we take a stance on that? --_{Piotr Konieczny aka Prokonsul Piotrus| reply here} 11:29, 4 April 2014 (UTC)

I see no need to do this. If I asked for anything, it would be this:

1. Join some kind of research registry, so that if some research project is disruptive, there is a central place for talking about that research. For this step give a name to the research
2. Put a real name on the research registry
3. Put an institution name on the research registry
4. Say, "I am conducting research in accord with my institution's policies"
5. optional - write a few sentences about the research

To talk about sock accounts, COI editing, disclosure, and all these other things are so much more complicated than necessary to address the usual problems which come from research. My primary motivation for wanting a registry is to make researchers think twice about intentionally disrupting Wikipedia by spamming users for interviews and surveys, or stalking users in a way that discloses their identity. I am not even saying that spamming should be prohibited; I just want to ask the people whose institution tells them not to do this to follow their own institution's rules. Those universities which allow disruption should have their staff allowed to disrupt until and unless we need more complicated bureaucracy on Wikipedia. University and foundation-supported disruption does happen, but not on a large scale and it does not need to be addressed now. Blue Rasberry (talk) 12:34, 4 April 2014 (UTC)

Merge from meta:Notes on good practices on Wikipedia research

I just found https://meta.wikimedia.org/wiki/Notes_on_good_practices_on_Wikipedia_research . I suggest this page should be merged and redirected here. --_{Piotr Konieczny aka Prokonsul Piotrus| reply here} 02:32, 30 July 2014 (UTC)