Wikipedia talk:Research/Archive 2

Archive 1

Archive 2

"Bureaucracy"

One of the common things I'm seeing in this discussion is the word "bureaucracy". However, no one has defined what exactly is bureaucratic about this policy and SRAG. As one of the authors of this proposal, I feel that this policy is very narrow in scope, and minimally bureaucratic. The basic functions are:

1. Recruitment must be done via SubjectRecruitmentBot (run by SRAG)
2. A centralized location where members of the community can discuss things before recruitment is done, and to which people can later be directed. This discussion centers around a small set of requirements that were drawn from concerns researchers have previously encountered in earlier attempts at recruitment on Wikipedia.

The result of the discussion allows or prevents the recruitment from occuring. There is much less bureaucracy involved in this policy than, for example, the policy on bot approval by the Bot Approvals Group. -- PiperNigrum ^(hail|scan) 14:49, 13 April 2010 (UTC)

I'd like to mention as well that the bureaucratic work (new proceedures/processes) that this policy introduces will be handled by researchers and those interested in research in Wikipedia. This will not introduce bureaucracy to editing practices in any way whatsoever. The only people who will have to jump through the new hoops WP:Research introduces are researchers and they are on board because they see the need. --EpochFail^(talk|work) 15:22, 13 April 2010 (UTC)

Exactly. A neat little bureaucratic fiefdom. Gigs (talk) 15:59, 13 April 2010 (UTC)

I don't see how this is anything like a feifdom. Keeping control in the hands of the community is the purpose of WP:Research and WP:SRAG. Researchers are willing to jump through hoops defined by the community to ensure that the community's needs are met. I don't want to make an enemy here, but I'm concerned that you are no longer adding to the discussion. --EpochFail^(talk|work) 16:15, 13 April 2010 (UTC)

I don't see the community here. I see a small group of "researchers and those interested in research" declaring new policy and heavyweight processes without telling the community about it until after it's already done. Gigs (talk) 17:11, 13 April 2010 (UTC)

The policy proposal went through all of the recommendations at WP:Policy#Proposals. It was also advertised on WP:VPP during its construction and RFC. This point has been brought up regularly in the discussions here. I think I'll start working on an FAQ to the top of this talk page so it doesn't get missed again. --EpochFail^(talk|work) 18:45, 13 April 2010 (UTC)

If you would, the notice at CENT described this proposal as "as proposed by the WMF". A link to that proposal would be much appreciated. Nifboy (talk) 19:15, 13 April 2010 (UTC)

I have spoken with representatives from the WMF about creating WP:Research, but I was not aware of their direct involvement in the work or discussion. It appears that Ipatrol was the editor who originally added the topic to {{cent}}. I think your best bet is to ask him/her directly. --EpochFail^(talk|work) 20:35, 13 April 2010 (UTC)

(edit conflict) What is substantially missing is community input. There's a huge difference between "Approval from SRAG" (a fiefdom) and "Approval from the community, through SRAG" (a representative meritocracy). Nifboy (talk) 17:17, 13 April 2010 (UTC)

The policy explicitly states that discussions are public and anyone, SRAG member or not, can participate in the discussion. Approval from SRAG comes from the community consensus obtained through discussion. That is the very notion of community involvement on Wikipedia. -- PiperNigrum ^(hail|scan) 18:42, 13 April 2010 (UTC)

This language could be improved to make the community involvement portion more obvious. I've made a quick pass at it with the intent of not changing much of the content. Please post on the talk page there if it is still confusing. Hopefully we can get the wording hashed out correctly. --EpochFail^(talk|work) 18:48, 13 April 2010 (UTC)

I was told somewhere that the WMF was involved. It might have been offwiki, likely IRC. Please prove to me if that is false so I can understand to what extent we can work here.--Ipatrol (talk) 22:34, 13 April 2010 (UTC)

I'm not sure what you are asking for here. I've brought up the fact that I've had (off-wiki) discussions with some members of the WMF about recruitment and their suggestion was to create a policy like WP:Research. If I've given you the impression that WP:Research holds any official endorsement by the WMF, please accept my apologies. --EpochFail^(talk|work) 16:39, 14 April 2010 (UTC)

I have changed the posting on WP:CENT to not incorrectly imply that this proposal has WMF backing. Gigs (talk) 20:00, 14 April 2010 (UTC)

This page does a very poor job explaining the ramifications

My primary objection to this proposal actually has little to do with SRAG (of which I am of two minds about, one being that user talk: is no big deal and the other being a passive opt-in of all editors to unsolicited not-related-to-content messaging). It's the fact that this page does nothing to inform researchers of the community norms violated in the cited examples of unmediated interactions, nor does it adequately inform editors of just what is going on in this policy. Having a policy page to point editors to after they've gotten angry isn't going to stop them from getting angry in the first place.

For editors, it needs to be crystal-clear that this policy is about researchers contacting editors, that vetted research will contact editors through User:SubjectRecruitmentBot, and that it is opt-out through {{bots}} or {{NoSubjectRecruitmentBot}}. The fact that I had to go to another page to find out how to opt-out is an indication to me this page is not nearly ready for prime-time. The questions this page needs to answer with respect to researchers are not "who are they", "what are they doing", and "why are they here" (these are self-evident) but rather "Why is the community letting these clueless researchers run amok in our Wiki?" and "What does the community get out of it?"

For researchers, it needs to be equally clear that we do not tolerate disruption and community vetting is not a guarantee nobody will be mad at you. Similarly, although the principles of merit and full disclosure are adequately explained in SRAG requirements, I'd like to also note disclosure is generally preferred even for projects that don't involve "Recruitment of research participants" (a phrase that sounds much more sinister than it needs to be).

As an interface for researchers, I'm not convinced SRAG is better than an official but less formal point of contact for researchers (WP:SUP mentioned above). Nifboy (talk) 19:13, 13 April 2010 (UTC)

An essay or guideline that merely described best practices would be sufficient guidance for researchers. I'm unconvinced that we need a small group of academics dictating who is or isn't allowed to contact me. Gigs (talk) 12:49, 14 April 2010 (UTC)

I think you have misread how SRAG works, which is understandable because the section about who is involved in discussions was poorly worded. SRAG is just there to organize discussions. They are not there to make decisions internally, but rather to offer advice from experience to researchers and help them advertise the discussion to the wider community. Discussions in SRAG should take place like RFC discussions with SRAG members being the regulars who make sure that a good process if followed. A subset of SRAG members would also have the responsibility of helping researchers use the SubjectRecruitmentBot to recruit for approved requests. If you are concerned about SRAG's processes, it would probably be easier to have a discussion there. --EpochFail^(talk|work) 15:29, 14 April 2010 (UTC)

I'll try to break up your list of objections below so that we can discuss each one separately. Please make modifications if I have missed something or have not worded your objections fairly. --EpochFail^(talk|work) 16:04, 14 April 2010 (UTC)

WP:Research does not inform researchers of the community norms violated in the cited examples of unmediated interactions.

Copied from Nifboy's post above:

For researchers, it needs to be equally clear that we do not tolerate disruption and community vetting is not a guarantee nobody will be mad at you. Similarly, although the principles of merit and full disclosure are adequately explained in SRAG requirements, I'd like to also note disclosure is generally preferred even for projects that don't involve "Recruitment of research participants" (a phrase that sounds much more sinister than it needs to be).

--EpochFail^(talk|work) 16:04, 14 April 2010 (UTC)

I agree that a section should be added that explains Wikipedia's concerns in relation to researchers, although I don't think that WP:POINT (the guideline you linked to) has anything to do with talk page postings, the idea that we do not tolerate disruption is a very good one to include.

As for the disclosure of projects that don't involve recruitment, how would you like to be notified when the study is taking place? Would you like the studies that only use publicly available database dumps to be disclosed or is disclosing only those studies where a researcher modifies Wikipedia or interacts with editors sufficient? --EpochFail^(talk|work) 16:12, 14 April 2010 (UTC)

Re:Disclosure, a central listing of interactions with live Wikipedia a la WP:SUP would be fine. Since SRAG as it is currently designed only deals with a subset of researchers, this page doesn't provide any guidance to all the other researchers out there who would like to interact with Wikipedia. For instance, linking to the database dumps would also be helpful to researchers. My impression was that, with a name as general as Wikipedia:Research, it would be (or ought to be) a central hub for all kinds of research-related information and not just something that might be more accurately called Wikipedia:Researchers contacting editors. Nifboy (talk) 17:22, 14 April 2010 (UTC)

I agree that WP:Research is too broad of a name for the content of the article as it stands. It is my intention to expand the article at a later date with policy for other interaction types as well. For example, I've been pushing interested ethnographers' to help write about good practices for disclosure, but the time/energy they have had to spend on it over the last couple of months has been nonexistent. Since best practices/policy related to recruitment was distinct from ethnography and there was time/energy to get that part hashed out, this currently the only prescriptive section within WP:Research. From my reading of WP:Policy this is acceptable and we'd probably want to re-RFC once language about ethnography was ready to be added. --EpochFail^(talk|work) 19:25, 14 April 2010 (UTC)

I'm not sure I'm comfortable adopting WIP pages as global policy anymore. This is being nitpicky, but if it were me I probably would have started a page like Wikipedia:WikiProject Research/Best practices, created a WP:Shortcut to it, make it a recommended-but-not-policy guideline due to its limited scope, wait for it to gain traction/be done, and then propose moving it into the Big Book of Official Policy. This page isn't necessary for the SRAG to exist, hence my initial confusion as to its purpose. Nifboy (talk) 20:25, 14 April 2010 (UTC)

Epoch, the whole page is prescriptive. It orders ("prescribes") an entirely new process and a set of standards that is not being used by the community. It does not describe the community's existing systems and standards, so it is not descriptive. The difference between description and prescription is its relationship to the wider community's actions, not merely word choice.

Your proposal might be "just what the doctor ordered" for the problem that concerns you, but it is a prescribed solution, not a description of the community's current response to that problem. WhatamIdoing (talk) 21:38, 14 April 2010 (UTC)

Thanks, you identified exactly what it was that didn't "feel right" about this proposal: It's asking the community to take a new, untested process and accompanying set of principles and turn it directly into policy. Nifboy (talk) 23:10, 14 April 2010 (UTC)

WP:Research does not inform editors about what is going on in this policy.

Copied from Nifboy's post above:

For editors, it needs to be crystal-clear that this policy is about researchers contacting editors, that vetted research will contact editors through User:SubjectRecruitmentBot, and that it is opt-out through {{bots}} or {{NoSubjectRecruitmentBot}}. The fact that I had to go to another page to find out how to opt-out is an indication to me this page is not nearly ready for prime-time.

--EpochFail^(talk|work) 16:04, 14 April 2010 (UTC)

From WP:Bot policy:

• Bots which deliver user talk messages are encouraged to provide a method of opting out of non-critical messages, and advertise that method on the bot user page.
• Bots which edit many pages, but may need to be prevented from editing particular pages, can do so by interpreting {{Bots}}; see the template page for an explanation of how this works.

When drafting this part of the article, it seemed reasonable to explain that there was a way to opt out and (given WP:Bot policy) direct them to the bot's page to learn how. In our draft of the recruitment message postings that SubjectRecruitmentBot might make, instructions on how to opt-out are included. From what I have seen, this is very common approach for other bots that make talk page postings. With that in mind, I'm not against adding an explanation of exactly how to use {{bots}} to opt-out in the policy. I'm just worried that it doesn't really belong there. --EpochFail^(talk|work) 16:24, 14 April 2010 (UTC)

My primary concern here is to answer the questions an editor might have immediately after being contacted for a study through SRAG's bot. Opt-out is one, telling vetted projects from phishing is another. Nifboy (talk) 17:57, 14 April 2010 (UTC)

The recruitment message should also have a link back to the sub-page of SRAG where discussion on this request took place. This somehow didn't make it into the message mockup. -- PiperNigrum ^(hail|scan) 12:31, 16 April 2010 (UTC)

With the instructions on how to opt-out, the link to the discussion, links to relevant policy/guidelines and post via approved bot (fingers crossed), it should be straightforward for an editor to tell which messages are phishing and which have been vetted. My own personal opinion is that it wouldn't hurt to include the information on how to opt-out wherever people might approach WP:Research, WP:SRAG, SubjectReqruitmentBot or the messages that are posted as well. --EpochFail^(talk|work) 15:08, 16 April 2010 (UTC)

WP:Research should answer the question, "Why is the community letting these clueless researchers run amok in our Wiki?"

Wouldn't this be best left to an essay that can be linked to from the lead of WP:Policy? Also, I don't think "run amok" is fair. Whenever researchers ran into trouble when interacting with users in Wikipedia, they were often being quite careful about their actions and never actually caused a problem with the content or processes of the encyclopedia. Instead, I think an essay could simply answer the question, "How does research of Wikipedia benefit Wikipedia?" --EpochFail^(talk|work) 16:30, 14 April 2010 (UTC)

I admit this was a bit of hyperbole, but the main idea is there; namely, that the community, by way of this policy (if accepted), accepts and actively encourages researchers to interact with Wikipedia, as opposed to the current status quo which tends to view everyone as an editor first and any other role second. Others have made reference to "special treatment"; it's something I feel should be acknowledged. Nifboy (talk) 17:34, 14 April 2010 (UTC)

If Wikipedians were only editors first, then there would be no place for administrators who primarily perform janitorial work and never actually edit articles. I know this statement is a little off-base, but I'd like to use it to make a point. These hypothetical admins are still good Wikipedians since they are doing work that helps Wikipedia. Their work may not directly result in improvements to articles, but overall, the community benefits from their efforts. Most Researchers of Wikipedia, in a general sense, are just a different breed of Wikipedian (whether intentionally or not) since the results of their work (increased understanding, new tools, etc.) directly benefit the project. Now, this isn't always the case. For example, market research or other commercial endeavors that merely seek to use Wikipedians as subjects without trying to increase knowledge about anything important to Wikipedia is probably not going to benefit the encyclopedia or the community in any way. Please notice that requirements have been explicitly listed in WP:Research under "Requirements for SRAG approval" that limit acceptable recruitment for research to the first category, studies with the goal of increasing understanding about Wikipedia. I see this policy as simply describing and making room for another type of work that benefits the community. I see no "special treatment" happening here. --EpochFail^(talk|work) 19:11, 14 April 2010 (UTC)

A less formal point of contact for researchers (WP:SUP mentioned above) might be better than SRAG.

Informal points of contact (see Wikipedia:WikiProject_Wikidemia and Wikipedia:WikiProject_Research) have already been created and they were apparently ineffective with regards to the problem of recruitment. --EpochFail^(talk|work) 16:34, 14 April 2010 (UTC)

Yeah, that's a fair point. Like I said, I'm more concerned about getting this policy to communicate effectively to both researchers and editors about each other and what best practices are instead of a very broad set of principles followed by a set of strict-sounding rules about a topic that isn't apparent from its heading. Nifboy (talk) 17:38, 14 April 2010 (UTC)

Support

I just wanted to say I think the general idea is good. I'm not worried about the details either way. Maurreen (talk) 20:24, 16 April 2010 (UTC)

Are you serious?

What's next? A policy on how to create a user account or edit as an IP when beginning to edit Wikipedia? A policy on how to use Jimbo's talk page? A barnstar policy? An April Fools Day policy? I reject this creepy page as a policy or guideline. It makes at best sense as an essay. Hans Adler 00:19, 16 April 2010 (UTC)

Thanks for swinging by to share your thoughts. It is usually more constructive to read and engage in the conversations we are currently having than just dumping your own ideas in a completely new topic. It gets time consuming to keep re-hashing the same discussions. If you read above, you'll see that the people working on WP:Research have been gearing up for some change that could address some of your complaints. If you could add your thoughts about the specific issues where they are applicable, I'm sure we can figure out how to best address them. --EpochFail^(talk|work) 20:00, 16 April 2010 (UTC)

This page is a complete waste of time. And it is precisely the kind of thing I would expect to see from editors who are only interested in the technical, Nomic and social networking aspects of Wikipedia, and have next to no first-hand experience of how it works. You may not have noticed (the fact that less than 10% of your few edits are to article or article talk space seems to suggest that), but this is a project to build an encyclopedia. If someone wants to research us, this can be handled with common sense, and without bureaucracy, rules, or other waste of time.

The only positive thing I can say is that it is at least more reasonable than WP:POLICE. Hans Adler 20:16, 16 April 2010 (UTC)

I would tend to agree. This policy seems like an attempt (well meaning) to layer BURO where it isn't needed. Obviously we have reason to worry that researchers may conduct research on wikipedia in a manner that might be deleterious to the encyclopedia; even where that research is approved by an IRB. But there are serious (fatal, if you ask me) execution problems. No one here has any authority over a prospective reader or editor to wikipedia. An attempt to assure that authority, even in innocuous ways is bound to fail. Unlike closed organizations, there is no ability to grant or deny entree to prospective researchers--this is important because we are highly unlikely to meet researchers before the research begins. Odds are we will run into them during the experiment or well into the setup. If we were a traditional organization, we would have been approached a while earlier in the process. Researchers would have to design their experiments around our organizational constraints. Obviously we would prefer that researchers respect our organizational constraints, but that is unlikely to go over smoothly if we interrupt them in the middle of an experiment in order to suggest that some "working group" has grave concerns about their ongoing work. An intervention like that is likely to engender both a culture shock and a rebuke and rightly so. More to the point, projects like this tend to accrete a certain crowd of editors who are uniquely concerned about chapter and verse of policy--policy which barely matters on en:wp, let alone in the real world. There are a dozen other problems, not least the complication of experiment design in order to confront ezposure of ongoing research to subjects. Etc. Protonk (talk) 22:59, 16 April 2010 (UTC)

Thanks for your comments. If you review the discussions on this page, you'll see that many of your concerns have been addressed quite thoroughly. The contributors of this page have been interpreting the recent rush of feedback (that finally came when the policy RFC was closed) and we are working out a plan to modify the content of WP:Research and WP:SRAG to account for this feedback. It would be more helpful to us if you would apply your comments to the work plan that so that we can be sure that your questions and concerns are addressed. --EpochFail^(talk|work) 16:09, 19 April 2010 (UTC)

This is an essay

See Wikipedia:Policies and guidelines for further guidance. If the editors of this page wish the community to look at the content and see if there is a consensus to upgrade this essay into a guideline or policy, then wider community input needs to be sought than has previously been the case. An RfC initiated which follows the Wikipedia:Requests for comment procedures - you'd need to place a {{rfctag|policy}} tag on a section of this talkpage marked something like: "Proposal to promote Research to guideline/policy", and then advertise the discussion via WP:Cent and Wikipedia:Village pump (policy) with clear wording. SilkTork *^YES! 11:31, 21 April 2010 (UTC)

Having looked at the page again, it seems that the proposal would be better off being organised as a Wikipedia:WikiProject - a gathering point for people with a similar aim. Those who wish to conduct research can place announcements on the WikiProject Research talkpage, and those Wikipedians willing to take part in that research can offer their time. SilkTork *^YES! 11:37, 21 April 2010 (UTC)

Ah - I see that has already been done - Wikipedia:WikiProject Research. SilkTork *^YES! 11:38, 21 April 2010 (UTC)

I removed the essay tag. This went through an RFC, objections arose after that, and work is in progress to resolve those objections. Thus, the "disputed" tag. Maurreen (talk) 11:51, 21 April 2010 (UTC)

SilkTork, please click here, read the first item, and then click here] and read the item at the top of the screen, and then come back and try again. WhatamIdoing (talk) 20:20, 21 April 2010 (UTC)

For good measure, please also click here. --EpochFail^(talk|work) 19:54, 22 April 2010 (UTC)

Clarification of the status of this page

Following a RfC, which was appropriately notified here and here, this page was marked as an official policy. Since then there have been various questions regarding the page's status as policy. There have been concerns that the community were not fully aware that this proposal was taking place, and that not enough people were involved in the decision to upgrade this to a policy. The aims of the project are to allow a bot to contact users for assistance in research projects outside of Wikipedia. SilkTork *^YES! 23:21, 21 April 2010 (UTC)

• Reject I do not support this as a policy nor as a guideline, and I would not be happy for a bot to spam my talkpage looking for assistance in non-Wikipedia matters. I feel that the project would work better as a WikiProject in which volunteers came forward to offer their time for the researchers. It could be advertised through SignPost, and approaches could perhaps be made to other WikiProjects, though as this Research project is essentially an off-Wiki project even that would need to be done with care and consideration. SilkTork *^YES!</Wikipedia_talk:WikiProject_Research#Examples_of_unmediated_interactionsspan> 23:48, 21 April 2010 (UTC)
  • A WikiProject has existed (see Wikipedia:WikiProject_Research and Wikipedia:WikiProject_Wikidemia) that did not end up meeting this need. This proposed solution has been brought up regularly in discussions on this talk page. Please review those discussions before you come to conclusions. --EpochFail^(talk|work) 19:51, 22 April 2010 (UTC)
• Support this remaining as policy. From a first reading, I think this is a useful, well-thought-out approach to an important issue: how to recruit Wikipedia editors as research participants. The researchers will not necessarily be Wikipedians, but that does not mean that their research will be of no value to our community. Self-selected WikiProject volunteers might be a useful basis for some research projects, but for results that apply to Wikipedia editors as a whole, a more controlled selection will usually be necessary. And you do not have to run any risk of being spammed, if you feel that research requests are objectionable: there is an opt-out mechanism described at User:SubjectRecruitmentBot. Perhaps this should be highlighted more prominently on this page. --Avenue (talk) 03:36, 22 April 2010 (UTC)
• Merge to Wikipedia:Subject Recruitment Approvals Group and allow that page to live or die by its own merits. There's no need to tie a policy to a process that cannot be described as "best practice" because it hasn't been put into practice. My opinion on SRAG itself is to give it a trial run and see if external researchers show interest; If there is interest, we can refine (or reject) the process as issues come up. Nifboy (talk) 05:47, 22 April 2010 (UTC)
• Keep disputed pending modification per the concerns that have been raised. This isn't the time take a poll over the current state of the article. The active contributors on this page are working on building consensus about the necessary changes and making those changes before holding another RFC. Please don't do anything rash until we have had a chance to complete our work. With that in mind, I do not feel that WP:Research is ready to be considered official policy at this time, but a poll to either accept or reject it is not constructive. --EpochFail^(talk|work) 19:51, 22 April 2010 (UTC)
• Keep disputed. I encourage those on both sides who are working to resolve the concerns and hopefully reach a broader consensus. Give the discussion process and the people involved a fair opportunity. Maurreen (talk) 22:56, 22 April 2010 (UTC)
• Mark historical and stop wasting your time. Demolishing an under construction house that was built on a floodplain in a place where no one wanted to live anyway is a smart thing to do. Gigs (talk) 15:56, 23 April 2010 (UTC)
  • It shouldn't be marked historical, because it was never implemented. The purpose of the historical tag is to label out-of-date procedures and advice, not rejected ones. WhatamIdoing (talk) 20:23, 23 April 2010 (UTC)
• Merge to SRAG. I think that proponents dramatically overestimate the value of a {{policy}} tag on their advice, and it is a little silly to call this untested idea either widely supported or the community's best practices. My advice is: Do something good in the world, and worry about how to classify it later.
  I don't believe that SilkTork has adequately considered the alternatives: Is being spammed by a bot (that you can easily opt out of, and that experienced editors can refuse to deploy if the scale or approach seems likely to be disruptive) really much worse than being spammed by individual newbies who've never heard of WP:CANVAS, who don't care if they get banned tomorrow (so long as they can spam a thousand editors today), and who have received zero feedback about their work? WhatamIdoing (talk) 20:23, 23 April 2010 (UTC)
• Keep disputed / Reject If a newbie is sending spam because they haven't heard of WP:CANVAS, what is the change that they know WP:RESEARCH? If anything i would say that this project is mainly intended for researchers with a somewhat broader understanding of the workings of Wikipedia, and not for the new people who do not know anything yet. However, there are two other things i notice:

The first issue is that the policy upgrade discussion doesn't seem to represent wiki-wide consensus. I noticed that a few editors had virtually no edits so i decided to check every editor against 3 criteria:

• Does the voter have more then 250 edits?
• Is the voter a researcher?
• Is the voter a member of WP:Research?

The edit count limit is arbitrarily chosen, but this was done to determine if an editor has reasonable wiki experience. The researcher bit was added to identify people who may have a conflict of interest with the subject. And the WP:Research part was added to see if the editor is closely involved with the project (Lack of third-part input can be a bad thing).

What i conclude is that only two voters meet none of these criteria. I count 6 researchers \ research students by only looking at their userpages, i see 5 users with less then 250 edits and another 5 who are members of the research project. I would also note that most people defending the policy somehow meet these criteria, and i cannot help but notice that there is a huge amount of GroupLens Research people here. The wikipage lists 25 people in that organization, and yet we have 5 or 6 people from that organization on this particular page being in favor of it. Nothing against GroupLens, but the amount is exceedingly high compared to the rest of the wiki population on this page.

Last, the idea behind the bot is likely unworkable. Bots which send message's tend to be a bit controversial, and therefor i doubt the bot will receive approval from BAG. The difference between this bot and other bots is that its message's are unsolicited; AFD patrol bot, CSD bot and so on also send messages to a lot of users, but those messages contain feedback on the users actions while this bot sends unsolicited messages at random; I don't think that an opt-out system is an excuse to assume that every wikipedian is a willing participant unless they refuse. Even if the bot gets past BAG i predict that a set of wikipedian's will complain, eventually resulting in shutting down the bot.

To conclude: I see a lot of good idea's here, but i think the execution is mediocre at best. I received mails before asking if i could participate in Wikipedia related research and i really don't mind doing that provided the research seems sensible. However the points of failure of this page are that researchers would have to know about this page before they can act upon it (Ask yourself: How many wikipedian's actually saw this discussion?), and the suggestion that a bot should be used in this way Excirial ^{(Contact me,Contribs)} 18:59, 24 April 2010 (UTC)

• Oppose policy or guideline status. It's not the kind of page that's appropriate as policy or guideline, because it offers no guidance, and it's not clear how it's connected to the improvement of the project. SlimVirgin ^{talk contribs} 21:07, 24 April 2010 (UTC)

• Oppose not convinced at all that there's a need for a guideline or policy, nor for the bureaucratic WP:SRAG. It seems to me that the best course of action is to simply make an information page for researchers and suggest/recommend that any mass notification of editors should seek approval (and warn of possible consequences of not doing it). If a user is willing to operate a bot for this kind of tasks, then we can make an informal arrangement that researchers request it to the bot operator which then can request approval to WP:BAG for the task. Anyone can comment in requests for task approval. Cenarium (talk) 17:53, 26 April 2010 (UTC)

I'm confused by your proposal. You suggest that an information page should direct researchers to seek approval for their actions, but then link to WP:BAG's approval group (which operates similarly to the way SRAG might, but is not designed to vet proposals to recruit editors). To me, it sounds like you are proposing that WP:SRAG exist, but that it should be incorporated into WP:BAG's responsibilities somehow? I just don't see how that makes any sense. --EpochFail^(talk|work) 15:12, 27 April 2010 (UTC)

One potential option in that vein would be to skip the SRAG, and have the SubjectRecruitmentBot (or CanvasBot or whatever it is called) require approval via a BAG process for each run, not just its existence. I'm not sure if that's a good idea or not, but it does roll the supervisory responsibilities into a group that already deals with such things. Elehack (talk) 16:14, 27 April 2010 (UTC)

BAG's purpose is to approve the existance of bots, I don't know that they'd be interested in taking on the approval of each use of a bot. Even if they did, there would still need to be (IMHO) some centralized place to propose and discuss each use of the bot where those with knowledge of research and those with knowledge of Wikipedia can comment, which is basically the framework of SRAG without the SRAG members who oversee the bot. Adding those members doesn't do anything remotely like adding "bureaucracy". -- PiperNigrum ^(hail|scan) 14:37, 28 April 2010 (UTC)

I think it's up to BAG whether they want to individually allow each use of the bot versus giving a newly-formed group carte blanche to approve requests into perpetuity. Nifboy (talk) 00:05, 29 April 2010 (UTC)

True. For every new task a new BAG request has to be made. I would argue that every new request to use this bot equals a new task, since every request has different parameters such as magnitude and message. Excirial ^{(Contact me,Contribs)} 13:34, 30 April 2010 (UTC)

Yes. BAG already considers all kind of requests and I suppose requests from researchers to recruit editors would be quite rare so it can fit without problem in BAG. When a bot is approved it is only for a specific task, the bot needs to be approved again for sensibly different tasks, except when completely uncontroversial. BAG members know when requests may receive opposition, need more advertising, etc, and anyone can comment in the discussions so the worthiness of the research for WP can be considered, so it makes sense to require BAG approval for each task. All this makes SRAG completely unneeded, and hazardous to try. Cenarium (talk) 15:05, 30 April 2010 (UTC)

I don't think the fact that BAG may have to approve every set of recruitment messages is the problem we were discussing. The problem is that it isn't BAG's job to interpret a researcher's plan. The reason we proposed SRAG in the first place is because having a discussion group that is actually familiar with research, IRBs, risk to subjects, statistical significance, good practices and the state-of-the-art would probably yield a better decision-making mechanism. Sadly, research isn't just something you can expect anyone to be able to approach and comment on. Now, I'm not trying to say that the average Wikipedian would not be welcomed/encouraged to participate in discussions and let their concerns/ideas be heard. It would just be important that the regulars (who are familiar with the research, recruiting, etc.) would be available to put the discussion into context, dispel misconceptions or make salient the subtle points of great interest to the community. --EpochFail^(talk|work) 14:58, 30 April 2010 (UTC)

(after e/c) Well, there's no need to set up a specific group for that, if people want to comment they can do so at the request for approval, and notification can be sent to the research wikiproject. Also, BAG members do have to consider such factors from time to time, bot requests can be quite peculiar on occasions, and are varied. Cenarium (talk) 15:34, 30 April 2010 (UTC)

Your proposal seems like it would only extend BAG's responsibilities (and its beaurocratic processes) to cover a subject matter they are likely to be completely unfamiliar with. It seems to me that this solution is more strongly subject to your concerns about WP:SRAG than WP:SRAG itself. --EpochFail^(talk|work) 20:46, 3 May 2010 (UTC)

Exactly what part of "use of a bot to contact editors" would BAG not be familiar with? If anything, I'd trust BAG more because they're more familiar with Wikipedia norms than researcher norms. Nifboy (talk) 00:30, 4 May 2010 (UTC)

• Mark as rejected. This proposal suffers from a complete disconnect from community standards and has no chance to be implemented. Yes there was an RfC, yes it was properly advertised, but no, it didn't bring in enough feedback, or feedback of sufficient quality, to turn something into a guideline, much less a policy. This can happen when something has been worked on in a quiet spot by a small number of enthusiastic editors, and the only gets announced as an RfC and on the Village Pump. Most experienced editors do not watchlist RfC pages, and the Village Pump is so full of noise that it's easy to miss something even if you haven't taken it off your watchlist for this reason. These things used to happen more often, and to address this problem we have WP:CENT, which unfortunately wasn't used in this case.

In addition I must say that I was at first sure that this was April Fools stuff that was originally planned to be advertised on the 1 April but for some reason had been continued after that date. Other editors, who saw this in March, may have had a similar impression and therefore refrained from alerting a wider circle to what was going on here. Hans Adler 15:42, 30 April 2010 (UTC)

The policy RFC was advertised multiple times on the Village Pump and mailing lists as it was being worked on, long before the RFC. Please cease repeating this falsehood. If you feel that a WP:CENT should be required for policy RFC discussion, I recommend that you start a discussion about adding it to the recommendations there. --EpochFail^(talk|work) 20:52, 3 May 2010 (UTC)

My comment wasn't about you failing to get some details of bureaucracy right. I think I have made it very clear that the problem is that, for whatever reason, your RfC didn't get the kind of high-quality feedback that it deserved. As a result you were left under the impression that there was consensus for your proposal when actually there is no such thing. This can happen, and it's unfair to you. It's not my problem. You can simply ignore me and try to push this through. This will be my last comment here because you don't seem to be listening anyway. If you want to see how a project that is out of tune with the general community can fail miserably after a while, have a look at WP:Esperanza.

You are trying to fix a problem that doesn't even exist in the first place by introducing a large amount of bureaucracy. That's not what Wikipedia is about. Wikipedia is about writing an encyclopedia with, theoretically, as little bureaucracy as possible. (It keeps creeping in and is hard to get rid of, but theoretically we try.) Sorry if this is a bit personal, but your editing statistics (less than 10% of your relatively few edits were in article or article talk space) easily explain why you haven't got a clue how this project works on the social side. Hans Adler 00:59, 4 May 2010 (UTC)

Part of the problem here is that terms such as "problem that doesn't exist" and "bureaucracy" are being tossed around ad nauseam, when (from our point of view) there is a clear problem, which this policy and accompanying group are addressing with a narrow focus which, if anything, puts up a hurdle for researchers, and not Wikipedians. One which, as can be seen from above, researchers seem willing to accept. I have yet to see a single example that demonstrates that this is not a problem, nor any explanation of how this is some sort of bureaucracy. -- PiperNigrum ^(hail|scan) 02:58, 4 May 2010 (UTC)

Has anyone ever tried to do the kind of research envisioned in this proposed policy using the existing methods? It seems to be talking primarily about unsolicited messages to editors, so to make the question a bit more precise: Has any researcher ever asked for approval from WP:BAG for a bot that invites a random sample of editors to help with a research project, and was rejected or set unacceptable conditions? The procedure would be: Write the bot, or find another bot that can do what you need, and then ask the BAG for approval of a bot account that uses the bot for precisely what you need it for (e.g. once, in May or June 2010, contact a random sample of 2,000 editors among those who were active as of 1 May 2010 and who have made between 1,000 and 10,000 edits; leave each of them the following message: [...]). It is precisely the BAG's job to estimate whether such a proposal is going to cause an uproar or not. They are the experts for that kind of thing, and I don't see why we would need parallel structures before the existing structure has even been tried. There is a real danger of independent communities developing that have mutually inconsistent norms and standards.

You are proposing a new process and new rules based on what you think is reasonable and could work rather than flowing naturally out of existing practices. That's very unusual and unlikely to work. Hans Adler 09:50, 4 May 2010 (UTC)

Firstly, I see that there appears to be another misunderstanding regarding the policy and the bot. The bot should be immaterial to this discussion. The policy is not about the bot, the bot assists in applying the policy. Consider it this way -- in WP:Research, and WP:SRAG, replace all instances of what the bot does with "a member of SRAG" doing that task. The bot merely simplifies that execution of the task, making it easier on the implementer, and BAG's job is to approve the existance of the bot, and not its individual use.

Yes, this policy is being sketched out based on what is reasonable, which is why the focus was narrowed to address only one aspect of research. We have had input from non-researchers on their perspective of what we felt was reasonable, and believe we have a good framework which does appear to need a little refinement. We anticipate that what will flow naturally is the extension of this policy in covering other aspects of research on Wikipedia that are not addressed by this policy. Again, as shown in the examples of prior interactions (which are merely a few examples of those which I am immediately aware), what is currently occuring is not working, and thus something needs to be made into policy. -- PiperNigrum ^(hail|scan) 14:56, 4 May 2010 (UTC)

No. It makes no sense to do such notifications by hand because it would be too tedious. Therefore we can safely assume it will be done by a bot, and for bots we have the BAG. The BAG's job is not just to check the source code or whatever you imagine it is. Just look at WP:Bots/Requests for approval to see some of the real discussions and what parameters they are interested in. The BAG must evaluate the overall context of a proposal, because they don't want to block anything beneficial but will be blamed if they approve something that causes uproar. But of course it's more attractive for Wikipedia researchers to build their own new project, which they can control, where they can approve each other (or themselves), than to work with an existing process. This kind of thing is closely related to the problem known as a WP:walled garden.

The problem you have had with your RfC demonstrates that your strategies for getting feedback from the community were not sufficiently effective. Hans Adler 17:05, 4 May 2010 (UTC)

Yes, it is highly likely that these tasks will be done by a bot, which is why it is in the policy, however, whether they actually are done by a bot, or some individual with a lot of time on their hands is, again, immaterial to the WP:Research policy, and WP:SRAG. The current state of affairs is more akin to a walled garden with Wikipedians on one side, and researchers on the other. This policy works to break down that wall, and have both sides working together for the benefit of both.

The problems with the RfC have nothing to do with our strategies for getting feedback. We did everything we could to get additional feedback from the community, and their "ineffectiveness" is a failing of the community to respond, and not of the process of seeking that response or its application. This is a phenomenon not uncommon to Wikipedia. Many discussions on RfC, AfD, MfD, ANI, etc only involve a small number of participants, and it doesn't pose a problem. However, perhaps this is an area for futher research to make such situations more effective, which is beyond the scope of this policy and a discussion for another time. -- PiperNigrum ^(hail|scan) 18:15, 4 May 2010 (UTC)

I give up. You are obviously not interested in how this project works but simply coming here with your preconceived notions and trying to explain everything that happens by fitting it into them. I hope you are not actually one of those people trying to research Wikipedia. Hans Adler 18:23, 4 May 2010 (UTC)

Perhaps I do have some preconceived notions about how I believe things work. So does every single contributor to this discussion, yourself included. We have been working to dispel your (the general your) preconceived notions that this policy was not well advertised, that it is not necessary, and that it's some sort of bureaucracy. Not a single person on the other side has made any arguments supporting any of these statements, only repeating them ad nauseum, so we have no other option than to correct your misconceptions. -- PiperNigrum ^(hail|scan) 18:40, 4 May 2010 (UTC)

The way to dispel preconceived notions all around is to show, not tell. That's why Hans asked if there had been any research-related BAG requests before: to show that there was prior consensus for such a task. That's why I'm asking for a trial run, below: to show if there's at least some demand for the task and there's consensus for the task on a one-time basis. You're right that the task could be done manually, but approval from BAG to do it shows that a group whom the Wikipedia community already trusts has given the task the go-ahead. Nifboy (talk) 20:52, 4 May 2010 (UTC)

"Under development" compromise?

I think everyone who has commented recognizes that this does not have wide consensus. As far as I know, it isn't being used yet. My understanding is that one or more of the proponents are working toward something that will be more satisfactory to more people.

Thus, regardless of the tag on the page, it's fair to call this "under development." Maurreen (talk) 19:12, 24 April 2010 (UTC)

Agreed. I've switched the tag to {{Draft proposal}} so that we can finish the work plan without confusing any readers. --EpochFail^(talk|work) 20:00, 13 May 2010 (UTC)

A trial run?

My primary objection at this point is that this proposal has been generated primarily from whole cloth and submitted for approval without having undergone any kind of testing or trial run. In particular, nobody seems to have any idea of what kind of volume SRAG would be getting. Ideally, I'd like to see some success stories equivalent to Wikipedia:WikiProject Murder Madness and Mayhem for WP:SUP, that really highlight the good that can happen when researchers interact with Wikipedia, not just the unsuccessful attempts. Remember, SUP generates real duds too, and they're also a problem the same way unsuccessful interactions with researchers generates ill will amongst the community.

In my mind, the basic plan of action goes something like this:

1. Find a researcher who wants in on this; spend as much time on this step as you feel appropriate doing the things SRAG would be doing (Vetting the research etc)
2. Create (or request) a one-time-use bot and submit it for approval
3. If the bot is approved, run it.
4. Iterate based on the feedback you receive, both from researchers and the Wikipedia community.

In my humble opinion, only once SRAG becomes a really good idea in practice will it be eligible for becoming a permanent part of Wikipedia. I've seen too many half-baked, inactive process proposals to trust one with a {{Policy}} tag without seeing it in action first. Nifboy (talk) 05:18, 4 May 2010 (UTC)

To answer your earlier question, I have used BAG when I wanted to use a bot and RfC to pester editors via their talk page. See Wikipedia:Bots/Requests_for_approval/InactivityEmailBot Wikipedia_talk:Requests_for_adminship/Inactive_admin_email. This is the way that it should be done, via consensus at a normal RfC/BAG, not via some group of academics that aren't in touch with the community. Gigs (talk) 20:35, 6 May 2010 (UTC)

The bot in question would need only one approval; the list of users to contact would by necessity not be part of the approval process. The list of users to contact and the contact message would typically be the end result of the discussions mediated by the SRAG. By the same logic you're applying here, many existing processes could be abandoned and replaced with RfCs... but the structure applied by a formal process ensures that all appropriate considerations are addressed, that experts in the area are paying attention, that the approvals process can undergo a continual process of improvement and refinement, and that a historical record is maintained in a centralized location. Josh Parris 09:58, 7 May 2010 (UTC)

My question then becomes, why can't we have that same discussion at BAG? If each individual request requires a separate approval process, why not just treat it as a new task for BAG? They have technical expertise and the trust of the community, and I doubt we would be swamping it with requests. Nifboy (talk) 16:21, 7 May 2010 (UTC)

Although BAG does have the technical expertise to apply the discussions, they are unlikely to posses a subject (Research, itself, is a subject) expertise. --EpochFail^(talk|work) 21:49, 7 May 2010 (UTC)

Any researchers who need the research expertise of the SRAG should probably be prevented from bothering Wikipedians with their research anyway. Wikipedia is not the playground for incompetent would-be researchers. The important thing that the BAG possesses, and the SRAG does not, is social expertise. They know how a wiki functions. They have experience in judging whether mass edits such as notifying hundreds or thousands of users on their talk pages about some research project are likely to create disruption or not. They know what they need to do when they have misjudged a situation, in order to minimise the disruption. The last thing we need here is a cuckoo's egg project, backed up with a policy, and consisting of "editors" with no edit history to speak of who are causing disruption by rubber stamping each other's projects with no real checking of social acceptability. Hans Adler 22:15, 7 May 2010 (UTC)

I think you'll find that the depth of understanding required for scientific analysis of the reality (and the evaluation of such analysis) to be quite nuanced and complex. It's only reasonable to assume that someone whose career is not in that field would have a few misconceptions. Generalizing researchers and suggesting that they are "incompetent" does not add anything to this discussion besides hurt feelings and negativity. SRAG's discussions would be open to the public and only coordinated (not closed, decided, etc.) by actual SRAG workers. The same people that participate in BAG discussions could participate in SRAG discussions. --EpochFail^(talk|work) 22:38, 7 May 2010 (UTC)

My point was that it's the researchers who are supposed to have the research experience. They had better know what they need, and they had better be able to say what they need. Then someone else, who doesn't have to be a researcher, can say whether it's feasible or not and can make alternative proposals which may or may not work for the researchers. And so on. It's called dialogue. It's one of these magical things that humans manage to do using the technology called "language". We don't need a new process for this. Hans Adler 22:49, 7 May 2010 (UTC)

Finally! I see the confusion. For better or worse, most research today (in any field) is done by graduate students who are in the process of learning how to become researchers. It is a career path not unlike an apprenticeship between an advisor and a graduate student. Most of the researchers approaching Wikipedia will be newbies to the system. It is unreasonable to expect that they understand Wikipedia at the level of an insider (Wikipedian). However, they bring with them subject expertise in their field (social psychology, data mining, user interface design, etc.) that allows them to inspect and bring insight into a small part of the Wikipedia ecosystem (e.g. article deletion decisions, vandalism detection, administrator review, asynchronous article editing, detecting sock-puppetry, etc.). --EpochFail^(talk|work) 19:34, 10 May 2010 (UTC)

I'd support this proposal if it offered something significantly different from what has been attempted in the past and failed. I can't imagine a researcher being willing to spend the time and energy to design an experiment around Wikipedia and its users only to take the chance of being banned and having all evidence of their existence deleted from the system (see this and [1]). I myself have hopes of bringing state-of-the-art technologies to Wikipedia and using the scientific method to understand where these technologies are make a difference, but I'm not about to spend all of my time constructing a system that editors would be able to use without a process put in place to let me recruit subjects. If we try the trial run without a policy/guideline to fall back on, no one will want to waste the time organizing an experiment. --EpochFail^(talk|work) 21:44, 7 May 2010 (UTC)

We are at an impasse, then, if no researcher wants to work without written policy and the Wikipedian community does not want policy that hasn't been tested. Nifboy (talk) 22:25, 7 May 2010 (UTC)

How do you test a process without putting it in place and seeing what happens? As you know, nothing in a wiki is written in stone. If something isn't working right, then the process can be changed. What you are suggesting is a test run without a crucial part of the system in place. Would you do several months of work if you were unsure if that work would mean anything once you needed to recruit editors? It is not unreasonable that researchers would not want to waste months of work. Research of Wikipedia is very valuable to Wikipedia--so valuable, in fact, that the WMF has been supporting us for years (see meta:Chief Research Coordinator). We are not trying to re-invent some critical part of Wikipedia's beaurocracy here. We are just trying to figure out a mechanism for researchers to do their work. --EpochFail^(talk|work) 16:46, 8 May 2010 (UTC)

This is Wikipedia, you don't "put process into place", you just DO IT. Once you've DONE IT and people can plainly see whether it's a good idea or not, then you can propose making it into policy. Nifboy (talk) 18:54, 8 May 2010 (UTC)

Surely there must be exceptions to "just going it". BAG itself must have started with the Bot flag/Bot policy. Otherwise, there would be no obligation to go through BAG to run a bot. I think that WP:BOLD only applies in cases where the right choice seems obvious and/or the success of a bold decision/action does not depend on the formal process being in place already. In the case of recruitment, I think that the formal process (whatever the process is) is necessary for success. When it comes to bots, I'm sure that BAG would be less effective without the bot flag and the WP:Bot policy that states all bots must be approved and which characteristics should be considered before approval. --EpochFail^(talk|work) 17:49, 11 May 2010 (UTC)

Prior to BAG (est. 2006), bot approvals were done for years on Wikipedia talk:Bots, and even to this day bot flags can only be given by a 'crat. The difference between getting approval to run a bot and getting approval to contact editors is that a bot can do a fragton of damage to the Wiki in a short period of time. I'm not convinced researchers need a mini-cabal to do something for which there isn't a lot of evidence for an overwhelming volume of requests and the immediate problem is a handful of researchers being bitten by the community. I think if you were to think of SubjectRecruitmentBot as an optional aid to researchers instead of a mandatory hurdle, and redesign the process accordingly, the process would become much more palatable to both sides. Nifboy (talk) 21:17, 11 May 2010 (UTC)

I'm starting to agree that going through SRAG would benefit from being recommended rather than manditory. Although I think it may benefit the community and researchers in the long run for the process to be mandatory (such a strong word) for the category of research recruitment, I agree that, right now, the process of going through SRAG could be strongly recommended and still result in significant good. If this change was implemented and the rest of the work plan completed, would you support this becoming policy in the next RFC? --EpochFail^(talk|work) 19:41, 13 May 2010 (UTC)

To my business training it makes more sense to think of it as a service being offered to researchers: "Here, let us be a cultural ambassador for you, all we ask is you respect us and make public your findings". Mandating it might make sense down the road, but until it's a proven idea all it is is an idea. As such, convincing the community to give you an "official" recommendation through policy, before it's seen the process in work, is not the way I'd go. Nifboy (talk) 22:08, 13 May 2010 (UTC)

The only reason I would like to refer to it as a "recommendation" is that I'd like researchers to be able to point to the fact that they followed policy and recommendations if their activities be called into question later. If we can find another way for them to express that they did exactly what was asked of them (demonstrating diligence and good faith), I'd be in favor of that. --EpochFail^(talk|work) 15:39, 14 May 2010 (UTC)

That would be SRAG's/your job as mediator; to field inquiries and complaints, by knowing the policies and giving recommendations to the researchers. A human being makes a better shield than policy, since some people will argue policy all day if they want. Nifboy (talk) 16:06, 14 May 2010 (UTC)

There is an obvious problem with researchers: They come here, knowing nothing about our community norms, and do things that some of us don't appreciate. There is also an obvious problem for researchers: They don't want to become editors and learn a lot about Wikipedia before they can start their research. After all, they think of themselves as proper researchers, not anthropologists.

Fortunately there is a very simple solution to both problems: Create a project for experienced Wikipedians who are willing to support the researchers. A place where researchers can go and explain what they are planning. And then the experienced Wikipedians can say: "Oh, no, if you do that you will be banned. But how about this minor change: [...] If that's acceptable for you, then I can help you get approval for a bot that will do it. The bot approval will immunise you against any pedantic teens who might try to enforce the letter of the canvasing policy against you even though it makes no sense. Oh, and you had better email this guy at the Wikimedia Foundation so that you later say they knew what you were doing."

The beauty of this solution is that it doesn't need any written rules. But after a year or so some practices may have crystallised, so people will write a guide to doing research on Wikipedia, and then it might become a guideline and if necessary even a policy. Hans Adler 23:00, 7 May 2010 (UTC)

Please cease your attacks and generalizations on the personal qualities of researchers. Such statements could never contribute to a conclusion of this discussion. Your proposed solution has already existed and failed to meet the need. See Wikipedia:WikiProject_Research and Wikipedia:WikiProject_Wikidemia. This fact has been brought up regularly on this talk page. Please see previous discussions. We might need to start an FAQ. --EpochFail^(talk|work) 17:00, 8 May 2010 (UTC)

Did it fail because of a lack of an approvals process or a lack of anyone knowing it existed? If the latter, creating SRAG isn't going to change anything. Nifboy (talk) 19:22, 8 May 2010 (UTC)

In my opinion, the two WikiProjects failed in this role because they lacked a formal process that could ensure that the researcher's and community's concerns are addressed before recruiting took place. The fate of the researchers involved was left to stretched interpretations of WP:CANVASS and some assumptions that, if it isn't expressly allowed by policy, it is forbidden. (This was, of course, fuelled by many people's misconceptions about who academic researchers are, but that is besides the point.) SRAG would offer a formal process that could be refined. SRAG's decisions would also offer some form of support for a researcher's action. If the community finds that a particular recruitment style, methodology, etc., was/is inappropriate, then the process by which the study was approved can be changed to ensure the problem does not happen again rather than having the researcher censured. --EpochFail^(talk|work) 19:23, 10 May 2010 (UTC)

Okay, just be aware that awareness is necessary. If researchers aren't aware of SRAG and try to carry on without it, SRAG just becomes another reason for the community to bite them. If the community isn't aware of SRAG, then the community's overreaction isn't going to change. If nobody's aware of it, SRAG just dies a death of inactivity. Nifboy (talk) 00:35, 11 May 2010 (UTC)

I completely agree that it will be necessary that researchers are aware of SRAG in the same way it is very important that bot developers know about BAG. Even if "the community" is unaware of SRAG, that fact that there is an approved-via-consensus process which the researchers followed should have a calming effect and redirect frustration from the researcher and their work towards SRAG and the process by which recruitment is approved. Rather than having a researcher banned, concerned editors can join the discussion at SRAG to make sure that their concerns can be raised in a forum where they'll have a lasting effect of future concensus. --EpochFail^(talk|work) 17:43, 11 May 2010 (UTC)

I am not against the researchers. I want to support them in a way that is consistent with how we approach other things. I am not attacking them. I am analysing the problem and pointing out an acceptable way to solve it. In my opinion it's juvenile (at least in most situations) to attack a researcher for openly asking editors to help with a study – at least so long as this doesn't happen so often that it keeps editors from working on Wikipedia. This is obviously not what the canvassing and spamming rules were made for. Unfortunately we have lots of immature pedants who are looking for excuses to hit adults with rules, and we must always take this into account here.

It's not my fault that you have decided with your very little experience that you can simply approach things the way that seems best to you and ignore the feedback you are getting. Hans Adler 22:38, 8 May 2010 (UTC)

The feedback that has been given before and after the RFC has not been ignored. If you'll notice the work plan topic that I started on this talk page, you'll see that I proposed a plan for taking the feedback that was given and incorporating it into WP:SRAG and WP:Research. At the time, the only active editor with criticisms (Nifboy) approved of those changes and I have been hard at work writing essays and trying to keep the discussion alive and civil here. You are right to say that I have limited experience with Wikipedia, but who doesn't? This was the first RFC I started, but I did it to the letter of WP:Policy with every recommendation met. I even went so far as to wait two weeks after the last !vote in the RFC to make sure it was closed by an independent editor. I've also remained completely civil throughout these discussions despite several attacks on my character and profession. All I want to do is find a way for important research to happen. We disagree on the best way to make that happen. Let's talk about that instead of each other. Alright? --EpochFail^(talk|work) 19:59, 10 May 2010 (UTC)

Work plan

Based on some ideas brought forward since the closing of the RFC and the 20-25 minutes of WP:Research's life as an official (at least technical) policy, I'd like to propose a plan of changes in preparation for the cycle of advertising for feedback again. Please feel free to edit. --EpochFail^(talk|work) 21:00, 16 April 2010 (UTC)

---

1.  Done An essay needs to be constructed to inform those who don't know what researchers are doing here or why their work benefits Wikipedia.
   • If there is to be room in Wikipedia for researchers and their work, it appears that an appreciable portion of the community must be convinced of this.
   • A careful line must be drawn between the types of research Wikipedians should be wary of (research for commercial gain) and research Wikipedians should embrace (research for scholarly understanding).
   • Researchers have a history of working with the WMF. An official endorsement of some sort might be helpful.
2. WP:Reseach needs to be refocused/expanded.
   • The article must talk generally about research of Wikipedia. For example, research must not be disruptive and must not break policies/guidelines. Researchers should take advantage of WP:VP, WP:HELP, WP:SRAG, Wikipedia:WikiProject_Research, etc.
   • Rewrite to consider the two interested audiences that are likely to approach it. Per Nifboy's comments, it must be crystal clear to someone even skimming the page what this would mean for editors and researchers.
     • We should keep in mind that when researchers read this, they may be unfamiliar with the structure of Wikipedia and the way the community expects them to behave. We should also keep in mind that when editors read this, they may have no idea what researchers are doing here or why they should be offered any accommodations whatsoever.
   • Change the SRAG process from being "mandatory" to "recommended".
     • Emphasize that receiving SRAG's approval before recruiting offers credibility to a researchers work and allows the researcher to point to the discussion and say that they both followed policy's mandates and followed the recommendations/best practices.
3. WP:SRAG needs to be reworded.
   • It must be painfully obvious that SRAG members perform purely janitorial functions and do not hold any more power over which recruitment requests are accepted than any other community member.
     • Most of the editors who participated in this discussion thought that SRAG was a self-appointed group of people asserting power. This cannot be the case, no matter how cursory a reading.

---

• No major complaints here. An observation with respect to SRAG is that it seems to sit somewhere between "research approval" and "permission to use the user talk bot", which seems to be part of the confusion about its scope and assertions of power. Nifboy (talk) 22:57, 16 April 2010 (UTC)

The (potentially anonymizing) user talk bot is one of my big concerns about this whole mess, honestly - it's just that there are many other more pressing problems with this before we even get to the bot. — Gavia immer (talk) 14:44, 17 April 2010 (UTC)

I'll start thinking about the essay and will ping those involved in this conversation when I have an outline of the general ideas. --EpochFail^(talk|work) 14:55, 20 April 2010 (UTC)

I've finally gotten the essays to a good place. Please see User:EpochFail/What_are_these_researchers_doing_in_my_Wikipedia and User:EpochFail/Don't_bite_the_researchers. --EpochFail^(talk|work) 17:27, 8 May 2010 (UTC)

I made some modifications to include changing the SRAG process from "mandatory" to "recommended" along with language about why it is recommended. I also marked the essays as done although they are still drafts beneath my user. I plan to make a posting about them to wiki-research-l and move them to the Wikipedia namespace within the week. --EpochFail^(talk|work) 19:57, 13 May 2010 (UTC)

Research related request for bot approval

I have submitted a bot request for approval to use SuggestBot in a research project, and would like to invite anyone interested to participate in the discussion on Wikipedia:Bots/Requests for approval/SuggestBot 3. Nettrom (talk) 01:31, 12 June 2010 (UTC)

Just wanted to remind everyone about this request and its discussion, in case there are additional issues that need to be brought to my attention. Nettrom (talk) 15:45, 21 June 2010 (UTC)

IRB-like guidelines?

It seems that this proposed policy/guideline/whatever might benefit from some its authors taking a look at how university IRBs handle research requests and approvals. I'm not proposing that we emulate IRB processes but that we learn from them. In particular, there are questions that are fairly standard on most IRB applications that we might consider adopting or modifying. Going even further, it might be good if some folks with formal IRB experience were to be consulted and brought into this process. Wikimedia and en.wikipedia are big enough entities with enough members that it would be good if we utilized some available experts in formulating or at least thinking about processes that are similar to other organizations' processes. ElKevbo (talk) 22:19, 19 June 2010 (UTC)

Can you give an example or two of the kinds of questions you are thinking of? Our intention has been to not create a whole Wikipedia IRB and/or replace institutional IRB's. That would be an overly bureaucratic process that we are trying to avoid here. It's still up to the IRB (or equivalent human-subjects research organization/institution/group in some countries) to give approval for the research. The purpose of this effort is to create a centralized avenue by which research requiring subjects from Wikipedia can get the samples they need for their studies while causing a minimum of disruption. -- PiperNigrum ^(hail|scan) 14:18, 21 June 2010 (UTC)

I think it would be irresponsible for us to not question the ethics of proposed research on Wikipedia, particularly when it involves contacting and working with editors.

However, I don't have any specific questions or suggestions drawn directly from IRBs. In part, that is simply because I am very ignorant of our current process(es). But my general point remains: Why reinvent the wheel or a wheel-like structure without drawing upon the wisdom and experience of other in the transportation business? ElKevbo (talk) 21:41, 21 June 2010 (UTC)

I don't think that the usual questions apply. For example, IRBs usually ask what the risks to the participants are. The risk we mostly seem to care about is "a potential participant will disrupt Wikipedia by spending hours griping about receiving a single, four-sentence message on his/her user talk page." You'd never put that kind of thing into an IRB proposal. WhatamIdoing (talk) 21:54, 21 June 2010 (UTC)

RFC: Researcher permission

Input is requested on developing a community-based procedure for assigning the 'researcher' userright.

Hello all,

we (the Wikimedia Foundation) recently added a new user group, "Researcher", on the English Wikipedia to be able to convey some additional data access to researchers. The group currently has the following permissions:

• Search deleted pages (browsearchive)
• Use higher limits in API queries (apihighlimits)
• View deleted history entries, without their associated text (deletedhistory)

The motivation was a specific request from Dario Taraborelli, a Research Fellow at the Centre for Research in Social Simulation, University of Surrey, and he's currently the only member of the group. We granted the request after having a brief internal discussion about what a reasonable set of rights would be, weighing the Foundation's privacy policy as well as other considerations such as potentially problematic revision text contents. We think the researcher group as currently devised meets many common needs, but it probably can use further refinement.

My intent in posting here is to motivate the English Wikipedia community to take ownership of the process for granting this permission, in the context of the ongoing discussions here and elsewhere about how to work together with the world-wide research community. I would suggest that this new group be assigned through some Wikipedia:Request for researcher permissions process, similar to other permission-granting processes. I will note that there's an existing request by User:West.andrew.g that I've put on hold for now, and it would be great if it could be handled through a community process.

It may be appropriate to have this conversation at the cross-language meta level as well, but I figure it might be easiest to start by piloting here before we go ahead and create researcher groups everywhere.--Eloquence* 02:22, 9 June 2010 (UTC)

• Comment - Can we get further clarification of exactly what this means? What does "view deleted history entries, without their associated text (deletedhistory)" mean? What does that page look like? It appears to mean that they can see when a history entry was deleted, but not see what it contained - is that correct? Can they still see the edit summaries? Basically, is this simply a higher level of searchability, in which case the process should probably be similar to Rollbacker rights, or is there a danger of them uncovering personal information which had been deleted to protect privacy, in which case it should be similar to Checkuser? - DustFormsWords (talk) 02:39, 9 June 2010 (UTC)

Yes, it means seeing the version history including edit summaries, but not actual revision text. There's currently no separate permission to manage edit summary visibility.--Eloquence* 02:48, 9 June 2010 (UTC)

• Well, if there's a potential to uncover information that was deleted on the grounds that it violates a user's privacy and anonymity, then it seems to me that it should be a similar process to Checkuser. Although I'm sure more experienced (and administrative) minds will weigh in shortly and I'd yield to their insight. - DustFormsWords (talk) 02:57, 9 June 2010 (UTC)

My two cents are that valid requests will be sufficiently rare that a community discussion like RFA/Edit Filter Manager followed by a Steward flicking the switch should provide an effective safeguard. MBisanz ^talk 04:44, 9 June 2010 (UTC)

• Could we get the researcher permission added to the testwiki:? I would like to experience it first-hand before commenting. –xeno^talk 13:00, 9 June 2010 (UTC)
  • Can you file a quick bug for that?--Eloquence* 22:05, 9 June 2010 (UTC)
    • bugzilla:23918. –xeno^talk 18:04, 11 June 2010 (UTC)
      • With this userright, I can view a user's entire deleted contribution history with edit summaries only; I can use Special:Undelete to do a prefix search for deleted pages, and I can look at the deleted history (edit summaries) of any page, including those hidden with revision delete which can be easily picked out of the deletion log. While none of the revision text is available, there is still a lot of things that administrators have deleted on the belief that only individuals with at least admin-level clearance would be seeing. I share the hesitation expressed in many of the earlier comments below, and I'm not even sure the community is comfortable seeing this userright granted in the first place. Keep in mind that for any researcher that just needs the 'apihighlimits' without the deleted content, we can give them a bot-flagged account not used for editing. –xeno^talk 02:27, 17 June 2010 (UTC)
        • I am not sure I like this idea at all. Having someone who is not even a trusted member of the community poke over my contributions systematically in order to draw conclusions while treating this project and its participants like a scientific lab from which to mine sociologic interaction data and other forms of data is not one of the things I signed up for when I first came here. I find this whole idea objectionable. In fact a user in the past was banned from the project in part for treating Wikipedia as a social experiment. I really do not think this is a good idea at all, never mind the giant privacy issues that can ensue from these research activities. In fact I think this page is not named properly. Its current title is misleading because it is too general. Research is something we all do in the process of building this encyclopedia. But the research this page refers to is research on Wikipedians. Therefore it should be renamed to "Research involving activities of Wikipedians" Dr.K. ^{λogosπraxis} 18:55, 19 June 2010 (UTC)
          • What did you sign up for when you came here? If you do not want your contributions to be visible to (or searchable by) other people on the internet, putting them on Wikipedia is probably not a good start. Personally, I want wikipedia to be a useful source of data to other people - not just regular wikipedians, but the general public - so I support this "researcher" suggestion. bobrayner (talk) 11:46, 2 July 2010 (UTC)

• I don't know you tell me. I thought everyone here was called a "user" and their actions here were called "contributions". It never occured to me, at least until this RFC, that our common designation may be in reality "research subjects" and our "contributions", even the deleted ones, may be subject to data-mining by unknown researchers in order to arbitrarily investigate our actions and characters as if we are fish in a pond. Enshrining these activities into a "policy" with a misleading title to boot is another thing that bothers me. Don't get me wrong, I am ok with you liking this system. I just hope you are as ok with me despising it. And please spare me any GFDL lessons. Dr.K. ^{λogosπraxis} 15:30, 3 July 2010 (UTC)

• Even through I strongly opposed it, it would be neglectful to not point out the recently failed Wikipedia:Research and Wikipedia:Subject_Recruitment_Approvals_Group. Gigs (talk) 14:18, 9 June 2010 (UTC)
  • It's inaccurate to label WP:Research and WP:SRAG as failed. They both passed the RFC and SRAG is currently being given a trial run. On that note, any (non-inflammatory) input is welcome. --EpochFail^(talk|work) 22:15, 9 June 2010 (UTC)
    • Of course it's failed. There was very significant objection to the entire idea. Pretty much every non-researcher that came here after the invalid RfC opposed it. Gigs (talk) 00:35, 12 June 2010 (UTC)
• A question I have would be who would have the ability to assign this permission? I'm not sure I'm comfortable with administrators assigning this, perhaps bureaucrats or stewards are more trusted to assign this bit to those who have a genuine interest in research. –MuZemike 19:51, 9 June 2010 (UTC)
  • It's up to y'all to figure out - I agree that it's probably sufficiently rate that it could be handled by bureaucrats or stewards without causing undue additional workload.--Eloquence* 22:05, 9 June 2010 (UTC)
    • So this would be handled like a application for rollback/edit filter access and not like an RfA? Judging from the name of the user group, I'm assuming that it would be given to researchers. How can we easily verify credentials (and make them public?) Netalarm^talk 07:03, 10 June 2010 (UTC)
      • I would say OTRS would be the best venue in which to verify credentials and authenticate the need for the bit. –MuZemike 21:17, 13 June 2010 (UTC)
• I'd like to have a fuller explanation of what a researcher would be doing with material that we have purposefully suppressed from public view. How would the material be represented in a researcher's publication? Would we need to impose restrictions on researchers not to republish such material? How would we be able to enforce such restrictions? Would the researchers be restricted to only viewing material that was deleted under standard notability concerns rather than CopyVios or BLP issues?

I think we might need an example to see how (or even if) the community gets involved in this. While I understand the motivation to involve the community, this may be something that is best handled on a case by case basis by the Foundation. The rationale for gaining permission to examine deleted content might be fairly complex and the implications of what is done with that material fairly subtle - perhaps too much to gain sufficent members of the community to examine the application appropriately. Approval may ultimately fall to a small group of interested parties who may be inclined to grant (or deny) permission based on reasons more linked to the researching world than the Wikipedia world, and this may be to the detriment of Wikipedia.

If I was a researcher I feel my first move if I wanted to request additional access would be to contact the Foundation, so all requests would naturally go to the Foundation via Wikipedia:Volunteer response team which contains community members who have the experience and knowledge to better deal with this.

Unless there is a specific reason why the general community get involved in this, I feel it may be best dealt with under the volunteer response team and OTRS. SilkTork *^YES! 10:07, 11 June 2010 (UTC)

• My view is that, because this would be useful for only a very select few people, it should be handled via bureaucrat (or steward, if need be), and done through some sort of BAG/RfA-type procedure, with the whole community. It would ideally involve the person applying for the right to write out a summary of what they want to research, how they viewing deleted pages will take a part in that, and they must be able to report back to the community after some period of time about what they have accomplished. I particularly want this latter bit in the proposal because if we become too lenient, someone could feasibly create a false request and misuse or abuse the researcher flag. Community oversight is needed, especially as some of the people applying for this may be fairly new to Wikipedia, and the community may not have interacted with them as much as one would expect in, say, and RfA. —fetch·comms 15:14, 11 June 2010 (UTC)
• I would prefer if the WMF were to handle this, because for example we can't as a community verify credits, I personally wouldn't like to. Cenarium (talk) 17:51, 11 June 2010 (UTC)
• I'm not at all interested in having a bureaucratic process created solely because the WMF decided to give someone some extra read-only options. Let the Foundation handle it; they can be a liaison for anyone interested in projects other than en.wikipedia as well. Nifboy (talk) 22:42, 11 June 2010 (UTC)

• i deal regularly with scientific requests on the english (here i need support to have access and I'm gratefull for the service of local adminstrators) as well as on the german Wikipedia (where i have access as local adminstrator). as long as i can see - based on my correspondences with universities in europe and north america - there is a permanent need. therefore, many thanks to Erik for this initiative. one point: sometimes it's necessary to review the proposed projects from the point of our copyright and privacy policy guidelines (generel as well as the local project ones), best regards --Jan eissfeldt (talk) 15:36, 13 June 2010 (UTC)

• As a researcher and active en.wikipedia editor, I agree with some of the comments above asserting that the foundation or its appointed representatives should handle this. Each request for extraordinary access should handled on a case-by-case basis by those qualified to make the necessary judgments. Beyond questions of competence, there is a chilling aspect to having research subject to approval by the entire community. The Foundation probably gets enough requests of this nature to warrant standardizing its processes but this is a step in the wrong, direction, IMHO. ElKevbo (talk) 07:08, 14 June 2010 (UTC)

• • Supporting ElKevbo's position. Each experiment will have unique requirements, and their procedure will need to be reviewed by a competent ethics board before the information is made available. Their identity would also have to be confirmed. This kind of per-experiment detailed process should not be conducted by the community, but by an official representative who can be held liable if private information is abused. Dcoetzee 15:11, 24 June 2010 (UTC)

It looks like consensus here is (surprisingly) biasing towards a WMF-blessed process, rather than a community process. What we might end up doing for now is bless a small selected cadre of qualified volunteers to review and approve these kinds of requests.--Eloquence* 02:32, 18 June 2010 (UTC)

I'd be ok with that, or any way where the WMF is handling it. If we handle it there will be people applying just for the sake of having the ability to see these things, which wouldn't be very productive and isn't the idea of the right. Prodego ^talk 19:46, 19 June 2010 (UTC)

No thanks, Erik. This is something that the WMF should be doing. If you want to have an advisory group to the WMF on what research should and should not be approved, go for it, but it shouldn't be linked directly to any specific project. I would also urge you to look at some of the things that the Toolserver can do, and see if it is something that can be worked out from there. Several Toolserver participants regularly assist researchers in extracting information from WMF databases. It strikes me that different researchers will want to have different levels or areas of access, so it is not sensible to have a one-size-fits-all permission with that name.

As an aside, you have on your userpage that you will always clearly indicate when you are posting in your role as a WMF staff member; I don't see you clearly declaring that here. Please seriously consider starting an account under your own name or as "Erik WMF" so that it is always clear to all readers when you are posting as a staff member. I would urge all WMF staff who post on any project to have a separate staff account for when they are acting in their staff role. Risker (talk) 22:05, 19 June 2010 (UTC)

I don't expect that the permission in its current form will present serious problems (assuming that it is handed out judiciously - meaning that the researcher has been at least superficially vetted in some way and specified a need for this kind of access, which I presume to be the case here).

However, it should be kept in mind that there also is a demand by researchers for more sensitive information, such as Checkuser data (example, see also a 2007 discussion about reader IPs). The permissions for the researcher user group should not be extended to those; if such a request is ever granted, the data should be provided in a different way, and only with the knowledge of the community. Or consider this recent case where the ArbCom found that data derived from the non-public watchlist table had been given out inappropriately for "experiments" that would fail basic ethics criteria.

Regards, HaeB (talk) 21:14, 21 June 2010 (UTC)

Let WMF handle it. Why do we force a researcher to go through RfA-ish process to obtain this right when their account probably has 0 or few edits to begin with? We should be helping the researchers, not putting up roadblocks to hinder their progress. OhanaUnited^{Talk page} 03:26, 28 June 2010 (UTC)

This seems like it would be a useful tool for WP:Deletion review. People with this ability could go through deleted articles, and if they find articles that, based on their content, should not have been deleted, they can put it up for deletion review and discuss it. That is something I would be interested in doing and would probably be good at. --WikiDonn (talk) 17:50, 8 July 2010 (UTC)

Proposal to let BAG handle individual requests

I propose, as previously discussed, that we let the well-established BAG handle each request for messaging/recruitment by researchers, and abandon the idea of SRAG which is too bureaucratic for handling a few requests a year. BAG will decide to approve or not each request, everyone can comment as usual, the nature of the research, potential impact on WP and editors, etc, can all be taken into consideration. If the specific request is approved, the User:SubjectRecruitmentBot or a similar bot will send the messages. Each such request should be advertized in the appropriate places like Wikipedia talk:WikiProject Research. This page (Wikipedia:Research) can then be made into an information page recommending researchers to use this process for recruiting editors. Cenarium (talk) 18:19, 11 June 2010 (UTC)

Can you chill out for a while so we can finish our work plan and trial run before you ask for consensus about what should be done with WP:Research and WP:SRAG? This discussion should be postponed until the WP:Research is ready for another RFC. --EpochFail^(talk|work) 18:41, 11 June 2010 (UTC)

This didn't kick in, there's been very little outside input. Cenarium (talk) 19:31, 11 June 2010 (UTC)

• BAG would have to approve this kind of bot anyway, so this proposal is essentially a call to forcibly "shut down" SRAG without changing anything. So in that sense I oppose this proposal even though I think it's the right way to proceed with respect to our relationship with researchers, i.e. baby steps. Nifboy (talk) 22:30, 11 June 2010 (UTC)
• Yes, forcibly shut down the SRAG which failed to gain consensus in a reasonable amount of time. Gigs (talk) 00:50, 12 June 2010 (UTC)
  • Wikipedia:There is no deadline. --EpochFail^(talk|work) 15:00, 14 June 2010 (UTC)

Oh, but there is a deadline for policy proposals: Proposals that failed to gain consensus in a reasonable length of time (i.e., at the end of a discussion about the proposal) are {{failed}} proposals. We don't want rejected proposals hanging out for years and years, on the off chance that maybe some day a consensus in favor of the proposal might appear. WhatamIdoing (talk) 21:48, 21 June 2010 (UTC)

This proposal was accepted by the only RFC that took place. It was brought back to draft status because some of the details were brought into question. --EpochFail^(talk|work) 13:58, 23 June 2010 (UTC)

Recruiting via talk pages

Hi all, I just wanted to make a record of a recruitment issue that came up during a wikimania planning session earlier today. One complication when it comes to recruiting research participants is protecting the confidentiality of their information--including the fact that they participated at all. Because I tend to do studies that involve strategic sampling (I identify a small pool of potential participants who have experiences that I want to learn about), I generally do not recruit via talk pages, because that would leave a public record of my solicitation. Hope to have further discussions at the Wikimania panel on research ethics, methods and policies. --Andicat (talk) 16:50, 23 June 2010 (UTC)

I don't see a public record of solicitation (not necessarily participation) as an invasion of privacy. Can you give a hypothetical example that demonstrates the problem? Do you see a talk page as inappropriate in all cases or just a certain category of recruitment? One of the Wikipedian members of SRAG said in a recent trial run that "Many people view unsolicited email as an intrusion. I suggest that you initially contact users through their talk pages and obtain their approval to then wp_email them [...]". It seems that the sentiment towards this approach is at least mixed. --EpochFail^(talk|work) 17:11, 23 June 2010 (UTC)

Naturally different study designs have different constraints and degrees of risk. The bottom line is, researchers must be principled in making decisions that honor the trust placed in them by participants. If I target a small sample of participants, say 20 in-depth interviews, and I post a public record of solicitation on the Internet, I can't really do a good job of protecting their identities, which is something I promise to do. It is part of my job as a researcher to understand that, whereas yes, some people prefer email and some prefer talk pages, talk pages are public and email is not and that has potentially long-term implications for my participants. --Andicat (talk) 18:00, 23 June 2010 (UTC)

Epoch, I'll give you a real-world example:

Imagine that you're a person with a stigmatized medical condition or in a stigmatized group. Perhaps you have a serious psychiatric condition, you had an elective abortion, or you have HIV; perhaps you're an ex-con, a drug addict, or in a sexual minority. The researcher wants to invite you to participate in a study -- and sends you a postcard emblazoned with something like "Former drug dealers like you wanted for scientific survey. Confidentiality guaranteed."

Simply sending that message is an inappropriate intrusion. WhatamIdoing (talk) 20:00, 23 June 2010 (UTC)

Why would you include in the message that you are recruiting them for embarrassing reason X? How would you even know that embarrassing thing X is relevant to them unless they've made it public knowledge? If you're releasing private information about an individual before you've recruited them, the problem isn't that you are posting on their talk page, the problem is that you've already invaded their privacy. --EpochFail^(talk|work) 21:50, 23 June 2010 (UTC)

Wikipedia:E-mailing users is the guideline on contacting users via email. Users opt in to receive emails so they are no more or less likely to feel hassled by an unsolicited email than an unsolicited talkpage message. Many users have set up a special email account for Wikipedia messages. If people do feel hassled by an email, they can report the situation which will be investigated, and if there has been abuse, then the sender of the email would be blocked. A record is kept of every email that is sent from Wikipedia.

It seems it would be appropriate to use the email system in certain cases, and members of SRAG who are familiar with appropiate Wikipedia policies and processes would be able to assist and guide researchers through all aspects of contacting potential research subjects. It might be useful to have a place within SRAG and/or Research for both users and researchers to ask questions, raise concerns, and alert potential misuses. Be useful to have some users who are admins as part of SRAG in case an account needs to be blocked.

A possible approach might be for SuggestBot to leave a message on talkpages of identified subjects and X number of random others asking people to email the researcher. Another approach would be for SuggestBot to alert identified subjects and X number of random others they might be contacted by email. There are various methods that could be discussed. Email is viable. SilkTork *^YES! 18:39, 1 July 2010 (UTC)

And it would be appropriate for let people know they can contact certain members of SRAG by email in cases where there has been a potential invasion of privacy. SilkTork *^YES! 18:44, 1 July 2010 (UTC)

The Wikimania session invited to discuss research ethics and recruitment issues on the mailing list wiki-research-l. Please join! --Jojoona (talk) 11:32, 12 July 2010 (UTC)

Information about users who have opt-in

The page says "Only users who have previously opted in for suggestions from a research bot or opted in as a research volunteer will be approached."

My question is: is there a way to get a list of the users who opted in? Thanks! phauly (talk) 14:39, 31 August 2010 (UTC)

There is none; this particular page stalled out about two months ago after objections were raised with regards to contacting users, particularly via the proposed bot. I may repurpose this particular page to link to various research activities on-wiki, since this page name is the first obvious place to look for such. Nifboy (talk) 01:37, 1 September 2010 (UTC)

Thanks for the information, Nifboy! I add this fact to the page so that other Wikipedians (and researchers) can know about it. phauly (talk) 11:41, 1 September 2010 (UTC)

Political access risk

There is one issue that will be divisive and contentious but does need discussion. There are some countries where the government either has implicit access to all research or could probably arrange it or arrange research as a "front". For example we might find that even genuine researchers from Iran, China, and a number of other places would be seen as an unsafe risk due to the political realities of those countries and some could be uncomfortable with anyone from a university there, even a bona fide researcher, having access for "research", simply because the risk to editors of government access to data (despite undertakings) is unacceptably high.

Then again all that would be needed would be to edit, gain admin access, and the data is readily available anyway, so maybe the risk is not that high. Also deletion is not oversight and deleted material is not our major source of privacy breaching content. Most is very mundane, just not suitable for public.

This issue needs serious consideration though it's bound to be thorny. The bottom line is, does WMF need to maintain a blacklist of countries where it will not grant the researcher right, due to insufficient separation of government and academic research or a risk of the access being subverted for government/political reasons? Or is this over reacting given that deleted material is not usually privacy breaching, adminship is not hard, not supervised, and can be obtained easily anyway if so minded?

FT2 ^{(Talk | email)} 11:58, 8 October 2010 (UTC)

Organization/page structure

The structure about the WP: space articles about research could use some improvement. Please see my analysis and suggestion here: Wikipedia_talk:Researching_with_Wikipedia#Organization.2Fpage_structure Please weigh in there! -Pete (talk) 22:53, 5 July 2011 (UTC)

Researcher

Wikipedia:User_access_levels#Researcher needs to clearly specify what are the powers of that flag, and how can one obtain it (and lose it). --_{Piotr Konieczny aka Prokonsul Piotrus| reply here} 04:05, 18 August 2013 (UTC)

I don't think it's possible to specify how to get the user right because there is no "Request for Researcher" process. The right is primarily given to WMF employees. Andrew³²⁷ 12:20, 20 August 2013 (UTC)== baseline requirements for researcher permission ==

In light of a recent request for the researcher permission, which apparently was not granted, and the subsequent mess, I think this is a good wake up call. The process needs greater transparency and some baseline requirements. 'Researcher' is not a generic name; it is a job title for many people, and has real world connotations. Granting people a permission called 'researcher' when they do not meet reasonable criteria would be like changing the name 'sysop'/'administrator' to 'policeman'. I'm not disputing the RFC above which concluded that it was best to leave this with the WMF, as I think that is quite appropriate. However it should be mandatory that the WMF has first confirmed that the research is being supervised, know the name of the supervisor, and I think it is necessary for anyone granted this flag to have first obtained ethics committee approval. John Vandenberg ^(chat) 08:39, 8 October 2010 (UTC)

How about 1) "have confirmed receipt of approval from the researcher's applicable ethics committee dealing with human research" (or words to that effect). Witty Lama 09:55, 8 October 2010 (UTC)

• Comment First comment is that deletion is not oversight - this is about access to histories of disruptive posts, vandalism, copyright breach, and inappropriate articles, much more than privacy breaching revisions. The rest is merely convenience, ability to use high API limits and so on. (It's true that older deleted material may contain usernames and edit summaries that would have been oversighted if possible at the time; against that these fields are not the main place privacy breaches took place, would have been seen by thousands over the years and are already old.)

  Given concerns above, I would wish WMF to require at least the following -

1. Bona fide research, by reference to an established academic or knowledge-based institution, a well defined research goal endorsed by that institution, and credible researchers with acceptable past research.
2. A formal research plan, explaining exactly what they propose to do, what data they will draw, what they will do with it, and so on - a propose proposal covering their Wikimedia data-related actions.
3. Suitable ethical clearance, that the researchers have obtained ethical clearance from their institution;
4. Undertakings to WMF, that the researchers have each signed an appropriate set of undertakings related to access to data, purpose of access, anonymization and non-publication (without aggregation) of certain data, and destruction of certain data after the research has been completed. The undertakings should cover how data will be stored, who will see it, (all those who will see may also need to provide undertakings).

I think on that basis I wouldn't have any problems (or else viewed pragmatically, the risks are probably lower than the risks we already accept due to 1700 admins being able to see the data). FT2 ^{(Talk | email)} 11:40, 8 October 2010 (UTC)

• Revision deletion is increasingly being used instead of oversight so I would say there are privacy issues with providing this permission and that only genuine researchers with a clearly disclosed reason for access should be considered. Perhaps we should have a rule that established users should stand at RFA rather then apply for this flag through the foundation. Spartaz ^Humbug! 17:12, 8 October 2010 (UTC)
  • That would presuppose that administrators are allowed to use their access to deleted material to subsequently use the material in research. To my knowledge they're not (absent the blessing of the Foundation). –xeno^talk 17:14, 8 October 2010 (UTC)

Spartaz, I think you're mistaken here. You're talking about suppression mode as the tool used for oversight, but as I understand it, researchers would not have access to that mode. Nor would they be able to see the content of deleted/revdeleted revisions. Most revisiondeleted material is routine stuff - crude vandalism, disruptive posts, copyright breaches, and inappropriate/AFD'ed articles. FT2 ^{(Talk | email)} 12:54, 9 October 2010 (UTC)

My opinion on the whole "Researcher" mess can be summed up by noting that I attempted to MfD this page, though it didn't get very far. I have to see this present controversy as the rotten fruit of a rotten tree - something easily anticipated to happen, eventually, with certainty once you accept the idea of this policy in the first place. Of course, the underlying premise is that the Foundation is going to push "Researchers", regardless, because having researchers dedicated to researching Wikipedia is a point of prestige. As such, I would actually suggest that we avoid implementing any baseline policy that is the product of Wikipedia consensus - and even remove any language in the present policy that implies that we have baseline requirements. We still ought to (aggressively) enforce the privacy policy, and the policy on deceptive sockpuppetry, and the policy on disruption of Wikipedia, etc. - but the problems with the Researcher permission come from the Foundation mandate that there is such a thing as a Researcher permission, and the issue of behavior on Wikipedia comes from the Foundation implementation of that permission as a thing used on the live wiki rather than on a database dump or replicated live copy. Since it's a Foundation mandate, we ought to ensure that responsibility for the consequences of the mandate remains with those who mandated it - on their own heads be it. We ought to stay clear. — Gavia immer (talk) 17:42, 8 October 2010 (UTC)

FT2, do you actually mean to ban all independent researchers? Because the actual effect of "obtained ethical clearance from their institution" is to ban all researchers that don't have an institution that is both willing and capable of providing formal ethical clearance—a class that includes most book authors, freelance journalists, unemployed academics, MediaWiki coders, etc.

Put another way: If STiki's author starts his own consulting business when he graduates, rather than staying in the ivory towers, do you think he should have to stop researching vandalism on Wikipedia? WhatamIdoing (talk) 22:51, 8 October 2010 (UTC)

A good point. I tend to think of "research" in terms of formal research by some accredited body. The question with independent researchers, journalists etc is, who is supervising their research? To whom are they answerable if it is poor quality or draws unfounded conclusions or makes biased claims not supported by the data. I think everyone who has dealt with the media is aware of the situation where material is presented in a partisan or non-neutral manner to grab attention.

My question is if these are "researchers", who supervises the quality of their findings and ensures they do in fact fairly reflect the data? Who, professionally, are they answerable to and takes a strict view on how data is held, stored and (ultimately) destroyed? Who has power to call them to a professional disciplinary meeting if their research or handling is grossly inappropriate.

These seem to be key controls I would expect a genuine "researcher" to operate within. I am concerned that researchers lacking these may be ethical and well intentioned, but we have no way to be sure of it and no certainty of an independent body that ensures these issues are carefully thought out, and treats lapses as a professional standards issue. FT2 ^{(Talk | email)} 13:03, 9 October 2010 (UTC)

The scientific community has a peer-review based publication system. This publication system is designed to deal with many of the issues you've raised. Also, there is nothing we as editors (note that I'm a researcher as well) can do to stop someone from purposefully misrepresenting Wikipedia related data, regardless of the researcher flag. --97.127.8.80 (talk) 14:36, 12 October 2010 (UTC)

I think, as a practical matter, we should assume good faith of all such requests, but also to treat any request to view deleted material as equivalent to "view and make public" the requested material. So we do need a level of trust above and beyond what an unemployed academic can provide when it comes to the "researcher" user right, which can view all deleted material (edit) can search for deleted articles' titles and see deleted revisions in the history but not the text thereof. I think both the community and the Foundation understands that, but it was just such an odd conversation to have when the Foundation was like, "Hey, we created and granted this user right to exactly one person, do you want to do anything with it?" Nifboy (talk) 23:27, 8 October 2010 (UTC)

I didn't know this existed. How is it granted and who can do so? How are researcher rights different from other rights? — Rlevse • Talk • 11:32, 9 October 2010 (UTC)

Quoting from Wikipedia:User access levels: "The 'researcher' group was created in April 2010 to allow individuals explicitly approved by the Wikimedia Foundation to search deleted pages and view deleted history entries without their associated text." There was an RFC about this wherein the Foundation put forth the idea of creating a new community process for granting the right, which the community primarily rejected. The user right still exists but nothing's been done with it since the time of its creation. Nifboy (talk) 14:36, 9 October 2010 (UTC)

@WhatamIdoing, it is not correct to say that requiring ethical clearance restricts access to only gainfully employed academia. Many ivory tower institutions accept applications for ethical clearance their undergraduate students, associated institutions and from people outside their institution, and there are commercial organisations which provide ethical clearance services that conform with the relevant regulations (e.g.).

Allowing journalists to rummage around in the dark recesses of the deleted content is a very scary idea. Journalists are supposed to report rather than create research ('research' here being the noun rather than the verb). Important distinctions being that their work is edited, rather than peer-reviewed, and journalistic ethics are a far cry from research ethics.

MediaWiki Coders are not researchers; if they want to see stuff only available to admins on English Wikipedia, they should go through RFA.

Researchers on Wikipedia:STiki and similar tools would have very little trouble applying for ethical clearance, because there would be very few ethical implications involved. However the ethical review process would involve them articulating to the ethical committee that they are not archiving or publishing the restricted data, and that they will follow appropriate data anonymising & disposal requirements. John Vandenberg ^(chat) 01:41, 11 October 2010 (UTC)

The problem is -- where does the ethical committee come from? If an existing STiki researcher leaves his academic institution and sets up his own one-man consulting business, then where exactly does he get an ethics committee? It's not like you can pick one up at the office supply store. So, sure: He could theoretically "articulat[e] to the ethical committee" all kinds of things -- except that that no ethics committee exists in his "institution". (A one-man consulting business is actually incapable of having an Institutional Review Board, since US federal standards require a minimum of five members.)

Does the community really want to have a rule that says "Solo researchers not welcome" or "You must currently be associated with an institution big enough to have a formal ethics committee, and if you lose that affiliation for any reason—whether due to retirement, becoming disabled, or the economic mess—then you're no longer welcome"? WhatamIdoing (talk) 04:05, 11 October 2010 (UTC)

It's not a question of welcoming, it's a question of trust. An unknown one-man organization cannot be trusted any more than a given member of the public, and deleted material is stuff that we by definition have taken off the public record. Nifboy (talk) 05:00, 11 October 2010 (UTC)

I just want to point out that requiring researchers be associated with a larger organization is not uncommon. Many philanthropic foundations who provide grants for research require that applicants be associated with larger institutions. So it would not be at all unusual if we were to have such a requirement, too, although I think that this requirement is often tied to financial management and oversight in addition to ethical oversight. ElKevbo (talk) 05:09, 11 October 2010 (UTC)

As I said in my last post, with an example even, researchers who are working either for themselves or small research organisations can, and do, seek ethical clearance from either an institutional ethics committee or a ethics approval consulting firm. External research organisations usually develop a formal tie with the ethical committee at their local university, and their clearance requests are treated identically to those of internal staff members. John Vandenberg ^(chat) 05:59, 11 October 2010 (UTC)

Nifboy, in my (hypothetical) STiki example, we're talking about exactly the same human. This rule says, in effect, that the author of WP:STiki) can only be trusted so long as he's at U Penn (or a similar institution, large enough to have an ethics committee). According to this proposal, if he leaves U Penn to set up a private consulting business tomorrow, then he instantly becomes a terribly risky person who shouldn't be trusted at all. Bluntly, it seems extremely insulting to say that we think so little of his moral character that he (the individual human) can't be trusted unless he has filed routine paperwork with a bureaucratic committee.

ElKevbo, I agree that this is a common enough rule (although it often has far more to do with taxes than anything else). I'm just not convinced that Wikipedia is best served by adopting the common rules. (If we did everything the common way, Wikipedia wouldn't exist.) WhatamIdoing (talk) 05:55, 11 October 2010 (UTC)

Well, yeah. The point of asking the ethics committee is that we trust without prior experience the ethics committee, who in turn trusts STiki, who we normally would not trust but for the ethics committee's trust in him. If he leaves, that chain of trust is effectively broken. Nifboy (talk) 06:43, 11 October 2010 (UTC)

It's not that the person in your hypothetical example would become "terribly risky" once he or she leaves Penn but that we wouldn't have a convenient way of being reassured that research proposed by this person has been vetted by persons knowledgeable in and experienced with research ethics. It's not really an issue of trust but of simply trying to ensure that many angles have been thought of and as much experience as possible brought to bear on the situation. My IRB is fickle and pedantic and I believe that our fear of lawsuits causes them to routinely overstep their bounds but at the same time I often appreciate their questions and guidance as their experiences and training make them a very valuable resource. ElKevbo (talk) 08:09, 11 October 2010 (UTC)

It's worth noting that 'self-certification' for ethics approval isn't uncommon outside of the US, and that even if some institutions might work with non-academic researchers that is not to say it's true in all places. Showing consideration (self-certification), and mitigation of, ethical issues is obviously important but it would be easy to exclude some legitimate researchers depending on accreditation required Sjgknight (talk) 16:58, 25 August 2013 (UTC)

• In the US at least, the sort of research board referred to is one that gives permission for "biomedical and behavioral research involving humans with the aim to protect the rights and welfare of the research subjects," in the word of Wikipedia, which summarizes the situation accurately. In the US at least, this is now often interpreted as not applying to the ordinary sort of non-invasive survey or questionnaire based research, which does not affect the welfare and interest of the research subjects. I think almost all research proposed for Wikipedia would fall totally outside the sphere of a IRB. Most research will be about Wikipedia articles, and the grounds for approving or disapproving such research are entirely within our domain. In any case, it would be inconsistent with the overall egalitarian traditions of Wikipedia to require any particular sort of formal academic or other affiliation--what we need to have is trust, and that would in each case be up to the committee. The wellbeing of Wikipedia and its users are our concern equally, and unless it would fall clearly within the scope of a researcher's IRB also , we acting through the research committee are the relevant RB. DGG ( talk ) 22:34, 9 September 2011 (UTC)