Wikipedia talk:WikiProject Editor Retention/Retain new editors

Epoch Fail's comment

I'm hoping to start exploring some techniques for improving retention. Specifically, I've been designing (very preliminary) an analog of Huggle (code named Snuggle) to help Wikipedians identify new users who are doing good work. I have some sketches and mockups that I plan to pull together as soon as I get back. In the short term, I want to try to get a sense of what makes a desirable newcomer desirable and explore some effective approaches for detecting them computationally and showing their work to Wikipedian mentors. In the meantime, I think it would be helpful to build a summary of What We Know(tm) about newcomer retention and effective techniques (if any) that we might borrow from other systems. --EpochFail^(talk|work) 21:24, 11 July 2012 (UTC)

Copied from his talk page. Ryan Vesey Review me! 14:41, 12 July 2012 (UTC)

The decline is caused, at least in part, by increasing rejection of good-faith newcomer contributions

• This conversation copied from Project page`Buster Seven Talk 22:10, 12 July 2012 (UTC)

Hey folks! I'm the author of the recent blog post: Kids these days: the quality of new Wikipedia editors over time. I'm posting to direct you to some work I'm doing to identify the cause of the decline and test potential solutions. I have a writeup of a multi-method analysis of the English Wikipedia's newcomer retention issues. This write-up is a summary of a larger paper (PDF linked in summary) accepted to the scholarly journal, American Behavior Scientist. For your convenience, here is a super brief summary of the results:

• The proportion of newcomers who are trying to be productive has not decreased since 2006.
• The source of the decline is decreased retention of these good-faith newcomers.
• When newcomers have their revisions reverted, they leave.
  • Newcomers are more likely to get reverted (for similar edits) today than in 2005.
• The use of tools like Huggle appears to be exacerbating the revert problem.
  • Getting reverted by a Huggle substantially increases the probability that a good-faith newcomer will leave quickly.
• Young editors (post newcomers, but pre Wikipedian) used to be able to contribute to policies and guidelines, but not anymore.
  • Everyone is writing essays now, but essays are not enforceable.

I'm excited about this WikiProject and hope to work with you guys to try make a dent in the declining detention. --EpochFail^(talk|work) 19:56, 6 July 2012 (UTC)

One other thing, not mentioned above, also comes to mind. For most of the content newer editors would be interested in developing, there already exists a substantial, generally well sourced article, and, possibly, substantial prior discussion regarding the material they want to add, particularly if it isn't news-related. And, yes, with the sometimes incredible amount of archived discussion on some of these most popular pages, it can be really difficult to see if what you want to add has already been dsicussed. I have a feeling that many editors, when they see, effectively, that other editors are saying "Yeah, we old hands already talked about that, and found it wasn't worth including, at least to the weight that you want," will feel, understandably, somewhat offended and leave on that basis. Having said that, however, I'm not sure how to deal with instances of that sort of problem. John Carter (talk) 20:08, 6 July 2012 (UTC)

These are all good points but how can we use them to change the status quo? Can anyone come up with viable proposals which have a chance of being implemented? And is anyone interested in pushing them through? --Ipigott (talk) 20:33, 6 July 2012 (UTC)

I think the suggestion that is hidden in the section above this one might solve a portion of this problem. The Teahouse is also made to solve that very issue. Ryan Vesey Review me! 20:39, 6 July 2012 (UTC)

@Ryan Vesey: I have looked at the Wikipedia:Contribution Team project. It looks to me as if it is currently doing a good job in trying to make new editors feel more welcome. The Editor Retention project, on the other hand, it more focused on keeping people who are already contributors but who are frustrated by the often hostile treatment they receive. Maybe you could present the aims of this project to the Contribution Team and investigate how we can work together? --Ipigott (talk) 20:57, 6 July 2012 (UTC)

While the Contribution Team still has a few new members, it appears to be defunct. I just pointed it out because some of the ideas may be useful. I also disagree with any notion that this project should focus on established contributors. I am of the opinion that increasing retention of new users may be our best shot. I've also talked to Dennis, and I've got another user creating a table from the list of editors who placed {{retired}} on their talk pages so we can contact them to ask them if they are interested in rejoining the project and/or if they have an idea to fix whatever problem caused them to leave. Sorry if I'm going off on a bit of a tangent here. Ryan Vesey Review me! 21:16, 6 July 2012 (UTC)

Just butting in here, while I see where you're coming from Ryan re: not focussing solely on established editors I'd echo Ipigott's suggestion, in a slightly different way. In order to help long term retention one thing this project could do is ask users who are reducing their time on WP why so, and what would help restore them to the site (you're unlikely to see much return for your efforts in looking fro responses from those who have already left for good). The reason for this is twofold:
1) if we figure out how to help editors who know the ropes, procedures and policies we can educate new editors in ways that a) help them integrate into WP's community and b) help them stick with it for years (as some of us have). There's no point in retaining new editors for months only to see them catch the same rot & leave.
2) John makes a very good point above. We have nearly 4 million articles written today - WP is a vastly different experience now as to 2006. There are fewer new articles to write. There are an increasing number of featured articles that require a standard of writing and research to make positive & constructive contributions to. We need not only to be nice to new editors but to equip them with an understanding of how to do good research and what pitfalls to avoid.
Many of us on site are education and higher education professionals in various fields. There is a pool of knowledge on how to help ppl learn to research and there are ppl with the skills to deliver that here. Also as Epochfail points out if there are issues with widely used tools (like Huggle or Twinkle etc) we should look at this, get the evidence, and open an RFC on disabling these features in order to retain new good faith users--Cailil ^talk 21:38, 6 July 2012 (UTC)

One small point. There are still a huge number of articles out there which haven't yet been created. I can think of the mind-boggling number of small, but minimally notable, Christian denominations, groups, religious orders, churches, and what all out there - I know of at least several thousand we don't yet have articles on. However, in a lot of cases, it can be harder to find RS on some of these smaller topics, so editors, including some older ones, try to add the material on these less-notable topics to already extant articles, where the material often won't meet WEIGHT requirements. When it gets removed there, they might leave. Wikipedia:WikiProject Missing encyclopedic articles might be a good place to start listings of some of these missing articles, and I am still trying to get together a list of the encyclopedias reviewed by journals on the JSTOR site. When that gets done, we might be able to have some of the WikiProjects on major topics, like for instance Christianity, have lists of articles which are apparently notable enough, and reference works which have material on them to help such articles. That might help a few editors stay around. But, for others, who, for instance, think Elvis killed Kennedy as per some book or other and want it prominently placed in those two biographies, those editors can, sometimes, seem reasonable, but are probably not long to stay here anyway. John Carter (talk) 21:55, 6 July 2012 (UTC)

They're great suggestions John - I'll see if I can get a convo or two started at the wikiprojects I'm involved in about notable but unwritten articles--Cailil ^talk 15:10, 7 July 2012 (UTC)

We've (WMF researchers) been studying ways to improve the effects of Huggle reverts. See Wikipedia:UWTEST, meta:Research:Warning_Templates_in_Huggle and the ICWSM publication. I really think these are stopgap approaches though. I want to figure out how to reverse the trend and fixing warning messages appears to make marginal improvments at best. I don't think that shutting down Huggle is a good option due to the massive efficiency and effectiveness of the tool at combating vandalism. Instead, I'd like to learn from huggle's efficiency for using human judgement to identify and remove vandalism by building a similar system (code name: snuggle) that uses human judgement to identify and support good new (or old) contributors. I hope to have more on that soon. --EpochFail^(talk|work) 13:39, 9 July 2012 (UTC)

@John Carter, one of the studies I cite in the paper is one of my own from last summer that supports your intuition about newbies editing increasingly complete articles and getting reverted. meta:Research:Newbie_reverts_and_article_length --EpochFail^(talk|work) 13:22, 9 July 2012 (UTC)

Bingo, I believe this is one of the top areas we need to address: What do we do with 'finished' articles, that probably should not be being edited that much any more? I think the answer is to identify FA articles as such and to direct editing efforts away from them and toward areas that still need work. Zad68 13:28, 9 July 2012 (UTC)

The report makes an incorrect assumption, a very common one, that Assume Good Faith is policy, it's not policy.

This study has many important implications for community and Wikimedia Foundation efforts to engage and retain new editors. To begin, it reasserts the centrality of one fundamental policy on the project, “Assume good faith.”

Reminds me of the quote from hitch-hikers guide on my talkpage,

It is very easy to be blinded to the essential failure of Wikipedia by the sense of achievement you get from getting it to work at all. In other words - and this is the rock solid principle on which the whole of wikipedia's success is founded - its fundamental design flaws are completely hidden by its superficial design flaws. Penyulap ☏ 07:35, 7 Jul 2012 (UTC)

That's what you get for reading the pre-print ;). I should have cautioned about that more carefully. I edited that out when I caught it as I was making the HTML summary. There's a strange bit of legal nuance around what I do and don't get to publish from my journal submissions. It seems safer to maintain the separate summary than to continue to edit my pre-print so the HTML version will be more up-to-date. --EpochFail^(talk|work) 13:22, 9 July 2012 (UTC)

Reminds me of "Everything has already been Invented", an 1899 quote by the Patent Office Commissioner. ```Buster Seven Talk 13:16, 7 July 2012 (UTC)

The biggest change (edit break)

• The biggest change since 2006 for Newbies is that a newbie who added an unsourced fact then was more likely to see it {{fact}} tagged and less likely to simply see it reverted as unsourced. I can see that this would be offputting for new editors, but it is only one stage of the process, we are losing not just newbies but editors who only last a few months. The community has become something of a clique with relatively few newbies making it into the core of active editors. These are in my view two distinct and separate problems. We need a better way to handle newbies who add unsourced content, and we probably need some research to establish how often that unsourced content is typically True, False or unverifiable by Google. If its mostly True then we need a program to encourage those who "revert unsourced" to only do so for genuinely contentious stuff. If its mostly False then that just boosts the case to implement Flagged revisions and process the vandalism more effectively and efficiently. If its pretty mixed then we need a more complex solution. As for how we make the community less cliquey, I'm not sure, though it can't help that those of us who started editing after 2006 are still a minority of the admin cadre. I might start a new thread lower down about retention of those who get past that early stage. ϢereSpielChequers 13:58, 7 July 2012 (UTC)

• I wonder if a modification to huggle that allows you to tag an unsourced section with {{fact}} and notify the contributor that they should attempt to find a source would do any good. Ryan Vesey Review me! 14:18, 7 July 2012 (UTC)

• Again that's a good idea Ryan but that wont help on FAs and GAs (or high profile articles that were under probation like Sarah Palin or Barrak Obama) - the standard for drive-by editing is getting higher (that's not inherently a bad thing) we shouldn't be dumbing down policy or letting article content slip on the off chance that that IP will open an account. Another option would be to put up an info box for all IPs on the "Dos and Don'ts" and a prominent create an account button--Cailil ^talk 15:17, 7 July 2012 (UTC)

Getting past that early stage

Most of us here have been envolved for some time. We have long ago past that "early stage". But why? Why did we persist while others simply stopped and left the game. Maybe there is some insight we can glean if we all tell our brief early history. Plus, it should be fun looking back. No time right now, but later I will create ..../Retain new editors/The early stage or something like that. ```Buster Seven Talk 15:24, 17 July 2012 (UTC)

I'm sorry that this thread didn't get picked up. I actually have some insight into some of the patterns (from a data perspective) that suggest why some editors stay. In the models of new user retention that I've built, there is a term that I always use to control for a strong predictive characteristic that is orthogonal to the effect I want to measure. I like to call this term "investment": the number of edits a newcomer saves when they first sit down to edit the encyclopedia. This is one of the strongest predictors that a new editor will stick around despite a negative reception. I suspect that high "investment" newcomers are generally more excited about the project than low "investment" editors. On this note, I'd like to ask, how much did you do when you first started editing? Do you remember how you felt about Wikipedia at the time? --EpochFail^(talk|work) 20:57, 5 September 2012 (UTC)

In my view

• • copied from main project page

In my view, some of the largest potential improvements in terms of the effects of reverts, will come from anything we can do to educate new editors before they make those edits, or while they're making those edits, not after. Improvements to the article editing interface, automated testing for potential warning signs in an article and giving users real time feedback, those are things that matter. "Hey, it looks like may not have included any references. That could be a problem that could lead your article to eventually be deleted, want some help with that?" To draw an analogy, consider the difference between, as I type (this) message, an interface that tells me that I forgot to leave an edit summary, and one that, an hour later, removes my comment and leaves a note on my talk page. Is there really any question which is friendlier? Good-faith edits that are problems from our point of view can be reduced by education, and everything we do to help that situation helps reduce the number of good-faith edits we reject. --j⚛e decker^talk 15:58, 13 July 2012 (UTC)

I agree. The more upfront normal contact we can create, the sooner the new editor will feel at home. Its like telling a country bumpkin, new to the city, to "look both ways before crossing". He appreciates your concern for his safety. (No harm to country bumpkins or new editors intended) Is what you are suggesting (improvements to the interface) possible? How about "Speedy Sandbox" instead of Speedy Delete for editors with, lets say, under 100 article edits? Again,a much friendlier "We're all in this together" conversation. The Deletion Process is very painful and heartless. It may be 100% accurate but it lacks a sense of community and comradship. There has to be a better way! ```Buster Seven Talk 00:07, 17 July 2012 (UTC)

Note

Just a note that I am working on a draft for a New editor buddy program (name subject to change). I'll move it to project space once I have some better context. Right now it is a bunch of loose ends in my head. Ryan Vesey Review me! 14:17, 13 July 2012 (UTC)

Some updates on Snuggle mockup

Mockup of the Snuggle interface visualizations

Hey guys! I'm sorry for the silence. I've been working on some mockups for an interface for visualizing newcomers and the work they are doing in the encyclopedia (which I've codenamed "Snuggle", a play on Huggle). I'm hoping to build a tool (possibly on the toolserver) that allows Wikipedians to quickly examine the pool of recent newcomers and identify their recent activity in the encyclopedia. Note the mockup to the right. In a nutshell, I'm hoping to produce a list of recent new editors based off of recentchanges with a visual summary of their daily activity (color coded by namespace) that can be examined at a glance and expanded for more in-depth inspection. Note that some of the edit rectangles have a red dot in the center. I hope to use such a marker to signify when edits have been reverted. The icons on the right side of each user-row represent User_talk page messages. [V] denotes a vandalism warning, [W] a welcome, [(C)] a copyright warning, etc. I hope to bring more on this project soon! --EpochFail^(talk|work) 21:10, 7 August 2012 (UTC)

I just realized that I never mentioned that the intention of this whole thing is to be a powerful, distributed, human-cognition system (read: quick and easy for you and me to work together) for detecting desirable newcomers and interacting with them on a personal level. --EpochFail^(talk|work) 00:07, 9 August 2012 (UTC)

Do you have an estimated time line? Are you testing this on another wiki or here? Ryan Vesey 01:07, 9 August 2012 (UTC)

Does it come with training wheels? ```Buster Seven Talk 04:52, 9 August 2012 (UTC)

Only if your name is not Willy. Ryan Vesey 05:08, 9 August 2012 (UTC)

Sorry for the late response. I must have accidentally unwatched the page. I won't be able to make good estimates until I find a feasible way to solve some of the harder problems (e.g. building revert data from recent changes). I'm hoping to have the system ready for some testing by the end of Sept., but at this point, that's more of a guess than an estimate. --EpochFail^(talk|work) 17:23, 14 August 2012 (UTC)

Hey folks, it's a bit of a lame update, but I've finally had a chance to get my thoughts together on this Snuggle thing, so I put together a project page and proposed a three stage plan for development. I've added a signup list for anyone who is interested in receiving regular notifications about new development. I'm hoping to a build a bit of an interest group around the project so that I end up putting together something that someone might find useful. I'm currently working on strategies for efficiently consuming Special:RecentChanges via the toolserver in order to detect newcomers and visualize their activity. I hope to have more on that and a source code repository in about a week. --EpochFail^(talk|work) 21:54, 5 September 2012 (UTC)

Suggestion

How about a policy/guideline to admins to userfy all newly deleted articles save a few obvious vandalisms? That, and a (friendly) message explaining what has been done will do a lot good to new ediots who get their articles regularly destroyed by deletions, and dont come back. TheOriginalSoni (talk) 16:18, 17 December 2012 (UTC)

The Wikipedia Adventure, alpha-testers needed

Hi folks, I've been working for the past 7 months on an interactive guided tour for new editors called The Wikipedia Adventure, as part of a WMF Individual Engagement Grant. The game is an experiment in teaching our aspiring future editors in an educational but playful way.

• This week I need some alpha-testers to kick the tires and basically try to break it. I'm interested in general impressions and suggestions of course, but I'm really looking for gnarly, unexpected browser issues, layout problems, workflow bugs, and other sundry errors that would prevent people from playing through and having a positive experience.

• If you're able to spend 1-3 hours doing some quality assurance work this week, you would have: a) my sincere gratitude b), a sparkly TWA barnstar, c) special thanks in the game credits, and d) left your mark on Wikipedia's outreach puzzle and new editor engagement efforts

• Please note that the game automatically sends edits to your own userspace and it lets you know when that will happen. If you want, you can register a new testing account just for the game, but it won't work properly unless you're logged-in by step 8 of mission 1 when it lets you register on the fly.

If you're interested, please add your name below and have at it. You can post feedback to WP:TWA/Feedback. Thanks and cheers! Ocaasi ^{t | c} 20:51, 16 October 2013 (UTC)

Try out The Wikipedia Adventure

I'm interested and on the bug-hunt. Will report back this week

1. Add your name here
2. Or here
3. Or here...