Wikipedia talk:Talk pages not indexed by Google

First remarks[edit]

About time. all our rejected ideas, trial runs, and the contributions of miscellaneous trolls and POV pushers are now indexed in google, and show up ranked highly if anyone searches for them. We work in public and want to continue to so, and therefore it is right that they remain searchable in context by someone who wants to look for t hem, using the in-wikipedia search function. We're not hiding anything--we';re removing the artificial emphasis an outside agency that we cannot control has put on our scratch pad. I would add user space as well. We say we are not a web host--in that case, lets act accordingly. DGG (talk) 23:02, 29 April 2008 (UTC)[reply]

The in wikipedia search function is no where near as good as google.Genisock2 (talk) 23:53, 29 April 2008 (UTC)[reply]

There has to be a balancing test. This stuff being highly ranked on google is causing inordinate harm, and outweighs any potential benefit from searching some old talk pages. We were supposed to have developed an improved search feature a long time ago, and this would be a great opportunity to finally do so (I think the scroll-down suggestion box was a good first step).--Pharos (talk) 00:29, 30 April 2008 (UTC)[reply]

we can improve our search function; we can not affect the operations and influence of google. so lets do what we can, not refuse to improve because if the rest of the world were better we wouldn't need to. DGG (talk) 18:47, 30 April 2008 (UTC)[reply]

Developers are actively working on improving the search function, the search suggestion box just recently added is only the first step. Putting our own convenience ahead of the potential real-world harm that may be caused by discussion archives appearing in the top of Google results seems a little selfish to me. Mr.Z-man 19:18, 30 April 2008 (UTC)[reply]

I support this idea as a good place to start. See this blog post I made earlier today. See also [1] this post on the mailing list and many subsequent ones. See also bugzilla:13864 (which was marked as invalid pending community consensus). ++Lar: t/c 21:32, 30 April 2008 (UTC)[reply]

Ideally we would have some sort of magic word that could be used to either remove a page from indexing, (or perhaps preferably have the default be no indexing for certain namespaces but have a magic word that allows indexing). I'm thinking it's useful to have the vast labyrinth of policy pages searchable by Google, but it's pointless to delete articles with BLP concerns when oftentimes the information is essentially preserved, at the same place in search engines, merely in the form of a deletion discussion. Would there be any reason to avoid implementing the ability to unindex Wikispace discussions that are about BLPs? --JayHenry (talk) 01:31, 1 May 2008 (UTC)[reply]

The need for policies and essays and so on to be searchable is why I'm proposing we do this to talk pages. We could, of course, create a Policy: and Essay: Namespace. I generally think applying this to User: is probably worthwhile, but for a trial run, I don't think it's the correct way to go. Wily D 11:46, 1 May 2008 (UTC)[reply]

Yeah, I agree with and support that for the trial. Looking forward, I think noindexing deletion discussions of contentious BLPs and ArbCom cases with BLPs and editors using their real names will also be a necessary step. Not meaning to sidetrack this proposal though. I definitely support trial run of noindexing on the talk pages. --JayHenry (talk) 03:35, 2 May 2008 (UTC)[reply]

Those new namespaces sound like a good idea. In addition to those, what about Guideline: and Proposal: namespaces (I was looking at WP:POLICY under the Policy-related pages section). I'm suggesting this because it can get confusing when Policies, Guidelines, Proposals, and Essays are all under the Wikipedia: namespace. -- Imperator3733 (talk) 21:20, 13 May 2008 (UTC)[reply]

Looks like we're getting pretty broad approval here. Are we going to propose this on the Village Pump soon?--Pharos (talk) 04:17, 4 May 2008 (UTC)[reply]

I figure it could use some better writing and such, but more input might be worthwhile. Wily D 18:59, 8 May 2008 (UTC)[reply]

Article talk pages[edit]

I was under the impression that article talk pages are no longer to be indexed by search engines (a change in the robots.txt file)? Sorry I can't be more specific at the moment, but perhaps someone else could check. -- John Broughton (♫♫) 01:57, 9 May 2008 (UTC)[reply]

I did some test-searching with phrases from different talk pages of prominent articles. Some don't show up at all for whatever reason (probably because they're not linked to much), but I got definite results for Talk:George W. Bush.--Pharos (talk) 04:33, 11 May 2008 (UTC)[reply]

Credibility makes it unnecessary[edit]

I don't think talk pages should be unindexed. My reasoning is that it's evident and clear that our talk pages and project pages are for discussion, and not intended to be a reliable source of information - therefore most readers will interpret them that way, just like they would if a random web forum popped up in their web search results. There's only a concern at all because of the manner in which PageRank functions, which really shouldn't be our concern, as long as we're honest in our presentation. By noindexing the pages, you are making the implicit and questionable assumption that "people who search the web don't want to find these pages," which in many cases is simply untrue - they can glean insight from our discussions, even if that's not their primary purpose. Dcoetzee 03:52, 9 May 2008 (UTC)[reply]

If people want to "glean insight from our discussions" (for example, academic researchers), then they can and should search on Wikipedia. These personality-damaging discussions popping up in Google is an embarrassment both to the persons affected and to us as a project; as an encyclopedia and as a top ten website, we should not hold ourselves to the standards of a "random web forum".--Pharos (talk) 04:08, 11 May 2008 (UTC)[reply]

A discussion damages nobody - it's clear that they are opinions expressed by individuals, and the discussion makes it clear the level of support their ideas have. Our articles aim to be reliable sources about a topic and present their ideas as facts; our discussions are presented as discussions and therefore do not need any warning, hiding, or qualification. The only question to my mind is whether people searching the web benefit from talk pages turning up in search results, whether they can find information there they might not elsewhere, at least in some cases. I believe the answer is yes. Just telling people to search on Wikipedia is not a solution, as they may not realise when they make a query that a discussion page on Wikipedia is relevant, and even if they do, Mediawiki's ranking algorithm is simplistic and frequently unhelpful, and many types of searches are not possible. Dcoetzee 08:30, 11 May 2008 (UTC)[reply]

No, because not everything on a talk page is an opinion. Discussion pages are very often full of facts. If we have an article about a man named John Onions with an unusual appearance who was arrested for a sex offense, and on the talk page someone says "this article on John Onions is just an internet meme about an ugly sex offender. I think it should be deleted." Only the second sentence is an opinion. The first sentence is still a set of facts and Mr. Onions may very well consider them damaging, when they remain at the top of a search engine. 95 percent of the time, what you're saying is correct, which is why I'd prefer to see some way to remove only certain pages from indexing. --JayHenry (talk) 16:23, 11 May 2008 (UTC)[reply]

Support to preserve privacy. —TreasuryTag—t—c 18:37, 13 May 2008 (UTC)[reply]
I would also like to add my support for this proposal, due to privacy concerns and this that we allow information on talk pages which we would never allow anywhere else. --PinchasC | £€åV€ m€ å m€§§åg€ 23:11, 13 May 2008 (UTC)[reply]

Great idea, one qualm[edit]

I just want to say that I heartily support this proposal. The benefits for users clearly outweighs the possible negatives. If people really want to find talk pages, they can search our site quite easily. We don't need them showing up in Google. I do wonder, however, what this is projected to do to our Google juice for the site as a whole? If there is any possibility this would bump our articles way down in rankings, I'm not for it. I'd rather sacrifice talk pages to Google if it ensures people find articles. VanTucky ^{Vote in my weird poll!} 18:57, 15 May 2008 (UTC)[reply]

I agree with that statement as much as the policy; it's an excellent idea, but has downsides, as VanTucky says. If it does decrease Google views, I don't think that will be too much of a problem, although it will certainly affect Wikipedia. weburiedoursecrets inthegarden 19:03, 15 May 2008 (UTC)[reply]

Yes, i agree with the above proposal. People like me who edits under real name finds lack of privacy and sometimes stay away from hot debates. I think this policy should be immediately implemented. Amartyabag ^TALK2ME 02:13, 17 May 2008 (UTC)[reply]

Infact, it talk pages are indexed by google, that would affect wikipedia adversely. One of the reasons being that these talk pages do not contain any information worth value(from search perspective) & would go /down as irrelevant/spam content.

Great Idea[edit]

I support this idea in its current form, but would also like to encourage the non-indexing of Wikipedia-space pages, particuarly XfD pages, which sometimes can have discussion of uncomplimentary material that has been inserted in various articles. Courtesy blanking is a good way to minimise the amount of harm done here, but I can't really see any downsides at the moment to a blanket ban. Lankiveil ^{(speak to me)} 05:46, 17 May 2008 (UTC).[reply]

Support as originally proposed. Good start, do it now. Then worry about XfD. Don't let the best be the enemy of the good. PaddyLeahy (talk) 14:05, 18 May 2008 (UTC)[reply]

Support -- XfDs should definitely not be indexed - they aren't encyclopedia articles and they can have various bad things on them. -- Imperator3733 (talk) 18:23, 19 May 2008 (UTC)[reply]
Support, I've seen this on Google results pages too and he suggested solution is really needed for talks, XfD and all behind-the-scenes goings on here at Wikipedia. +Hexagon1 ^(t) 08:19, 24 May 2008 (UTC)[reply]
Support - Great idea! I'd like my wiki life to stay separate from meatspace, thankyou... Anxietycello (talk) 17:57, 30 December 2008 (UTC)[reply]
support -- at least as refers to XfD. The current situation with XfD is that sometimes the leading article at Google is a discussion that we've deleted the article--which does seem a little counter-productive to trying to prevent use for publicity. DGG (talk) 00:13, 1 January 2009 (UTC)[reply]