Wikipedia:Bots/Requests for approval/Mdann52 bot

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was

Withdrawn by operator.

Mdann52 bot[edit]

Operator: Mdann52 (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 12:21, Monday October 21, 2013 (UTC)

Automatic, Supervised, or Manual: Automatic

Programming language(s): Pywikibot

Source code available: Standard Pywikibot

Function overview: Checks for dead links on pages

Links to relevant discussions (where appropriate): Previous BRFA for task - Wikipedia:Bots/Requests for approval/Legobot 4

Edit period(s): daily/weekly

Estimated number of pages affected: Depends on demand, ~~but no more than 100 per run.~~ probably a lot more than I think.

Exclusion compliant Yes:

Already has a bot flag No:

Function details: Using weblinkchecker.py, scans links on pages, per request sent to me. Then, either reports on the talk page or a subpage (depending on request) after a second run.

Discussion[edit]

This request is seriously lacking in detail. What talk page/subpage does it report to? — HELLKNOWZ ▎TALK 12:29, 21 October 2013 (UTC)[reply]

Depends what the requestor wants; I intend for this to be requested on behalf of Wikiprojects, so either a subpage or the talkpage, depending on what they want. Of course, if consensus here is for a subpage, I will of course honour that. (eg. Wikiproject example may want it put at WP:Wikiproject example/dead links, or on the articles talk page)--Mdann52talk to me! 12:36, 21 October 2013 (UTC)[reply]

So how does no more than 100 pages per run work when certain WikiProjects have tens of thousands of pages? — HELLKNOWZ ▎TALK 12:39, 21 October 2013 (UTC)[reply]

Yes, but not all of them will have dead links; It doesn't make any physical edits to articles. At most, it may make some edits to article talk pages. 100 was just an estimate; I am not 100% sure what the dead link:Article ratio is. --Mdann52talk to me! 12:42, 21 October 2013 (UTC)[reply]

It's very very high. Does this report links that are already archived (or tagged as dead)? — HELLKNOWZ ▎TALK 12:46, 21 October 2013 (UTC)[reply]

The script checks the link twice, and sees if they are dead. It seems (from my tests not commiting edits) that it ignores links in {{cite web}} with an |archive url parameter, and does not flag them as dead. I also set it up to ignore anything already tagged as {{dead link}}, so it will not tag it in the report. --Mdann52talk to me! 12:56, 21 October 2013 (UTC)[reply]

Any progress? --Mdann52talk to me! 19:47, 29 October 2013 (UTC)[reply]

Your link in Function details 404's. The previous BotReq linked has no community/editor request, nor does yours. Do you have a particular request you plan to fulfil, or is this a speculative BotReq? Josh Parris 02:11, 5 November 2013 (UTC)[reply]

Link fixed. I just felt that, as dead links are such a problem, a bot to help detect them would help a lot. Although it is unlikely to fix the problem outright, there have been several requests for archiving/dead link detection bots (eg. WP:Bounty board#A working WebCiteBOT). --Mdann52talk to me! 08:25, 5 November 2013 (UTC)[reply]

It's a real problem; that's widely known and acknowledged. I was wondering if there was a particular plan of attack that you're going to apply once approved? If you're looking to make an on-going impact, a broad based approach would be to use the Recent Changes function of the program. Josh Parris 06:35, 6 November 2013 (UTC)[reply]

That is a possibility in the future, once I get round to sorting that side of it out. However, in the more immediate future, I plan to run it on request. If no one asks, then I may consider prioritizing that suggestion (Which may need another BRFA I guess, although if you can approve me doing that on this one....) This is implemented in the script with the -recentchanges command. If you feel this is appropriate, I will try and stick this on tools/labs, and start it as soon as I get permission. --Mdann52talk to me! 08:28, 6 November 2013 (UTC)[reply]

Let's see what happens when the code is let loose

Approved for trial (5 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete.

Have you read and understood WP:BOTPOL? 10:24, Josh Parris 6 November 2013 (UTC)

As an additional note, there are a lot of small details concerning dead link checking that you can see from previous BRFAs. None of them are running because botops basically ran into constant annoying problems with the task (myself included). Since 2008, the number of dead links has grown so fast and we expect so many edits to be made, that these problems start to make up a notable portion of edits. I am not specifically listing all the problems again, because you do not edit the articles, only report to the talk pages. However, if you should edit the articles, then those will become much more pertinent.

As another note, we already have (had) bots that straight away repair the dead links and tag them in the article, so this seems like a step backwards. However, due to the scope of the problem and the fact that none of the bots are running, I don't object on that basis (I can't speak for everyone though). — HELLKNOWZ ▎TALK 10:51, 6 November 2013 (UTC)[reply]

Ok. It may take a while for this to finish; I will override the 7 day limit if possible. I have read the policy. Looking at the code, I am not sure if it takes {{dead link}} tags into account; I will try and run it on some of my subpages to determine this, and modify the code if needed. --Mdann52talk to me! 11:09, 6 November 2013 (UTC)[reply]

Just FYI, this may have to be closed. I have found out the script ignores all <ref> tags, so it will scan all links on the page. This means it is impossible to stop it ignoring links tagged as dead, and I lack the time (and probably the knowledge needed) to completely rewrite this to make it work; The closest I can get is to make it ignore any pages with ^{[dead link]} on them. Should I go ahead with the task, or withdraw it? --Mdann52talk to me! 19:49, 6 November 2013 (UTC)[reply]

Oh dear, that is a show-stopper.

Withdrawn by operator. due to inadequate code in the pywikipedia supplied program.

I'd encourage you to make a request at WP:Bot requests for someone to alter the code to be useful against our current template-base and editing norms. If you can get someone to do so, re-opening this BRFA shouldn't be a problem. Alternatively, there might be other link-fixing bots around that you can use yourself - ask around, poke around, check approval historys. Josh Parris 20:39, 6 November 2013 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.