User talk:Full-date unlinking bot/Archive 2

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 1 Archive 2

Bot breaks

The bot appears to take breaks from time to time (it hasn't been running for around 7 hours now). Is there a problem?

Also, the current pattern of 6 edits in a second or two followed by a pause of a minute, seems an odd way to minimise the strain on the servers (and we may be worrying about that unecessarily anyway - see Wikipedia:Don't_worry_about_performance). Could it be perhaps be set up to perform an edit every few seconds continuously, and overall run a bit faster? One edit every 5 seconds, for example, would seem unlikely to overstress the servers. There's a maxlag parameter in MediaWiki that's designed to allow bots to run at full speed in quiet times but slow down at busy periods. Would this be a useful addition? Colonies Chris (talk) 20:45, 24 November 2009 (UTC)

From time to time, the bot will get a "bad token" (despite the fact that the bot renews the token every time it edits). My solution is to start it up again each time (until I can implement a more sustainable solution). By the way, it does not make six edits per minute then pause; it makes an edit, it pauses for six seconds, and it goes to the next article. I will consider using the maxlag feature instead. @harej 21:13, 24 November 2009 (UTC)
Thanks for the response, James (even though I don't understand it :-}). My comment about the editing pattern was based on misreading the 'User contributions' list - apologies for creating confusion. Colonies Chris (talk) 22:31, 24 November 2009 (UTC)

Minor edits

Now that people are not pulling their hair out and jumping off cliffs any longer about date de-linking, please make this bot flag its date cleanup edits as minor. It's getting pretty annoying to have my watchlist full of an endless stream of essentially identical, [now-]uncontroversial edits that no one is going to care about on a case-by-case basis.  :-) — SMcCandlish [talk] [cont] ‹(-¿-)› 18:23, 26 November 2009 (UTC)

They are flagged as minor. @harej 20:20, 26 November 2009 (UTC)

Article order

Just wondering, what is the order that the bot edits articles? Looking at those recently delinked, they seem completely haphazard. Thanks, Reywas92Talk 04:38, 27 November 2009 (UTC)

I believe it's going in chronological order—the bot delinks all articles that have links to January 1, and once it finishes those, it goes on to articles that have links to January 2, and so on. Dabomb87 (talk) 04:50, 27 November 2009 (UTC)
See also #The bot now runs full-time. Dabomb87 (talk) 04:51, 27 November 2009 (UTC)
Okay, thanks. So I guess by the time it gets to December most of the articles will have already been taken care of. Reywas92Talk 05:04, 27 November 2009 (UTC)

Commas

Is the bot smart enough to recognize and remove incorrect commas? While [[27 May]], [[2007]] correctly autoformats to either [[May 27]], [[2007]] (with a comma) or [[27 May]] [[2007]] (without a comma), does the bot know to remove both the brackets and the incorrect comma from [[27 May]], [[2007]] to form 27 May 2007? Reywas92Talk 05:01, 27 November 2009 (UTC)

I believe so. See User:Full-date unlinking bot#Edit summary codes, specifically edit summary BRodd. Dabomb87 (talk) 05:04, 27 November 2009 (UTC)

Progress

Thanks for running this bot. Really appreciated. Makes Wikipedia look a lot more professional. Just wondering, would it be possible to state somewhere on the bot-page where it currently works? I mean, it goes from 1 January until 31 December, right? Where in January is it now? Thank you. Mark in wiki (talk) 08:14, 27 November 2009 (UTC)

Currently up to January 16. @harej 08:15, 27 November 2009 (UTC)
I have added a progress indicator onto the bot's user page. @harej 08:35, 27 November 2009 (UTC)
Beautiful. And very quick too! Mark in wiki (talk) 08:37, 27 November 2009 (UTC)

(outdent) thanks & praises, harej - but i'm curious: for me the "progress indicator" says January 1, and the bot seems to have gone back to January 3 and 4. is that the way it's supposed to be? Sssoul (talk) 10:11, 27 November 2009 (UTC)

It's on January 17 but changed the date displayed on the bot page to Januar 1 right before having hiccups. It's all working again, though... Hekerui (talk) 14:38, 27 November 2009 (UTC)
right: it went back to early-January dates for about 35 minutes this morning (from 8:33 to 9:08) before taking some time off and then returning to normal. i'm glad it's back on the ball, but still curious. and the progress indicator still says January 1 for me - is that what everyone sees, or am i just lucky? Sssoul (talk) 15:19, 27 November 2009 (UTC)
Sadly, the progress indicator seems to be stuck, now shows January 1 Chris the speller (talk) 16:53, 27 November 2009 (UTC)
It still says Jan 1 as I sign this. Darrenhusted (talk) 17:06, 27 November 2009 (UTC)
It starts to go back to doing January 1 now, look. Edit: It's now on January 2, looks like it goes through the process all over again and thereby cathes some new full dates. Hekerui (talk) 20:49, 27 November 2009 (UTC)

At this rate it's never going to get past January (though I understand it does all the years and dates on each page). Would it be crazy to suggest a second bot starting on December 31 and working backwards, or target certain dates (like Feb 14, March 17, Dec 25, April 1). Otherwise it may as well just be called "January Date unlinking bot". We have multiple Smackbots and others, maybe an unlinking bot for each quarter (Jan-Mar/April-June/July-Sept/Oct-Dec)? I understand we have millions of articles, and possibly billions of links but this bot may never get the bulk of the dates delinked if it left to do it alone. Darrenhusted (talk) 21:06, 27 November 2009 (UTC)

It only resets when I have to start the bot over. Now that we have the counter in, the bot will keep going. I can assure you the bot will make it beyond January. @harej 21:07, 27 November 2009 (UTC)
Also bear in mind that it takes far less time to go through an already-processed date than if a date has never been processed before. @harej 21:08, 27 November 2009 (UTC)
So once it's past January the speed should pic up because it has already done all the articles that include a January date combined with any other date? Hekerui (talk) 21:17, 27 November 2009 (UTC)
That, and the fact that there should only be 1 or 2 inappropriate links for a date where the bot already removed like 5,000 of them. Therefore, as the edit history of the bot's user page will tell you, it only takes a few minutes to go from January X to January Y (as January X backlinks had already been processed). @harej 21:35, 27 November 2009 (UTC)
I think I know what you mean, although the reply is slightly ambiguous: as I understand it, the bot is trundling on at 30 miles per hour in a 70 mph zone. Unless we get the bot working on more articles per minute, or get a more bots on the job, it will still be processing 6 articles per minute. However, as you said, the rate at which it moves through the calendar should speed up. Ohconfucius ¡digame! 13:28, 28 November 2009 (UTC)
Right. OK. Good to know. Darrenhusted (talk) 21:25, 27 November 2009 (UTC)
  • Also, I observed that the action on 1 January was to a fresh edit which only created date links on 27 November. I think it's good/important that the bot goes back to mop these up. Ohconfucius ¡digame! 00:46, 28 November 2009 (UTC)
  • Oh, I think Darren's idea of a second bot starting from the other end is an excellent idea. Or we could request permission to increase the speed limit to 15 edit/min so the task can be completed within a reasonable time (it's easy to predict the duration based on current daily rate).

    What would also be good is using MAXLAG, as suggested above, which will allow us to use unlimited speed. Maxlag will cut in to reduce speed when servers are busy. Ohconfucius ¡digame! 00:56, 28 November 2009 (UTC)

Mirror script

Due to popular demand, I am now running a second script which goes through the calendar backwards, starting from December 31. @harej 21:32, 28 November 2009 (UTC)

Wow, I was only thinking out loud, but thanks for doing that. Darrenhusted (talk) 22:47, 28 November 2009 (UTC)
Brill! Ohconfucius ¡digame! 01:24, 29 November 2009 (UTC)

Revert

This is the only significant one I have found so far, re-checking FDULB's edits, apart form a handful by deb in the first few days.

Rich Farmbrough, 01:51, 1 December 2009 (UTC).
I won't be the one to raise a ruckus. It's one isolated incident. @harej 01:59, 1 December 2009 (UTC)
No, not a problematic edit at all, it was simply contested. Ohconfucius ¡digame! 02:37, 1 December 2009 (UTC)

Article rescans on restart

I've noticed that after a restart the bot is taking longer and longer to return to the point it left off. I believe the bot is retrieving and scanning many articles that have month-day links for the dates already processed (i.e., January 1-30-), but have no qualifying fully linked date triples. Since there is no action taken, the article is not added to the processed list, and the bot is revisiting each of these articles with each restart. I estimate that there are between 1000 and 2000 articles per date that are being rescanned which means that the API is being called to retrieve roughly 30,000 and 50,000 over a about a five hour period before any real progress is made. That's a lot of extra overhead that will continue to increase.

I'd suggest resolving this in either of two ways: (1) add articles with no changes to the been-here-done-that database or (2) on bot restart, query the bot user page to extract the last-processed date and skip processing of all dates leading up to that. As an interim measure, January can be dropped from the outer loop so that a restart will immediately start with February. If you post the latest bot code (with the user-page status logic), I would be happy to assist with adjusting the restart logic. -- Tom N (tcncv) talk/contrib 05:05, 2 December 2009 (UTC)

I could actually work this in myself, I think. I just thought of a way, and it would not be that hard. @harej 05:44, 2 December 2009 (UTC)
 Done @harej 00:24, 3 December 2009 (UTC)

Bot's note in Summary

Hi, I just now see that you, the bot, link "Details here" to Full-date unlinking bot's user page where no explanation or details are given, as of now. I think this should be repaired, shouldn't it? [w.] 13:07, 3 December 2009 (UTC)

that was a temporary glitch, W. - the page had been blanked accidentally when you looked in. it's visible again now Sssoul (talk) 13:38, 3 December 2009 (UTC)

Consensus

It might be helpful to provide a link to the actual Consensus regarding date unlinking rather providing a link to how the consensus process works.--Kmhkmh (talk) 16:22, 2 December 2009 (UTC)

here's the RFC for this particular bot's deeds - that's a far less daunting document than the assorted RfC's on date-unlinking in general. Sssoul (talk) 17:41, 2 December 2009 (UTC)
Thank you, I'd suggest to put that next to the consensus-link on the bot's userpage.--Kmhkmh (talk) 18:46, 2 December 2009 (UTC)
sounds reasonable! harej, do you concur? Sssoul (talk) 19:23, 2 December 2009 (UTC)
Go ahead. @harej 21:02, 2 December 2009 (UTC)

(outdent) i'll wait until you've sorted out whatever caused the bot to blank its own page ... thanks harej Sssoul (talk) 07:09, 3 December 2009 (UTC)

It look like the code is missing getpage call before the logic that updates the processing date.
 		$userpage = $objwiki->getpage("User:Full-date unlinking bot");  // <<<----- Add this
 		$userpage = preg_replace("/\{{2}ombox\|text='''Currently processing backlinks to this date:\s?'''.*/i", "{{ombox|text='''Currently processing backlinks to this date:''' " . $months[$i] . " " . $d . "}}", $userpage);
 		$objwiki->edit("User:Full-date unlinking bot",$userpage,"Now processing: " . $months[$i] . " " . $d,true,true);
 		unset($userpage);
-- Tom N (tcncv) talk/contrib 08:42, 3 December 2009 (UTC)
I did not think it was needed, then I saw that I unloaded the variable within the loop without reloading it thereafter. @harej 21:15, 3 December 2009 (UTC)
if it seem to be fixed now, i'll add that link to the RfC. ongoing thanks & admiration for all your good work Sssoul (talk) 21:49, 3 December 2009 (UTC)

date formats

You've probably been asked this before, but shouldn't the bot be replacing linked dates with a template that displays of the reader's preferred date formate? Since the year's generally most important, I prefer YYYY-MM-DD. kwami (talk) 20:31, 5 December 2009 (UTC)

No such template exists. Although there is a magic word, {{#formatdate:date}} that attempts to do this, the magic word does not work correctly in many situations. There seems to be a consensus that any mechanism is worth using when it does not work for readers who have not logged in (this group forms the vast majority of readers). Indeed, there is no clear consensus that it is desirable to present different date formats to different readers under any circumstances. --Jc3s5h (talk) 20:54, 5 December 2009 (UTC)
But our user prefs allow us to decide how we see dates, so there obviously is a consensus to present different date formats to different readers. kwami (talk) 23:21, 5 December 2009 (UTC)
Many wikis still use that autoformatting function, and that preference has been there for a long time (doesn't mean it has consensus). Even on the English Wikipedia dates will still autoformat if you link them, but the practice is deprecated so we don't do it. More importantly, there is no consensus (see Wikipedia:Date formatting and linking poll) to present dates in different formats. Dabomb87 (talk) 23:25, 5 December 2009 (UTC)
The bot was approved for a narrowly defined task: removing links from linked full dates (and uncontroversial date errors such as ordinals, punctuation and capitalization). Therefore, it would go against the community's wishes for the bot to do something that it was not approved to do, especially since there is no consensus (there may even be a slight consensus against it) on the general idea of autoformatting dates. Dabomb87 (talk) 22:45, 5 December 2009 (UTC)
Okay, makes sense. kwami (talk) 23:21, 5 December 2009 (UTC)

Query

I notice you delinked dates at South Australian state election, 1912. I realise it's the new MOS and I don't have an issue with it, but just wondering why you managed to pick 1912 as a starting point when elections in SA (on wikipedia anyway) go as far back as 1890? Timeshift (talk) 07:20, 8 December 2009 (UTC)

Because it's going by day, and 10 February was apparently the earliest election. --NE2 07:22, 8 December 2009 (UTC)
Well there you go :) Thanks. Timeshift (talk) 07:30, 8 December 2009 (UTC)

Query "backlinks"

Perhaps I'm being thick; should the info at the top use the terms "links" instead"? Could be confusing to users. Tony (talk) 11:37, 8 December 2009 (UTC)

January 14

18430 Balzac I suspect the bot is missing these minor planet articles because they look date related. If so, no problem, they will all be done soon anyway. If that's no the reason, well just thought you should know. Rich Farmbrough, 01:55, 1 December 2009 (UTC).

I think it has to do with how they resemble years somehow, yeah. @harej 02:00, 1 December 2009 (UTC)
It looks like the "/^\d{1,4}(st|rd|th|nd|s)?\s?(century|millennium)?( BC)?/i" regex filter is responsible. It will match any article name that begins with a digit. All other components of the regex are optional and the regex is open-ended on the right (no "$" to match end-of-string).
Adjusting the filter is not trivial, since it filters a wide variety of articles that qualify as "intrinsically chronological articles" such a 1900s (decade), 209–200 BC], and "(year) in (topic)"-like articles. Year-qualified recurring events such as 2001 Tour de France and one-time events such as 1933 Outer Banks hurricane probably would not qualify and should be processed, as should non-dated articles such as 49th Parallel, 401(k), and 1790 Naval Air Squadron. Tweaking the filter and finding the proper balance may take some time and a good bit of testing, but there is no rush. The bot can easily rescan for missed articles at a later date. I expect an updated filter could include 10,000 to 20,000 articles that are currently being excluded. -- Tom N (tcncv) talk/contrib 04:32, 2 December 2009 (UTC)
It's not necessary, I am catching these articles. Rich Farmbrough, 21:33, 2 December 2009 (UTC).
They are done. Rich Farmbrough, 19:46, 10 December 2009 (UTC).

Bots using Maxlag

I did some googling for Maxlag, and may have found several php bots which have code which use the function. These are:

Is the code for any of these compatible with FDUB? Ohconfucius ¡digame! 09:53, 2 December 2009 (UTC)

AWB uses maxlag too. Rich Farmbrough, 19:46, 10 December 2009 (UTC).
Tx for that information. Ohconfucius ¡digame! 01:28, 11 December 2009 (UTC)

A very special thing has occurred!

The bot is now processing my birthday! @harej 04:23, 10 December 2009 (UTC)

smile: that must tickle Sssoul (talk) 07:10, 10 December 2009 (UTC)
Don't overdose on cake; we still might need you! But enjoy yourself! Chris the speller (talk) 02:22, 11 December 2009 (UTC)

#dateformat:... magic word

Why can't the bot replace linked dates with the {{#dateformat:...}} magic word? This would give date autoformating for users who have a preference, but without cluttering up the article with links. Hgrosser (talk) 19:18, 11 December 2009 (UTC)

It was approved to to one thing, remove [[ and ]] from all dates and occasionally clean up dates, nothing more. Darrenhusted (talk) 19:42, 11 December 2009 (UTC)
Please see above discussions, as well as the WT:MOSNUM archives and WP:DATEPOLL. There is no consensus to add date autoformatting templates. Dabomb87 (talk) 00:18, 12 December 2009 (UTC)

ISO unlinking in body text is unhelpful

Unlinking ISO-style dates in tables might sort of be OK. Edits like this one however that unlink ISO-style dates in body text are unhelpful. By unlinking, and thus removing autoformating, they obscure dates for those who previously used the autoformatting feature AND ARE HELPFUL TO NO ONE. Please undo all such edits performed by this and other bots. (sdsds - talk) 08:00, 21 December 2009 (UTC)

It actually is a mixed effect. By unlinking ISO dates, editors who previously had date preferences set will now see dates in the same form that the majority of (anonymous) readers see. In the case of the ProtoStar article, these readers have been seeing ISO dates since article creation, which might not have been apparent to you as an editor. This highlights the need to convert the dates to a more appropriate format in the article body. -- Tom N (tcncv) talk/contrib 09:42, 21 December 2009 (UTC)
Exactly, the edits don't obscure, they reveal, and that revelation should eventually be helpful to our unregistered readers, since once a regular editor who might not have previously noticed the weird date format will fix them, unregistered readers will at long last see properly formatted dates.
I would have thought the bot capable of at least converting ISO dates, but we're now way past that point of discussion. If you notice any ISO dates, please just fix them. Amalthea 10:39, 21 December 2009 (UTC)
Sdsds, you made this argument at Wikipedia talk:Manual of Style (dates and numbers)/Archive D6 on 00:43, 17 August 2008 (UTC) when you stated
  • Disagree - The work being done, albeit well intentioned, to remove markup that indicates some text may be automatically formated, is a step in the wrong direction. What wikipedia (enwiki, and elsewhere) needs is more markup indicating text which can be autoformatted, not less!
Your position has been rejected by the community in that discussion and in 2 subsequent RfCs, and in the approval process for this bot. You are now required to give up and accept the community consensus, or face the consequences. --Jc3s5h (talk) 15:26, 21 December 2009 (UTC)
Well, it's not quite as black and white as "rejected by the community", and yes, continuing to argue against it is tilting at windmills, but I don't see any consequences unless anyone starts mass-reverting the bot or mass-linking dates outside a few specific areas while they actually know better. Amalthea 16:19, 21 December 2009 (UTC)
I would expect that repeatedly stopping the bot because the bot is doing exactly what it is approved to do would also lead to consequences. This has not happened; let us hope it does not happen in the future. --Jc3s5h (talk) 16:27, 21 December 2009 (UTC)
Ah, I didn't realize that the bot was shut down by Sdsds. I've removed the shutdown password since this is clearly not an emergency and the bot works as designed, and I too ask Sdsds to not use the shutdown page unless the bot is actually malfunctioning, i.e. not working as intended and approved in the BRFA. Amalthea 16:45, 21 December 2009 (UTC)

Tom N writes, "These readers have been seeing ISO dates since article creation, which might not have been apparent to you as an editor. This highlights the need to convert the dates to a more appropriate format in the article body." Yes, but it violates WP:POINT to make an article worse for a portion of the readership. Changing body text ISO dates hurts readability for some and makes readability no better for anyone. (Regarding windmills, who was it who said, "Give me a windmill and a place to tilt....") (sdsds - talk) 05:39, 22 December 2009 (UTC)

sdsds, WTF do you mean??? The autoformatting function has been disabled for months already, and delinking ISO 8601 dates has a completely neutral effect on the date format which appears on your screen. It just eliminates the blue associated with a linked article. Ohconfucius ¡digame! 05:45, 22 December 2009 (UTC)
I believe the decision to turn off the date autoformatting ($wgUseDynamicDate) option was deferred until at least after this bot has finished processing, since the bot fixes some punctuation problem such as "[[December 22]][[2009]]" (no comma-space) and "[[22 December]],[[2009]] (improper comma)" that are currently adjusted during autoformatting. Once the bot completes its tasks, and any residual punctuation problems are cleaned up (such as might exist in the bypassed articles), turning off the autoformatting option will again be considered.
As for readability, which of the following scenarios is preferred?
  1. Unregistered readers see ISO formatted dates. Nothing changes. Unregistered readers continue to see ISO formatted dates.
  2. Unregistered readers see ISO formatted dates. Bot delinks dates. Primary editor recognizes a problem. Article is updated. Unregistered readers now see the same MDY (or DMY) formatted dates that the editor sees.
I think the answer is clear. -- Tom N (tcncv) talk/contrib 06:35, 22 December 2009 (UTC)
This is indeed old ground, and covering it again adds little. Had this been approached correctly, the bot could have transformed ISO dates in body text into MDY or DMY -- either would have been better than what it is currently doing. But instead the "community" went forward with a bot that breaks more than a few eggs along the way towards making its omelette. That's a shame, but I join with the community in hoping this effort will one day result in a better encyclopedia, and acknowledge that for many articles, it at least makes them no worse. Too bad that isn't true for all articles, though! (sdsds - talk) 07:18, 22 December 2009 (UTC)
Well we did change a whole bunch of them, just after FDUB started running. But there is no reason someone (you?) can't search for and change ISO-style dates in text. Rich Farmbrough, 22:48, 22 December 2009 (UTC).
  • "This is indeed old ground, and covering it again adds little." Indeed. You're complaining about the eggshells in your omelette, then?? ;-) Ohconfucius ¡digame! 07:31, 22 December 2009 (UTC)

I did consider having the date change from ISO to something else, based on the plurality of date-styles that appear within the article. That, however, would open up a can of worms on which to choose and why. We ultimately decided that the bot should not make any judgments, and to just remove link brackets and correct date-related punctuation. @harej 23:50, 22 December 2009 (UTC)

That was certainly an understandable approach given the desire to get moving on the task quickly. And it means the bot doesn't need to detect and special-case ISO dates in the sortable columns of tables (which have to remain ISO format for sorting to work as intended). Relatedly, I'm thankful for the editors who manually fixed up the article that brought this issue (back) to my attention. Is there a place where whines about other bot-edited articles would get similar personalized attention? (sdsds - talk) 03:22, 23 December 2009 (UTC)
I am not sure what you mean. This talk page is dedicated to the operations of this bot and is as relevant to the bot as you're going to get. @harej 03:40, 23 December 2009 (UTC)

bot speed

James, any reason the bot is running so slowly, compared with the maximum speed that would be allowed, or why it runs so intermittently? Tony (talk) 11:49, 18 December 2009 (UTC)

First of all, 10-12 edits per minute is by no means slow. Secondly, this is not an urgent task and we literally have until the end of time to complete this. Furthermore, the API likes having weird errors that the bot cannot account for, so it has to shut down (believe me, I have tried to resolve this). My wish is that with the script I proposed above, there will more or less be continuity in the bot's operation. @harej 12:20, 18 December 2009 (UTC)
Is the mirror script intended to make the task complete sooner? Is there a significant risk of edit or database conflicts between the non-mirror and mirror scripts? — Richardguk (talk) 13:01, 18 December 2009 (UTC)
This is not a significant risk, while the edit conflict could happen, edit conflicts can happen anyway. The system resolves them up to to a point, the bot can check the version number when it submits the page. If the bot overwrites itself it is less of a problem than if it overwrites someone else. The mirror script is speeding the task upm, and if you read back speed was the reason. Rich Farmbrough, 17:09, 18 December 2009 (UTC).
Out of curiosity, if speed was the reason for introducing the mirror script (I'm not sure that's conclusively stated in the previous sections), why instead was the sleep() argument not simply reduced from the current value of 6 seconds, sidestepping the complexity of dual runs? Incidentally, I note that the transition from February to March has demonstrated that the bot assumes 31 days in every month, which seems a harmless quirk. Is it intentional that dates without 4-digit years are left unfixed? The listed exceptions cover dates without years but aren't explicit about years prior to 1000. — Richardguk (talk) 18:32, 18 December 2009 (UTC)
I think the mirror script pre-dates my mechanism for tracking the resumption of the script. Yes, the bot assumes every month has 31 days; this is just a lot easier and has no real negative effects. And it may be intentional that dates without 4-digit years are unfixed; what say you, date-experts? @harej 20:50, 18 December 2009 (UTC)
I think Richardguk may be correct in that it would have been simpler to reduce the sleeptime from 6 seconds than to run a mirror script, and Harej is the best person to tell you why he chose the solution he did. I am among a number of people who would have preferred to let the bot completely loose, except capped by MAXLAG (to protect server loading).

As to the dates with years prior to 1000, I think the consensus implicitly included delinking these dates, if they are full dates. If the bot is written only for four digit years, it should be tweaked. Ohconfucius ¡digame! 02:26, 19 December 2009 (UTC)

With the exception of ISO-8601-like dates, the bot is specifically coded to recognize linked years with 1–4 digits. Are you seeing cases of three digits or less that are not being unlinked? -- Tom N (tcncv) talk/contrib 17:48, 19 December 2009 (UTC)
Apologies, I thought I'd seen several unfixed medieval dates on February pages but on looking at March 2 cannot find any so this does not seem to be a problem after all. Sorry for the confusion. That said, it might be worth noting the following which remain unfixed for March 2:
— Richardguk (talk) 19:28, 19 December 2009 (UTC)
In each of the above example, the month/day is linked but the year is not. The bots charter is only to delink fully linked date triples such as "[[March 2]], [[2008]]", but not "[[March 2]], 2008" or "March 2, [[2008]]". The philosophy was that date triples were most likely linked solely for the purposes of date autoformatting, or by editors that linked all dates because that appeared to be common practice. I believe it was judged that solitary month-day links and solitary year links have a (slightly) greater likelihood of having been intentionally linked based on relevance, so these are being left intact for manual review and cleanup. -- Tom N (tcncv) talk/contrib 20:36, 19 December 2009 (UTC)
Ah, thanks, and further apologies for inadvertently stirring. All's well that ends well. — Richardguk (talk) 00:33, 20 December 2009 (UTC)

Having two scripts, each with a sleep time at 6 seconds, and having just one script with a sleep time of 3 seconds would have the same effect of making the editing faster. Therefore it was a matter of preference; I feel like more progress is made when two dates are being done at once. Once I figure out how best to implement maxlag. The issue is that on the client side, the maxlag value is basically a test; if the lag is below the maxlag, all is good, otherwise it will result in an error message. Because I have not found a way to properly deal with error messages in a way that allows the bot to continue editing, my only recourse is to have the bot stop. I might be able to put an end to such perfenander by integrating edit conflict handling within the bot classes file itself. I will let you know if I accomplish anything. @harej 03:08, 19 December 2009 (UTC)

Thanks for indulging my idle curiosity, and for all your good work. Perhaps you should organise a sweepstake as to which date is being unlinked when the two bots eventually meet! — Richardguk (talk) 03:52, 19 December 2009 (UTC)
I'll go for 11 May. ;-) Ohconfucius ¡digame! 04:07, 19 December 2009 (UTC) Sorry, I mean 11 June. Ohconfucius ¡digame! 05:48, 21 December 2009 (UTC)
Whoever is closest wins... money. Maybe. @harej 04:47, 19 December 2009 (UTC)
I'll take July 15 (a family birthdate). Interestingly, after the two bots meet, they will continue on in their respective directions to make one last pass through previously unprocessed articles that might have recently added linked full dates. -- Tom N (tcncv) talk/contrib 17:48, 19 December 2009 (UTC)
Hey this is Wikipedia, I think we're only allowed to gamble for barnstars! July 12 is the current midpoint but I think the earlier dates will somehow have more links so I'll go for July 3. (I don't want to be around when they meet though: unless one bot has nearly finished when the other arrives, the two bots will end up battling over the same articles, probably in the same order, probably causing thousands of edit conflicts.) — Richardguk (talk) 01:24, 21 December 2009 (UTC)
"10-12 edits per minute is by no means slow". Actually, if we take a look at the last 5000 edits, it has averaged 7-8 edits per minute, and I've not seen it

get to 12 edit/min for some time. Can the delay be tweaked to bring it back up to 12 edit/min? As I understand it, the reason for constraining speed is to ensure that servers aren't overloaded by a bot that doesn't measure and respond to server load. It isn't because slow is good. The value is arbitrary and plenty of bots run faster than 12 edits per minute without overloading servers, I'd like to see 30 edit/min. Harej, your work is appreciated and I hope you don't mind these comments. Ohconfucius ¡digame! 14:16, 23 December 2009 (UTC)

November 7

Took no time at all to process. Very odd. Rich Farmbrough, 22:29, 23 December 2009 (UTC).

Yep – 33 seconds to be more precise! I'm guessing the list of linked articles did not get returned so was treated as blank, or perhaps was severely truncated. Special:Contributions/Full-date unlinking bot shows that no articles were edited for November 7 but that November 8 and November 6 articles were successfully edited immediately before and after 21:49 UTC. There are certainly articles that link to November 7 to be processed, for example Chris Karamesines and Upstage (film). — Richardguk (talk) 23:08, 23 December 2009 (UTC)
Same problem with November 5: the bot finished November 6 at 01:19:42 and started November 4 at 01:20:14 UTC. — Richardguk (talk) 02:19, 24 December 2009 (UTC)
How are you getting timestamps that are precise to the second? @harej 14:48, 24 December 2009 (UTC)
Taken from the times when {{/currently...}} was updated on the User:Full-date unlinking bot page history. Strictly, therefore, the earlier time is not the end of one date but the start of the skipped date. Cross-checked against the contribution history. Hope you're not making it skip dates to rig the sweepstake! Keep up the great work. — Richardguk (talk) 15:42, 24 December 2009 (UTC)
I meant how your timestamps kept count of the seconds, whereas Wikipedia only shows minutes for me. I guess that's just how my timestamp settings work. Also, my programming skills are barely good enough to run a full-date unlinking bot; I am not nearly adroit enough to use said skills to rig a contest. :D @harej 15:56, 24 December 2009 (UTC)
Ironically, timestamps with seconds are an option under Special:Preferences, apparently only available if you opt for the ISO date format ("2001-01-15T16:12:34") under "Date and time". Yay ISO! — Richardguk (talk) 16:56, 24 December 2009 (UTC)
Problem with a third date: October 28: the bot finished October 29 at 20:31:19 and started October 27 at 20:36:34 UTC. Curiously, this time it did start processing but only amended one article for October 28: Java coffee. Perhaps the 5 minute interval was a timeout? — Richardguk (talk) 21:03, 24 December 2009 (UTC)
The last time the mirror script had to re-start was over two hours ago. @harej 21:12, 24 December 2009 (UTC)
Not sure how that would cause the dates to be skipped. But there seem not to be many articles that link to October 28 so presumably the date was (mostly) fixed already, either by Fdub or someone else. The Java coffee date was formatted without a space so might not have been picked up on an earlier run: "...Accessed on [[28 October]][[2005]]</ref>..." — Richardguk (talk) 22:34, 24 December 2009 (UTC)

Scheduled downtime

The bot will not be operating while I implement and test revisions to the code regarding operation. @harej 13:03, 24 December 2009 (UTC)

The maintenance is done. I have posted the changes to the code here. @harej 14:45, 24 December 2009 (UTC)

Merry Christmas

This edit was the bot's first edit on Christmas Day. Merry Christmas to my appreciated development team. Yours, @harej 01:56, 25 December 2009 (UTC)

skipped?

I see no reason why the bot seemed to have skipped Maynard Ferguson. There were a few linked dates, including 1 January 1950 and February 27, 2005. These have now been delinked. Ohconfucius ¡digame! 05:26, 22 December 2009 (UTC)

Looks like a false hit on the article-name filters. In the bot code, $regex1 is designed to filter "Month" and "Month day" article names, but also filters anything that begins with a month name, such as "Maynard Ferguson". We have previously identified a similar problem that filters any article that begins with a digit (see January 14 topic above). -- Tom N (tcncv) talk/contrib 06:14, 22 December 2009 (UTC)

The bot seemed to have skipped Mayen. There were a few linked dates, including 12 December 1944 and 2 January 1945. These have now been delinked. I presume it's the same as with Maynard Ferguson. Ohconfucius ¡digame! 02:57, 24 December 2009 (UTC)

Yes. Same reason. Any article name that happens to start with the prefix "May", "March", or any other month name will be false hits for the article name filters. -- Tom N (tcncv) talk/contrib 06:03, 24 December 2009 (UTC)

People's creativity knows no bounds: I have just come across an article with piped ISO dates that your bot missed. Piping is along the lines of | accessdate=[[2009]]-[[January 2|01-02]]. It is not the first example of such I have come across. Ohconfucius ¡digame! 03:18, 24 December 2009 (UTC)

Amazing. -- Tom N (tcncv) talk/contrib 06:03, 24 December 2009 (UTC)

Any reason why were Samit Bhanja and Melvin Holwijn passed over by the bot? Ohconfucius ¡digame! 05:26, 24 December 2009 (UTC)

Both articles link to [[02 January]], which is not a defined article or redirect in Wikipedia, so the above-mentioned articles were not included in the what-links-here lists for [[January 2]]. It seems that date formatting overrides the actual defined link and directs it straight to January 2. -- Tom N (tcncv) talk/contrib 06:03, 24 December 2009 (UTC)
I just generated 18,900 possible date like items - but reduced it to about 6,500, I'm checking these out now, for April (Stuff like 03rd Apr ) it picks up about 300 articles. Rich Farmbrough, 12:13, 24 December 2009 (UTC).

Any idea why UK Mail was not fixed by Full-date unlinking bot when it contained "...on [[1 January]] [[2006]]. It..."? Unlike the above articles, this one seems to have a regularly formatted date and the title does not contain a date element. I notice that it was fixed today by SmackBot. — Richardguk (talk) 22:19, 24 December 2009 (UTC)

It's slightly frustrating, I see FDUB picking up stuff SmackBot should have got too. But on the other hand we will get them all in the end, and the code is largely throw-away. Rich Farmbrough, 22:38, 24 December 2009 (UTC).
  • Another one missed? This one was linked to 3 January 1968. Ohconfucius ¡digame! 13:36, 27 December 2009 (UTC)
    Trailing space. I will be picking up these items tonight, probably, wierd dates that aren't redirects, mainly tomorrow.(But not 3th March and the like.)) Rich Farmbrough, 21:16, 27 December 2009 (UTC).

Suggestion

Hi. Do you think it would be possible to format date automatically by using {{date}} template, while unlinking dates by bot? Beagel (talk) 05:52, 28 December 2009 (UTC)

  • We've been over this repeatedly: it's outside the scope of this bot. In any event, the consensus appears to be that autoformatting is not desirable. What is more, nobody has deigned to respond as to what practical use {{date}} has outside of displaying today's date. Ohconfucius ¡digame! 07:08, 28 December 2009 (UTC)
    • It's useful in the following circumstance: You have a template: (such as {{Bolognia 2}}) which displays, in this example a reference to a book. You want the publication date, 1 January 1999 (say) to display in the appropriate format for the page it is used in. You do {{Bolognia 2|15|234|format=dmy}} and then use the date template within {{Bolognia 2}} to ensure that the style is consistent across the page. Rich Farmbrough, 13:30, 28 December 2009 (UTC).

how many bots are delinking dates on wikipedia?

I have noticed user:SmackBot is also delinking. are there any others?  Dr. Loosmark  13:34, 23 December 2009 (UTC)

Just noticed the same. SmackBot seems to be delinking at a rate of around one article a second. Though not significantly harmful, I'd have thought it be better for consistency and server resources for SmackBot to stand back and instead have the throttling on User:Full-date unlinking bot reduced. Incidentally, Full-date unlinking bot has speeded up from around 7 hours per date to less than five 4 hours per date within the past day or two, perhaps because of a reduction in load with the holidays – or has the throttling been reduced? — Richardguk (talk) 13:57, 23 December 2009 (UTC) (edit: currently under 4 hours per date — 14:31, 23 December 2009 (UTC))
Smackbot runs using AWB, which has a built in MAXLAG function (which slows the bot if server loads become heavy), thus should be of no concern for server load. Ohconfucius ¡digame! 14:49, 23 December 2009 (UTC)

FDUB only does full dates, leaving an odd mix of unlinked dates and linked partial dates. The other bots are stepping in to fill the gap. --NE2 18:29, 23 December 2009 (UTC)

Since it appears that the ArbCom's mass date delinking restrictions have expired (as of 14 June + six months), are we tossing the "community approved" aspects and resumed unrestricted operations? If that's the case, maybe we should just shut down this limited-scope bot, since we now have several overlapping efforts underway. -- Tom N (tcncv) talk/contrib 20:44, 23 December 2009 (UTC)
FDUB will be finished soon. It has done a great job of phase 1, I am hoping James and the coders will be back with PDUB, this is a more sensitive area - well years are, most of the other constructs are not. There are some 30,000 Month Year links I believe. Rich Farmbrough, 07:06, 24 December 2009 (UTC).
indeed: FDUB is doing a beautiful job with task specifically tailored to avoid oversensitive territory. let it rock! Sssoul (talk) 11:05, 24 December 2009 (UTC)
Not sure why we would need a PDUB at this point. It appears that SmackBot has already pulled out all the stops by implementing and executing Partial Date Unlinking - without BAG approval (application submitted but never approved). I'm sure many applaud the effort and the results, but I think it sets a bad precedent to allow even one of the more experienced bot operators to proceed unchecked – especially in an area with a history of controversy. -- Tom N (tcncv) talk/contrib 02:11, 30 December 2009 (UTC)

Maxlag in operation?

I notice the code for FDUB was amended with a comment "Maintenance, reliance on maxlag instead of arbitrary throttle". OTOH, I notice that the sleep parameter remains at (6), and it is delinking about five articles per minute. Please could you explain what this all means? Ohconfucius ¡digame! 16:05, 25 December 2009 (UTC)

The sleep(6); is commented out; it is not executed during runtime. That means it operates under the same speed with the maxlag mechanism as with the throttle, which means I got it right the first time. (It's still wiser to use maxlag, though, so that the bot can slow down as needed). @harej 20:28, 25 December 2009 (UTC)
If it's under maxlag and no longer has a 6 second sleep, what could explain the current bot speed – it made only 50 edits in 20 minutes, and it's delinked 9,252 articles on Friday (6.4 edits per minute)? Ohconfucius ¡digame! 05:15, 26 December 2009 (UTC)
Because with maxlag in place (set at 5 seconds), it just happens to take that long. Are you saying I should give the bot a higher sense of priority and double its maxlag? @harej 15:47, 26 December 2009 (UTC)
Well, I'm querying it because I just find it very surprising. Maybe I've got misplaced expectations, but it's holidays, there's minimal server activity, yet the bot's trundling along, at an even slower rate than before Maxlag was implemented (6,634 articles on Saturday or 4.6 articles per minute). I was expecting it to be zapping through dates at 10-50 articles a minute now that the sleeping policeman has been bypassed. ;-) Ohconfucius ¡digame! 05:57, 27 December 2009 (UTC)
But it's getting through dates faster. I just picked off one more template that makes loads of unrequired links, that should give another couple of minutes saving per date. Rich Farmbrough, 11:59, 27 December 2009 (UTC).

Should be finished by now... still any time today is fine.. . Rich Farmbrough, 13:32, 28 December 2009 (UTC).

Indeed, we are quickly approaching the rendezvous point. It will be in August, like I predicted it to be.[citation needed] @harej 16:17, 28 December 2009 (UTC)

Woot!

We have crossover! Rich Farmbrough, 17:24, 28 December 2009 (UTC).

August 9, the date the two scripts met. Here is the permalink to that occasion. @harej 17:54, 28 December 2009 (UTC)
Congrats! Interesting to see the small number of articles being picked up on the remaining sweep, mostly newly-created articles. Only teasing with the {fact} edit above. — Richardguk (talk) 23:37, 28 December 2009 (UTC)

Look at this. Rich Farmbrough, 13:40, 29 December 2009 (UTC).

That is seriously cool, in a very geeky fashion... Happy‑melon 13:57, 29 December 2009 (UTC)
bravi! very nice indeed 8) Sssoul (talk) 14:29, 29 December 2009 (UTC)

And the mirror script has finished with January 1. Task done, I think. Darrenhusted (talk) 00:43, 30 December 2009 (UTC)

More skipped?

Ohconfucius ¡digame! 05:24, 30 December 2009 (UTC)

Sigh, I'd work on them but I have some "corrections" to do and SB has other issues right now. Rich Farmbrough, 09:21, 30 December 2009 (UTC).

Task completed

After being worked upon for most of this year, I am glad to announce that the full-date unlinking bot has completed operations. Mission accomplished. @harej 00:56, 30 December 2009 (UTC)

I don't want to blow any trumpets or anything but on the timeline it might be nice to note when the mirror script was added. And congrats! on the whole thing harej. Darrenhusted (talk) 01:02, 30 December 2009 (UTC)
Congrats! Awesome. --Apoc2400 (talk) 11:39, 30 December 2009 (UTC)
Well done! Hekerui (talk) 14:46, 30 December 2009 (UTC)

Next step:turn off date autoformatting

Is anyone going to pursue having the variable changed that will turn off date autoformatting in the English Wikipedia? It's been so long, I don't remember the details of how this could be done. --Jc3s5h (talk) 17:19, 30 December 2009 (UTC)

More work to be done!

Now that the work on articles is complete, Full Date Unlinking Bot should be tasked to process non-articles such as:

I believe this still falls within the original remit of the bot, so further authorisation should not be necessary for this. The objects linked can be found by looking at 'what links here' for each of the days of the year (that is how FDUB found articles). There are too many for an individual but it would be easy for FDUB. Ohconfucius ¡digame! 13:21, 3 January 2010 (UTC)

The bot is only authorized to work in the main namespace; I believe allowing it to work in those namespaces would require additional approval. @harej 20:42, 3 January 2010 (UTC)

Template:NRDB species

Several hundred reptile articles have autoformatted dates because they transclude this template. I'm not sure how exactly to go about modifying the template though, as simply removing the square brackets would break any MDY-formatted dates. Any suggestions? Colonies Chris (talk) 20:33, 3 January 2010 (UTC)

Change the template to accept an alternate parameter "accessdate" that takes a full date, and then ask someone nicely to go through all 700 transclusions of the template and change the date/year parameters into a full accessdate parameter, either by a straight conversion that takes the old autoformatting rules into account or preferably, if one format is highly prevalent on the page in question, by changing it into that one. Amalthea 20:53, 3 January 2010 (UTC)
I have moved the autoformatting for now. Rettetast (talk) 21:05, 3 January 2010 (UTC)
Although a non-issue here (I checked), but a possible issue for similar templates, is that a template might expect date auto-formatting to insert a comma when needed in code like "[[{{{1}}}]] [[{{{2}}}]]", when the first parameter is in month-day form rather than day-month. I scanned a slightly out-of-date dump for NRDB template usage, and found that all of the dates used in this template appeared to be in day-month form. -- Tom N (tcncv) talk/contrib 03:15, 6 January 2010 (UTC)
That's what I meant with "taking the old autoformatting rules into account", thanks for checking! Amalthea 13:10, 6 January 2010 (UTC)

Could someone look at Template:Video game release. The syntax is to intricate for me. Rettetast (talk) 01:20, 6 January 2010 (UTC)

The key nested template appears to be Template:Dts vgr. I posted a proposed change on the talk page for SharkD (talk · contribs) to review. -- Tom N (tcncv) talk/contrib 03:03, 6 January 2010 (UTC)

Dangling Braces

In Lori Foster, the bot left opening double braces on the birthdate line, thereby breaking the page layout. -- Schewek (talk) 19:35, 19 January 2010 (UTC)

No, the problem was created in this edit by an anonymous editor using the IP address 74.83.79.203 --Jc3s5h (talk) 19:42, 19 January 2010 (UTC)

Template:CalendarCustom

Could someone look at changing this template to not link the bare month? The linking must be buried somewhere in a nested template, it's not obvious. Colonies Chris (talk) 09:07, 29 January 2010 (UTC)

Monthly bot run

New articles are created each month by editors who may be unaware of our linking protocols and policies. Could we perhaps schedule a monthly bot run, or to program the bot to patrol recent changes, particularly to delink dates of these recently created articles, and any other articles which may have been linked by editors unaware of our practices? Ohconfucius ¡digame! 10:30, 29 January 2010 (UTC)

At least some of the relinking was done because the bot was still processing "year in topic" and "decade in topic" articles. If that hasn't been resolved, it shouldn't be rerun, although it could still be run on recent articles. — Arthur Rubin (talk) 10:40, 29 January 2010 (UTC)
Rerunning is outside the scope of the bot approval and would have to be approved by the bot approval group. --Jc3s5h (talk) 11:20, 29 January 2010 (UTC)
And doesn't SmackBot handle this stuff anyway? harej 00:35, 30 January 2010 (UTC)

Template:Infobox SCOTUS case

This one's a mystery. It appears to be linking 'day, month'; I say that because articles that transclude it show appears in a 'what links here' list for a specific date e.g. links to October 11 include Seminole Tribe v. Florida, which has a Date_Argued of October 11, 1995. However, there's no obvious linking in the template and no date link appears in the article itself. Colonies Chris (talk) 12:30, 3 February 2010 (UTC)

From Template talk:Infobox SCOTUS case#Datelinks?:
The template tests several parameters to see whether there exist articles of the same name, using {{#ifexist:...|...|...}}. Even though the pages are not in fact linked, the test is sufficient for the connection to be recorded on WhatLinksHere.
If a relevant parameter is assigned a value and there exists no wiki page of that name (perhaps because a date is misspelled or badly punctuated), then the template flags the argument as invalid by categorizing the transcluding article under Category:Flagged U.S. Supreme Court articles.
The parameters tested are: SubmitDate, SubmitYear, ArgueDate, ArgueYear, ArgueDateA, ReargueDate, ReargueYear, ReargueDateA, ReargueDate2, ReargueYear2, ReargueDateA2, DecideDate and DecideYear.
— Richardguk (talk) 23:01, 5 January 2010 (UTC)
The validation could be done without using #ifexist. But I'm not sure that the benefit would justify the effort, unless someone is planning to recode the template anyway. — Richardguk (talk) 13:28, 3 February 2010 (UTC)
Yeah, this is bugzilla:12019. --MZMcBride (talk) 19:24, 3 February 2010 (UTC)

Limited approval for trial run in category and portal space

I refer to approval at Wikipedia:Bots/Requests for approval/Full-date unlinking bot 2. Ohconfucius ¡digame! 05:20, 22 February 2010 (UTC)

Retrieved from "https://en.wikipedia.org/w/index.php?title=User_talk:Full-date_unlinking_bot/Archive_2&oldid=1142549324"