Wikipedia:Bots/Requests for approval/Lowercase sigmabot III 2

Add links
From Wikipedia, the free encyclopedia
The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was Approved.

Lowercase sigmabot III 2[edit]

Operator: Σ (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 02:49, Monday October 14, 2013 (UTC)

Automatic, Supervised, or Manual: Automatic

Programming language(s): Python

Source code available: User:Lowercase sigmabot III/Source.py

Function overview: Replace the MiszaBots.

Links to relevant discussions (where appropriate): WP:BOTN#MiszaBots_down

Edit period(s): Continuous

Estimated number of pages affected: Any page that transcludes the appropriate template. [0, ∞) 14181

Exclusion compliant (Yes/No): No

Already has a bot flag (Yes/No): No

Function details: Archive pages that transclude {{User:MiszaBot/config}} if the MiszaBots become inactive.

Discussion[edit]

This task uses my own code. →Σσς. (Sigma) 02:49, 14 October 2013 (UTC)[reply]

Yay <3. Have you implemented the key feature that archivebot.py had? To prevent bot induced vandalism, if a page had an archive that was not a subpage of the current page, Misza had to generate a hash that the bot would verify with a salt and then archive it. See also Category:Pages archived using a key. Legoktm (talk) 03:55, 14 October 2013 (UTC)[reply]
Yes. I haven't gotten around to testing it, though, so we'll see whether or not my implementation is sound. →Σσς. (Sigma) 03:56, 17 October 2013 (UTC)[reply]
The key works. →Σσς. (Sigma) 07:38, 26 October 2013 (UTC)[reply]

I set up a {{User:MiszaBot/config}} on my talk page just yesterday (so of course it hasn't archived yet). It's a bit of an odd situation, since there are 53 pages of manual archives already, so a bot should create the 54th. (I have "counter=54" in the config template.) If you want to try the bot out on an unusual case, you're certainly welcome to try my page. – Quadell (talk) 23:02, 17 October 2013 (UTC)[reply]

  • Can we get the ball rolling on this? Werieth (talk) 13:05, 21 October 2013 (UTC)[reply]

I'm still a member of BAG, though I haven't been active in a long time. Still, unless there are any objections, I see no reason that I can't approve a trial here. (It would be best if the bot operator could try a variety of different kinds of pages.) Approved for trial (50 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. – Quadell (talk) 22:08, 21 October 2013 (UTC)[reply]

It looks like you used User:Legobot, rather than User:lowercase sigmabot III, for this test, correct? – Quadell (talk) 10:53, 22 October 2013 (UTC)[reply]
Er, no, that's me. I've been running one off's of Misza's code to keep the backlog down until someone can actually take them over (lcsb3!). Legoktm (talk) 20:18, 22 October 2013 (UTC)[reply]

Trial complete., and slightly overdone while I was going around in circles chasing a few bugs. →Σσς. (Sigma) 05:16, 28 October 2013 (UTC)[reply]

{{BAGAssistanceNeeded}} I'll need to test my bot on more unusual talk page and archive configurations. A time-limited trial would be appreciated. →Σσς. (Sigma) 05:28, 28 October 2013 (UTC)[reply]
My spotchecks indicate that the bot has worked correctly on the pages it has attempted to archive. What sorts of "unusual talk page and archive configurations" are you hoping to test, and what are pages you would like to make sure your bot archives correctly? – Quadell (talk) 14:32, 28 October 2013 (UTC)[reply]
I don't know. It's entirely possible that there are talk pages more unusual than User talk:Jasper Deng's, which surprised me and gave me a great deal of trouble. My intent for the next trial is to find out what else breaks the bot, fix the bot accordingly, and then decide whether the bot is functioning the best it can. The best way to do this is to let it edit more freely, under a trial. →Σσς. (Sigma) 04:36, 29 October 2013 (UTC)[reply]

If you're willing to check every edit it makes and clean up after any accidents, then let's do this: Approved for trial (3 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. – Quadell (talk) 11:57, 29 October 2013 (UTC)[reply]

Day 1 ends with me finding a few bugs, and learning that I forgot to implement the minthreadstoarchive parameter. Sorry. →Σσς. (Sigma) 07:57, 3 November 2013 (UTC)[reply]

  • I love this bot, and recommend that it get approved when ready. Meanwhile, I've blocked it per, "Administrators: if this bot is making edits that appear to be unassisted to pages not in the operator's or its own userspace, please block it." The bot is all over the place loading up people's watchlists with edit notices. Before you continue testing, please flag it as a bot so it doesn't annoy people. There have been a bunch of complaints. Jehochman Talk 17:51, 4 November 2013 (UTC)[reply]
    And admin fail, the bot was doing an approved test run. It shouldnt be flagged until after its been established running correctly. Werieth (talk) 17:55, 4 November 2013 (UTC)[reply]
    Admin fail? I don't know what you mean. Could you please speak English, and be polite while you are at it? I was following the directions on the bots page, which said, "Administrators: if this bot is making edits that appear to be unassisted to pages not in the operator's or its own userspace, please block it.", in case you missed it the first time. Moreover, the bot was annoying lots of people by cluttering their watchlists, which is good reason #2 for it to be blocked, and good reason #3 is that it's automatic shutoff switch doesn't work. Please rethink your testing regimen. If a bot is going to be released to make a high volume of edits, can it be flagged temporarily? If it's going to be out there, can it please have a functioning shut off switch? Jehochman Talk 18:03, 4 November 2013 (UTC)[reply]
    I think Werieth meant I had failed to set the bot's flag when I approved the test, not that you had failed, Jehochman. He may be correct; I'm a little rusty at bot approvals. Jehochman, if I set the bot's bot-flag to true, will that satisfy your complaint? – Quadell (talk) 18:08, 4 November 2013 (UTC)[reply]
    Absolutely. Thank you, Quadell. Werieth, I am very sorry for sniping at you. No hard feelings? Jehochman Talk 18:10, 4 November 2013 (UTC)[reply]
    Sure thing. But it appears that I cannot set lowercase sigmabot III's bot flag; only a bcrat can do that. Is this something that should be done during a wide-scale test like this? And who usually does sets this flag? (I'll page for another, more knowledgeable BAG member.) – Quadell (talk) 20:41, 4 November 2013 (UTC)[reply]

{{BAG assistance needed}}

  • Bug at WT:TW (source, archive): It removed a level3 heading, which is not how Miszabot did it, and I note that it didn't add that section to the archive, which looks like a separate bug (but might not be relevant if the other one is fixed).
    On second thought, it may be the missing timestamp that's confusing the logic? You are parsing timestamps to find sections to archive just like Misazbot did, correct?
    Cheers, Amalthea 19:07, 4 November 2013 (UTC)[reply]

{{OperatorAssistanceNeeded|D}} – Quadell (talk) 20:41, 4 November 2013 (UTC)[reply]

I'll look at that shortly. I think the more pressing issue is the one about watchlist spam. We seem to have general opposition to the fact that people cannot hide the bot's edits in their watchlists. I believe that this warrants granting lcsb3 the bot flag even though it hasn't been approved. →Σσς. (Sigma) 00:33, 5 November 2013 (UTC)[reply]
I'm not comfortable with handing out the bot flag for testing. The bot flag means "I'm doing something well understood and highly structured in a reliable and tested way" not "I'm doing lots of stuff". If this was the existing MiszaBot's code being run by another operator, then fine, it would just be a matter of evaluating the op and letting them go. This is a new code base, and its activities need to be inspected. If you can't do this in a way that doesn't disrupt the community, then perhaps it ought not be done at all. Why not scan the pages for "odd settings" and then test against them, in a sandbox? Why do you need to test against the live system? Why not only archive pages that aren't "odd"? Why not run the code against MiszaBot's output, and halt on differences? Why not have a code review done? Why aren't you striving to minimize the angst created by the bot? Josh Parris 01:41, 5 November 2013 (UTC)[reply]
When people began to push for my bot marking its edits as bot edits, I had the exact same concerns you have right now. A bot should not be marked as a bot when we are not sure whether it should be a bot. I have decided that because I won't have much time this week, a code review by experienced Python programmers, such as @Legoktm: and @The Earwig: would be the most effective path to take today. And, Amalthea, I've just fixed that bug. →Σσς. (Sigma) 10:19, 5 November 2013 (UTC)[reply]
I'm going to declare Trial complete., and we can start talking about next steps in verifying this is a safe bot to run. How about you arrange for a code review, and come back here with the results. Have you considered running existing code instead of something you've re-invented? Josh Parris 11:30, 5 November 2013 (UTC)[reply]
Have you seen the MiszaBot code? its been in need of a fairly major rewrite for several years. Werieth (talk) 12:08, 5 November 2013 (UTC)[reply]
The summary (up top) suggests the code is not available for review. Please correct that if it is now available. Josh Parris 03:48, 7 November 2013 (UTC)[reply]
  • May I suggest that for testing purposes you recruit a group of willing editors and have them tag pages they want the new bot to archive. That would allow you to do a more limited test that would not annoy so many people with the watchlist spam. Either that or if further testing is done, can you throttle the number of transactions to be 1/10 as many? That would also resolve the watchlist spam problem. Thank you for the good work! This bot is going to be really useful. Jehochman Talk 12:13, 5 November 2013 (UTC)[reply]
    One other suggestion, leave the user talk archiving to the last, and keep it as minimal as possible. That would avoid excessive echo notices. Werieth (talk) 13:21, 5 November 2013 (UTC)[reply]
    Regarding Jehochman's suggestion, I am willing to assist with that, Σ, if it would be helpful. – Quadell (talk) 13:40, 5 November 2013 (UTC)[reply]
    I'll look into it. →Σσς. (Sigma) 09:33, 9 November 2013 (UTC)[reply]

Legoktm and Earwig have reviewed the code. In general, it is sane, but there could be some optimisations made to it. →Σσς. (Sigma) 09:33, 9 November 2013 (UTC)[reply]

I make the following observation: if the bot stuffs up, it will cause outcry and the gnashing of teeth. I've read the code, and I'm not confident it won't screw up. Equally, if it does screw up and you fix that, I'm not confident the code won't break somewhere else because of that. I speak from experience.
I'm not going to approve this for any further trials without seeing the following:
  1. a suite of tests exercising all the template parsing options found at User:MiszaBot/config, plus invalid values for all those options
  2. a suite of tests containing all kinds of crazy talk page layouts that a vandal could leave to intentionally screw up the bot - invalid month names, no level two headers, a header that reads "==the equal symbol is ===", time-dates separated by thousands of characters (11:37 ..... 9 November 2013 - and variants), year 9999, year 0000, no dates in a section, template in the wrong place, section bigger than the maximum archive size, empty section, every section old enough to archive, etc etc etc
  3. tests for where the:
    • talk page is protected
    • archive page is protected
    • talk page has something that will trip an edit filter/abuse filter (as an example of something that was written before they were implemented and is now being forbidden)
    • talk page has something that will trip the url blacklist (same reasoning as above)
I don't need these things because I'm a pedant. I need them because people will scream and yell and shout - at you. And then they'll start pointing fingers at the dope who approved the bot. Josh Parris 11:47, 9 November 2013 (UTC)[reply]
I "reviewed the code" in the sense that I looked over it, gave some comments on specific things, and ensured there were no glaring problems. I'm in agreement with Josh Parris here, in that I feel test cases are more reliable than someone just looking over code that we know already works for standard cases. — Earwig talk 21:22, 9 November 2013 (UTC)[reply]
I described my bot as literally MiszaBot with a different implementation. I called it a bot that does exactly what MiszaBot does, and it is a bot that does exactly what MiszaBot does. What you're asking me to do is upgrade MiszaBot, in a sense. Gracefully "unarchiving" threads if the bot cannot edit the page was not something Misza ever implemented, and seeing that nobody has complained about MiszaBot, I think it's safe to assume that nobody will complain about my bot either.
Notwithstanding, my bot now gracefully deals with your third point. Your second point has already been addressed at mwparserfromhell's test cases, the try/excepts in DiscussionPage.parse_stamps, and Arrow's strptime test cases. Your first concern has been met with the try/except in Archiver.generate_config and str2size.
So it was never really a question of test cases for the config template or timestamp format. Rather, one of how the bot would exit when it encountered an error. I hope this addresses the question. →Σσς. (Sigma) 09:48, 12 November 2013 (UTC)[reply]

I'll give you my opinion, for what it's worth. As an experienced developer myself, I might cynically interpret "literally MiszaBot with a different implementation" as "literally MiszaBot with a whole new set of bugs nobody's found yet". Please, please tell me you didn't rewrite the bot from scratch - that really is a disaster waiting to happen. If MiszaBot is down, and cannot stay up reliably, surely the best option is to find an alternative host that runs exactly the same implementation as before with no changes, then work on a replacement in parallel. Ritchie333 (talk) (cont) 11:38, 12 November 2013 (UTC)[reply]

Do not take this as a project hastily put together in the space of 3 weeks. I began development on a talk page archive bot as early as July 2012, and by the time MiszaBot went down, it was ready to deploy, although it was far less sophisticated than it is now. My code is no more of a mess of bugs nobody's found yet than ClueBot III or Werdnabot were in their time. →Σσς. (Sigma) 01:10, 13 November 2013 (UTC)[reply]
To clarify, I'm not saying that at all, I'm just saying that if the old code works, and consensus seems to be it's worked pretty well on large amounts of live data for some considerable time - far and beyond what any trial period would do, why not just build on that instead of trying to do a like for like replacement? Seems like a lot more work - as you can see, you'll have to get the bot through the entire approval procedure, whereas just redeploying the old MiszaBot code and then doing an incremental update some time later would get the urgent problem resolved and passing new functionality in a separate approval task would be simplier. As it is, Josh is suggesting the request be suspended if not flat out denied, while MiszaBot is still non-functional, which isn't really a satisfying result. Ritchie333 (talk) (cont) 10:03, 13 November 2013 (UTC)[reply]

My two cents: this particular bot is so desperately needed that even if it does happen to have some minor glitches at this time it should nevertheless be approved. Possible major glitches should be fully checked for prior to approval, of course. Also, I strongly disagree with the comment by User:Ritchie333 above as it seems to imply that bot code should not be rewritten from scratch no matter what, and part of it doesn't seem to make sense at all - how would the code be different if developed in parallel? Dogmaticeclectic (talk) 19:50, 12 November 2013 (UTC)[reply]

If MiszaBot was malfunctioning left right and centre, people were complaining about it on AN / ANI all the time, and multiple admins had blocked the bot, then that might be cause to rewrite - otherwise I agree with what Joel Spolsky said some 13 years ago (and, indeed, Microsoft said about 25). As for "how would the code be different if developed in parallel" - it will contain different bugs! Nobody, but nobody, ever writes 100% bug free code. That's not slamming developers, that's just being realistic and pragmatic. Ritchie333 (talk) (cont) 10:03, 13 November 2013 (UTC)[reply]
User:Ritchie333, could you explain what you mean by developing code in parallel? I don't think I understood what you meant correctly. Dogmaticeclectic (talk) 18:09, 13 November 2013 (UTC)[reply]
"Parallel code development" is when the same program is written by two different, independent parties. It's a technique used in high-reliability systems, typically with three systems developed in parallel, and their output feed into a fourth system which has the job on deciding what the majority opinion is and actually doing that. It's intended as a solution to various design and implementation errors.
However, "work on a replacement in parallel" meant in this case, "while the old program is executing on a new computer system, write a replacement program that is more reliable before then replacing the old program". Josh Parris 04:40, 16 November 2013 (UTC)[reply]

Could another run of the MiszaBot functionality be authorized? There is significant constipation throughout WP (I just edited a 500K talk page article). (I had tried ClueBot, but I somehow misconfigured it to archive to the wrong place, and I'd rather not spend more time on it.) vzaak (talk) 18:02, 13 November 2013 (UTC)[reply]

Certainly. I'm awaiting a through test harness from the operator, then we'll be up for a trial, probably followed by approval. Alternatively, another BAG member might approve for trial as-is, but I'd be surprised - presumably they have the same concerns I do. Josh Parris 06:21, 15 November 2013 (UTC)[reply]
I think we can all agree that this bot is desperately needed. I think we can also agree that if it messes up, it could do a lot of harm and make a lot of people very angry. So we should proceed as quickly and carefully as we can. I think it's prudent to insist on the sort of "test harness" Josh Parris recommends, while giving the bot operator as much support as we can. I don't think it's useful to introduce other hoops for him to jump through; what matters is that the bot should work. – Quadell (talk) 15:39, 15 November 2013 (UTC)[reply]
See Test harness. Josh Parris 04:40, 16 November 2013 (UTC)[reply]

Editing outside of trial[edit]

I have noticed that the bot is editing outside of it's approved parameters. Are you familiar with WP:BOTPOL? Josh Parris 09:53, 13 November 2013 (UTC)[reply]

It is not a mistake that I meant for my bot to skip non-user-talk namespaces.[1] These people have given me permission to test my bot on their talk pages. For this, I treat these pages as an extension to my usual sandboxes. →Σσς. (Sigma) 10:03, 13 November 2013 (UTC)[reply]
(edit conflict)... except these requests, for te most part, were made off-wiki. I see how this could be confusing. Perhaps, we could arrange for them to confirm this? →Σσς. (Sigma) 10:10, 13 November 2013 (UTC)[reply]
(re the ec) No, I believe I've seen their names on-wiki putting up their hands, or at least expressing a desire for the bot to operate. I believe the edits are being made in good faith, and with the target's approval. I'm just making my Marge Simpson growl, and ensuring you've got plenty of WP:ROPE. Josh Parris 12:19, 13 November 2013 (UTC)[reply]
I understand that. Do you understand WP:BOTPOL? Josh Parris 10:08, 13 November 2013 (UTC)[reply]
Yes. →Σσς. (Sigma) 10:10, 13 November 2013 (UTC)[reply]
It's possible to make such a request!? Where can I sign up? Dogmaticeclectic (talk) 10:32, 13 November 2013 (UTC)[reply]
You tell me that you want it, and then I add you to my list. →Σσς. (Sigma) 06:04, 15 November 2013 (UTC)[reply]
@User:Σ: Yes, please! Dogmaticeclectic (talk) 14:06, 15 November 2013 (UTC)[reply]
@Σ: can I have it too please? --Mdann52talk to me! 08:40, 22 November 2013 (UTC)[reply]
@Mdann52 and Dogmaticeclectic: Place {{Experimental archiving}} on your talk pages. →Σσς. (Sigma) 08:54, 22 November 2013 (UTC)[reply]

Trial idea[edit]

Σ, I have an idea. If you can confirm that the bot works on relatively simple cases (with incremental archives, and with typical parameters), then the bot could be tested and approved for just those pages that use User:MiszaBot/config in the most common ways. For each level of complexity (having a minthreadstoarchive greater than 1, or using a date-based archive, or with a "secret key", or with a malformed template, etc.), these could be tested separately and approved when ready. This could allow the bot to run quickly, at least on some pages, while minimizing the chance of errors due to unexpected configurations. Does this sound like a good way to proceed to you? – Quadell (talk) 15:39, 15 November 2013 (UTC)[reply]

That's an interesting idea. I'll think about it and get back to you about this tomorrow. →Σσς. (Sigma) 04:18, 16 November 2013 (UTC)[reply]

Approved for extended trial. Please provide a link to the relevant contributions and/or diffs when the trial is complete. The bot may only operate on pages it would otherwise operate on, which also have {{Experimental archiving}} - no other editing is allowed. The BAG reserves the right to pull this trial at any time for any reason. The bot bit will not be given for this trial. This trial is intended to operate during the development of a rigorous test harness. Regardless of the duration of the trial, no more than 50 pages per day are to be operated on. The operator is expected, as in all trials, to inspect every edit the bot makes and report misbehaviour here. Josh Parris 04:29, 16 November 2013 (UTC)[reply]

Regarding the trial, let me clarify: archive any page with (what you have now renamed to) {{Experimental archiving}}; on- or off-wiki communications aren't necessary - or permissible - to opt-in. Only the owners of those talk pages ought to be adding the necessary template(s). Josh Parris 10:32, 20 November 2013 (UTC)[reply]
Tests have been written. See the source. →Σσς. (Sigma) 04:35, 18 November 2013 (UTC)[reply]
As best I can tell, you've built test cases for 42 lines of your program (functions str2time and str2size). Excluding test cases, your program extends to 539 lines. The things you tested were the things I had greatest confidence in, as they were small, simple functions that one could eyeball quickly and form a strong opinion as to their correctness. As you've discovered, it's easy to exhaustively test a simple function to discover flaws and build your confidence in the code.
Please test the gnarly parts of the program with the same diligence. Josh Parris 06:30, 20 November 2013 (UTC)[reply]
Greatest confidence?
  • a suite of tests exercising all the template parsing options found at User:MiszaBot/config, plus invalid values for all those options
The only inputs that actually required parsing (beyond converting to an int) are passed into str2time and str2size.
I left comments to denote where something that could have gone wrong was being handled. And in addition, there is a history of tests located at User talk:Σ. The description of the test can be found in the lead section of each revision that wasn't a rollback or a bot edit. →Σσς. (Sigma) 07:32, 21 November 2013 (UTC)[reply]
By "greatest confidence" I meant those functions will parse fine; it's what the bot does with those values after it parses them that becomes more opaque. Combinations, permutations.
I'm detecting that you're finding this insistence on testing exasperating. I understand. I've explained earlier my rationale.
I'm very pleased to see the testing you've done against testwiki. I see you've been using real-world data, testing various combinations of archive size and archiving conditions, and you've tested various page-protection errors the bot could come across. I believe you have demonstrated that the bot's ability to deal with write failures is tested.
The sanity-checking code in the bot is a comfort.
I see a lot of Common coupling between your class methods. This may be why you're finding it burdensome to unit test. If you don't want to decouple those class methods, you could exercise the classes by passing them chunks of wikitext.
The things I'm most uncomfortable about at the moment are if a header that reads "==the equal symbol is ===" will ever get archived, and knowing that the bot will operate correctly for the various combinations of configuration template parameters. Josh Parris 10:18, 21 November 2013 (UTC)[reply]
That has been covered here, and as I am unable to come up with any odd archive configurations, feel free to suggest a few to me. →Σσς. (Sigma) 05:55, 22 November 2013 (UTC)[reply]
Give https://test.wikipedia.org/wiki/User_talk:Josh_Parris a go. That's got the weird = in it. Josh Parris 10:39, 22 November 2013 (UTC)[reply]
The bot won't archive threads if the archives are not subpages of the talk page. In order to demonstrate the = heading, I have fixed that for you. See this for details. →Σσς. (Sigma) 01:29, 23 November 2013 (UTC)[reply]
Appreciated, thanks. Josh Parris 09:46, 23 November 2013 (UTC)[reply]
In fact, give every talk page created by https://test.wikipedia.org/wiki/Special:Contributions/110.174.86.241 a go. Josh Parris 11:51, 22 November 2013 (UTC)[reply]
Done.

Promising results.

  • Do you have any interest in immediate archiving per ArchiveNow?
  • User talk:Obiwankenobi - the edit summary has the target pages listed in a surprising order.
  • have another go at testwiki:User talk:Ultraexactzz - there are less z than you gave, plus I've jiggled it a little.

Josh Parris 09:46, 23 November 2013 (UTC)[reply]

Ah, nice catch. I've archived that now. Anyway, I don't foresee myself having plans to implement ArchiveNow, but I'm not completely opposed to it. Also, frozensets in Python are not ordered. The edit summary is hardly a pressing issue. →Σσς. (Sigma) 03:31, 24 November 2013 (UTC)[reply]
sorted(self.page.title) would solve the ordering issue. Josh Parris 08:15, 24 November 2013 (UTC)[reply]

Miszabot misbehaviour[edit]

This is.... interesting, to say the least. →Σσς. (Sigma) 05:55, 22 November 2013 (UTC)[reply]

Interesting, huh? I've been reassured that while... quirky... Miszabot wasn't bug-ridden. Yet that was wrong. This observation has certainly pushed matters in your favour. Josh Parris 10:39, 22 November 2013 (UTC)[reply]

Thank-you for your persistence. Feel free to continue to improve the bot. Keep a close eye on its early runs.  Approved. Josh Parris 08:15, 24 November 2013 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:Bots/Requests_for_approval/Lowercase_sigmabot_III_2&oldid=1192505275"