User talk:BilledMammal/ARBPIA discussion statistics

Additional criteria?

@BilledMammal: you may want to consider a definition of "bludgeoning" for these statistics purposes that goes beyond just "X number of comments" and adds some additional criteria to better represent problematic discussions. The two obvious suggestions that come to mind would be a minimum number of participants in the conversation, and a minimum percentage of replies in the conversation by the user. For instance, "Editors are assumed to have bludgeoned a discussion if they made more than twenty comments in it" might be better off as "Editors are assumed to have bludgeoned a discussion if they made more than twenty comments in it, amounting to at least 25% of the total number of comments, and the discussion contains at least 5 participants". This would have the effect of weeding out scenarios where there's a very active back-and-forth between two editors and nobody else is interested; as well as scenarios where there's a very long, sprawling or multifaceted discussion in which 20 comments is simply not going to move the needle. ⇒SWATJester ^{Shoot Blues, Tell VileRat!} 23:03, 19 August 2024 (UTC)

Just noting, since I have replied to the below, that I will reply to this - however, it takes more thought and effort than the other reply took and so will take longer. BilledMammal (talk) 00:17, 20 August 2024 (UTC)

@Swatjester: I didn't end up implementing either of those criteria. For the first, I believe it is possible to be disruptive in a discussion while making less than 25% of the comments; this is particularly true when multiple editors are bludgeoning.

For the second, while it's less likely to be bludgeoning, I think it can still be disruptive and discourage the participation of uninvolved editors. For example, this discussion, which only had five participants, but two of whom had such an extensive discussion that they discouraged uninvolved participation and made the discussion very difficult to follow without either contributing much of worth towards the end.

However, I have provided additional information, about the number of participants and the number of comments in the discussion. I hope this will make it easier to review and addresses some of your concerns? BilledMammal (talk) 20:05, 25 August 2024 (UTC)

They're less concerns than they are suggestions which you can take or leave as you like -- I'm not going to be the end user of this data, after all. To clarify one thing: the suggestions of specific numbers of (25%) and (5 participants) are arbitrary and you could substitute whatever you'd like: I don't disagree that it's possible to disrupt a conversation with only a smaller share of comments but a) it's going to be less frequent than someone using more comments to make that happen; and b) I'd personally choose to err on the side of being more stringent and weeding out false positives than being looser in order to weed out more false negatives, given the subject matter. Similarly with the number of participants in a conversation -- while it's true that bludgeoning can occur in a conversation of just two people, the small size of the conversation is also a strong indicator that any bludgeoning in that conversation was not particularly impactful (as it implies that neither "side" brought in additional members, which goes to the heart of the issue of tag-teaming). ⇒SWATJester ^{Shoot Blues, Tell VileRat!} 20:54, 25 August 2024 (UTC)

That's a good point; I've limited the statistics to discussions with more than five participants.

However, I still haven't applied a percentage limit, because it results in us missing the most disrupted discussions. For example:

In this discussion there were at least four very disruptive editors, and several more who were less disruptive. However, because of the number of editors bludgeoning the discussion none of them made more than 9% of the comments. BilledMammal (talk) 22:10, 25 August 2024 (UTC)

That's a fair point as well. The software PM part of my brain looks at that and says there's ways to dynamically scale the share-of-comments filter by the size of the discussion, but I suspect that's way more complexity than this is worth.⇒SWATJester ^{Shoot Blues, Tell VileRat!} 22:27, 25 August 2024 (UTC)

Value of BilledMammal's stats

A user-designed statistical tool that counts only edits (presumably whether they be to remove a linebreak, agree with someone, or make a substantive comment?) is unlikely to be of much relevance.

When someone brought this page to ANI calling it an attack page, I provided the evidence of the WMF-designed statistical tool, which shows that 18% of BM's contributions to mainspace have been reverted. These are his stats, mentioned in the title of this section. Suddenly, he wanted to look at the content of those reverts and explain why that 17.7% was not representative of his general reliability, rather than focus on the global numbers. Here on this page, the global numbers may include fixing spelling mistakes and typographical concerns (there is no explanation of the methodology, but if it was automated we can assume it does count such fixes)...

In conclusion, I would suggest that if BilledMammal wishes to use his hand-made statistics at ArbCom he should also include the WMF-made statistics showing that nearly 1 of every 5 of his edits (947 of 5327 to be exact) to mainspace have had to be reverted. -- SashiRolls ^{🌿 · 🍥} 23:57, 19 August 2024 (UTC)

You misunderstand how the tool works. It counts comments, not edits.

As for the reverted edits, those were part of WP:BRD, and are outside the scope of ARBPIA. If you want to discuss them further, please come to my talk page. BilledMammal (talk) 00:16, 20 August 2024 (UTC)

Feel free to explain the methodology for counting to 20 comments on the page then, as is normal in any paper on statistics. You say that you did not simply count edits to a specific section. Did you manually count the comments?

As for your 18% reversion rate, I agree that this is an overall statistic which suggests there may be some more general issues at play that ArbCom should consider, potentially involving the WP:BOLD & the WP:RECKLESS -- SashiRolls ^{🌿 · 🍥} 00:29, 20 August 2024 (UTC)

I used regex to identify signatures within each section, and then counted the number of signatures for each user. This does have an issue with renamed users (eg. Kowal2701) but they are rare and the method is otherwise very accurate.

If you want to discuss my species edits further you are welcome to come to my talk page. Please don’t continue to discuss it here as it distracts from the purpose of this page, which is to assess the level of bludgeoning in the topic area. BilledMammal (talk) 01:32, 20 August 2024 (UTC)

Some stats can be explained without any policies or guidelines necessarily being violated; others not so much. How can one possibly explain 50+ comments in one discussion without it being bludgeoning? — xDanielx ^T/_C\^R 00:32, 20 August 2024 (UTC)

Indeed, the number of edits by Kentucky Rain24 on Talk:Zionism#Colonial project? is astronomical. Yet, they do not figure in the table. Were blocked sockpuppets automatically excluded from your statistics? Did you also remove replies to bludgeoning sockpuppets from your bludgeoning stats? (It doesn't look like you did, but I'll let you reply...) -- SashiRolls ^{🌿 · 🍥} 00:48, 20 August 2024 (UTC)

Editors were only included if they made 20+ comments in 4+ discussions. While Kentucky Rain24 did bludgeon some discussions, they didn’t bludgeon enough for inclusion.

Replies to bludgeoning sockpuppets were not excluded, as while bludgeoning under those circumstances is more understandable, it is still disruptive. Perhaps it will help to clarify the purpose of this analysis; it isn’t to get specific editors in trouble, but to demonstrate that this is a common issue in the topic area that needs a broad remedy similar to what ScottishFinnishRadish and Red-tailed hawk are discussing.

Let me know if you want to expand the inclusion criteria; as I said at the bottom of the page, I’m happy to rerun the analysis at any editors request. BilledMammal (talk) 01:23, 20 August 2024 (UTC)

Yes, please rerun your analysis excluding any response to sockpuppets like Kentucky Rain24 and Tombah. This will need to be done manually of course as you will need to determine whether the responses are indeed responding to the sockpuppet or just follow the sockpuppet's comment but are ignoring it. As it is, your data for Talk:Zionism#Colonial project? and Talk:Israel/Archive_93#2nd lead paragraph are meaningless. (This is going to be a very time-consuming procedure. Good luck! Just cleaning up the strike tags was a major hassle!) Alternately, it would be honest to acknowledge in your methodology section that while there were multiple pages being bludgeoned by sockpuppets, you didn't control for that. -- SashiRolls ^{🌿 · 🍥} 02:18, 20 August 2024 (UTC)

Sorry, I meant if you wanted me to include all editors that had bludgeoned a discussion, rather than just those that has bludgeoned at least four. I don’t believe that it is useful to remove responses to sockpuppets, as bludgeoning in partnership with a sockpuppet is still disruptive and evidence of an issue in the topic area that could be addressed by broad remedies. BilledMammal (talk) 02:28, 20 August 2024 (UTC)

As you are no doubt aware, the problem is not bludgeoning in partnership with a sockpuppet it is being dinged as a bludgeoner because you responded to aggressive sockpuppets.-- SashiRolls ^{🌿 · 🍥}

FWIW: after further research I have been able to determine that it was the decision to set the cutoff at 20 comments rather than at 18 which kept Kentucky Rain24 (NoCal100) from appearing on the list. That said, and as others have already said, showing that people engaged in discussion, provided RS, debunked silly arguments, responded to sockpuppet provocation, etc. does not show that people "bludgeoned" anything. As the explanatory essay says: Participating fully isn't a bad thing. -- SashiRolls ^{🌿 · 🍥} 01:34, 26 August 2024 (UTC)

Weaknesses and alternatives

I saw the ANI discussion. I don't have any involvement in the Arbcom(?) issue, and don't want to even look at it long enough to form a view on it. I have glanced at some of the discussions you've identified, but only to consider stats. I am worried that if you present the stats as they stand at an evidence stage, then shooting them down will be a noisy distraction - and successful.

That "assumed to have bludgeoned a discussion if they made more than twenty comments" is vulnerable first because it seems to be your invention; I don't see it at WP:BLUDGEON. One might proceed by first selecting for further analysis all discussions in which editors A, B or C made more than 20 comments, but whether that turned out to be bludgeoning would depend on the further analysis. The first one on your list, Talk:Gaza genocide/Archive 2#Requested move 3 May 2024, reports 557 comments; 20 would be a drop in that ocean and 83 is only 15% - not clear domination and unlikely to be replying to every comment (to use a couple of WP:BLUDGEON's terms) .

So what weight should you give to a large number of comments? Some editors are in the habit of making little typo/phrasing tweaks to their posts, 5 edits to make one comment, 20 to make 4. It's a nuisance if they're doing it while you're responding, but not bludgeoning. So, size matters. But the editor who makes only two monstrously large posts, constituting half the volume of the discussion but only at the start and then walking away, isn't bludgeoning either. There's no simple formula for the combination of size and number, but merely describing them together might be helpful. Here's part of what I found myself writing three years ago, replying to one editor and pinging the other: "In the last few days, more than half the edits to that talk page, by count or volume, have been from you or P. To be exact, of 95 contributions in 37,351 bytes on 17 August and since (up to this edit and excluding bot edits), 45 were yours (6,450 bytes) and 16 P's (13,038 bytes).) One other editor has made 8 (4592 bytes), the rest up to 3. You're both in WP:BLUDGEON territory and I'm surprised neither of you has noticed and backed off without needing stats." One of them found that a little persuasive.

Your stats as they stand would be vulnerable in other ways too. Is it bludgeoning or constructive to post WP:NOTFORUM <sig> repeatedly? It might be bludgeoning if it was unjustified and always done to "opponents", but if justified and even-handed it's constructive - even if it's opponents who have the greater tendency to veer into FORUM and need steering back, so that the figures look unbalanced. How about reverting posts by editors that aren't extended-confirmed? It messes up the stats, for starters; all those negative sizes, and are the reverted comments included in the 557-comments figure? If all non-EC posts get the same treatment, it's surely not bludgeoning.

It's not impossible to make a statistical case of bludgeoning; that comparison I offered of two editors with the rest should have been persuasive - IMHO. But to evaluate against the descriptions at Wikipedia:Don't bludgeon the process#Bludgeon (replies to many, goal of getting each to change their !vote, having the last word) as well as a set of creditable acts (respectful discussions that kept moving forward and exploring the issues, even-handed housekeeping), you'd need to assess interactions. It'd be a lot of work; you'd have to be sure it was worthwhile. But presenting statistics as simple as the ones I've seen won't be convincing and may itself look like hitting someone with a blunt instrument. NebY (talk) 01:26, 20 August 2024 (UTC)

To clarify, my analysis counts comments, not edits.

Regarding the rest, I’ll consider it along with Swatjesters comments and reply in more depth later; you both make some good points. BilledMammal (talk) 01:44, 20 August 2024 (UTC)

Thanks. Do you have a control group? I read your This is a common issue in the topic area, and it can’t be resolved with targeted sanctions - instead, broad remedies are required and wonder if your method would also indicate editors were "bludgeoning" in protracted discussions on other subjects, in other topic areas, that were polite or even amicable, hard-working and constructive, and led to positive outcomes. NebY (talk) 02:04, 20 August 2024 (UTC)

No, I’ll get one - do you have a suggestion for a topic area? BilledMammal (talk) 02:28, 20 August 2024 (UTC)

@NebY: I've added some additional contextual information that I believe addresses many of your concerns, including a list of which editors were replied to. Do you want to take a look?

I haven't produced the control group yet. BilledMammal (talk) 20:07, 25 August 2024 (UTC)

This feels wrong.

I don't think it goes against WP:NPA since there are multiple links proving bludgeoning, but it still feels wrong. Maybe it's just me, I dunno. LilianaUwU ^{(talk / contributions)} 07:12, 20 August 2024 (UTC)

I’m speaking only for myself, but I think the list is fine. It’s just important to acknowledge what it is: it’s not a justification for sanctions, as no objective measurement can reasonably be used to verify a violation of WP:Bludgeon, and certainly not a numerical one. However, it is pretty good at showing that there is pattern caused by a significant number of the „regulars“ (counting myself here too), that, while IMO not inherently a violation of the current policies (though it can be an indication), could nevertheless be remedied to improve the topic area. While I personally think that the list could be improved by providing a relative percentage (for example, 20 comments in a 40 comment discussion with 10 participants are worse than 20/40 in a discussion with 3 people, and 20/400 is almost certainly not a problem per se) and that civility (probably including the identifiable off-Wiki variety, but I’m guessing that this one will go down like a lead balloon) is easier and more productive when it comes to cracking down on the mess that the area is, we aren’t restricted in the number of available remedies, and a comment number restriction of some kind is potentially useful enough to warrant discussion. FortunateSons (talk) 11:10, 20 August 2024 (UTC)

Size of text, not number of responses

I think BilledMammal is mistaken by considering the number of responses, not whether the response is a excessively long, as a metric for blugeoning. Back and forth responses are a necessary part of any discussion and they should be encouraged. The problem becomes when some people write an essay in response to every comment. I'd be more interested in those stats.VR _{(Please ping on reply)} 11:15, 20 August 2024 (UTC)

I think this requires an amendment. For example, the template originally created by me and later significantly improved by @Cdjp1 was quite long (and pretty useful IMO), and so is any other source analysis. Perhaps excluding any linked words and anything in a table would remedy that issue? So just counting by some metric won’t do IMO. FortunateSons (talk) 11:26, 20 August 2024 (UTC)

Oh yeah, Template:Expert opinions in the Gaza genocide debate was incredibly useful. But such lists should be hatted in any discussion, and thus should not contribute to bludgeoning. I also think the length should not be measured by bytes added (which becomes bloated by lengthy URLs, which is a sign a user is citing their sources) but by amount of visible prose. VR _{(Please ping on reply)} 11:31, 20 August 2024 (UTC)

Well, I feel left out not making the list, but I do actively try to avoid Talkpage discussions, as I would inevitably end up being an asshole.

But besides that, per WP:Bludgeoning, it would seem that the number of comments takes priority, with length also (secondarily) being considered. Though of course that analysis of stats can only highlight a potential case of bludgeoning, as number of comments and length can also be indicators of being part of an active constructive discussion, following best practices of citing sources and logical reasoning. -- Cdjp1 (talk) 11:41, 20 August 2024 (UTC)

You’re right, but that’s probably the weakness of any hard metric, it often ends up as a judgement call based on specific conduct.

I‘m sure we can find a few discussions for you to bludgeon, don’t worry ;) FortunateSons (talk) 11:44, 20 August 2024 (UTC)

Hmmm, that makes sense. If we can only include prose, and exclude all links and tables, it could definitely be a useful metric. FortunateSons (talk) 11:41, 20 August 2024 (UTC)

I've just implemented a check for the median number of words in each comment. However, it currently doesn't exclude non-visible prose, as that can be difficult to identify.

I think that issue is partly addressed by looking at the median comment length, rather than the total or the average, but we could also exclude all text within templates? I could use the same process I used for User:BilledMammal/Average articles. BilledMammal (talk) 11:49, 20 August 2024 (UTC)

@Vice regent: I've added the median number of words by that editor in the relevant discussion to the table. I'm not sure how useful it is, but perhaps you'll find some use for it. BilledMammal (talk) 20:09, 25 August 2024 (UTC)

Refusal to include acknowledgment of LTA activity

Twice now you have reinstated leading language when neutral language has been substituted, and have removed any acknowledgment that LTA sockpuppets are a problem that you have not controlled for. Why this lack of transparency? Note that this has already been mentioned at ARCA and will again be mentioned if the case reaches an evidence phase. Granted, it's a page in your user space, so rigor is not required. -- SashiRolls ^{🌿 · 🍥} 20:35, 25 August 2024 (UTC)

I disagree with your belief that your version is neutral; it puts excessive focus on a non-issue, suggesting it is more significant than it is.

If you look at the "replies" column, you’ll see that very few replies were to LTA's - and as a general note, I think the justification "I only bludgeoned because an LTA was also bludgeoning" is a very poor one. BilledMammal (talk) 20:44, 25 August 2024 (UTC)

Bludgeon is not a neutral word. "made 20 or more comments" is neutral. I think that "I replied assuming good faith to a long-term abuser who made 48 comments in that discussion" is a good justification for having made more than 20 comments. YMMV.

Would you be willing to use your algorithm to provide here on the talk page the stats for NoCal100 qua Kentucky Rain24 on the pages in question as well as for any other blocked sockpuppets, like Tombah for example? Thank you. -- SashiRolls ^{🌿 · 🍥} 20:48, 25 August 2024 (UTC)

The only editor whose "number of discussions" would be affected by excluding sock puppets would be Iskandar323 - it would decrease their number from 16 to 15. No editor would have the "number of discussions" bludgeoned affected if sockpuppets were excluded. The impact is trivial, and not worth considering.Updated 22:23, 25 August 2024 (UTC) - Tombah isn't a sockpuppet or LTA

Tombah was a sockpuppeteer, rather. I saw that they were blocked for abusing multiple accounts and did not investigate whether they were the master or the sock, so your point is taken :) -- SashiRolls ^{🌿 · 🍥} 22:46, 25 August 2024 (UTC)

If you can provide me a list of sock puppets you are interested in, yes. BilledMammal (talk) 20:52, 25 August 2024 (UTC)

Interesting. How do you know that? Are you saying the only person who ever responded to Kentucky Rain24 was Iskandar323? I don't think that is accurate. I have already provided two pseudos to get the ball rolling. Others more familiar with the problem may provide you with others. I look forward to seeing those results. -- SashiRolls ^{🌿 · 🍥} 20:58, 25 August 2024 (UTC)

Look at the "replies" column. The only editor with a discussion that would drop below 20 comments if replies to socks were excluded is them. BilledMammal (talk) 21:01, 25 August 2024 (UTC)

Yes, and would the artificially inflated number of comments in a discussion drop for other editors? Say from 60 to 30? Let's see the data... -- SashiRolls ^{🌿 · 🍥} 21:04, 25 August 2024 (UTC)

That data is already available in the table? BilledMammal (talk) 21:05, 25 August 2024 (UTC)

Where do you show the number of comments responding to provocation by sockpuppets? I admit that I missed that if it exists... Also, as you admit above your analysis does not count every comment made, but only counts comments made by a select group of users actively engaging in discussion, sometimes with bad faith actors. -- SashiRolls ^{🌿 · 🍥} 21:14, 25 August 2024 (UTC)

As I said above, the "replies" column - it’s in the discussions column.

Also, as you admit above your analysis does not count every comment made, but only counts comments made by a select group of users actively engaging in discussion, sometimes with bad faith actors.

I have no idea what you are trying to say here. BilledMammal (talk) 21:18, 25 August 2024 (UTC)

OK I see that you've added a new feature since last I looked at the page, which partially takes into account my criticism concerning LTA problems. Let's look at how you've done that. By a quick count of the struck comments, Tombah made 17 of them on the page Talk:Israel/Archive_93#2nd_lead_paragraph. Because of your system (which is not explained), this fact does not appear... instead you list Tombah as having made 1 comment (in Levivich's data) and 7 comments (in Nableezy's data). 1+7 does not equal 17. -- SashiRolls ^{🌿 · 🍥} 21:36, 25 August 2024 (UTC)

You've edited the page twice since I added it?

One of Levivich's comments was in reply to Tombah, and seven of Nableezy's were. FYI, Tombah is a sock-puppeteer, not a sock puppet or LTA. BilledMammal (talk) 21:45, 25 August 2024 (UTC)

Another example of this same problem: Talk:2024 Nuseirat rescue operation#Requested_move_9_June_2024 where Kentucky Rain24 made 25 comments. You mention that Selfstudier replied to 12 of them, but also note that Selfstudier replied to themself 5 times... in fact these 5 replies were all discrete comments replying immediately after the LTA and providing RS. Why didn't you count them as such? As it turns out that suggests Selfstudier only made 17 comments in that discussion which were not immediately responding to NoCal100. (And some of those remaining 17 could certainly be counted as responding more distantly to him...) You've made some progress, but there are still flaws in your design. I look forward to you providing the complete data on Kentucky Rain24.-- SashiRolls ^{🌿 · 🍥} 22:30, 25 August 2024 (UTC)

Look at the indentation level of those comments.

Regardless, this is turning into an irrelevant distraction.

There are just three editor-discussion pairs where a non-trivial number of replies were to a sockpuppet or LTA:

1. Levivich, 26 replies out of 59 at Talk:Zionism#Colonial project?
2. Selfstudier, 15 replies out of 45 at Talk:Zionism#Colonial project?
3. Selfstudier, 12 replies out of 34 at Talk:2024 Nuseirat rescue operation#Requested move 9 June 2024

Even if you exclude all of these discussions, it barely changes the overall number of discussions bludgeoned, and doesn’t change the rankings. As such, I don’t see any benefit of fixating on it. BilledMammal (talk) 23:03, 25 August 2024 (UTC)

I also looked through your data on yourself and noticed you did not respond to Kentucky Rain24 even once. -- SashiRolls ^{🌿 · 🍥} 22:53, 25 August 2024 (UTC)

Can you clarify what you mean by this, because by my reading it is coming very close to an aspersion, particularly when read in the context of comments like this, and I’m hoping it is not. BilledMammal (talk) 23:03, 25 August 2024 (UTC)

I just noticed that you never -- to your credit -- made any replies to NoCal100 in any discussion you listed yourself as bludgeoning. Why would that be an aspersion? It's a fact that emerges from the data you provided. In looking at the page, I also noticed that Kentucky Rain24 just missed your cutoff with 18 comments at Talk:2024 Nuseirat rescue operation#Proposed merge of Nuseirat refugee camp massacre into 2024 Nuseirat rescue operation. You do agree that they made 47 at Talk:Zionism#Colonial project? and 25 at Talk:2024 Nuseirat rescue operation#Requested move 9 June 2024, right? Are there any others you know of? -- SashiRolls ^{🌿 · 🍥} 23:13, 25 August 2024 (UTC)

It looks like KR24 made 26 comments on Talk:Nuseirat refugee camp massacre where you had 10 comments, but again, to your credit, 0 of them were responses to KR24. Also 26 KF24 comments on Talk:Israeli war crimes. -- SashiRolls ^{🌿 · 🍥} 23:49, 25 August 2024 (UTC)