User talk:Rsjaffe/Archives/2023/February

Guild of Copy Editors 2022 Annual Report

Guild of Copy Editors 2022 Annual Report Our 2022 Annual Report is now ready for review. Highlights: Overview of Backlog-reduction progress Summary of Drives, Blitzes, and the Requests page Membership news and results of elections Closing words – Your Guild coordinators: Baffle gab1978, Dhtwiki, Miniapolis and Zippybonzo To discontinue receiving GOCE newsletters, please remove your name from our mailing list.

Sent by Baffle gab1978 using MediaWiki message delivery (talk) 00:30, 6 February 2023 (UTC)

AI generated articles

Hello, Rory,

I've come across a couple of articles you tagged as having been created with AI and I just wondered how you coud tell this was true. They weren't sourced and were pretty vague in their content but I was wondering if there was a script or tool you used or whether it was a judgment on your part. I work a lot with draft articles these days and it would be helpful to be able to tell whether or not the content was original. Thank you! Liz ^{Read! Talk!} 23:05, 3 February 2023 (UTC)

There are several tools I use. The first is made by the company that produces ChatGPT. It is based on an earlier version but seems to work well. I don't count it as AI-created unless the certainty is above 99%. It usually is above 99.9%. There isn't a bell-curve distribution of readings. Most are either close to 0% fake (that is, human-produced) or 99.98% fake (LLM model, e.g., ChatGPT produced). I've never had anyone argue that the tag was incorrect, and I've probably tagged about 40 articles so far.

The tool is at https://openai-openai-detector.hf.space/. I paste in the text, omitting headings and inline ref tags. A second detector, if you're interested, is https://detector.dng.ai/. I almost always just use the first. Sometimes the AI text is embedded within other text (e.g., a custom written opening paragraph), in which case I omit the custom-written text when testing.

I've been looking for "tells" before testing. The text is more "lifeless" than human text typically is, tends to have uniform length sentences, and may have a paragraph at the end that is a summary paragraph. The text is also unlikely to have in-line references. This doesn't catch all of them but tends to be a high-likelihood way of finding them. Some of these AI-generated articles also have AI-generated references. The references are almost always fake: the AI confabulates reasonable-sounding references! — rsjaffe 🗣️ 00:49, 4 February 2023 (UTC)

I'm just stumbling across this after a heads-up in the newsroom for The Signpost. We included a teaser about AI generated articles in a recent issue. Can you tell me how to find the articles you tagged? ☆ Bri (talk) 16:20, 13 February 2023 (UTC)

@Bri Search for hastemplate:AI-generated in article and draftspace. I think several of them did make it to article space but were then draftified. I'd have to go through the edit history to figure out which ones those were. — rsjaffe 🗣️ 16:39, 13 February 2023 (UTC)

Also, a couple have had the template removed after the author re-edited the page to remove the AI-generated text. — rsjaffe 🗣️ 16:40, 13 February 2023 (UTC)

Here we go: Draft:Nordic states game industry was in article space. It was subsequently moved to Draft. — rsjaffe 🗣️ 16:41, 13 February 2023 (UTC)

Your take on AI content quality

Thanks for the pointer to the draft. Do you have a personal reaction to the quality of the generated articles? Briefly, are you for or against this method of creation? ☆ Bri (talk) 16:46, 13 February 2023 (UTC)

Strongly against. These language generators are good at stringing together reasonable-sounding text, but:

1. Don't rely solely on what Wikipedia recognizes as reliable sources.

2. Don't have a sense of what is real and what is fake.

3. Will fabricate information to fill in gaps.

4. Cannot identify where any specific bit of "information" came from.

So the text can fail WP:RS and WP:V, yet may look very convincing.

However, they're good at helping to break "writer's block", by giving an example of how to write something. So I see having some text generated by ChatGPT would be helpful as a writing prompt, but without using any of the generated text directly—just looking at it and then writing a well-researched article similar to the prompt. — rsjaffe 🗣️ 17:06, 13 February 2023 (UTC)

Cool, let's follow up more at The Newsroom, I see one of the other editors contacted you by email and I'm not sure who was first (doesn't really matter now). ☆ Bri (talk) 21:11, 13 February 2023 (UTC)

Before I do anything else, could you affirm that it's OK if I use the conversation above in The Signpost as a mini interview? Also, do you prefer we refer to you by the name on your userpage, your on-Wiki username, or both? ☆ Bri (talk) 21:44, 13 February 2023 (UTC)

OK, to use conversation. Refer to me with both username and my real name (Rory Jaffe). — rsjaffe 🗣️ 22:37, 13 February 2023 (UTC)

Feedback request: Maths, science, and technology request for comment

Your feedback is requested at Wikipedia:Reliable sources/Noticeboard on a "Maths, science, and technology" request for comment. Thank you for helping out!
You were randomly selected to receive this invitation from the list of Feedback Request Service subscribers. If you'd like not to receive these messages any more, you can opt out at any time by removing your name.

Message delivered to you with love by Yapperbot :) | Is this wrong? Contact my bot operator. | Sent at 21:30, 16 February 2023 (UTC)

Jon Nordmark page

Did you message me about this page? You were wondering if I have written wikipages before? I assume you see my history, yes? Davejenk1ns (talk) 18:29, 24 February 2023 (UTC)

It was in error. I had put a speedy delete tag on the page but then withdrew it. — rsjaffe 🗣️ 19:41, 24 February 2023 (UTC)

Okay thanks Davejenk1ns (talk) 03:47, 25 February 2023 (UTC)