Topic: AI-generated post discussion thread: how to identify & report - page 8. (Read 2717 times)

I do like the idea, but think that in most cases it will not good enough for using beyond doubt.
Of course, there are exceptions:
* sudden change in the quality (and fluence) of the user's posts
* sudden change in how knowledgeable somebody is in various fields

And here there can be a doubt too between use of AI or account changing hands, for example, however both should come at least with red tag.


... Maybe this new development would also trigger equivalating the gravity of plagiarism, use of AI and buying accounts, to make everybody's life easier.

----
I like rat03gopoh logic, but that will probably not work on long term (AI will start writing even more human, the users will learn to edit such keywords, ...) and I'm not sure it could be proven (although I totally agree it's unnatural to talk here like that).

I think I know the criteria for AI generated posts. For some reason I suspect posts that contains this sentence: "it's important to note/remember that..."
That's like saying in my mind, it's a part of the disclaimer that must always be included in every fenerated answers.

Look https://ninjastic.space/search?content=it%27s%20important%20to%20note%20that
Lately, this sentence has been used a lot by newbie accounts.

That's true. From now on we'll just post links with no names. With this exception: if you report a post for being "AI spam" and the report gets marked as Good, then share the archived version of the post here so we can get an idea of what "AI spam" looks like according to the mods.


That's also true. What I have so far is:

1) sudden shift to perfect English is the #1 indicator
2) detection tools can provide additional evidence if the majority of them indicate "probable AI," or >50% likelihood.

Of course running them through all 7 is too time consuming. Perhaps in the future we can narrow it down as we gain a better understanding of their accuracy.


When I have a bit more time I will put LoyceV's text through all 7 of the detectors.

Written with the help of: Google Translate
The idea itself is good in general and we have to be aware of the posts that arise that may become suspicious.

But this statement is not the best. I remember reading recently that LoyceV did a test of its own text and those detectors said it was AI text. Therefore, these systems saying that the text has a 50% or more probability of being written by an AI, could be 100% wrong.

Another point is that not everyone is fluent in English or has the ability to write in English, and can therefore use translation and writing help systems. For example, to write here in the forum in English, I use Google Translate and sometimes I use the Grammarly tool to try to improve my text. The probability of being considered a text written by AI is high.

That's why I wrote this post, using 4 methods: write only using Google Translate; write with ChatGPT doing the translation; write only using Google Translate with Grammarly help; write with ChatGPT translation and Grammarly help. Do your analysis.

---

Written with the help of: Google Translate & Grammarly
The idea itself is good in general and we have to be aware of the posts that arise that may become suspicious.

But this statement is not the best. I recently read that LoyceV did a test of its text and those detectors said it was AI text. Therefore, these systems saying that the text has a 50% or more probability of being written by an AI, could be 100% wrong.

Another point is that not everyone is fluent in English or can write in English, and can therefore use translation and writing help systems. For example, to write here in the forum in English, I use Google Translate and sometimes the Grammarly tool to try to improve my text. The probability of being considered a text written by AI is high.

That's why I wrote this post, using 4 methods: write only using Google Translate; write with ChatGPT doing the translation; write only using Google Translate with Grammarly help; write with ChatGPT translation and Grammarly help. Do your analysis.

---

Written with the help of: ChatGPT
The idea itself is generally good and we need to be attentive to posts that may become suspicious.

However, this statement is not the best. I remember reading recently that LoyceV tested their own text and those detectors said it was AI-generated. Therefore, these systems claiming that a text has a 50% or higher probability of being written by an AI may be 100% wrong.

Another point is that not everyone is fluent in English or has the ability to write in English and may therefore use translation and writing assistance systems. For example, I use Google Translate to write here on the forum in English and sometimes use the Grammarly tool to try to improve my text. The probability of being considered an AI-generated text is high.

That's why I wrote this post, using four methods: writing only using Google Translate; writing with ChatGPT doing the translation; writing only using Google Translate with Grammarly's help; and writing with ChatGPT's translation and Grammarly's help. Make your own analysis.

---

Written with the help of: ChatGPT & Grammarly
The idea itself is generally good and we need to be attentive to posts that may become suspicious.

However, this statement is not the best. I recently read that LoyceV tested their text and those detectors said it was AI-generated. Therefore, these systems claim that a text that has a 50% or higher probability of being written by an AI maybe 100% wrong.

Another point is that not everyone is fluent in English or can write in English and may therefore use translation and writing assistance systems. For example, I use Google Translate to write here on the forum in English and sometimes use the Grammarly tool to try to improve my text. The probability of being considered an AI-generated text is high.

That's why I wrote this post, using four methods: writing only using Google Translate; writing with ChatGPT doing the translation; writing only using Google Translate with Grammarly's help; and writing with ChatGPT's translation and Grammarly's help. Make your analysis.

---

In Portuguese

I personally avoid using those detection tools because of false positive and (at least for now) I think that the sudden increase in English level is probably the best piece of evidence that we can provide to prove someone is using chatbot.

Its probably harder to catch those brand new account that use chatbots right from the start, but so far I had some success with those as well.


AI generated text for sure. 4 days ago he managed to wrote that and then yesterday he wrote the post below. There's simply no plausible explanation for such a drastic decrease of his English.

Thats good idea but maybe we should keep it as lenient as possible or dont put names within the suspected since w arent sure yet if the content or written post are AI generated. It could be worse if we accuse someone of doing this and later on proven his innocent. Maybe a rule to what grounds a post can be submitted here. Should the post itself must be quoted including a member's name, or his/her plain post only.

Seems AI-assisted posting is becoming more rampant, and we should work together to brainstorm some concrete criteria for identification of AI-written posts.

For me, the best indicator if a post is AI-generated is if it is written in flawless English. Even the best posters who are native English speakers don't necessarily write every post using the precision in punctuation, spelling & grammar executed by ChatGPT. If a user's native language isn't English (as can be evidenced by their posting history), and they suddenly start posting in flawless English, it means one of two things:

1. The text is plagiarized, or
2. The text is AI-generated.

Its worth noting that not all plagiarized texts are written with the same mastery of English as ChatGPT. In fact, they seldom are.

The second best indicator is taking an average of separate AI text identification systems; the more, the better. This is the one created by the makers of ChatGPT and is thus probably the best for identifying such text:

https://platform.openai.com/ai-text-classifier

Here are 6 other ones.

https://writer.com/ai-content-detector/ (max. 1500 chars)
https://gptzero.me/ (min. 250 chars)
https://contentatscale.ai/ai-content-detector/ (min. 25 words)
https://x.writefull.com/gpt-detector (50-2000 words)
https://hivemoderation.com/ai-generated-content-detection (max. 8192 chars)
https://paraphrasingtool.ai/ai-content-detector/ (max. 4900 chars)

Seems like too much work to run a text through all 7, and some probably use extremely similar methods.

Of course, they are not always accurate, but if the majority of them say "likely" or "over 50% likely", then I'd say its compelling evidence the text is AI generated.

For example, this post - which I'm almost certain is AI-generated - comes back as "unlikely" to be AI-generated according to OpenAI's Text Classifier, yet it is "99% likely" AI generated according to the Hive Moderation checker.

Here is how you ask ChatGPT to generate such a list:

Prompt:


It generated an extremely similar list which comes back as "unclear" according to the OpenAI Text Classifier and "99% likely to contain AI generated text" according to the Hive Moderation site.


The problem is people can now be lazier than ever on the forum and get paid for it, which actually hurts them in the long run. Instead of thinking or learning anything for themselves, they just plug in a prompt, and prompt-plugging is the only actual skill they are gathering. Has nothing to do with Bitcoin, cryptocurrency, the blockchain, etc.


TL;DR - What are the rock-solid criteria for detection & definition of AI-generated text?


I suggest we put examples of what are believed to be AI-written posts in this thread to help evaluate them and come up with a refined consensus on how to detect them.