Topic: AI-generated post discussion thread: how to identify & report - page 8. (Read 2498 times)

I personally avoid using those detection tools because of false positive and (at least for now) I think that the sudden increase in English level is probably the best piece of evidence that we can provide to prove someone is using chatbot.

Its probably harder to catch those brand new account that use chatbots right from the start, but so far I had some success with those as well.


AI generated text for sure. 4 days ago he managed to wrote that and then yesterday he wrote the post below. There's simply no plausible explanation for such a drastic decrease of his English.

Thats good idea but maybe we should keep it as lenient as possible or dont put names within the suspected since w arent sure yet if the content or written post are AI generated. It could be worse if we accuse someone of doing this and later on proven his innocent. Maybe a rule to what grounds a post can be submitted here. Should the post itself must be quoted including a member's name, or his/her plain post only.

Seems AI-assisted posting is becoming more rampant, and we should work together to brainstorm some concrete criteria for identification of AI-written posts.

For me, the best indicator if a post is AI-generated is if it is written in flawless English. Even the best posters who are native English speakers don't necessarily write every post using the precision in punctuation, spelling & grammar executed by ChatGPT. If a user's native language isn't English (as can be evidenced by their posting history), and they suddenly start posting in flawless English, it means one of two things:

1. The text is plagiarized, or
2. The text is AI-generated.

Its worth noting that not all plagiarized texts are written with the same mastery of English as ChatGPT. In fact, they seldom are.

The second best indicator is taking an average of separate AI text identification systems; the more, the better. This is the one created by the makers of ChatGPT and is thus probably the best for identifying such text:

https://platform.openai.com/ai-text-classifier

Here are 6 other ones.

https://writer.com/ai-content-detector/ (max. 1500 chars)
https://gptzero.me/ (min. 250 chars)
https://contentatscale.ai/ai-content-detector/ (min. 25 words)
https://x.writefull.com/gpt-detector (50-2000 words)
https://hivemoderation.com/ai-generated-content-detection (max. 8192 chars)
https://paraphrasingtool.ai/ai-content-detector/ (max. 4900 chars)

Seems like too much work to run a text through all 7, and some probably use extremely similar methods.

Of course, they are not always accurate, but if the majority of them say "likely" or "over 50% likely", then I'd say its compelling evidence the text is AI generated.

For example, this post - which I'm almost certain is AI-generated - comes back as "unlikely" to be AI-generated according to OpenAI's Text Classifier, yet it is "99% likely" AI generated according to the Hive Moderation checker.

Here is how you ask ChatGPT to generate such a list:

Prompt:


It generated an extremely similar list which comes back as "unclear" according to the OpenAI Text Classifier and "99% likely to contain AI generated text" according to the Hive Moderation site.


The problem is people can now be lazier than ever on the forum and get paid for it, which actually hurts them in the long run. Instead of thinking or learning anything for themselves, they just plug in a prompt, and prompt-plugging is the only actual skill they are gathering. Has nothing to do with Bitcoin, cryptocurrency, the blockchain, etc.


TL;DR - What are the rock-solid criteria for detection & definition of AI-generated text?


I suggest we put examples of what are believed to be AI-written posts in this thread to help evaluate them and come up with a refined consensus on how to detect them.