Topic: AI-generated post discussion thread: how to identify & report - page 2. (Read 2717 times)

Well, Adams0001 decided to write me a PM saying that it was an occasional case, that he is good and never uses AI. He occasionally used the same words as in some other text in Internet. Main mistake is not to delete all AI written posts. If I said that I'm too lazy to catch all his AI written posts and prove it, it doesn't mean it is so hard that no one will see.

I just took randomly one of his recent posts more.


First let's look at the grammar. As you can see English is not my mother tongue so I can't be an expert, but it looks like Adams0001 writes in nearly perfect language. Native speakers can confirm or not, but online grammar checkers say he is great.

And unexpectedly we see that there are two incomplete sentences in this post (marked with yellow; looks like he deleted some parts not to be detected). Strange, right?

Let's ask some detectors if this is human written? Copyleaks says no, 72.3% of probability fir AI, Hivemoderation says no, 58.1% likely to contain AI Generated Text, Sapling says no, 86.3% for fake.

Okay, let's detect for plagiarism with text.ru. And what was expected, we see that this post is copied from invstr.com/risks-and-rewards-of-investing/ for 47%!!! Nearly half of this text entirely from some single other source!



And moreover each time we see that a plagiarism tool doesn't say that there are many others who read some texts in Internet and used the same words. No, it finds an exact text which was a base for AI to generate this exact post. But there are hundreds of millions users in Internet who writes in English and read some texts. How can they not say some way the same things with the same words? It doesn't happen so usually, and even if here was an outstanding case, then not two in a row.

---

And you know, I decided to go through the challenge. I decided to check all my own words from the text above with the same 4 tools I used for Adams0001's post. Copyleaks gives 79% of probability for Human, Hivemoderation gives 0% likely to contain AI Generated Text, Sapling gives 7.6% for fake, Text.ru says 100% unique. As expected, because I really wrote everything of this by myself. And reading other texts in Internet didn't make me writing like AI.

And not only that post created by A.I are plagiarized, but are also totally useless, as they add no value to forum discussion. Because though they may sound super brilliant, they lack that human-to-human feeling (expression) that a normal conversation ought to have, and as such I wish to thank our forum moderators who have always been working hard just to flush away such people off this forum who add no value here.

However, I think the job of detecting an A.I poster can best be done by human and not any online tool.

That's right. If they think that the worst thing they can face is being detected as AI writers and then getting neutral or negative tag, then no. As I showed, AI uses ready patterns for writing so there are several cases when AI writers were reported as plagiarists (who they really are, because posting what was written by AI not saying that, is plagiarism anyway) and were permanently banned. So my advice for all of them is to stop using AI for posting and delete all AI posts they left. Because otherwise sooner or later they all will be caught and get negative tags or ban.

If the post beginning sounds like the answer to a prompt, then there's a high chance of it being an AI post. I mean, come on, "It is difficult to determine if cryptocurrencies will survive indefinitely"? What kind of person on this forum would write that? (Besides Peter Schiff's known trolls) For everyone here, it is very easy to determine that at least the majority of popular coins are here to stay, so if you're writing something that sounds like what an Oxford professor in a research paper would write, then your posting days are strictly numbered.

He uses, and we know it, and he knows it (he just deleted the mentioned post when he saw that he was caught). But even if someone will wish to spend time for making more proofs (there are some options of proving even for those who make small edits of texts, but it takes time) that he uses AI all we'll get is deleting of posts with proven AI usage. Too much time to spend for not too big result.

But you talked not about how to ignore that someone uses AI if he edits his posts good enough not to be caught by automated detectors, but how to prevent spamming forum with AI texts, right? Here we have an example of why is it not so easy. Automated tools are not as good as we could wish. So some part of the ones who use AI are still here and use AI, and it takes time for catching them, and no automated tool can fully automate this process.

If a user, let's take the case of this Adams0001 if he keeps editing without being found as an AI user, can't it be seen as someone who's not using AI because of the editing?
Since the tool you're using failed to detect his post while using AI, are there no other means to find him guilty of using AI?

This thread seems to have mixed debate about the AI tools and whether to rely on them or not. It's pity that we created the AI tools to do the stuff and now we are not able to identify whether it was our tool that created it or not. AI simply nailed it.

I am not sure how useful it could be but they are making various tools and also experimenting with the AI Content Checker. I have recently found AI Content Detector Chrome Extension brought to us by Copyleaks. Obviously it might be just another addition to the list of OP and is easy to use tool.

You just add it to the chrome as extension and whatever you copy on screen will get pasted and detected by the extension.

They have actually segregated the detection in following types:
GPT4 Content
ChatGPT
Bard
Human
AI + Human.

What caught my eye is the last one. AI+Human text detection could be very useful on the forum since we are seeing examples where people are just modifying some text with their own text. 

Will this extension be very useful?

It's not so easy. Let's see on this user Adams0001. Usually he makes some edits to the texts so detectors don't easily catch him. You can probably catch him manually, but it will take lots of time. If you'll read his texts you'll see that some of them are kind of technical, people don't speak that way usually. So as human beings we can see that it is not natural. But AI detectors sometimes don't see the same.

But he made at least one big mistake:

This post is easily detected by each detector with 99%+. Moreover we can check it with a plagiarism tool like text.ru and see results which are characterful for AI written: big parts taken from some other texts sometimes with whole phrases.


So now we are sure he is using AI, but how can a detector know it about some other texts with automated results of 0% probability?

I tried to get three posts of him for AI Spam Report Reference Thread, but changes he makes (sometimes he just deletes some parts of text) make it harder. Some texts have 60%+ of AI probability, some have 50%+, and I'm not motivated to spend too much time on him. So automated detectors failed.

How I wish the Forum would have something that can identify an AI post without stressing mods or other members who check to identify such users.
Or when posting using AI the post shouldn't be allowed, like the Forum should have an inbuilt AI detector that won't allow the user to drop the post and it should be having a red sign that the post is an AI post.

I wouldn't say AI is evolving at all by itself, there are people behind who are changing the code and they feed it with new information all the time.

Yeah but AI is not inventing anything new, it only uses old stuff that other people wrote before, and it combines this from several sources.
This is nothing else than mass plagiarism, and it's all coming only from bias government approved sources, so you can't get different opinion from AI.
Easy way for me to identify if something is AI or not would be talking with it for few minutes, and I can do the same thing with AI suspected members in bitcointalk forum.

I'm not saying there aren't some patterns. But as you say, they are not easy to detect, and have to become evident throughout several texts. But a quality AI, and with its correct use, can write things in different ways, making it difficult to create patterns. Therefore, I believe that over time it will be increasingly difficult to detect.

I'm not offended by AI text that passes in my eyes for human text. I'm offended by the AI text that is obviously so, and as has been mentioned in this thread extensively, there are giveaways for such text. There are clear patterns for observing AI-written text but to see them you need to be a native English speaker and also be good at detecting patterns.

If somebody manipulates the ChatGPT / GPT-3 output to make it less detectable, I'm more-or-less fine with that as it means they have bothered to put some time into making it less mechanical and more human. Most of the time when people use AI, its because they are lazy, so they won't bother spending the time editing their posts to make them less AI.

Well, you can't. That's why I've always said that these tools aren't efficient, and certainly (unfortunately for us) they never will be. We have to find other ways - than simply copying a text and checking it in a tool, to understand what comes from an AI or not.

Honestly, I don't know how we can do this. The only idea that comes to mind is the interaction with the person. If it's an AI, sooner or later it will respond in a way that doesn't make much sense and most likely a human wouldn't respond that way.




I don't think it has much to do with the fact that AI is evolving day by day. Yes, this fact is important, but it will not be the main element.

There are 1001 ways to say the same thing. Unless there is a clearly identifying pattern, it is very difficult to find out whether a text is made by an AI rather than a human. If we already know how the person habitually writes, it will be easier to detect if this way suddenly changes. Otherwise, it gets pretty difficult.

Even so, we have as an example cases of users who sold their accounts, and only several months (or years) after they were discovered. If we have a hard time figuring out the difference between the writing style of two humans. Let alone an AI that can take on multiple personalities.

They are not reliable enough, maybe because AI text generators are constantly changing and it's not easy to keep track with all changes and updates.
Using several sources in combination with common sense can give better results in detection, but even them don't expect to have high precision detection.

What's worse than having tools that don't accurately determine if a text is made from AI or not is the repercussion that they may have, the so called "false positive". That could easly happen in here - Imagine that a text from a user, with a shady posting behaviour, is marked as being AI generated - surely he'll face some criticism from the community (and with reason). But what if it's a "false positive"? His reputation will still be stained because AI detectors have determined that his text was "positively AI generated". Sure there are always indicators that one can take from such posts, but should we give the result of these AI detectors as proof when one of the biggest AI companies has admitted that they failed in detecting AI generated text?

Very well observed.

I am of the opinion that detecting AI text is extremely difficult, especially if the AI used does not have a writing pattern. And I believe that the more AI writes and evolves, the less standard it has, writing more and more in a very natural way.

So it will be more and more difficult to find out if a text is made by AI or not, just by reading it. Here on the forum, it will be the interaction with the user that will allow you to understand whether or not you are an AI.

How full circle it is when OpenAI has difficulty determining the AI-generated text they created. It makes it even more impossible if you think about it because it's improving. Isn't it somewhat contradicting in a way? Getting better in AI text generation makes it harder to detect it even more.

Perhaps some users haven't noticed yet but OpenAI has shut down their AI detection tool[1] that is mentioned in the OP[2]. Interestingly enough, more than the fact how silent this removal was, was the reason underneath it[3]:
It makes me wonder - If OpenAI isn't good at detecting AI generated text, how reliable are other detectors?

---

[1]https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool
[2]https://platform.openai.com/ai-text-classifier
[3]https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text

That's right, The easiest indicators to know an article was made with AI is the perfection of the language. (maybe a typo is important so it's not considered AI generated)

However, if you copy paste and it is detected by an AI detector the result is the AI generated. AI detector can only show AI generated or not, but cannot detect plagiarism. Must be done 2 times. with AI generated and plagiarized text

AI generated is still not accurate. AI and AI generated are still in the development stage.

AI is still in the development stage, it's already troublesome, especially if AI technology is getting more sophisticated. Maybe we'll be eliminated.

Welcome new word

I found one here: loyce.club/badposts/spam.html, and it got edited after posting.
Original:
The user has a neutral tag about it already, but hasn't been banned. It looks like 18 posts have been deleted already, but the user is still spamming.