Topic: AI-generated post discussion thread: how to identify & report - page 2. (Read 2498 times)

If a user, let's take the case of this Adams0001 if he keeps editing without being found as an AI user, can't it be seen as someone who's not using AI because of the editing?
Since the tool you're using failed to detect his post while using AI, are there no other means to find him guilty of using AI?

This thread seems to have mixed debate about the AI tools and whether to rely on them or not. It's pity that we created the AI tools to do the stuff and now we are not able to identify whether it was our tool that created it or not. AI simply nailed it.

I am not sure how useful it could be but they are making various tools and also experimenting with the AI Content Checker. I have recently found AI Content Detector Chrome Extension brought to us by Copyleaks. Obviously it might be just another addition to the list of OP and is easy to use tool.

You just add it to the chrome as extension and whatever you copy on screen will get pasted and detected by the extension.

They have actually segregated the detection in following types:
GPT4 Content
ChatGPT
Bard
Human
AI + Human.

What caught my eye is the last one. AI+Human text detection could be very useful on the forum since we are seeing examples where people are just modifying some text with their own text. 

Will this extension be very useful?

It's not so easy. Let's see on this user Adams0001. Usually he makes some edits to the texts so detectors don't easily catch him. You can probably catch him manually, but it will take lots of time. If you'll read his texts you'll see that some of them are kind of technical, people don't speak that way usually. So as human beings we can see that it is not natural. But AI detectors sometimes don't see the same.

But he made at least one big mistake:

This post is easily detected by each detector with 99%+. Moreover we can check it with a plagiarism tool like text.ru and see results which are characterful for AI written: big parts taken from some other texts sometimes with whole phrases.


So now we are sure he is using AI, but how can a detector know it about some other texts with automated results of 0% probability?

I tried to get three posts of him for AI Spam Report Reference Thread, but changes he makes (sometimes he just deletes some parts of text) make it harder. Some texts have 60%+ of AI probability, some have 50%+, and I'm not motivated to spend too much time on him. So automated detectors failed.

How I wish the Forum would have something that can identify an AI post without stressing mods or other members who check to identify such users.
Or when posting using AI the post shouldn't be allowed, like the Forum should have an inbuilt AI detector that won't allow the user to drop the post and it should be having a red sign that the post is an AI post.

I wouldn't say AI is evolving at all by itself, there are people behind who are changing the code and they feed it with new information all the time.

Yeah but AI is not inventing anything new, it only uses old stuff that other people wrote before, and it combines this from several sources.
This is nothing else than mass plagiarism, and it's all coming only from bias government approved sources, so you can't get different opinion from AI.
Easy way for me to identify if something is AI or not would be talking with it for few minutes, and I can do the same thing with AI suspected members in bitcointalk forum.

I'm not saying there aren't some patterns. But as you say, they are not easy to detect, and have to become evident throughout several texts. But a quality AI, and with its correct use, can write things in different ways, making it difficult to create patterns. Therefore, I believe that over time it will be increasingly difficult to detect.

I'm not offended by AI text that passes in my eyes for human text. I'm offended by the AI text that is obviously so, and as has been mentioned in this thread extensively, there are giveaways for such text. There are clear patterns for observing AI-written text but to see them you need to be a native English speaker and also be good at detecting patterns.

If somebody manipulates the ChatGPT / GPT-3 output to make it less detectable, I'm more-or-less fine with that as it means they have bothered to put some time into making it less mechanical and more human. Most of the time when people use AI, its because they are lazy, so they won't bother spending the time editing their posts to make them less AI.

Well, you can't. That's why I've always said that these tools aren't efficient, and certainly (unfortunately for us) they never will be. We have to find other ways - than simply copying a text and checking it in a tool, to understand what comes from an AI or not.

Honestly, I don't know how we can do this. The only idea that comes to mind is the interaction with the person. If it's an AI, sooner or later it will respond in a way that doesn't make much sense and most likely a human wouldn't respond that way.




I don't think it has much to do with the fact that AI is evolving day by day. Yes, this fact is important, but it will not be the main element.

There are 1001 ways to say the same thing. Unless there is a clearly identifying pattern, it is very difficult to find out whether a text is made by an AI rather than a human. If we already know how the person habitually writes, it will be easier to detect if this way suddenly changes. Otherwise, it gets pretty difficult.

Even so, we have as an example cases of users who sold their accounts, and only several months (or years) after they were discovered. If we have a hard time figuring out the difference between the writing style of two humans. Let alone an AI that can take on multiple personalities.

They are not reliable enough, maybe because AI text generators are constantly changing and it's not easy to keep track with all changes and updates.
Using several sources in combination with common sense can give better results in detection, but even them don't expect to have high precision detection.

What's worse than having tools that don't accurately determine if a text is made from AI or not is the repercussion that they may have, the so called "false positive". That could easly happen in here - Imagine that a text from a user, with a shady posting behaviour, is marked as being AI generated - surely he'll face some criticism from the community (and with reason). But what if it's a "false positive"? His reputation will still be stained because AI detectors have determined that his text was "positively AI generated". Sure there are always indicators that one can take from such posts, but should we give the result of these AI detectors as proof when one of the biggest AI companies has admitted that they failed in detecting AI generated text?

Very well observed.

I am of the opinion that detecting AI text is extremely difficult, especially if the AI used does not have a writing pattern. And I believe that the more AI writes and evolves, the less standard it has, writing more and more in a very natural way.

So it will be more and more difficult to find out if a text is made by AI or not, just by reading it. Here on the forum, it will be the interaction with the user that will allow you to understand whether or not you are an AI.

How full circle it is when OpenAI has difficulty determining the AI-generated text they created. It makes it even more impossible if you think about it because it's improving. Isn't it somewhat contradicting in a way? Getting better in AI text generation makes it harder to detect it even more.

Perhaps some users haven't noticed yet but OpenAI has shut down their AI detection tool[1] that is mentioned in the OP[2]. Interestingly enough, more than the fact how silent this removal was, was the reason underneath it[3]:
It makes me wonder - If OpenAI isn't good at detecting AI generated text, how reliable are other detectors?

---

[1]https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool
[2]https://platform.openai.com/ai-text-classifier
[3]https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text

That's right, The easiest indicators to know an article was made with AI is the perfection of the language. (maybe a typo is important so it's not considered AI generated)

However, if you copy paste and it is detected by an AI detector the result is the AI generated. AI detector can only show AI generated or not, but cannot detect plagiarism. Must be done 2 times. with AI generated and plagiarized text

AI generated is still not accurate. AI and AI generated are still in the development stage.

AI is still in the development stage, it's already troublesome, especially if AI technology is getting more sophisticated. Maybe we'll be eliminated.

Welcome new word

I found one here: loyce.club/badposts/spam.html, and it got edited after posting.
Original:
The user has a neutral tag about it already, but hasn't been banned. It looks like 18 posts have been deleted already, but the user is still spamming.

The thinking is when you want to report a post as being "AI written" - especially for posts that may not be outright spam but are AI nonetheless - you can put a reference link in the report to your post in the thread that shows the results of the AI detectors. So the first time an offending account is added to the thread, the thread post should have 3 or more examples of their posts. Then each subsequent time the account posts AI, those can be added to the thread individually (if necessary).

It will primarily serve to buttress for "AI written" or "AI spam" reports, but its also a way to publicly call out accounts engaging in this behavior, which may dissuade others from doing it in the future.


Holy moly are these bad:


 

Agreed! You're a Global Mod, so why isn't nesty banned yet?

Let's look at the definition:

I've reported one of this posts again:

I think it's a good idea. But how should we report it to moderators? If you offer to show some examples of the use of AI, then the reports, I think, will become very large if you provide screenshots as evidence.
Are three examples enough?
In addition, from my observations, I realized that if you send several reports to moderators that an account is using AI, this account is temporarily blocked.
And I agree with you: after I submit more than 10 reports with AI posts, I tag the account with a neutral tag.

Here is an example of such a spammer.  https://bitcointalksearch.org/user/miner1448-2132496
Post creation speed and use of AI

Ran into some more AI-generated posts today. Not even by sig campaigners, lol.

I'm going to open a thread in Reputation which acts as a reference for reports on AI posts. Before doing that, I'd like to get any final feedback on how to proceed. Note this is simply for attaching references to reports, not for any kind of tagging (although if several of a user's posts are deleted as spam then a neutral indicating they are an AI spammer may be warranted).

After much deliberation and testing, here are the criteria I will be using to report posts as AI spam:


A. These 3 AI content detectors must all give a result of the post having a strong likelihood of containing AI-written material (with "strong likelihood" defined for each one):

1. Hive Moderation - 80% likely to contain AI generated text or above
2. Writefull GPT Detector - 80% likely to come from GPT-3, GPT-4 or ChatGPT or above
3. Writer AI Detector - 20% human-generated content or below

(I decided against including detectors that make you pay or register to keep verification accessible to all)


B. There must be at least 3 posts by the account that meet the requirements specified in A.

So, if its just a one-off or even two-off, I think the poster deserves some leniency, but if they make a habit of it, I strongly believe they should be named n' shamed and their posts reported.


C. Clues to help identify an AI poster before putting their posts through the detection process include:


and if their posts are frequently riddled with phrases mentioned here.


Anyone who thinks this shouldn't be done, or should be done in a different way, please feel free to chime in now or forever hold your peace. Thanks.

I have been testing this tool undetectable.ai for some time now. It's accuracy is far better than other tool available for free at the moment. Does detection from all relevant AI content writing apps. I did tested it for BARD and it worked fine. Detection is done for 10000 characters. At the moment it is not charging a penny for AI detection. The only problem here is that it requires registration through email. I did use temp mail and they allowed the email address.