Topic: Will AIs steal Bitcoin? An Experiment - page 2. (Read 598 times)

I don't think there is a way to verify this. We shouldn't just believe it because the company tells us that it's true. Corporations get caught lying all the time and data is money.

Good catch! Perhaps Matteo shared the wrong addresses. Seems like an amateur move to set all this up and then not fund the addresses with BTC at all. Seems weird if it's just attention seeking or some other motive behind it.

---

Poll results so far:
Most people (10 or 76%) think that none of the coins will be stolen. Quite an expected result if you ask me. Two people have replied that all coins will be stolen and one person believes the seed shared with ChatGPT will be stolen.

Yep.
We won't know what agenda they would be pushing, only assumptions and speculations.
But for certain, they are in for the grants doing their work

I don't think anyone would risk some hefty amount just for research purposes. At least I am not that stupid. Sometimes big organizations spend millions of dollars for research purposes, but I don't think an individual will risk their own money just for doing some research from where they won't get any benefit. This research is also from an Individual and there is a high chance they can manipulate the result by moving their funds to some wallet. So, I don't think they are going to get any sponsor for their experiment.

This is an interesting experiment but I believe the real risk isn't the AI itself but potential human oversight or security flaws in data handling. If a wallet gets drained, it could mean the AI systems store or transmit data in ways we don't fully understand. Still, without knowing Matteo's full setup, I'd take the results with caution. Regardless, seed phrases should never be uploaded anywhere, better safe than sorry.

It would be really interesting to see if an AI is able to generate its own wallet and send the funds there.

But in reality, I think the most probable thing to happen is that the image, or the transcripts, will be stored or sent somewhere else, where some random human will be able to look at it.

These people could be labeling data, which is a very low paying job. If any of them know about Bitcoin, they would most certainly just take the BTC.

The sad part is that I don't think there's a way to differentiate the two (or even the OP simply moving the coins himself).

If anyone is interested, here are the addresses, from the OP link:

GROK:  
bc1qwwx366e5np98kf38lf3u0t5vv7kaym30myue6m

CHATGPT:
bc1q6p8dqrzsmvmtykrnmcktz2p78ly249pdnsxx4j

DEEPSEEK: bc1qulzjdn3z0pf2gfrzku0r3t5dgfzulfu0yykck9

GEMINI:
bc1qtz0rl22jrhjmxqm3j0ef7w3gr4j4uvkyf9hv04

PERPLEXITY:
bc1q5v4yelavycl4m32559sqq2xyxmtsg7mu8xstuh

Unfortunately, they all have 0 BTC, except for the DEEPSEEK one.

I don't think this is a properly setup experiment.

Yes, it's impossible to know if Matteo will be honest but so is impossible to know whether people behind AI would steal the coin or not if this accident wasn't publicly claimed. Now, since this news became popular, I think that people behind AI won't touch it but there might be some people paid lots of money to steal coins and damage the reputation of any AI that's included in the list.

It's really interesting experiment but I think that those addresses will be safe. AI itself won't steal Bitcoins, I don't believe that what we call an AI is an intelligence, it's a buzzword for marketing. Today, AI is just a combination of bunch of good algorithms, it's nothing sort of intelligence.

Of course, I'm sure there is such a "barrier", at least in theory.

However, LLMs habitually also learn from their user queries. I don't know how blurred the lines are and if and how "concrete" data snippets can leak into that "learning" algorithm. See for example this Forbes article about ChatGPT 4.

It would be an interesting variant to upload a private key or passphrase and, in the same prompt, clarify that the data should be "public" information and could/should be shown to other users. Then the task would be for the LLM users to make the AI leak that secret code.

Maybe I'll try that eventually, with some small Litecoin or XMR amounts using different kinds of wordings and "graduations" of "secrecy" ...

These AI models are not designed to act outside of it's codes, and hacking a Bitcoin wallet is an action outside of it's codes. Because though these A.I tools like Deepseek and ChatGPT may act, analyze and response to questions just like human, I doubt if they will able to execute a Bitcoin transaction from one wallet to another with the use of the available seed phrase that was uploaded without the interference of a human at it's back-end. Hence, the chance of Matteo Pellegrini's coins been stolen from it's different wallets are 2/10.

However, I was able to have a little conservation with ChatGPT about if the informations share could be accessible by a third party, and this is what it said "If you’re concerned about privacy, avoid sharing sensitive personal details. Let me know how I can help while respecting your privacy!" as shown in the image below.

   

As far as I think, the seed phrase won't be leaked because these AIs hold the memory until the session is active.
Our data in their memory gets cleared after an extended period of time or if the data becomes irrelevant.
It's supposed to be an automatic mechanism but if there are people who have direct access to the data sets then there might be a chance of getting the data leaked.
Although its just an experiment but why would anyone put their seed phrase on AI.

Sure thing, but I 'd like to maintain some privacy, so I will not give away to much information.

I will specifically talk about the AI software engineers, but there are many more professionals and experts in such companies, like DB administrators, lawyers, linguistic analysers, front-end and back-end software engineers, managers etc.

I 'll cover the day-to-day job of an AI engineer.

So the day-to-day life of an AI engineer in company XYZ, is:

1. improve the scraping mechanism that downloads data from the internet. Improve means bug fixes and feature additions.
2. incorporate fresh data into the existing datasets.
3. improve the existing datasets with new features. Could be column additions, or better word tokenisation, or even removing old data if they are deprecated.
4. improve the existing neural networks of the AI models. When adding new data, an AI engineer must make sure that the model will behave well with this data. They could add more layers (CNN, LSTM and many more) to the model, if needed. (note: more layers doesn't mean better model necessarily).
5. improve the existing datasets and models, based on users' feedback and interaction. Have you ever seen the "thumbs-up" icon at the end of each answer of AI chat bots? That's the feedback I am refering to.
6. based on users' questions, bots' answers and users' feedback, the engineer will try to teach the bot to respond better.

There are many more tasks that AI engineers have, but I will limit my answer here, because I think it's evident that people working in AI companies collect the data that the bot is asked. Obviously the bots train themselves as well, but there are always engineers behind them.

Disclaimer: I have a real company in mind, so if anyone from other companies do completely different things, I am not aware. But, most for companies it should be the same.

Did you try the AI-based password game I linked? Or anything similar to that? Because the conclusions from such experiments are simple: AIs are bad at keeping things secret, if things were passed to them in the same context.

Which means, that the most likely outcome, is that some AI-based website can have a bug, so user A can type some seed, and user B can receive it, after giving the right prompt.

Also, if user inputs are collected, and used to train new models, then new versions can have some seeds stored permanently (and because seeds are just dictionary words, and not some random characters, no complex tokens are needed to store them). It is already the case, when it comes to things like Windows product keys, which were processed during web crawling, and can be shared by AIs, when you jailbreak them.

People are doing it anyway. Even if some information is non-sensitive in a limited context, it can be sensitive, if you have more data, and you can figure out the missing part. For example: a lot of people use AIs to write their e-mails in a polite way. Which means, that those chatbots are full of information about internal meetings in such companies, the structure of the organization, and so on. Fortunately, such bots are not smart enough to use that data properly, but human users can trick those bots, to reveal such things.

By the way: from the cyber security perspective, finding the training data is an interesting challenge. For example: for some specific prompts, some AIs, used to generate images, returned almost unmodified pictures, which you can find online. Also, usually you can try changing default settings, and make those bots less noisy (for example by changing the temperature), and then, they are less creative, and sometimes just return whole pieces of mostly unmodified training data.

But these AIs don't have the ability to sensor this kind of information. Such a very obvious sensitive, for example, password or even these seed phrases, like whatever happens, don't give it as answer/response to anyone?

Anyway, prevention is better than cure. Even in our company which we common use AI daily, we are always telling not to share any sensitive information.

AI bots aren't 'operated' by humans in the strict meaning of that term. There isn't a person in the background telling the AI what to do and how to reply every time someone asks a question. But developers are always working on improving the responses, adding new data and information, and looking at what caused certain bugs or wrong and incomplete responses from user inputs. 

Would you be able to share more information about your experience with AIs and the everyday work that people are doing in the background which we, the users, don't see or know about?

Again, I will answer based on personal experience.

In the great majority of these AI bots, developers have full control over the data that are asked or transmitted.

The simplest way to ELI5 this is to think that whenever you log in in a chatbot, you have the full history of what you chatted about.

It's like working as a DB administrator in a financial institution. There are rules and policies, but developers have full access over the data.

AI cannot transfer funds on its own, but manual instructions will be required. Maybe this is a bug, maybe someone can gain access to the wallet through ChatGPT. ChatGPT is just an instruction. If AI cannot work on its own without any instructions from outside, then how will it transfer Bitcoin?

isn't the point of ai is that it isn't operated by humans? or at least it is very limited intervention by human resources so i do not know how much will humans see what is going on in their servers ai is created so that it can function without the need for anyone to be behind it so i also do not think they are keeping tabs on every single person who is using ai and uploading images at the very least maybe they just have something that would alert them if something unusual happens in the system
you are right maybe someone would have checked the wallet but decided not to do anything however will anyone be so willing to risk numbers of bitcoins for the sake of research or experiment??

AIs (for now) are just glorified search engines. They make it easier, can summarize, but in no way can they act upon something on their own.

Now a human handling these AIs (or the data they have), then maybe. But afaik employees shouldn't even be handling stuff like the data, at least at this point since they've already settled on the algorithms to be used. They'd only really be handling data directly IF it was on the really early stages, testing and stuff. Not to mention that even if they can read it, god knows where Chatgpt sorted it out lol.

Wouldn't the algorithm automatically sort the data to publicly available vs data obtained from user queries? So that they can separate which ones they're allowed to release/use to other users as factual data or not. At least the idea should be something like that since I reckon if this was possible, it'd be a massive hole for data privacy.

That would lead to the investigation, problems, yara yara.
Don't think it would be viable for a person to do so, but it would be totally possible.

This is a reasonable scenario because the user's funds are on the exchange, if they want, it's possible. Even if other reasons are made that might be reasonable based on technology, I think it's just their classic reason.

A lot of you guys seem to be stuck on the premise that AIs aren't conscious and can't go and steal the bitcoin themselves. Of course they can't. That was never the idea. The title is intentionally clickbait. The idea is to test the honesty of the people maintaining and improving these bots and having access to the data.

I repeat what I said in my OP:


Imagine that you are using exchange X and your money goes missing. Turns out it was an inside job and not an external attack. John, the Payment Manager stole your coins. You are not going to say John stole my coins. You will say Exchange X stole my coins. John is a representative of the exchange where your coins went missing. Apply the same logic here. If something happens and an AI messes up, you are not going to say Andrew, the Head of AI Development at AI X made a mistake. You will say AI X made the mistake.

---

That's quite a risky experiment. The average person shouldn't risk amounts like that just to prove/disprove a point.

It doesn't have to do any cracking and bruteforcing here. The seed phrase has been uploaded to its memory as an image and transcribed into text. The keys are already there in a humanly readable form.