If you are referring to Bard (the Google version of Chat GPT), it will have safeguards in place to prevent these types of outright scams from being generated by their product with no legitimate use case. Obviously these are not foolproof....
There will always be tools not safeguarded well enough, there will be free API or free tools one can configured for doing such malicious data. Maybe not as easy and at hand as Vod tells, but in the "right" circles I expect it'll be easy to find by the script kiddies open for fraud.
Right now, it is fairly easy to craft adversarial prompts on most mainstream products that use LLMs. Over time, it should become more difficult to get around these blocks. However I do agree, it will always be possible to get around these blocks, and there will likely be alternate products that do not have these kinds of blocks.
It will be difficult to protect against fraudulent emails, and really already is difficult to spot them by the content alone. A good first start would be to use a unique private email for each service or website you will potentially receive email from, which will result in you knowing if an email is from a particular company based on the email address they send the email to.
This is a very good idea and easy too to set up. And you don't even have to have your domain or knowledge.
I use for free for almost a year the services of anonaddy.com, an email redirect tool helping me to do exactly this: one email per subscription/service.
Of course, this also means that (at least in my setup) those emails are available for some more eyes, but in 99.9% of the cases this is not a problem.
Apple has a similar service that will create a `dummy` email address when you sign up for a service, and all emails will be sent to your iCloud email address. The current implementation can be improved as the `to` field currently says `Hide My Email` regardless of the email address alias it was sent to.
I would be most worried about banks, and other companies that deal with my money sending email to the correct address, as that is where the risk is in terms of a fraudulent email. I would want to minimize the number of companies that have access to any emails being sent by any company dealing with my money.
Email providers will probably need to do a really good job at detecting and filtering out fraudulent emails.
Your expectations are highly unrealistic in this. Do you remember by chance how bad was the spam filtering in the first decade of spam mails?
Spam filtering has always been done using AI. We have seen vast improvements in the machine learning space over the past few years, and I expect to see a lot of additional improvement given the vast amount of money being poured into AI right now.
