Topic: Bot to automatically check signatures. (Read 1070 times)

Note : Translation of my French post.

Another quick update

Added information about the signature request in the bot message ( Address, message, signature, version ).

The bot is now able to detect the following signature formats :
- https://bitcointalksearch.org/topic/m.53276206
- https://bitcointalksearch.org/topic/m.53280675

On top of that, from the moment a message does not contain "standard" format signatures, the bot will automatically look for a Bitcoin address in the message.
If an address is found, it will automatically be attributed to the person who posted the message. The next ones posting this address will be considered as "verifiers".
Note that for this case, the bot will not post an automatic response.

Also note that for old signatures ( Without "standard" format ), this system is not perfect.
The bot may associate an address with a bad account.
Similarly, it is possible for the bot to detect addresses that are not really addresses.
All this is part of its "error rate" possible, and I consider this error rate not very embarrassing ( Too much work to optimize it even more ).

---

On top of that, I released a little update of the public bot.

Now, the messages of the users whom he quotes are between quote tag.

And I took the opportunity to fix a bug on the replacement of dates in messages.

Yes, it's the post where is staked my signature.
Ok I understand now, it should have been more formatted.

Thanks for the report, indeed the bot does not yet correctly detect all the signatures.

I did a search with your nickname, is this the signature you are looking for ?

https://bitcointalksearch.org/topic/m.22803423

Unfortunately, this kind of signatures are complicated to scrap with the bot.

I need to see to further improve this part.

Very interesting and convenient feature but unfortunately it doesn't work for my nickname.  
I don't know if it's because of the spelling of my pseudo or if it's because of the post where my address is staked...

Yes I understand what you mean

I'll see to change that ( This week is very busy for now )

I will see to propose an update of the bot by the end of the week.

---

Note : Translation of my French post.

A small update of the bot that handles the checks.

Now, the bot can check up to one request every 30 minutes.
Allowing, in case of several simultaneous requests, not to give the monopoly to the bot.

In addition to that, it adds a certain security :
In case there is a bug, the bot will not be able to post 10 messages in 10 seconds ( No reason that it happens, but we never know ).

Signature detection has been improved. Now the bot no longer detects the signatures of "examples" like this one for example.

In addition to all that, it is now possible to see the addresses linked to a nickname.

To do this, go to this url :
- https://signatures.bitcointalk.luc-mergault.fr/user/{your-username}.html

Example:
- https://signatures.bitcointalk.luc-mergault.fr/user/lulucrypto.html

And finally I corrected two more three bugs ( Which crashed the bot ).

The other solution is to request your signature checking bot to be whitelisted from the plagiarism reporting bot. Since this is not an official bot it won't be easy and in addition there might be few people (bots) reporting for copy/paste. The easiest solution is to add quotation tags, as LoyceV already gave example.

Just to explain it simple. AFAIK the plagiarism bot check all the recent messages and compares the posts to a database for similarities. If you don't use the quotes your messages will probable end up automatically reported. I guess the mods will look into each case and probably mark them as bad or your posts will be deleted but this is again extra unnecessary work for the mods. To stay on the save side just add quotes.

I've edited your post to what I think it should look like:
Just add [ quote]-tags, just to be sure.

Hello,

Thank you for your proposal, for cons I'm not sure I understand ( Thank you my bad English ).

If I understand correctly, you say that I should quote updated / deleted messages instead of putting them in plain text in the message ?

You speak well of the messages sent on this subject :
- https://bitcointalk.org/index.php?topic=5202039.new;topicseen#new

Regarding to the bot that checks signed messages for changes, I think you should consider posting the original messages in quotation tags, otherwise there is another bot checking for plagiarism which can accidentally report those posts as a plagiarism. Too many bots working the same time can have some issues.

Oh ok, but you could have adapted the project for another project for example

To tell the truth, I do not really know if I can afford, to my level, to offer this kind of paid services ( Who am I to do it ? ).

Anyway, I will think about the idea

I have not abandoned the project, but the waves campaign stopped at a given time, they were paying me a very high salary.

I would suggest to you, to make a campaign manager, it's hard to find campaigns who going to trust you, but you can at least develop the system,
bot to count posts and filter the bad ones, create an admin dashboard with good statistics and overview of all posts, and duties,
and let the campaign manager use it to make his job easier. So, in that case, you can ask for pay to develop for the new features + maintenance.

Ho, your project is very interesting too !

If I understand correctly you have abandoned it ?

I thought precisely as a next project, to develop an extension ( Or a website ), which would manage very easily signing campaigns.

Would you mind if I developed a similar idea ?

I have already worked on the same idea back in 2015, Built it with PHP:
https://bitcointalksearch.org/topic/m.10253433

It was worth it, I was considering to make it available for everyone for generous donations, but forum members warned that I should not expect donations  .
Finally, I decided to make it paid service, I was on behalf waves platform campaign bot, I managed to relay the bot with my personal address https://www.blockchain.com/btc/address/16pe2twTgrjKC8mioPjX3f2UbCMGvWbVz4,
so every week I receive bitcoins on my address the bot automatically distribute payments for signature campaign members.

I'm not sure if the idea is still applicable since theymos has set a limit for 1 second for requests.
Anyway, thank you for sharing your hard work.

Thank you, I was talking about the message archiving system built into the bot precisely

Like here :

https://signatures.bitcointalk.cryptos-currencies.com/

It's easy to install the bot and archive messages from subject ^^

https://github.com/luluwebmaster/bitcointalk-auto-verify-signatures-archive-and-alert

---

Little update of the bot !

For those who have missed out on the subject, there is now a dedicated topic to post alerts about the topics "Stake your Bitcoin address here".

The subject :
- https://bitcointalksearch.org/topic/m.53094495

In addition to that, the bot now integrates a system allowing it to automatically post in this same subject ( Only deletion alerts ).

For that it will be necessary to create an account on the forum dedicated to that, once created, it is enough for you to configure the bot following the file of config ( See README ).

As a reminder, if you install the bot, do not hesitate to say it here !

Reminder of the Github project link :
- https://github.com/luluwebmaster/bitcointalk-auto-verify-signatures-archive-and-alert

http://loyce.club/archive/posts/5304/53043222.html

Hmm, ok I understand,  it's good idea, I'll see what I can do

Edit : Do not hesitate to install the archive bot elsewhere, it would be cool to have other links available for archives ^^

Correct. That would instantly highlight possible suspicious activity.

Hmm, I did not understand everything


Do you mean to create a topic dedicated to all posts updated or removed coming from the subject "Stake your Bitcoin address here" ?


Yes, I know that

I saw the discussion in the French section about detecting edited or deleted messages (but can't really read it so let's continue here): if you scrape all posts just once a week, it's enough considering the long-term purpose of the Stake your Bitcoin address here thread.
It would be really nice to have a bot that updates all changes to staked addresses in a new topic in Reputation.

I also keep track of all posts (only since a few months) here (not updated by a cronjob yet).

I guess you're talking about this message.

Currently, the addresses "bc1" are not checked by the bot, it deals only with quoting and archiving.

Verification of these signatures is expected shortly.

---

Hello everyone,

This post is a translation of my French post.

A small point on the bot is needed ( In this post, I will refer to points raised on the subject FR and EN ) !

So I had the opportunity to chat with Theymos about bot security.
In the end, he does not see the benefit of preventing the bot from deleting his messages.
In the sense, or, there are already people who take care of archiving the posts.
In addition to that, if needed, he is able to restore a deleted message up to two years back.

So starting from there, you have to know that from now on, I archive all the messages of the subject in question.
This archive can be found easily here : https://signatures.bitcointalk.luc-mergault.fr/
In addition to serving as an archive, this site allows you to perform a very basic search among all messages recorded on the subject.

History to ensure a little more the reliability of the bot and more globally of all the messages of the subject, I propose now a small very simple script, which allows whoever wishes to use it to be alerted by mail in case of suppression / edit the subject or are posted addresses.
This script allows anyone to help ensure the reliability of the messages on this topic ( The idea is that if you receive an alert, it makes sense to alert the forum ).

Note that deleted message detection is not yet in place on the script.

In addition to serving as an alert script, it also allows you to archive all messages, and if you want to make this archive public !

For those who wish to install it ( Very simple ) : https://github.com/luluwebmaster/bitcointalk-auto-verify-signatures-archive-and-alert
If you install the script, do not hesitate to provide the link to consult the archived messages, I would be happy to list it in the first post !

By the way I took the opportunity to install it on the machine of Cryptos-Currencies.Com :
- https://signatures.bitcointalk.cryptos-currencies.com/

Note that this script is not fixed, and improvements must still be made ( Especially on loading archived messages ).

It is now possible not to have his messages checked by the bot, simply by adding "no-bot" some by your post.

In addition to that, the bot supports Bech32 addresses using this method:
- https://bitcointalksearch.org/topic/how-to-verify-segwit-signature-with-brainwallet-5198585

The P2SH addresses are not yet, but the bot detects them.
It takes care of quoting and archiving messages containing messages signed from this type of address.

So much for this "small" point !

If you have other proposals, do not hesitate !

---

Hello everyone!

Little update of the project

Today, I have more or less finished the public bot ( Available on Github ).

If you have installed it, I invite you to update it ( Please note that the DB will be reset given the structural modification of it for messages ).

I would like to recall an important point, if you have a VPS or a machine running H24, do not hesitate to install it, and share the link of archived messages, it would be a big plus for the security of the project !
And if you have installed the script, the same, do not hesitate to share the link, it would be great

Now, the bot also detects deleted messages, and you alert if you want !

I took the opportunity to detail a little more the README Github, so that it is more understandable ( If you have a question, do not hesitate ).

Also, tonight, I'll take care to update the first post of this topic to provide all the missing info since launching ^^

Edit : First post edited !

Just another note here, when the bot is quoting a message which could not be verified, you should post a warning like ( failed to verify), otherwise everyone can stake a fake addresses and possibly scam people.

The bot in its current state will not be made public ( The code is too rough ).

However, I am preparing a script ( Bot ) that everyone will install, and that will help with the security of the checks.

Is there any chance of you putting this bot on Github? It would be good to see other people being able to suggest modifications and even better if more than one instance of the bot could be archiving these entries. I think that something like this which is not-for-profit would benefit quite  a lot from being made open source.

Hello,

Yes I saw that, thank you for the report.

I'm still working on the project, new ones should come soon ... 

The archive link is not available in the quoted message, see below :

Yes, I understand the problem, I work to propose a solution soon

I read a while back about a scammer who years ago lost his edit-privileges on the forum, to stop him from removing evidence. If theymos can take away edit and delete power from your bot (and publicly confirms it), it can offer a very nice service.

Yes it is a good thing that you archive the messages too. For my part, all messages posted on the topic dedicated to "Stake" are also saved, and will soon be publicly available on a dedicated page.


Ideally, the bot account should not be able to delete its messages, and can only edit its messages once ( Or for a limited time, for example ). But already if it was possible to make sure that he can not delete his messages, it would be a very good point !


Thank you for that, I understand now !

I'll see what I can do with that

Here's a Javascript library for decoding bech32 - https://github.com/slowli/bech32-buffer

They even have a web demo - https://slowli.github.io/bech32-buffer/

This decoder returns a buffer, so depending on how you handle signature verification, you might need to convert that buffer into a Bitcoin address (like in pooya87's example).

The links to archive.fo and imgur will be very hard to find back without the post.
But indeed, I archive messages. See this link for the posts I have on that thread. It's not automatically updated yet, but it will be once I have the time to test it a bit more. I'm not sure how long I'll do this though, staking an address is meant to be used years later, and I will run out of space at some point.

An easy solution would be to move the entire "stake addy"-thread to Auctions, so that users can't edit or delete their posts anymore.

i would drop using these silly online tools and use a decent library but here is a workaround in case you were interested:
(first a simple explanation of what is "message signature verification") an ECDSA signature is two numbers known as r and s. the math behind it has some characteristics that allows you to recover possible public keys from signature by knowing the message that was signed. so the first step is to base64-decode the signature, take r and s out, hash the message and calculate public key(s).
the second step is to convert that public key to an address and then check it with what user gave you.

knowing that you can easily convert a bech32 address to a legacy address* to fool the "silly" tool to pass the second step. all you need is a bech32 decoder

example https://bitcointalksearch.org/topic/m.52763726
bc1qarr0w42t3z7xtrcgwxh6kcckzq47tme5xfnj4n ->
e8c6f7554b88bc658f0871afab6316102be5ef34 ->
1NDp7v4M1fueJDob7mrnJktn1SnT4yramy
now the silly tool passes the verification!

you can't use this for addresses starting with 3 (nested SegWit) because they are hash of a hash of public key (it is more complicated than that, just simplifying it) so you have to have the public key and can't use what i explained above.

* node that this is a  workaround and should not be done for anything else. the decoding is done by knowing bech32 encodes "OP_0 " and base58 encodes "version checksum"

---

how about an "opt-out" option so that user could post a string (eg. nobot) in first line and get the bot to ignore the message.

Hello,

Thank you for your interest about the project

To answer +/- in order:

1) I could have used a package to directly check the signatures, but it did not fit the idea I wanted. Namely, allow everyone to visually check (Screenshot), and provide the verification link.
2) The purpose of this bot is not necessarily to replace the users who help with the checks. But as you said, rather serve as a complement ( Everyone is free to continue to perform verification ).
3) Now, I understand the problem of hacking, and I will see to try to find a solution to this.
4) The idea of having multiple instances of the bot is interesting, but as it was said, it becomes complicated to manage double posts / who should post / etc ...

Edit : For information, the archive contain the original post and the quotation ( Exemple : https://archive.fo/9DYWr )

The project is clearly not over, so expect changes to come

It's true that a mass vanishing would probably be caught fairly quickly. My concern is that a single deletion could lead to a big exit scam on a bought/hacked account and it's very unlikely that it would be detected in time.

Obviously in an ideal world we wouldn't need to have addresses be staked in a thread, it would be much nicer if there was a function on the forum that allowed you to stake an address on your profile which would be verified by the forum software and from then on saved.

I don't think it's a big problem, LoyceV and maybe some other users archive all messages, and the bot itself posts link to imgur and archive.fo, so even in worst case of the bot account getting hacked, the quotes won't be gone forever. Plus, theymos could probably restore posts if the vanishing would be quickly detected.


If the project was open source, we could have multiple competing bots, but it would cause duplicate posting in some cases, so the bot would need some system for deleting its own posts if they duplicate a quote.

It's a nice project, but there really needs to be more than one account quoting and verifying signatures. If it ends up being that solely one person is quoting these it's too easy for a malicious actor behind the account to edit or delete quotes others have made. Say for instance that the bot account gets hacked and offers to delete quotes of hacked account's addresses for payment. My worry is that people will get complacent and only this bot will end up quoting signatures.

These problems will obviously be way less significant if more than one person runs this script, or if people still quote as normal. Just saying that this can't be something that entirely 'frees' users from needing to quote, but simply a complement to it.

Nice project, I welcome any automation that frees members of Bitcoin community from doing tedious tasks.

I got a question, do you plan to use Brainwallet for signature verification, or do you plan to eventually switch to some node library? I think it's unnecessary to rely on third-party service to do cryptography for you if you can do it yourself, Brainwallet's site can go offline or change their structure and they can have bugs.

Hello,

Thank you

1) Currently I use "-----BEGIN BITCOIN SIGNED MESSAGE-----", "-----BEGIN SIGNATURE-----" and "-----END BITCOIN SIGNED MESSAGE-----" to found user request.
2) It's pretty simple, any new address ( Not registered in the database, and detected as a signed message ) is saved as a request. From the moment when another user posts a message with this address ( Quote ), then the request is detected as treated ( I am sure that it is not necessarily the best way, it is possible that it changes thereafter ).
3) For the detection, it takes about 5 to 10 seconds, and for the verification process, it takes +/- 30 to 60 seconds ( It is very variable ).
4) Oh, ok thank you, and for "P2SH" addresses, is it possible ?

Edit : And for information, currently the bot can't verify signatures, he is currently saving all the messages already sent on the subject ( +/- 10 000 ).

Edit 2 :  A screenshot of the console currently

Wow, it seems you did a great job but I would like to inquire about some information:

 - What keywords does a bot search for? You talk about this formula, but is it "----" or "-----BEGIN BITCOIN SIGNED MESSAGE-----"?
 - How is it determined that it was handled by another user?
 - How long does it take to search and to respond?

So far, there has been no agreement on a standardized format for signature to act as a verification of Legacy's (P2SH) signature.
For more read this ----> About possibility to Sign messages in Segwit address in future

Hello to all,

Note : This post is a translation from my French post.

It's been a while since I'm this topic, as I understand how it works, I started to tell myself that it would be possible to develop a bot to automate all that!

So this project was born.
I did not really give him a specific name, but to put it simply it's a bot that will automatically check the signatures posted by users on the topic mentioned above.

The operation is quite simple, the bot detects the requests ( By analyzing the last messages of the topic ), when it detects a request not treated by another user, it will launch the verification process.
To do this, the bot starts in a first to check the signature using Brainwallet.
If the signature is correct, then a screenshot of the site is taken. Note that if the check is unsuccessful, the bot stops there and does not respond automatically ( it may change later ).
Once the verification is complete, the bot post a first message, with the Quote of the message of the person who made the request, as well as the Brainwallet link + a link to the Brainwallet screenshot.
After that, it goes on archiving the request of the user ( thus integrating the response of the bot ).
He then finishes by editing his post, integrating a link to the quote and a link to the archive.

Some tips to know for the proper functioning:
- Please respect this signature format, otherwise the bot will not detect your request.
- Currently, the original addresses ( Legacy ) and SegWit, are processed by the bot.
- SegWit addresses are processed using this verification method.
- You can ask that the bot does not check your message, simply by adding "no-bot" somewhere in your message.

By the way, I take this opportunity to ask you something, do you know a website to verify the signatures created by P2SH addresses ?

For those who are interested in the technical part, here are some details:
- The bot is developed with Node.JS.
- Packages used :
-- Puppeteer ( Manage the posts, take screens ).
-- Request ( Scraper the latest messages + API calls ).
-- Jsdom ( Help scraping ).
-- Jquery ( Help scraping ).
-- Mysql ( Save the data the proper operation of the bot ).
-- Fs ( To retrieve the screens ).

And so from now on, the bot is launched and functional !

Well that's for the party bot "private", but following the request of some, I decided to develop another bot, this public and open-source!
This second bot is not for verification, but will serve as additional "security". Let me explain.

The problem with my checking bot is that it will centralize a lot of checks ( And lots of quotes ). Which would make this account a real target for hackers.
To put it simply, the purpose of the bot is to archive all messages of the subject.
In addition to archiving messages, he also takes care of detecting message updated and deleted messages.
So I used this detection to develop an alert system by email, allowing anyone who wishes to be alerted by email in case of update / deletion of a message.
The goal is that you can then alert the forum that there has been a change / deletion and therefore should be wary ( Depending on the type of message assigned ).

With this bot, it will become much less interesting to try to hack the account of the bot, since even if the account comes to be compromised, it will be possible to find archives.
The advantage is that since this bot is public and open-source, everyone can run it ( Very easy to launch ).

So for those who want to install the bot, this is where it happens:
- https://github.com/luluwebmaster/bitcointalk-auto-verify-signatures-archive-and-alert

Here, I will list all the sites archiving the messages of the subject in question, thus starting with two sites ( Do not hesitate to share yours if you use the bot ) :
- https://signatures.bitcointalk.luc-mergault.fr/
- https://signatures.bitcointalk.cryptos-currencies.com/

Here is a link to see the addresses linked to a nickname ( Replace my username with yours ) :
- https://signatures.bitcointalk.luc-mergault.fr/user/lulucrypto.html

If you have not yet sent your address, do not hesitate to come and test ^^

https://bitcointalk.org/index.php?topic=996318.new;topicseen#new

If you wish, it is possible to support the project here:
- 1DSXQn7AankhmXUvExfZBbo8zWa3ie3jXc

Note : During the beginning of the operation of the bot, I would be there to check its functioning, and correct the possible errors / bugs of it.
Note : Before the launch of this project, I had permission from theymos.

See you soon !