How can I get data on where Bitcoin nodes are located?

fortysxity

newbie

Activity: 4

Merit: 0

Quote from: DaveF on April 12, 2023, 09:45:31 PM

Their code is here: https://github.com/ayeowch/bitnodes
And the install & configure guide is here: https://github.com/ayeowch/bitnodes/wiki/Provisioning-Bitcoin-Network-Crawler

Not much to it, if you have a fully patched and updated debian machine you can probably be up and running in under an hour.

-Dave

Thank you!

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: RickDeckard on April 13, 2023, 04:48:13 PM

The work being done on bitnodes.io

As far as statistics go, theirs is a decent work. But some of the nodes they list specifically in the first page are spy nodes not real full nodes that are maliciously and intentionally placed there.

Quote

I don't know if it is still being updated

Considering that the charts showing number of nodes is updated on daily basis, I would assume the links you shared are also up to date.
https://luke.dashjr.org/programs/bitcoin/files/charts/historical-dygraph.html

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: RickDeckard on April 13, 2023, 04:48:13 PM

--snip--
Right at the end of the page sits also an important information that is worth to be taken into account when looking at the ~17,700 active nodes:

Quote

Bitnodes uses Bitcoin protocol version 70001 (i.e. >= /Satoshi:0.8.x/), so nodes running an older protocol version will be skipped.

Satoshi:0.8.x (which represent Bitcoin-Qt 0.8.x) released 10 years ago[1]. And Bitcoin Qt version lower than 0.8 can't even perform initial block download anymore[2]. So i wouldn't worry too much about number of undetected node (which accept incoming connection).

Quote from: RickDeckard on April 13, 2023, 04:48:13 PM

As a closing remark, I once kept these[2][3] links shared by Luke Dashjr[4] that supposedly were scraping/listening to the nodes and recording each entry. I don't know if it is still being updated but considering who did scrapper, for sure it may be seen as reliable. I do like this[3] section of his website where we can see the amount of nodes that is running each Bitcoin protocol. I wonder if there is some way to verify these numbers?

It seems to be updated since current number is different from past snapshot on internet archive[3]. As for verifying the number, i guess it's impossible unless you run Bitcoin DNS seed node where your IP node/domain is added to list of DNS seed on Bitcoin Core (and optionally other full node software).

[1] https://bitcoin.org/en/release/v0.8.0
[2] https://blog.lopp.net/running-bitcoin-core-v0-7-and-earlier/
[3] https://web.archive.org/web/20230326174034/https://luke.dashjr.org/programs/bitcoin/files/charts/software.html

RickDeckard

legendary

Activity: 1148

Merit: 3117

Quote from: pooya87 on April 12, 2023, 11:18:14 PM

bitnodes.io can only detect nodes that are accepting incoming connections not all bitcoin nodes. In order to have a better view of the whole network and see all nodes (or at least a lot more than just those nodes accepting incoming connection) you'd have to run strong servers with multiple nodes accepting incoming connections and receive connection from a large part of the network.

And the way to detect the first node types is by using their public announcements (addr message they send out) even though the nodes you connect to and send a getaddr message are not going to send you all the announcements which means your "crawler" has to connect to a lot of nodes and for a long time to be able to construct as large a database as possible.

Their location is a matter of categorizing IP addresses and mapping them to geography.

The work being done on bitnodes.io is great since other companies/services use it as a data source for their website. Take, for instance, the case of bitrawr[1] who uses the data from bitnodes to showcase how Bitcoin is spread around the world. Right at the end of the page sits also an important information that is worth to be taken into account when looking at the ~17,700 active nodes:

Quote

Bitnodes uses Bitcoin protocol version 70001 (i.e. >= /Satoshi:0.8.x/), so nodes running an older protocol version will be skipped.

As a closing remark, I once kept these[2][3] links shared by Luke Dashjr[4] that supposedly were scraping/listening to the nodes and recording each entry. I don't know if it is still being updated but considering who did scrapper, for sure it may be seen as reliable. I do like this[3] section of his website where we can see the amount of nodes that is running each Bitcoin protocol. I wonder if there is some way to verify these numbers?

[1]https://www.bitrawr.com/terminal/bitcoin-node-map
[2]https://luke.dashjr.org/programs/bitcoin/files/charts/software.html
[3]https://luke.dashjr.org/programs/bitcoin/files/charts/services.html
[4]https://nitter.net/LukeDashjr

DaveF

legendary

Activity: 3500

Merit: 6320

Crypto Swap Exchange

Quote from: PawGo on April 13, 2023, 12:37:30 PM

Quote from: pooya87 on April 12, 2023, 11:18:14 PM

Their location is a matter of categorizing IP addresses and mapping them to geography.

Which of course cannot be treated as a definitive solution. There are companies which sell updated information about IP ranges and their locations, so if you want to rely on that it will not be for free. I do not think there are a lot of public (not-tor) nodes which use VPN to hide their real public IP, but it is possible too.

Due to the IPv4 exhaustion and the sale / transfer of IP space even between regions a lot of those services are best guesses.

Yes, the IP I have from my cable company is setup properly and the IP geolocation is accurate, BUT I have a friend using a small rural ISP who has an IP address that is still, as in YEARS later, linked back to an out of business ISP and their address & geolocation THAT IS NOT EVEN IN THE US. No matter what he does, a lot of places still think he is in Canada.

-Dave

PawGo

legendary

Activity: 952

Merit: 1385

Quote from: pooya87 on April 12, 2023, 11:18:14 PM

Their location is a matter of categorizing IP addresses and mapping them to geography.

Which of course cannot be treated as a definitive solution. There are companies which sell updated information about IP ranges and their locations, so if you want to rely on that it will not be for free. I do not think there are a lot of public (not-tor) nodes which use VPN to hide their real public IP, but it is possible too.

pooya87

legendary

Activity: 3472

Merit: 10611

bitnodes.io can only detect nodes that are accepting incoming connections not all bitcoin nodes. In order to have a better view of the whole network and see all nodes (or at least a lot more than just those nodes accepting incoming connection) you'd have to run strong servers with multiple nodes accepting incoming connections and receive connection from a large part of the network.

And the way to detect the first node types is by using their public announcements (addr message they send out) even though the nodes you connect to and send a getaddr message are not going to send you all the announcements which means your "crawler" has to connect to a lot of nodes and for a long time to be able to construct as large a database as possible.

Their location is a matter of categorizing IP addresses and mapping them to geography.

DaveF

legendary

Activity: 3500

Merit: 6320

Crypto Swap Exchange

Their code is here: https://github.com/ayeowch/bitnodes
And the install & configure guide is here: https://github.com/ayeowch/bitnodes/wiki/Provisioning-Bitcoin-Network-Crawler

Not much to it, if you have a fully patched and updated debian machine you can probably be up and running in under an hour.

-Dave

fortysxity

newbie

Activity: 4

Merit: 0

How does this site do it?

https://bitnodes.io/

How could I do it?

Topic: How can I get data on where Bitcoin nodes are located? (Read 126 times)