Topic: How Many Full Nodes Bitcoin Online ? - page 4. (Read 22454 times)

Our work is not related to making "mempools synced", though they are naturally similar.

Our work is exclusively related to eliminating the massive overheads from relaying transactions the first time through as they go around the network.
 
That doesn't happen in the Bitcoin protocol, no one has proposed for it to happen, and it isn't needed.

It's really a shame that people are forced to waste their time correcting you simply because you are so persistent and voluminous in your inaccuracies that you manage to confuse many people even though your posts are not very convincing.

Your statement here makes no sense.  Nodes prioritize transactions by feerate, none of that has been removed.  The only "individualizing" in practice is that a low memory host might reduce their mempool size. This is, in any case, totally unrelated to removing the relay inefficiencies.

Again, Bitcoin nodes don't interrogate nodes to list mempools nor pick up transactions again, no one has proposed they do, because there is no reason to do that.

Just pre-empting another confused tangent: There is a "mempool" p2p message which was added to the protocol by bitpay for the purpose of surveilling the network under a dishonest justification, which was later realized to be a privacy problem and the privacy leak was removed (and after that bitpay's staff recommended removing it from the protocol).  Bitcoin Core has no ability to send a mempool p2p request and never has had the ability to do so. It might be interesting to do so at initial startup to quick start the mempool and give miners something to mine after being offline for a while, but at the moment no one is working on that, AFAIK.


The problem we are addressing is that if you have 100 peers, each of your hundred peers will advertise (or have advertised to them) each of those 8 transactions, using 100x the bandwidth on those advertisements as if you had only one peer.

The need for nodes to potentially drop transactions has nothing to do with free market behaviour and everything to do with nodes not having infinite storage to keep the transactions.  But there is, again, no interrogation-- they don't need to go refetch them again.


You've misunderstood what we've accomplished here.

If at some point during the initial relay of transactions,  you receive from your other peers TX  A, B, C, D, E, F  and I get TX B, C, D, E, F In the historical Bitcoin protocol each of those 6 values would be sent across the link between us (potentially twice).

Instead, you could send me the single value X = A xor B xor C xor D xor E xor F,  or I could send you the single value Y = B xor C xor D xor E xor F.  

After the single value is exchanged whomever received it computes  X xor Y = A -- the missing transaction, even though neither of us knew in advance which transaction was missing.

Minisketch generalizes this to support any number of differences.  The data sent is exactly equal to the number of different values, regardless of how big the original sets are. (In fact, the first value in a minisketch is exactly what I described above: the xor of all the elements in your set).

So, if you have received in relay A, B, C, D ... X  and I have  already received B, C, D ... X, Y, Z;   Then I need send you only three values (or you me): The xor of all my values, the xor of all my values cubed, and the xor of all my values to the fifth power... and then you will know that I am missing A from you, and you are missing Y, Z from me.  And by doing this we send only three values on the link between us in the initial relay instead of 26 - 52 (depending on how much duplication there is from concurrent sends).

There has never been and can never been a "consensus priority formula", because priority by its very definition is external to consensus.  But the behaviour of existing nodes is consistent-- they keep and drop the same transactions, subject to having them in the first place, and subject to the restriction that anything configured to use less memory obviously can't keep as much.

There is no re-interrogation, no-rerelay in Bitcoin, nor is any proposed.  It exists only in the imaginary protocol that you spend your days attacking and confusing people with.  The inefficiency in Bitcoin that we're working to resolve exists in the initial relay itself, and would still exists even if nodes had no mempools at all.

firstly trying to get mempools synced is meant to be about if everyone has the same tx set before a pool mines a block, then all that needs to be sent as a confirmed block is the headers and list of txid's. thus reducing the data needed to be sent when a confirmed block is created.

but here lays the issues
1. "the tx fee freemarket flaw"
this is where certain nodes drop certain tx's at DIFFERENT dust/min relay levels. meaning different people will have different mempool sets.

so trying to get people to keep every tx again. is counter intuitive to the whole drop tx if under dust/min relay fee 'free market'
easier solution is get rid of the tx free market and implement a consensus (all node agree on) fee structure so when tx's are relayed initially they all keep the same tx's in mempool and not have to drop tx's using fee freemarket. to then suddenly need to grab hundreds of transactions from X and hundreds of tx from Y AGAIN

2. distributed mempool
highlighting not the freemarket part. but the individual picky nodes. some nodes such as those that are "downstream filtered"(gregs buzzword) non-segwit nodes dont relay non-segwit unconfirmed tx's,

also some nodes that are run by pools fill their blocks with their own zero fee tx data which they dont relay the tx's out pre confirm purely to cause latency issues for thir competition(BTCC pool done this a few years back). so it wont solve the issue because different nodes have different ways of doing things.

i do find it funny that it was these very same devs that wanted a fee freemarket by removing a fee priority mechanism to make individualising mempools, that are now seeing the flaw in it.. even if they dont want to admit it the cause/effect on their implementation

but allowing nodes to relay tx's and drop them due to "fee free market" but then have to interrogate nodes to list their entire mempools(actually causing more bandwidth) and pick up the tx's AGAIN(more bandwidth again).. is silly..

imagine it 3 different nodes with sat 5 tx(out of in each of thier mempools due to different tx fee level preferences
1,2, a,b,c
1,2,3,4,c
1,a,b,c,d

they all initially did get 1,2,3,4,a,b,c,d at initial relay.. but freemaket let them drop certain tx.
now the third node has to ask the first node for the list.. 1,2,a,b,c (more data than initial relay)
now the third node has to ask the first node for the missing.. 2 (more data than initial relay)
.
now the third node has to ask the second node for the list.. 1,2,3,4,c (more data than initial relay)
now the third node has to ask the second node for the missing.. 3,4 (more data than initial relay)

now the second node has to ask the first node for the list.. 1,2,a,b,c (more data than initial relay)
now the second node has to ask the first node for the missing.. a,b (more data than initial relay)
.
now the second node has to ask the third node for the list.. 1,a,b,c,d (more data than initial relay)
now the second node has to ask the third node for the missing.. a,b,d (more data than initial relay)

now the first node has to ask the second node for the list.. 1,2,3,4,c (more data than initial relay)
now the first node has to ask the second node for the missing.. 3,4 (more data than initial relay)
.
now the first node has to ask the third node for the list.. 1,a,b,c,d (more data than initial relay)
now the first node has to ask the third node for the missing.. d (more data than initial relay)

the solution is much more simple.. get rid of the free market that lets nodes drop tx's in the initial relay. thus they would ALL have them all first go-around. without having to interrogate EACH connected node, after dropping.. because their would be no drop in the first place.

3. issue
"Minisketch generalizes this by sending various types of ‘sums’ of the data. The result is that with N different sums, you can find N differences … As long as the number of differences between the sets is not more than the number of ‘sums’ sent, minisketch will always succeed in finding all the differences."

lets call (the node interrogation) or as they call it 'sending sums'  X
lets call bloom filtring the missing transactions Y
X)now the first node has to ask the third node for the list.. 1,a,b,c,d (more data than initial relay)
y)now the first node has to ask the third node for the missing.. d (more data than initial relay)

by getting rid of the "free market" and getting back to a consensus fee priority formulae/structure that everyone follows means that X is not needed. and Y becomes a smaller percentage of missing tx's that it becomes insignificant to need X

what makes me laugh though. is then saying that nodes can then do 16/24 connections instead of 8.. i laugh because imagine having to do the X command(sending sums to 2-3 times more nodes.....(facepalm)

again solution is much more simply. get rid off the freemarket that allows individualised mempools at the initial relay, to then not need to re-interrogate nodes and re-relay transactions.. then you will get to conect to 16-24 nodes as oppose to 8. and no need extra bandwidth and commands/sums playing around.

What do you think?

Synching Data Between Bitcoin Nodes Is About to Get Easier, Minisketch is a new solution that’s trying to solve an old problem.

Blockstream co-founder Pieter Wuille, Bitcoin Core contributor and fellow Blockstream co-founder Gregory Maxwell, and Blockstream software engineer Gleb Naumenko.

https://bitcoinmagazine.com/articles/synching-data-between-bitcoin-nodes-about-get-easier/

My personal Full Node up-time status doesn’t seem to fit into any of the provided options. I run a full node "every now and then", meaning I could go days or even weeks without turning it on (and then patiently waiting to the sync to take place when I do).
 
I have on an external 4TB USB 3.0 disk, so as not to clutter my smaller SSD main drive, and since I have little spare time, I’m happy following the above procedure to look into things whenever I get some linear time to do so (which is seldom currently).

if you can understand the history of how banks came about in the 18th-19th century by locking gold up(call it bitcoin 8 decimal point value transactions locked outputs) and then hand people unaudited promissory(bank) notes(call it 12 decimal unconfirmed non blockchain transactions of value pegged to multiple coins(dependant on which coin you locked/atomically linked to)) then you will be on your way to understand what LN is doing with its latest concepts and direction.

as we all know what happened to the gold:bank imbalance after people stopped wanting to withdraw gold(bitcoin) but take silver(litecoin) coins out instead.

just remember LN wont by paying with real confirmed guaranteed funds. the developers have made it clear use at own risk. there is no way to guarantee fund confirmation.

LN is also not user adoption ready. and wont be for years, by which it will only fit a small niche of those wanting to spend under $100 for purchases of far less daily.

LN is to make people move away from the bitcoin network.
by definition that is not scaling bitcoin network in any way. its diluting/offsetting/removing utility

alot of people are already questioning the whole have fiat. then get a crypto, then lock a crypto to then split up unconfirmed crypto into multiple accounts to hope for productive regular use.... when its far easier to just use fiat to buy a few giftcards. job done.
(i have seen many other questions asked by average joe questioning is crypto better than fiat)

yes i understand th utopia. yes i understand as a bitcoiner myself all the other benefits of it. but for the outsider looking in. see LN as another barrier wall against adoption. "another room inside a room that needs to be walked through" (one reply was) "just to spend crypto as fast as say applepay or bank notes"

^
Thanks for the analysis, your argument is very complete, I can not debate it because my knowledge in Bitcoin technology is very recent, but these innovations can serve to make Bitcoin adoption faster to the base of the population, which does not know how to save, buy, sell ​​or pay with Bitcoin.

They are projecting Bitcoin all over the Internet:

Tim Draper Backed OpenNode Startup Raises $1.25 Million



https://www.nasdaq.com/article/bitcoin-payment-processor-opennode-gets-125m-from-investors-cm1071687

Bitcoin Space:

The move is an interesting one, as it removes the dependency on a physical internet connection through traditional ADSL connections and fibre optic networks.

https://cointelegraph.com/news/bitcoin-from-space-blockstream-cso-explains-its-satellite-services

Maybe he's right, surely, if Bitcoin does not react, every time we will be less decentralized.

what you'll find is that in the network. the whole consensus of solving byzantine generals problem has been bypassed. with near everyone just following one general now(core) and any dev team that wants to propose a new rule/feature that opposes the core roadmap ends up getting knocked off the network.. there is less need of a decentralised network(their view). as the "generals"(plural) has been sidelined and relegated off network. and now its one general and tens of thousands of loyal soldiers blindly following.

its gone from a byzantine generals solution for a decentralised generals network. to a singular general for a distributed soldier network

people end up not wanting to archive 10 years of data. just to wade through all that data just to ensure their $100 is confirmed.
holding the data if they are just interested in a few addresses becomes a waste of hard drive utility so they end up prunning it, thus they no longer become able to be a node that others can sync from

.. also you being a DNS seed is not being a full node. its just being a yellow pages directory to guide users to each other, essentually. which if you without bias dont set it to just list core nodes(as all DNS should do), would help if a revolution occured to bring core down a peg or two back down to an equal playing field with other nodes that want to get back to a united community of byzantine generals,
so it has purpose for some independent people to be DNS seed nodes to ensure the network doesnt become full 100% core only centralised.
.....
but back to the users being full nodes...
the situation will get worse when things are really pushed to pursuade users off the bitcoin network and into using another network like lightning.

because lightning network is not a bitcoin layer. it is its own network that allows multiple coins.
(bitcoin layer is just the sponsorship advertising buzzword many people use to try faming up that their project is part of the crypto industry, to garner investment)

users using the lightning network will find in a couple years that funds from litecoin, vertcoin or bitcoin are locked to be unspendable on their respective networks in a smart contract with a "factory/watchtower" entity. and this entity would need to monitor those multiple chains as a masternode.(super bloated mega node) to then when convinced the locks are locked. offer out a un-chained(unconfirmed) 'payment' to users that allows users to open channels. so that users can use phone apps to make payments instead of lugging around laptops/desktops when they visit starbucks

these factories become the gatekeepers (i would say banks but people hate when i say it, even if factually correct). this is so that users dont need to wait for their locked funds timeout to close a channel and broadcast a tx to the respective coin networks. they can just close channel and let the factory audit and then refresh them with a new un-chained(unconfirmed) 'payment' to reopen a new channel at any time.

this allows users to open/close channels more frequently if their co-partner in-channel is offline
yea i know your thinking the optimism of how great and user friendly it is. but. if you think about the whole network security of phone app users needing to trust a factory/masternode(bank) who has the real privatkey co-sign control of the real funds
and that the factory has to be a masternode monitoring the networks and also many users channels...
its not healthy. or secure, especially when the channels themselves are not byzantine general solved either

the only reasons the bitcoin network is told to people that blockchains dont work and impractical and slow. is due to the limitations of a blockchain imposed by the developers.(bitcoin is not AI self coding.. devs code it. devs put in the limitations, devs can remove them too)

anyway, take the initial block download argument. people dont really object to the blockchain data size. as a 256gb data is less than a fingernail(microsd) size, not a server size
its the fact that its coded that users cant even check a unverified balance to be able to do anything until the chain is synced.
if only they realised they could bloomfilter(request specific data of specific addresses) from peers first. just to get a unverified balance instantly. and then make the verifying/syncing more of a background secondary task that goes on less noticably so that people are not twiddling their thumbs waiting to see if they even have a balance/imported the correct wallet/keys

many people are too optimistic about lightning(future multicurrency banking system) but dont realise that to "be your own bank" on LN requires being a masternode of multiple chains. which is much more than just full noding one chain
(if you still think LN is bitcoin only feature.. research chainhash registry code in LN aswell as atomic swaps)


many people are too optimistic about lightning(future multicurrency banking system) but dont realise that its not a 'scaling bitcoin' its a take people away from bitcoin and use 12decimal payment values PEGGED to bitcoin via a masternode

many people are too optimistic about lightning(future multicurrency banking system) but dont realise that its not scaling bitcoin because its taking people off the bitcoin network where people wont want to be bitcoin full nodes as they are never going to be using bitcoin network daily

but anyways.. we are already seeing the dilution of the community bing diverted away from bitcoin. we are starting to see where the term "bitcoin maximalists" are being used as a derogatory term even to make it fel like if you support bitcoin as a payment network and want bitcoin to be secure and to scale then your 'dirt'

Interesting article:

Pierre Rochard > "Lightning Adoption Will Basically Mirror Bitcoin Adoption"

https://bitcoinmagazine.com/articles/pierre-rochard-lightning-adoption-will-basically-mirror-bitcoin-adoption/

Easiest Bitcoin Lightning Guide!

https://medium.com/@pierre_rochard/windows-macos-lightning-network-284bd5034340

probably better if your gonna use lukes charts to atleast use the more useful one
https://luke.dashjr.org/programs/bitcoin/files/charts/branches.html ~65% are capable of being full nodes, but it doesnt really show how many actually are (not prunned)

and as for home users with bad internet or using it as an excuse i have seen many times when they post their logs they ramped up their connection count

home users that naively think connecting their (imagine worse case)1mb connection to 100 nodes is helping. it is not.
if you have low bandwidth its better to only connect to 10 nodes. thus the 10 nodes get the data relayed sooner.

no one wants to be connected at a speed of 0.01. when they could be connected to a node at 0.1
it puts less strain on the user AND less strain on the network this way.

imagine it this way. your house. you want to open it up as a homeless shelter. it can only accommodate 10 beds happily.
do you think cramming in 100 people is helping those people. or is it better to help just 10 people and let others find another house where if everyone offered a room for 10 people everyone becomes happy.

so in short if your bandwidth is tight. just connect to less nodes, better for you, better for those connected to you.
if 10 nodes is a problem. try 5, if that a problem try 2.. just dont go up to high numbers and then cry that your internet is super slow

Ok, I'm going to try the software bitcoin-seeder.

Bitcoin is only useful if it is decentralized.
The larger the hashrate a single miner controls, the more centralized Bitcoin.

One of the reasons why the increase in the size of the Bitcoin block is not a solution, is because if the block increases in size, the individual nodes will be excluded.
This is a danger for any network, not just Bitcoin.
The larger the block size the more centralization.

Start more individual Full Nodes is the solution.

This graphic is wrong, or, at least, not totally true about the real number. According to @gmaxwell: "that page only lists ones that accept inbound connections". The actual number could be over 50 thousand. Still, it may sound small, but it is not. This number already guarantees the necessary statistical reliability. And more nodes would be useful to ensure individual security and privacy, but would not be totally necessary or urgent at this time.

I do not know if bitcoin-seeder still works, but it could deliver better numbers: https://github.com/sipa/bitcoin-seeder

VB1001:
Please search the forum how to combine quotes from different posts from different members into one reply. Don't worry, it's easy
The above 2 posts has been made with 6 minutes difference, so it would be just easier to edit the first reply and insert the content of your second reply into the first post. You can also delete the second post, the topic would become more easy to read and you won't get reported in the end (it's against the rules to post several replies in a row into the same topic).

If, unfortunately, the connection to the Internet is indispensable, be it good, this is a difficult point to solve, I am sure that if I did not have these problems I would have an online node.


Your answer is a good reason to encourage everyone to install a Full Node,
I hope that this thread arouses curiosity and we increase the number.

https://bitnodes.earn.com/nodes/
it says 9737 differnt types of nodes at time of posting
if you then search for 1037 (tag that suggests running full node services)
it says 6295 at time of posting

thats 65% of nodes are full nodes


and going back to the near 10k diverse nodes.. if you search out the current block height
it says 5805 at time of posting
thats under 60% that are uptodate of all nodes.

which means that the 65% of full nodes are not all uptodate

oh. also out of the near 10k nodes 30% of them are all hoarded on amazon, hetzner and digital ocean servers. meaning not decentralised.

so the representation of decentralised, diverse, uptodate and fully validating nodes is more closer to 3k than it is to 10k

I get the reason for running a node, but I don't have the spare memory/computing power to do so--and there's no reward for running one, so I temporarily un-ignored this section and voted "no" because I saw your thread in Meta. 

Don't laugh at me or call me names, but I was experimenting with earning interest through PIVX and downloaded the whole blockchain and kept it running, and it just made everything else on my computer run really slow.  I'm not a huge computer guy, don't own any spare laptops or anything, and my internet connection isn't the greatest, so I nixed that whole idea.  As an aside, I think I bought the PIVX when it was around $3 and look where it is now: $0.49.  So I'd like to say screw PIVX in addition to everything I wrote above.

Good web on the state of the Lightning Network

Number of Nodes
Number of Channels
Network Capacity
and more information

Lightning Network Search and Analysis Engine https://1ml.com/

Thanks for your support.

Regards.

Running Full Node

The Bitcoin Core GUI are several icons. If you hover over the signal strength icon, it will tell you how many connections you have.



The icon won’t turn "green" until you have more than 8 active connections, which only happens if inbound connections are allowed.

Peer info in Bitcoin Core GUI https://bitcoin.org/en/full-node#gui-peer-info

Peer info in Bitcoin Core daemon https://bitcoin.org/en/full-node#daemon-peer-info

Thanks for the 2nd edition book, Antonopoulus.
Ventilation is essential for the indoor temperature, a good fan will improve sure.
Yes, I know, I'm going to add information about the number of visible nodes with more than 8 active connections and those who are only active as clients.