Topic: Why does there need to be a limit on amount of transactions? - page 3. (Read 3739 times)

I never said that that nobody could run a full node. Although 100,000 tps with 16 peers would require ~5 Gbps of bandwidth.  I doubt most small businesses could afford that type of server costs.  Even if they "could" the cost to run a SPV is "free" by comparison so you WILL see less full nodes than say a network capable of "only" 1,000 tps.

The current implementation of the P2P network is not particularly efficient in terms of bandwidth usage and storage requirements. There's a lot that can be (and is being) done to improve this and also as more businesses of all sizes become dependent on Bitcoin they'll have an incentives to operate full nodes and to promote the development needed to reduce the resources needed to run them.

Even at 100,000 tps, small companies could afford to run full nodes.

caveden I think you misunderstood I wasn't talking about miners but full nodes.  

Right now today there are x nodes.

If the average block size increased to 10MB the cost of those nodes (who receive no revenue) would increase.  It is likely the number of full nodes would thus decrease leading to an increased amount of centralization.
Now what if the average block size is 20MB, 50MB, 300MB, 1GB?  

I wasn't talking about miners.  Miners are already highly centralized.  I was talking about independent full nodes which the network requires a large number to operate effectively.  As the block size increases the (uncompensated) cost on those full nodes increases and the % of participants in the system running a full node (and thus paying the full cost) will decline in favor of SPV and other lite solution (which are "free").   If eventually the network handles 100,000 tps but the demands on a full node are so high that only the largest of the largest of companies operating in massive datacenters can run a full node then you might as well call those "full nodes" .... banks.  As the network grows those established large entities have an incentive to encourage more growth as it acts as a barrier to entry against smaller participants.  Barriers to entry = higher profits it is both rational for entities to attempt this and realistic to expect it.

The idea that miners would soft fork blocks which are "too big" is equally disturbing.  First of all it is unlikely it would work.  Nobody is going to make a block larger than what all but the tiniest fraction of miners says it "too big".  If 10% of miners say they will soft fork blocks over 5MB only an idiot miner would make a block larger than 5MB.  You are taking a 10% orphan chance by making the block even a few bytes over the limit.  Another element is that Bitcoin works on the concept (at least in theory) that miners are independent actors.  If various miners will soft fork at various levels you have created a disincentive to remain independent.  You NEED to know what all your peer miners are doing.  We shouldn't be building a system where independent miners (the desired state) is at a disadvantage to miners in coordination.  Lastly if various miners do have differing soft for levels than an attacker can exploit that to degrade the effective hashrate of the network.  This would open the network to a 51% style attack with less than 51% of hashpower.  What matters is effective haspower (i.e. hashpower applied to the longest chain).  If miner X has a soft fork level of x MB and miner y has a soft fork level of y then by planting various blocks of differing size the attacker could fragment the good miners into working on competing chains.  Anytime miners are on multiple chains the network is only as strong as the hashpower on the chain with the most hashpower.  Note miner in this case is the entity (pool for most hashers) that is making the strategic decision of what chain to extend and what tx to include.  Hash processors aren't truly miners in that they are already following an authority (the pool server).

Not only it would not propagate in time, but even if it did, it'd likely hit everybody's soft limits so everybody would just ignore this block and keep building on top of the previous one.

Really? Even you buy this story that without central planning we'll get an "evil cartel"?

You're very active in this forum so you're probably aware that miners could set their own soft limits to block sizes, and only accept a block beyond such limit if it's "deep enough" (that being whatever they want). This feature could be coded in bitcoind with some values by default, as the transaction fee policy, making it very easy for miners to perform such decentralized block size control.
So, how could this "centralization by bloat" actually happen?

Don't forget that the probability of being orphaned increases if you create extremely large blocks - even if we ignore the chance of hitting some soft limits, chance that we should not ignore. Just look at the examples we see today. I've heard some pools were voluntarily censoring SD just to avoid the extra propagation time.
Oh, and don't forget either that you generate storage costs for yourself too if you start bloating the chain.

Certain costs and risks in exchange of a hypothetical possibility to kick out a few pool operators with limited bandwidth? No way.
Actually, DDoSers have unfortunately already done much more damage than this hypothetical risk you claim could possibly do... many pool operators have to hide themselves behind DDoS resistant ISPs already, which means large bandwidth. And that doesn't seem to be creating any "unbreakable evil cartel". It's not that difficult to start a pool today.
You should check Hearn's calculations on bandwidth costs and bandwidth demands for running a full node, they are somewhere in this forum I think. You won't ever need millions of dollars to run full nodes. A few thousands per year would be enough, even with Visa-like volumes. That's far from being an insurmountable barrier of entry capable of protecting "evil cartels".

I'm with Gavin and Mike Hearn on this one: there's no reason for a hard limit. Even if such catastrophic scenario you describe ever materialize (it won't), it wouldn't be difficult to fix it by just going back and adding some limits.

Unless the miner controls a majority of hashing power such that his chain will always have the largest accumulated work, a block that large in today's network is guaranteed to be orphaned.

Even without a hard cap on block size 80 GB blocks will not be possible until the majority of mining nodes have sufficient bandwidth to move that much data in a timely fashion.

I think from an economic standpoint, it makes sense to scale block size based on Tx fees. For example, at each difficulty adjustment, calculate average Tx fees in a block over all blocks since last adjustment, and scale block size based on that. That way, the extra cost to miners of storing larger blocks is made up by larger Tx fees. When competition for block space increases and people start paying more in Tx fees, the block size scales accordingly. This of course opens an attack vector where a miner mines many blocks himself, collecting his own Tx fees, to raise or lower the block size artificially. Also the equation to calculate block size from average block fees would still be an arbitrary/magic component.

Assuming the outputs can't be spent (remember, this was an evil miner doing this): 80GB must be stored forever. To put that in perspective, I was hopping to buy an 80GB SSD for my full node, and simply upgrade as the block-chain gets too large. I was hoping that would be on the order of a year or more, not 10 minutes

Edit: since 80GB would take a long time for most nodes to download, such a large block may actually get orphaned because it does not propagate fast enough.

Really? If the 1 MB limit was lifted right now via a simultaneous upgrade of the entire network, what would actually happen if a miner tried to produce a 80 GB block 10 minutes from now?

The average Joe isn't going to run a full node so his options are limited to what those running a full node do.

Even at 1MB limit sustain the blockchain will grow by 87GB annually.  That puts a limit on blockchain at >300GB by the time of the next subsidy halving.  A well connected nodes (say 24 peers) would need ~192Mbps of upload speed to share the block in a timely manner (~1 second).  The CPU load for synced nodes would be minimal but for bootstrapping nodes the number of tx to validate annually would be ~220 million.  So a new node around the time of the blokchain halving would need to download and validate roughly 1 billion transactions to get synced.  Even at 1MB sustained the requirements become a decent server (storage, memory, cpu) that has high speed, low latency links running 24/7.  Not impossible for an enthusiast or business but your average Joe isn't going to run a full node.

Can the client not be updated now (or soon) to handle more TPS and larger block sizes but not yet create them?  Leave this change in place until needed - 1 or 2 years.  Then when miners actually implement larger block sizes most people will already have a client that can deal with it.

I think it is important to distinguish between A limit and the current limit.

So there are a couple scenarios:
a) limit totally removed from code. unlimited block size. Tomorrow a miner could produce a 80GB block if they desired to
b) limit remains but is raised to another limited value say 20MB.
c) limit remains but protocol is enhanced to provide a rising limit based on deterministic factors.
d) limit remains and is now and always shall be exactly 1 MB.

Personally I think a is impossible and has a lot of unintended consequences.  It would allow centralization through bloat.  Eventually a handful of companies would be the only ones with enough resources to run a full node.  This can be done by intentionally mining massive amounts of data to push the full node requirements (think TB per day of storage, the need for 24/7 high bandwidth low latency links, and 128GB of RAM) to kill off existing full nodes.  Eventually this cartel of full nodes could gain enough power to require governmental approval to add new full nodes (we will call this a charter) and the ability to charge users for access through their node.  Many users will deposit their coins directly at these full nodes.  Congratulations you just invented banking.

The last scenario (d) is the status quo.  I don't think it will remain "forever" as forever is a long time.  It is important to consider that the change is non-trivial.  Anyone thinking the solution is "easy" likely hasn't thought about it enough. 

The new fixed limit (b) gives Bitcoin some breathing room.  PayPal for example does about 50 tps so a 32MB block size would allow ~224 tps (or 5x PayPal).  At peak usage it would make Bitcoin one of the largest transfer networks in the world.  No it wouldn't be the one system for all transactions but I doubt anyone but the purists would consider Bitcoin a "failure".  It has the ugliness of a new larger "magic number".

The dynamic scenario (c) is the most complex and as such the one with the most potential for creating hard to troubleshoot scenarios that could cripple the network either through lack of understanding or by introducing a new attack vector that entities who wish to see Bitcoin destroyed could exploit.

Personally if this was a dictatorship I would go with something like:
* Announce unlimited blockchain is not viable and won't be attempted
* Keep 1MB limit until the next block halving at which point the limit would be raised to 32MB.
* Fund research into an optimal way to have block size expand deterministically.

Of course this isn't a dictatorship and Bitcoin requires consensus which makes things even more challenging.

The average Joe will just see "There is a update available for your Bitcoin Wallet" and just install it. I would rather worry that this will make nefarious changes easier.

Bitcoin was always intended to scale to high transaction rates. Satoshi talked about it frequently, and in the very early days of Bitcoin every time somebody brought up the block size limit they were always told it would be raised when needed.

Originally there was no specific limit on block size except for a 32 MB message limit that applied to all node communications.

The block size was lowered to 1 MB in late 2010 because of a concern that an attacker could fill up everybody's hard drive with spam. Again this change was noted to be temporary by Satoshi, who posted about it on this forum.

Somewhere along the line some people decided they want the limit to stay at 1 MB forever. The reasons for this vary from person to person and appear to include: a desire to make alternate currencies artificially viable, they invented some type of off-blockchain transaction that nobody wants to use and so need some extra incentive to make people adopt it, hostility to Bitcoin's potential as a payment system that anyone in the world can use directly, a desire to see Bitcoin fail, or they invested in mining hardware and never want to be forced to upgrade or change their business model in order to remain profitable.

Frequently those who are in favor of keeping the block size at 1 MB will resort to lies of omission, misrepresentation, and fallacious economic arguments to make their case.

Yes, but the longer we wait, the harder it's going to become to gather overwhelming support. You think the average joe is going to care about this?

Who do you think are paying the multi-billion dollar Visa makes every year then? And if Bitcoin really becomes that valuable, do you think the miners would not  just get off their lazy ass to change one or two lines of code(assuming devs won't do) to accept small-amount transactions?(Yes, it's that easy)

You can't simply change the client.  The existing protocol is incompatible with a larger blocksize.  Period.  You can't force that protocol to cease to exist.  Today we call that protocol Bitcoin.  The only option is a hard fork, an incompatible version of the protocol but that means the existing protocol still exists as well.  Essentially two incompatible Bitcoins.   The only viable solution is before making a hard fork you have such overwhelming support that a super super super majority of all users (not just miners but merchants, exchanges, service providers, full nodes, and active users) switch that the old fork dies off.  If you don't one of two things will happen:
a) most people won't leave and your forked Bitcoin will just die off
OR
b) both forks gain significant backing and people end up with differing wealth of both chains.  Both "camps" have a vested interest in their chain surviving so you end up with chaos.  Two Bitcoins both calling themselves Bitcoin but completely incompatible to each other.  (Yes this is the bad news scenario which is why hard forks should never be considered trivial changes).

The federal wire network processes about $1.6 quadrillion USD worth of transactions annually and it works out to roughly 7tps.  Not saying the cap can't be raised but the idea that all economic transactions must be on blockchain or bitcoin "fails" is a dubious binary distinction.  It is also possible the limit will be raised but there still will be a limit (as opposed to unlimited block size).


Actually VISA routinely charges $0.30 PLUS 2% so that retailer selling a loaf of bread (not sure what bread is only $1) would get ~$0.68.  The 0.0001 (it was lowered in 0.8.2) min tx fee is just to avoid spam.  The market will decide the price of fast confirmations.  Also low value tx could be processed off blockchain so the merchant potentially could end up more.


There is no ignore dust rule, there is a DON'T ALLOW CREATION OF NEW DUST.  Of course the "rule" has a default value one that smart miners will decrease over time.  Newever versions of the client will almost certainly adjust the default value for nodes which don't override the default as well.  I would point out the min tx fee (for low priority txs) was initially 0.01 BTC that was lowered to 0.001 BTC, 0.0005, and now finally 0.0001 as the value of a BTC has risen. 


Maybe?  Bitcoin is an experiment in progress.  It may be that:
a) you can use on blockchain tx for these transactions but merchants would prefer (and thus offer a discount) if you use off-blockchain txs for.
b) you end up using an alt-coin backed by Bitcoin specifically designed for high volume low value transactions.  Bitcoin remains used for larger transfers of value.
c) you can't economically use a crypto-currency for these types of transactions and Bitcoin still becomes the worlds largest value transfer network in the history of the human race.