Topic: Share your ideas on what to replace the 1 MB block size limit with (Read 6977 times)

There are some great replies inthis thread.

The impression that I get is that there are a number of objectives that have been discussed ( blocksize is a possible solution and not an objective).

- Speed up transaction confirmation for end users
- Reduce bloat in transactions
- Reduce the number of records required for each transaction
- Maintain the de-centralisation of mining.

At the moment block generation time is too long, and it can result in transactions taking up to an hour to be confirmed. Bitcoin won't remain competitive if this continues, or gets worse. I read the replies to my thread suggesting a reduction in the generation time, and I am grateful for the informed comments, and I accept the reasons for not reducing the time in the current environment. I think it is desirable for better minds than mine to address this problem to see if there can be a way to reduce the generation time.

Reducing bloat in transactions - one way to do this is to REDUCE the blocksize, maybe to 500K, and combine this with faster block generation. Obviously this is subject to the constraints mentioned above. SegWit should be  great help with this as well.

Reducing the number of records - there was an interesting suggestion about combining micro-payments  into one transaction within a wallet. It would be useful if this could be done with a zero fee transaction, but would miners be prepared to support this. There is not the same priority for the confirmation of these consolidations, so maybe they could wait for anything up to a week, Listing them in a consolidatation pool after they have been verified could give miners a handy alternative to mining empty or near empty blocks. This would help to speed up future payments from that wallet, and it would reduce the number of transactions waiting to go into a block. You wouldn't need a fork for this, as miners not supporting it would just ignore the second pool, and those who did support it would create blocks that were valid under the current rules - is this true, or am I misunderstanding the block filling process?

Maintaining the decentralisation of mining. I accept the point that reducing the block generation time can create "churning" of the blockchain, and would give larger miners an advantage when the block generation time is minimal. If it were possible to enforce a minimum time for the generation of  a new block ( say one minute) then surely that would cut out much of the churning, and may even attract a few small miners to return to mining.

Side chains are also beneficial, and maybe we can explore the way to increase their use more effectively. Maybe all faucet payments should be moved onto a specialist side chain for example.

Thanks for David Jerry bringing this up (https://twitter.com/digitsu/status/809152914023251968 ). Here is my reply


Yes, if we consider ONLY these 2 factors, this is the best solution. But the very obvious fact is these are not the only factors


Yes, I still think 1MB is an arbitrary limit. If Satoshi used 2MB, Bitcoin should have worked in exactly the same way as it was in 2014 (when I wrote the original message). However, if we had 2MB today, without all the optimization brought by Bitcoin Core in the past 2 years (libsecp256k1, compact block, pruning, etc), many full nodes would have been broken.

Had he chosen 0.5MB, it might be easier for people to reach consensus to increase it. However, I have to say the effect of hitting the limit is not as bad as I thought. One indication is the robust price uptrend in this year. OTOH, most tx confirmation problems are related to wallet design.


Note that when I say "1MB is definitely not a good one", the context is bitcoin would become a global settlement network with a trillion market cap. I think my estimation is still correct, but this is a long term vision.



This 100MB estimation does not take payment channel like LN into considerations. It also doesn't take better cryptography like Schnoor signature aggregation (saving lots of block space) into considerations. Schnoor signature alone could easily cut my estimated size by more than half (and it is totally on-chain). With more optimizations (weak block, UTXO set growth limit, fraud proof, partial block validation by SPV wallet), I don't think 50MB blocks is an insane idea. It just can't happen today.

Finally, as a not-so-related note, one may also note that I'm always a fan of softforks since I joined this forum, for examples:
https://bitcointalksearch.org/topic/auxiliary-block-increasing-max-block-size-with-softfork-283746
https://bitcointalk.org/index.php?topic=256516.10
https://bitcointalksearch.org/topic/opcheckcolorverify-soft-fork-for-native-color-coin-support-253385
https://bitcointalksearch.org/topic/partial-validating-nodes-1103281

Sorry for the late reply. I wasn't aware of this thread discussing Monero.

There was never any "crippling bloat." I'm not even sure what gmaxwell is talking about. There are certainly some forward-looking concerns about the future size of the blockchain (as with Bitcoin). But as of today there is no crippling. I run a Monero node on a smartphone-class nettop (just to see if I could), and it works fine.

There have been some growing pains. Early on the release one pool code didn't have a minimum payout threshold, so pools paid out on every block. Obviously that increased the volume of transactions, but the network still functioned normally and required no adjustment (save for fixing the pool code). And there was recently a spam attack on Monero and we responded by raising the (previously unrealistic <0.01 USD) default fee, but that only lasted a day or so and added 20 MB to the blockchain before the higher fee kicked in and the spammers stopped. Finally, the on-disk format is currently inefficient, and is being improved.

Still none of these issues have been crippling at all.

While this may not directly address block size, it will have an effect on it as well as other aspects of the entire system.

There is an inefficiency in bitcoin related to people needing to consolidate their inputs and it ends up costing the system more than the transaction fees are worth due to unnecessary traffic.   What we need to do is remove the need for people to consolidate all their tiny transactions, by doing it for them automatically.

Couldn't we add a feature to the wallet system that automatically consolidates all extant inputs to a particular address if there are any inputs of at least X days of age into one transaction, by adding a marker/token to a new block that effectively authenticates the presence of all these inputs from previous blocks, so there is no need to search further back down the block-chain for authentication of the inputs. Thus decreasing the data size of any future outputs from that address. I would make it so that it would, also, automatically re-consolidate the returned change inputs by adding them back to the address from which they originally came without having them travel the block-chain again.

To take this further, it effectively would be an auto-pruning measure, because at some point you would never need to go back to the genesis block for authentication.

This, ultimately, may or may not have an effect on difficulty, so care needs to be taken that we do not introduce inflation into the economy.

What size should each block be?  I'd say as compact as we can make them with as much data as we can cram into them. My proposal helps with that aspect.  Eventually, we will see the block-chain housing significantly more data than it already does.  

There are advantages and disadvantages to every proposal so far.  Does increasing/decreasing block sizes contribute to maintenance of current and future difficulty standards and thus have a positive or negative impact on inflation?  When we get a protocol in place to begin pruning we will then see a need to balance block size against chain length. To that end, I suppose, there will have to be a block size growth factor calculated in or the chain will become "unmanageably" long even with pruning and difficulty will skyrocket, especially as seen against the back-drop of the goal of bitcoin to become the currency of the world economy...

Just my ideas! What do you all think?

whitebeard

Letting miners _freely_ expand blocks is a bit like asking the foxes to guard the hen-house— it's almost equivalent to no limit at all. Individual miners have every incentive to put as much fee paying transactions in their own blocks as they can (esp. presuming propagation delays are resolved or the miner has a lot of hashpower and so propagation hardly matters)— because they only need verify once the cost of a few more cpus or disks isn't a big deal. In theory (I say because miners have been bad with software updates), they can afford the operating costs of fixing things like integer overflows that arise with larger blocks, especially since they usually have little customization— other nodes, not so much?

Since miners can always put fewer transactions in, it's not unreasonable for the block-chain to coordinate that soft-limit (— in the hopes that baking in the conspiracy discourages worse ones from forming). But in that case it shouldn't be based on the actual size, but instead on an explicit limit, so that expressing your honest belief that the network would be better off with smaller blocks is not at odds with maximizing your revenue in this block.

If you want to talk about new limit-agreement mechanisms, I think that txout creation is more interesting to limit than the size directly though... or potentially both.

Even for these uses— Median might not be the right metric, however— consider that recently it would have giver control to effectively a single party, at the moment it would effectively give it to two parties. You could easily use the median for lowering the limit and (say) the 25th percentile for raising it though... though even thats somewhat sloppy because having more than half the hashrate in your conspiracy means you can have all the hashrate if you really want it.

I think I'm not alone when I say we are dangerously close to hitting the limit already that we need a quick fix right away. The only easy and quick fix is raising the limit, and to not make people upset, just double it to 2MB.

Reading about soft fork vs hard fork, this appears to be more difficult than a simple user like me imagines. But something needs to be done soon. Getting tx stuck when the next boom hits is something we can all agree on shouldn't happen.



In related news, I also feel spamming the network is way too cheap right now. I can spam 10 tx/s for less than 9BTC for a full day (assuming 0.01 mBTC fee). If I was in a position planning to buy in big time, I would do this, expecting a price drop with probability high enough to make the whole endeavor worth it.

As far as I understand those schemes they are only good if you run a node with a current memory pool. The full transactions still need to communicated at some time, and still need to be written to the blockchain in full. You couldn't just write IBLTs to the blockchain, because no one without your memory pool could reconstruct the TXs.

Indeed, but it is now apparent (to me anyway) that simply increasing the block limit to allow larger and larger blocks to propagate is not necessary, as this is not the optimal long-term solution. The optimal solution takes advantage of the fact that most transactions are already known to most peers before the next block is mined. So, "highly abbreviated" new blocks can be propagated instead. This is beyond mere data compression because it relies on the receiver knowing most of the block contents in advance.

We see in the O(1) thread that there are excellent proposals on the table for block propagation efficiency:
A) short transaction hashes: as in block network coding, and similarly in the optimized block relay (Matt Corallo already has a relay service live)
B) IBLT blocks

Even better, they are compatible such that A can be used within B giving enormous efficiency gains. This must be the long-term goal.

The next question is: Can the max block size be made flexible (for example: a function of the median size of the previous 2016 blocks) as a phase in the process of introducing block propagation efficiency as a consensus change?

I am not going to cite to provoke unnecessary debate. As you are not one of them, let's stop here.


This kind of connection is available in Hong Kong for maybe 10 years. Today, with $20/mo we have 100Mb/s. We can even have an 1Gb/s fiber directly connected to the computer at home with only $70/mo.

Anyway, I know our case is atypical. However, I'd be really surprised if you can't do the same in silicon valley in 10 years. Also, I doubt one would have any difficulty to rent an 1U collocation space with 100Mb/s for $100/mo in silicon valley.


Most of them are here for profit so they won't do it anyway, no matter it's 1MB or 100MB.

Also, I can't see why we really need to verify every single transaction back to the genesis block. If there were no fork floating around, and no one is complaining in the last few months, it's really safe to assume the blockchain (until a few months ago) is legitimate.


Again, they won't do it no matter it's 1MB or 100MB, unless the protocol forces them to do so


Why that would take hours to broadcast a warning like that? Let say you are a merchant and you can't afford you own full node. You primarily rely on Blockchain.info but you also run an SPV client to monitor the block headers. As long as one of your peers is honest, you should be able to detect any problem in the data of Blockchain.info within 6 confirmations (since the block headers won't match).



I mean, a 100MB block is not a problem for bitcoin whales and altruistic players to run full nodes, and we will have enough honest full nodes to support SPV clients. For miners, however, as block propagation is crucial for their profit, a big block with O(n) propagation time will cause problem. Gavin's O(1) proposal gives some hope, but I have to admit I don't understand the maths behind it.


No matter that will be a hardfork, or an auxiliary block softfork, this will be the most dramatic change to the protocol. However, I can't see any real progress in reaching consensus despite years of debate.

Can you cite these people specifically?  The strongest I've seen is that it "may not be necessary" and shouldn't be done unless the consequences are clear (including mining incentives, etc), the software well tested, etc.

Great. I live in the middle of silicon valley and no such domestic connection is available at any price (short of me paying tens of thousands of dollars NRE to lay fiber someplace). This is true for much of the world today.

I agree with this partially, but I know it's at all the whole truth of it. Right now, even on a host with solid gigabit connectivity you will take days to synchronize the blockchain— this is due to dumb software limitations which are being fixed... but even with them fixed, on a quad core i7 3.2GHz and a fast SSD you're still talking about three hours. With 100x that load you're talking about 30 hours— 12.5 days.

Few who are operating in any kind of task driven manner— e.g. "setup a service" are willing to tolerate that, and I can't blame them.

There is no need to delegate your mining vote to a third party to mine— it would be perfectly possible for pools to pay according to shares that pay them, regardless of where you got your transaction lists from— bur they don't do this.

And tell them what?  That hours ago the miners created a bunch of extra coin out of thin air ("no worries, the inflation was needed to support the economy/security/etc. and there is nothing you can do about it because it's hours burried and the miners won't reorg it out and any attempt to do so opens up a bunch of double spending risk")—  How exactly does this give you anything over a centralized service that offers to let people audit it?  In both there can always be some excuse good enough to get away with justifying compromising the properties once you've left the law of math and resorted to enforcement by men and politics.

In the whitepaper a better path is mentioned that few seem to have noticed "One strategy to protect against this would be to accept alerts from network nodes when they detect an invalid block, prompting the user's software to download the full block and alerted transactions to confirm the inconsistency". Sadly, I'm not aware of even any brainstorming behind what it would take to make that a reality beyond a bit I did a few years ago. (... even if I worked on Bitcoin full time I couldn't possibly personally build all the things I think we need to build, there just isn't enough hours in the day)

That isn't the whole tool in the belt, but I point it out to highlight that what you're suggesting above is a real and concerning relaxation of the security model, which moves bitcoin closer to the trust-us-were-lolgulated-banking-industry... and it that it is not at all obvious to me that such compromises are necessary.

It's beyond infuriating to me when I hear a dismissive tone, since pretending these things don't have a decentralization impact removes all interest from working on the technical tools needed to bridge the gap.

I'm not sure why you think that— miners are paid for their participation. Some of them habe been extract revenue on the hundred thousands dollars a month in fees from their hashers. There is a lot of funds to pay for equipment there.

Oh, I wasn't trying to express any opinion/dislike on the inefficient use but to point out that to some extent load expands to fill capacity, and if the price is too low people will use it wastefully or selfishly.

Same cost for miners, who are paid for their resources. Not the same cost for verifiers, because not everyone has to verify everything.

In spite of all the nits I'm picking above I agree with you in broad strokes.

This is how I see the problem:

1. 1MB was just an arbitrary choice to protect the network at alpha stage. Satoshi made it clear that he intended to raise it.

2. With some calculation I think 100MB is a realistic target, to keep the whole thing reasonably decentralized, to charge a competitive transaction fee, and to offer enough profit for miners to keep the network safe.

3. To reach the 100MB target we should raise it gradually

4. We should consider to limit the growth of UTXO set if the MAX_BLOCK_SIZE is increased. For each block, calculate (total size of new outputs - total size of spent outputs) and put a limit on it.

5. We won't know the price of bitcoin in the future. Requesting miners to give up a fixed amount of bitcoin for a bigger block size could become problematic.

6. I have demonstrated that the block size could be increased with a soft-fork. I would like to know whether people prefer a cumbersome soft-fork as I suggested (https://bitcointalksearch.org/topic/auxiliary-block-increasing-max-block-size-with-softfork-283746), or a simple hard-fork as Satoshi suggested (https://bitcointalksearch.org/topic/m.15366). Either choice has its own risk and benefit.

Some of the 1MB-block supporters believe we should keep the limit forever, and move 99.9% of the transactions to off-chain. I just want to point out that their logic is completely flawed.




I've been maintaining a node with my 100Mb/s domestic connection since 2012. It takes less than 800MB of RAM now which I have 24GB. CPU load is <0.5% of a Core i5. Harddrive space is essentially infinite. I don't anticipate any problem even if everything scales up by 10x, or 100x with some optimization.

Therefore, people are not running full node simply because they don't really care. Cost is mostly an excuse. From a development and maintenance standpoint it's just easier to rely on Blockchain.info than running a full node. People are not solo mining mostly because of variance, as only pools could survive when the difficulty is growing by 20% every 2 weeks. This may sound bad but majority of the commercial players and miners are here for profit, not the ideology of Bitcoin.

At the end of the day, theoretically, we only require one honest full node on the network to capture all the wrongdoing in the blockchain, and tell the whole world. I'm pretty sure we will have enough big bitcoin whales and altruistic players to maintain full nodes as long as the cost is reasonable, say $100/month. I don't think we will hit this even by scaling up 1000x

The real problem for scaling is probably in mining. I hope Gavin's O(1) propagation would help a bit.


If the limit was 2MB, the load would be higher, but not doubled. Some people have to shutdown their bitcoind but we should still have more than enough full nodes to maintain a healthy network. Core developers may have a different development priority (e.g. optimization of network use rather than payment protocol). These are not questions of life or death.

I hate those spams too but I also recognize that a successful bitcoin network has to be able to handle much more than that.


Merge mining incurs extra cost, with the same scale property of bitcoin. I'm not sure how bitcoin mining could be substantially funded by merge mining.


I'm just trying to set a realistic target, not saying that we should raise the limit to 100MB today. However, the 1MB limit will become a major limiting factor much soon, most likely in 2 years.

I dont think the size per block matters, as long as we can improve the transactions per second cap.
As you said, 7 transactions per second is miniscule, and we should focus on solving the perfect balance for maximum speed.

Gee. And yet no one is suggesting that. Perhaps this should suggest your understanding of other people's views is flawed, before clinging to it and insulting people with an oversimplification of their views and preventing polite discourse as a result? :-/

Yes, sort of— fee requirements at major pools, monero apparently planning a hard-fork to change the rules, I'm not sure where thats standing— I'll ping some of their developers to comment.  Monero's blockchain size is currently about 2.1GBytes on my disk here.

The improvements Andrew and I came up with do not change the scalablity at all, they change the privacy (and do work for all possible sizes), and since its not scalability related it's really completely off-topic for this thread.

Blocks of 1MB combined with tree-chains could turn out to be a perfectly adequate solution. The trade space is larger than just changing the block size.

That step in the argument needs more work. Security isn't the only consideration, nor does it obviously trump all others. At some point the incremental value of additional security might not be worth it.

1MB-block supporters have 2 major arguments: decentralization and block space scarcity. By considering ONLY these 2 factors, however, the BEST solution is to limit a block to only 2 transactions: a reward transaction and a normal transaction. This will limit the block size to absolute minimal, and make sure everyone could mine with a 9.6k modem and a 80386 computer

The truth is that the 1MB limit was just an arbitrary choice by Satoshi without any considering its implications carefully (at least not well documented). He chose "1" simply because it's the first natural number. Had he chosen 2MB instead of 1MB, I am pretty sure that Bitcoin would have worked in exactly the same way as how it works now. Had he chosen 0.5MB, we might have already run into a big trouble.

We want to maximize miner profit because that will translate to security. However, a block size limit does not directly translate to maximized miner profit. Consider the most extreme "2 transactions block limit", that will crush the value of Bitcoin to zero and no one will mine for it. We need to find a reasonable balance but 1MB is definitely not a good one. Assume that we aim at paying $1 million/block ($52 billion/year) to secure the network (I consider this as a small amount if Bitcoin ever grows to a trillion market cap). The current 7tps llimit will require a fee of $238/tx, which is way too expensive even for a global settlement network among banks.

To answer the question of "what to replace the 1 MB block size limit with", we first need to set a realistic goal for Bitcoin. In long term, I don't think bitcoin could/should be used for buying a cup of coffee. To be competitive with VISA, the wiki quotes 2000tps, or 1200000tx/block, or 586MB/block (assuming 512bytes/tx). To be competitive with SWIFT, which has about 20 million translations per day, it takes 232tps, or 138889tx/block, or 68MB/block. Divide them by the $1 million fee/block, that would be $0.83/tx and $7.2/tx respectively. A fix rate of $7.2/tx is a bit high but still (very) competitive with wire transfer. $0.83/tx is very competitive for transactions over $100 of value. I think a reasonable choice, with the implications for centralization considered, would be around 100MB/block. That takes 1.5Mb/s of bandwidth in a perfect scenario. That would be a better equilibrium in technical and economical terms.

Not sure about whether bloat has been addressed to the extent that it would need to be in Bytecoin (BCN), but I do know that recent updates dropped the default fee from 10 BCN to .01 BCN (1000 times cheaper), but the updates also provide that the user specifies what the fee will be so that the higher the fee, the faster the transaction makes it in, like this:

In this example, I've shown the general format for the transfer command and I've shown the -f (fee) as 10 bytecoin:


My understanding is that gmaxwell and andytoshi (et. al.?) have come up with "substantial cryptographic improvements" to the BCN system which potentially are a "pretty straight forward to add to Bitcoin" as per gmaxwell, see:  https://download.wpsoftware.net/bitcoin/wizardry/brs-arbitrary-output-sizes.txt and previous comment(s) cited in this thread.  However, I still have my (unanswered) questions, to wit:

How would this output-encoding scheme work realistically for something of *every possible size?*  

Assuming this were applied to bitcoin as an option [much as SharedCoin is in blockchain.info], wouldn't it still come at a cost both in terms of size of the data corresponding to whatever transactions involved the scheme in the cases where users choose to utilize it, as well as corresponding additional fee(s)?  

How are the scalability issue(s) addressed?

Glancing at block explorers for Monero and ByteCoin... I'm not seeing crippling bloat right now. I see lots of very-few-transactions blocks.

Glancing at recent release notes for ByteCoin, it looks like transactions were not being prioritized by fee, which is a fundamental to getting a working fee market.

Have Monero and ByteCoin fixed the bloat problem, or did the transaction spammers just get bored and go away?

I just read this whole thread and found it very interesting. After reading it, I decided to go back and re-read this:

Output Distribution Obfuscation (posted July 16, 2014), by Gregory Maxwell and Andrew Poelstra. (Involves use of cryptonote-based bytecoin (BCN) ring signatures, described as a possibility for bitcoin: "Using Bytecoin Ring Signatures (BRS), described at cryptonote.org, it is possible to disguise which of utxos is being referenced in any given transaction input. This is done by simply referencing all the utxos, then ringsigning with a combination of all their associated pubkeys."
http://download.wpsoftware.net/bitcoin/wizardry/brs-arbitrary-output-sizes.txt
(This, in part, proposes "an output-encoding scheme by which outputs of *every possible size* are created alongside each real output (...) further requir(ing) that the "ghost outputs" are indistinguishable from the real ones to anyone not in possession of the outputs' associated private key. (With ring signatures hiding the exact outputs which are being spent, even spending a real output will not identify it as real.)" In this scenario, ghost outputs are chosen randomly, and users improve anonymity when selecting ghost outputs " by trying to reuse n for any given P, V."

(Background to this:)

So, my questions:

How would this output-encoding scheme work realistically for something of *every possible size?*   And assuming this were applied to bitcoin as an option [much as SharedCoin is in blockchain.info], wouldn't it still come at a cost both in terms  of size of the data corresponding to whatever transactions involved the scheme in the cases where users choose to utilize it, as well as corresponding additional fee(s)?  How are the scalability issue(s) addressed?  (Please also explain from both the scripting vs. no-scripting scenarios.)