Re sucking in more users, if it is precisely an increase in the number of users that threatens us with all these problems in the first place, maybe it is time that we stopped being so eager to suck in more users. We are working nicely so far with the users we already have. If more want in that is going to cost us a lot, so using freebies that cost the whole population of full nodes, including miners, a lot, precisely to attract more new users faster, does not seem that great an idea.
-MarkM-
Topic: How a floating blocksize limit inevitably leads towards centralization - page 13. (Read 71590 times)
February 21, 2013, 04:35:56 AM
February 21, 2013, 04:33:05 AM
You will probably not see miners accepting blocks above 250KB until 250KB blocks filled with only paying TX appear.
Yep. I mentioned that in my prediction for what will happen to the soft limit.
February 21, 2013, 04:26:13 AM
Again, we've already bumped up against the 250KB soft limit on several occasions; and amazingly, none of the miners have chosen to comment out that line of code.
It's been said a few times in this thread that there is little incentive to change the code as long as the block subsidy (25 BTC reward) is orders of magnitude larger than the total of transaction fees in the block.
I'm pretty sure that the 250KB limit has never been broken to date.
But it has. If you will go back to one of the earlier posts in this thread you will see that there is at least one mining pool with customized software that does not have the 250kb limit. You can also scan through the block chain and find plenty of blocks larger than 250kb. There are also a good number that have no transactions at all. The presumption is that these came from a miner whose only interest was in collecting the subsidy. As long as the subsidy is orders of magnitude larger than the total of transaction fees, this type of behavior will persist (as has been stated numerous times already).
Show me why you believe that this is so, not just your assumptions about what the future holds using overly simplified mental models.
Was this earlier post insufficient?
If we want to cap the time of downloading overhead the latest block to say 1%, we need to be able to download the MAX_BLOCKSIZE within 6 seconds on average so that we can spend 99% time hashing.
At 1MB, you would need a ~1.7Mbps connection to keep downloading time to 6s.
At 10MB, 17Mbps
At 100MB, 170Mbps
and you start to see why even 100MB block size would render 90% of the world population unable to participate in mining.
Even at 10MB, it requires investing in a relatively high speed connection.
At 1MB, you would need a ~1.7Mbps connection to keep downloading time to 6s.
At 10MB, 17Mbps
At 100MB, 170Mbps
and you start to see why even 100MB block size would render 90% of the world population unable to participate in mining.
Even at 10MB, it requires investing in a relatively high speed connection.
It seems that all of your relevant points have been answered at least once so now I'm just being redundant, and no longer adding value to the conversation.
February 21, 2013, 04:25:03 AM
If I recall correctly, pretty much the only, or at least the core, argument in favour of allowing any free transactions at all is as a propaganda / public-relations scheme to suck in more users. Basically a free loss-leader.
Aren't we getting to a point where anyone who wants to blow wealth on giving out free loss-leaders can do so themselves, simply by paying the damn fee themselves or lowering the prices of what they are selling by the amount of the fees involved in the actual selling of the thing?
For example, merchant terminals could add an input from their own float account to the transaction before presenting it to the buyer for signing of the buyer's input, or publish separately a transaction with fee that takes one or more recent payments of customers as input. (This latter though would make verifying blocks harder, since verifiers would not be able to tell whether a free transaction has sneaked into the block simply by checking whether the transaction contains a fee. So maybe free transactions are not yet being rejected by verifiers simply because verifiers are currently too simplistic to recognise cases in which the fee is being provided by a dependent transaction? We could however, make each block able to be verified as to its lack of free-riders by requiring any such method of providing its fee be in the same block.)
-MarkM-
Aren't we getting to a point where anyone who wants to blow wealth on giving out free loss-leaders can do so themselves, simply by paying the damn fee themselves or lowering the prices of what they are selling by the amount of the fees involved in the actual selling of the thing?
For example, merchant terminals could add an input from their own float account to the transaction before presenting it to the buyer for signing of the buyer's input, or publish separately a transaction with fee that takes one or more recent payments of customers as input. (This latter though would make verifying blocks harder, since verifiers would not be able to tell whether a free transaction has sneaked into the block simply by checking whether the transaction contains a fee. So maybe free transactions are not yet being rejected by verifiers simply because verifiers are currently too simplistic to recognise cases in which the fee is being provided by a dependent transaction? We could however, make each block able to be verified as to its lack of free-riders by requiring any such method of providing its fee be in the same block.)
-MarkM-
February 21, 2013, 04:24:05 AM
Please don't forget that what attracted most of us the users is the decentralized nature of bitcoin. We want a secure p2p network that serves as a store of value.
I feel your pain - managing the blockchain with increasing transactions per second will be a pain - but the top priority should always be avoiding centralization.
I feel your pain - managing the blockchain with increasing transactions per second will be a pain - but the top priority should always be avoiding centralization.
February 21, 2013, 04:17:42 AM
Sure it might impact miners who are too small to have direct dedicated high bandwidth pipes to the top 51% miners, but the top 51% miners will not even notice if every one of you irrelevant non-mining nodes all stick your heads in the sand at every block on the chain. You just don't matter to the big boys, the most you could do would be help the big boys to shake the small players out of the game faster.
Indeed, if 51% of the miners decide to work together we will have a problem. February 21, 2013, 04:13:47 AM
And what about having nodes build in some sorts of max block size relay delay, so they can punish miners that in their eyes produce too large blocks?
That's very similar to what I proposed earlier, except the relay delay would be infinity.
Yes I know, but I guess that could bring you in a situation that you won't be aware of any blocks in the blockchain because nodes around you won't relay you the blocks.
February 21, 2013, 04:12:16 AM
Only someone who assumes that only a hard limit is actually the only limiting factor. Again, we've already bumped up against the 250KB soft limit on several occasions; and amazingly, none of the miners have chosen to comment out that line of code. Obviously there is some kind of cost to the miners for doing so, or a presumption of a cost, that would exceed the benefits to the miner for doing so. One such cost is simply the work involved in commenting out that line of code and recompiling their mining programs over a relatively few number of tiny transaction fees. That condition is bound to change, certainly. This does not mean that miners will start to ignore the soft limit. It is a community convention that, while not presently enforceble by the running protocol, is still enforceable by the community. If some miner actually did this prior to the community deciding to raise that soft limit, the offender would prompt some kind of response. What kind of response, I won't hazard a guess; but those miners certainly don't desire to prompt the community to include rules more strict than what I'm proposing. It's not in the interest of the miners to invoke a response.
I don't really buy the community argument. I have seen blocks above 250KB btw.
If you look at the total fees for blocks hitting 250KB, it is almost never above 1BTC, and these blocks still include many free transactions, so it is safe to assume that most if not all the transactions that were left out were free ones. Why bother commenting out that line of code when it brings no additional benefit to you?
You will probably not see miners accepting blocks above 250KB until 250KB blocks filled with only paying TX appear.
It's been a while, but IIRC, the soft limit only counts fee paying transactions. I certainly could be wrong about that, but if not, a block with an actual size over 250KB is possible without violating the soft limit. Any such propagation rule would have to consider the blocksize in the same manner, and perhaps also add a fudge factor.
February 21, 2013, 04:08:29 AM
And what about having nodes build in some sorts of max block size relay delay, so they can punish miners that in their eyes produce too large blocks?
That's very similar to what I proposed earlier, except the relay delay would be infinity.
February 21, 2013, 04:06:36 AM
Only someone who assumes that only a hard limit is actually the only limiting factor. Again, we've already bumped up against the 250KB soft limit on several occasions; and amazingly, none of the miners have chosen to comment out that line of code. Obviously there is some kind of cost to the miners for doing so, or a presumption of a cost, that would exceed the benefits to the miner for doing so. One such cost is simply the work involved in commenting out that line of code and recompiling their mining programs over a relatively few number of tiny transaction fees. That condition is bound to change, certainly. This does not mean that miners will start to ignore the soft limit. It is a community convention that, while not presently enforceble by the running protocol, is still enforceable by the community. If some miner actually did this prior to the community deciding to raise that soft limit, the offender would prompt some kind of response. What kind of response, I won't hazard a guess; but those miners certainly don't desire to prompt the community to include rules more strict than what I'm proposing. It's not in the interest of the miners to invoke a response.
I don't really buy the community argument. I have seen blocks above 250KB btw.
If you look at the total fees for blocks hitting 250KB, it is almost never above 1BTC, and these blocks still include many free transactions, so it is safe to assume that most if not all the transactions that were left out were free ones. Why bother commenting out that line of code when it brings no additional benefit to you?
You will probably not see miners accepting blocks above 250KB until 250KB blocks filled with only paying TX appear.
February 21, 2013, 04:06:31 AM
And what about having nodes build in some sorts of max block size relay delay, so they can punish miners that in their eyes produce too large blocks?
there are things that can be done to disincentive miners from attempting to break that limit when it suits them. Did you not understand what I was trying to communicate concerning adding a 'soft limit' propagation rule to the clients? Truly harmless to the network and the client, and only somewhat damaging to the miner, unless a significant enough of a minority of clients participate in this rule, and the miner with an overlarge block ends up with an orphaned block as a direct result. It doesn't have to work every time, just enough for the miners to notice.
It is also truly harmless, that is, utterly uneffective in influencing them in any way, to the big miners.
Sure it might impact miners who are too small to have direct dedicated high bandwidth pipes to the top 51% miners, but the top 51% miners will not even notice if every one of you irrelevant non-mining nodes all stick your heads in the sand at every block on the chain. You just don't matter to the big boys, the most you could do would be help the big boys to shake the small players out of the game faster.
-MarkM-
February 21, 2013, 04:04:55 AM
there are things that can be done to disincentive miners from attempting to break that limit when it suits them. Did you not understand what I was trying to communicate concerning adding a 'soft limit' propagation rule to the clients? Truly harmless to the network and the client, and only somewhat damaging to the miner, unless a significant enough of a minority of clients participate in this rule, and the miner with an overlarge block ends up with an orphaned block as a direct result. It doesn't have to work every time, just enough for the miners to notice.
It is also truly harmless, that is, utterly uneffective in influencing them in any way, to the big miners.
Sure it might impact miners who are too small to have direct dedicated high bandwidth pipes to the top 51% miners, but the top 51% miners will not even notice if every one of you irrelevant non-mining nodes all stick your heads in the sand at every block on the chain. You just don't matter to the big boys, the most you could do would be help the big boys to shake the small players out of the game faster.
-MarkM-
What if those big miners also had that same soft limit rule? The mining pools of BTCGuild, 50BTC and Slush still collectively represent over 50% of the mining power of the bitcoin network, the rise of ASICs notwithstanding. While it's true that small miners can leverage their bandwidth best by directly connecting to these three mining pools. So, do these three mining pools have an economic incentive to disregard the community will and exceed the soft limit? On the one hand, yes; for they stand to gain more transaction fees just as you guys presume. On the other hand, no, for they depend upon their membership for that hashing power; and it's that hashing power that creates the other incentives to begin with. All three pools depend upon their good reputations with their membership in order to dominate. If even a minority of their membership disagreed with the owners of the pools exceeding the soft limits without the community's overall consent, the pools will suffer hashrate loss. Thus, all three of these pools actually have a greater incentive to participate in the convention set by the community, and resist any players that would not abide. They are, in fact, most likely among nodes to impose such a propagation rule upon their own systems; partly because of they control a large percentage of the mining power.
In fact, I would here propose that, at the same time that we are raising the soft limit to 500KB, we also ask these major mining pools to include my soft limit propagation rule. Just these three alone would make such a rule quite effective.
February 21, 2013, 04:02:12 AM
And what about having nodes build in some sorts of max block size relay delay, so they can punish miners that in their eyes produce too large blocks?
February 21, 2013, 03:57:37 AM
International bandwidth is much more complicated than this, with respect to a p2p network such as Bitcoin. Much like bittorrent, the bandwidth for Singapore is functionally replicated for all of the Bitcoin clients in all of Singapore, so it's not true that the international connection is divided among the many peers all trying to download the same block, it's effectively shared data as if there were a locally available seed on Bittorrent. Granted, it's not the same as saying that all of Singapore could be regarded as one node, and then the block replicates across the entire nation-state from one copy; but once one copy of the block has made it across the bottleneck, the local bandwidth effectively becomes the limiting factor to propagation.
Sure but you would be slower by atleast the time taken by the fastest node in your country. To mine profitably, you would want to be well connected with all other mining nodes worldwide, so international bandwidth matters a great deal. Getting information 2nd hand potentially doubles your network overhead timewise.
A serious miner would want to have as much upstream bandwidth as possible to get his block out to as many relay nodes as fast as possible, so the upstream requirements should ideally match the download requirements.
Ofcourse i believe some of these issues can be optimised.
For example you could request a block with only the hashes of the TX it contains, and you use the transactions in your not-included TX pool instead of downloading them. For your average transaction this wont help much, but for transactions with many inputs and outputs, this could help reduce burst BW requirements significantly.
The more i think of it, the more it seems to me that even at 10MB, most residential miners without an expensive connection will be out of the game. And if you are a small miner who was hashing 2-10ghz, the cost of a more expensive connection will be more than the profit you bring in making it unviable.
February 21, 2013, 03:53:45 AM
I think that this might be a great way to move the soft limit, but I still think we need to have a (much higher) hard limit.
Did the earlier posts not sink in? The soft limit is not a proper limit, it is merely a thin road block designed to function as an early warning system. There's no "moving the soft limit." See my earlier post about analyzing the block chain to see the effect of the soft limit on miner behavior. As far as you are concerned, you should pretend as if the soft limit doesn't exist.
As for having a much higher hard limit, again did you not read the earlier posts? A 10 megabyte hard limit (which isn't even "much higher" in your terms) would raise the minimum bandwidth requirements to 17Mbps. Do you have any idea how many miners would be knocked off the network? In an earlier post you were just explaining the limitations of international bandwidth and also that solo miners would like to actually use their internet connections while they are mining. How can you then claim that we need a much higher hard limit, with its accompanying much higher minimum bandwidth requirements? Go back and re-read the chapter and then try answering the test questions again.
Don't be a dick. I read your posts. I believe that I understood them, but apparently you didn't understand my arguments. The soft limit is a voluntary limit at present, but there are things that can be done to disincentive miners from attempting to break that limit when it suits them. Did you not understand what I was trying to communicate concerning adding a 'soft limit' propagation rule to the clients? Truly harmless to the network and the client, and only somewhat damaging to the miner, unless a significant enough of a minority of clients participate in this rule, and the miner with an overlarge block ends up with an orphaned block as a direct result. It doesn't have to work every time, just enough for the miners to notice.
would that basically be like checks and balances? miners are checked from overly large blocks?
February 21, 2013, 03:50:41 AM
The soft limit is a voluntary limit at present, but there are things that can be done to disincentive miners from attempting to break that limit when it suits them.
Why on earth would we want to discourage miners from breaking the soft limit? The soft limit was intended to be broken! I'll repeat that...it is intended that eventually miners will remove the soft limit. I explained the reasoning behind the soft limit in this post (anyone feel free to correct me if I got it wrong).
Quote
Did you not understand what I was trying to communicate concerning adding a 'soft limit' propagation rule to the clients?
Yeah I understood it. It was based on an erroneous understanding of the soft limit and its intention. And, it does nothing to improve the current situation. So I did not comment on it.
Then comment upon it now, for you are working on an erroneous understanding of my proposal and it's implications.
Well then, the way that he is presenting it becomes a strawman argument. Could you show me why this is inevitable? Are there really no other factors? Does an increase in resource demands not also incentivize the bandwidth starved mining operation to invest in better infrastructure?
Now you're saying exactly what he was saying. That Bitcoin mining and validation should have ever increasing minimum requirements of bandwidth, CPU, and storage. Every time the requirements are jacked up, it will put Bitcoin out of reach for some fraction of miners. Sure, some miners will remain (those who have the capital to invest in new equipment). But at each increase, there will be fewer miners. Continuing to its logical conclusion, mining will be an activity performed only by large entities with the capital necessary for the big up-front investment in equipment (data center, dedicated fiber line, server farms). That's exactly the example he was making with Google, et. al. being the sole providers.
That is a static view, and not one that is supportable. With each smaller increment, the percentage of marginal miners forced out would be smaller than otherwise, and thus the likelyhood that they would be able to return as both their own local infrastructure and the average infrastructure improves. The limits exist for several reasons, not the least of which is artificial scarcity, but we still have to balance the future against the present. Even without my proposed propagation rule, I'm pretty sure that the 250KB limit has never been broken to date. Why is that? Do you presume that it's only because there are not enough transactions to justify it to the miners? I would think that part of it is that the soft limit is an agreed upon rule, to be exceeded once that has been agreed upon also. My rule would only add a bit of risk to ignoring this rule that doesn't presently exist. I don't doubt that, eventually, some miners are going to test the soft limit. I want some of those tests to fail.
Lets think for a moment of all those Jalapinos that once upon a time sounded like they could decentralise hashing power into every home...
That's conjecture. Show me why you believe that this is so, not just your assumptions about what the future holds using overly simplified mental models. Thus far you have failed to present an argument for this perspective. I'm not even willing to claim that it's incorrect, only that it remains unsupported. Can you really not imagine any other incentives that would contradict the "logical conclusion" you have come to?I can think of at least three.
Quote
Someone suggested a "really high limit" for the block size of one gigabyte. Now we know that would require dedicated bandwidth of 1.7Gbps (yeah that's right, giga). This doesn't sound practical at all, and I cannot take anyone seriously who would give this a shred of legitimacy.
Only someone who assumes that only a hard limit is actually the only limiting factor. Again, we've already bumped up against the 250KB soft limit on several occasions; and amazingly, none of the miners have chosen to comment out that line of code. Obviously there is some kind of cost to the miners for doing so, or a presumption of a cost, that would exceed the benefits to the miner for doing so. One such cost is simply the work involved in commenting out that line of code and recompiling their mining programs over a relatively few number of tiny transaction fees. That condition is bound to change, certainly. This does not mean that miners will start to ignore the soft limit. It is a community convention that, while not presently enforceble by the running protocol, is still enforceable by the community. If some miner actually did this prior to the community deciding to raise that soft limit, the offender would prompt some kind of response. What kind of response, I won't hazard a guess; but those miners certainly don't desire to prompt the community to include rules more strict than what I'm proposing. It's not in the interest of the miners to invoke a response.
February 21, 2013, 03:39:13 AM
there are things that can be done to disincentive miners from attempting to break that limit when it suits them. Did you not understand what I was trying to communicate concerning adding a 'soft limit' propagation rule to the clients? Truly harmless to the network and the client, and only somewhat damaging to the miner, unless a significant enough of a minority of clients participate in this rule, and the miner with an overlarge block ends up with an orphaned block as a direct result. It doesn't have to work every time, just enough for the miners to notice.
It is also truly harmless, that is, utterly uneffective in influencing them in any way, to the big miners.
Sure it might impact miners who are too small to have direct dedicated high bandwidth pipes to the top 51% miners, but the top 51% miners will not even notice if every one of you irrelevant non-mining nodes all stick your heads in the sand at every block on the chain. You just don't matter to the big boys, the most you could do would be help the big boys to shake the small players out of the game faster.
-MarkM-
February 21, 2013, 03:10:24 AM
The soft limit is a voluntary limit at present, but there are things that can be done to disincentive miners from attempting to break that limit when it suits them.
Why on earth would we want to discourage miners from breaking the soft limit? The soft limit was intended to be broken! I'll repeat that...it is intended that eventually miners will remove the soft limit. I explained the reasoning behind the soft limit in this post (anyone feel free to correct me if I got it wrong).
Well then, the way that he is presenting it becomes a strawman argument. Could you show me why this is inevitable? Are there really no other factors? Does an increase in resource demands not also incentivize the bandwidth starved mining operation to invest in better infrastructure?
Now you're saying exactly what he was saying. That Bitcoin mining and validation should have ever increasing minimum requirements of bandwidth, CPU, and storage. Every time the requirements are jacked up, it will put Bitcoin out of reach for some fraction of miners. Sure, some miners will remain (those who have the capital to invest in new equipment). But at each increase, there will be fewer miners. Continuing to its logical conclusion, mining will be an activity performed only by large entities with the capital necessary for the big up-front investment in equipment (data center, dedicated fiber line, server farms). That's exactly the example he was making with Google, et. al. being the sole providers.
Certainly I will no longer be able to mine with my one Jalapeno (if it ever ships). The idea of hundreds of thousands of individuals solo mining with their commodity ASICs (like the Jalapeno) will go down the drain. Which he also pointed out in his post:
Lets think for a moment of all those Jalapinos that once upon a time sounded like they could decentralise hashing power into every home...
Someone suggested a "really high limit" for the block size of one gigabyte. Now we know that would require dedicated bandwidth of 1.7Gbps (yeah that's right, giga). This doesn't sound practical at all, and I cannot take anyone seriously who would give this a shred of legitimacy.
February 21, 2013, 03:08:27 AM
I think that this might be a great way to move the soft limit, but I still think we need to have a (much higher) hard limit.
Did the earlier posts not sink in? The soft limit is not a proper limit, it is merely a thin road block designed to function as an early warning system. There's no "moving the soft limit." See my earlier post about analyzing the block chain to see the effect of the soft limit on miner behavior. As far as you are concerned, you should pretend as if the soft limit doesn't exist.
As for having a much higher hard limit, again did you not read the earlier posts? A 10 megabyte hard limit (which isn't even "much higher" in your terms) would raise the minimum bandwidth requirements to 17Mbps. Do you have any idea how many miners would be knocked off the network? In an earlier post you were just explaining the limitations of international bandwidth and also that solo miners would like to actually use their internet connections while they are mining. How can you then claim that we need a much higher hard limit, with its accompanying much higher minimum bandwidth requirements? Go back and re-read the chapter and then try answering the test questions again.
Don't be a dick. I read your posts. I believe that I understood them, but apparently you didn't understand my arguments. The soft limit is a voluntary limit at present, but there are things that can be done to disincentive miners from attempting to break that limit when it suits them. Did you not understand what I was trying to communicate concerning adding a 'soft limit' propagation rule to the clients? Truly harmless to the network and the client, and only somewhat damaging to the miner, unless a significant enough of a minority of clients participate in this rule, and the miner with an overlarge block ends up with an orphaned block as a direct result. It doesn't have to work every time, just enough for the miners to notice.
February 21, 2013, 03:01:48 AM
I don't know that I'd agree that a 1% change per difficulty adjustment is going to be enough.
This is such a contrived situation that I have to reject it as a strawman argument.
It is not contrived, it is precisely what would happen if the minimum bandwidth requirements were jacked up by a factor of 100 (which happens when you increase the maximum block size to 100 megabytes). I think he's just explaining it in terms that a non-technical person might understand.
Well then, the way that he is presenting it becomes a strawman argument. Could you show me why this is inevitable? Are there really no other factors? Does an increase in resource demands not also incentivize the bandwidth starved mining operation to invest in better infrastructure? This is over simple, and does not consider that the infrastructure itself is not static. There are also other resource factors to be considered, which may or may not be of greater influence. I have, for years, predicted that the city of Reykjavík, Iceland will one day become a center of the Bitcoin mining industry; for the simple reason that they have both relatively low electric rates (due to the blessings of geothermal power) and a very high domestic heating demand. This has not yet come to pass, but it's not because Reykjavík has substandard international bandwidth connections, because that's not so.