Topic: Miners that refuse to include transactions are becoming a problem - page 7. (Read 16972 times)

The getwork size is the same, whether transactions are included or not.  The quantity of getworks is the same too, whether or not transactions are included.

If your calculations prove anything, then that its not a botnet.  "Because its too difficult to manage even a moderade sized botnet (ask Tycho)" - your own words paraphrased.

The fact that transactions are not included does not add one single jota of support towards/against the botnet theory.

If what you're saying is true, and I'm not knowledgeable enough about the various BTC miner software versions to know, then gmaxwell's proposal would not work at all, and would be a huge pain since it requires a protocol change.

It seems to me that if you're right, then it's most likely a bug, since it's inconceivable that anyone would purposefully omit tx unless as T&D stated it was to maybe to reduce server loads on an already stretched system. Unless "mystery" steps forward, assuming it's even one person, we couldn't know their intentions.

Also, if this is the case, then the method proposed by Gavin/Revalin would be necessary. If a block has far fewer tx than the average, then it gets delayed, and if it has only 1 or none it may be excluded altogether.

There are several reasons why the "I'm entitled to charge whatever I want" argument has no weight.

1: I can charge $1million USD for a single grain of sand on ebay. Nobody will buy it (I would hope), but I still have to pay for posting it at auction. In BTC if you do the same thing you still get paid, at the expense of everyone using the system.

2: Gavin/Revalin's system still gives miners a good deal of room in choosing what they want to charge. As long as the threshold for delay isn't too restrictive, and the miner is charging a reasonable enough fee that they actually get some business, they shouldn't be excluded or delayed. Unless your business model is crap, there's no penalty. Also, since it's not actually part of the protocol, there's no obligation of any party to delay or reject a block, nor any obligation to accept any block, and it could potentially be adjustable by the user. I think, however, that the sentiment of most miners is that they also are not interested in supporting cheapskates.

3: The blockchain is 2 fucking gigabytes, and climbing at an absurd rate. If I have to download a 2GB blockchain, and 15% (or more) of all the blocks are empty or nearly empty, then why have I wasted my time downloading 300MB+ of that? The only purpose of the blockchain is to be able to securely verify transactions that have occurred, and empty or nearly empty blocks add filesize without contributing to the purpose of the data. At the rate things are going it's already going to take further development just to figure out how to keep the size reasonable, and the last thing we need is a bunch of retards being paid in inflation to spam it up faster.

Finally, if it is just a bug in one of the clients rather than someone trying to take shortcuts, then implementing a screening system would make it obvious to the users of that client that something is wrong. In that case it probably wouldn't take long for it to be fixed. If nothing is done, then we have no way of knowing.

This is an excellent argument for pools disallowing clients which aren't turning in enough results to justify their bandwidth.  If low hashrate clients (botnet or not) want to participate, they can have one guy get work and distribute it to the others.  Letting them collect 50BTC per null block to save the pool the awkward responsibility of turning away unprofitable workers is completely backward.

I consider gmaxwell's approach to be short-term.  It will probably solve the current problem - whoever this is will notice their blocks are being orphaned and do something to fix it - and while it's intrusive (it requires protocol changes), it has little chance of other collateral damage.

I'm looking at ways to solve it in a more general manner to improve the long-term security of the network.  My proposed change is much simpler but the consequences are complicated.  I don't think D&T's fears will actually happen in reality - I think the economic incentives will actually result in overall more fees being paid and more latitude for miners to set them - but it's not easy to model, and obviously we disagree.

I also don't consider it a foregone conclusion that it's a botnet.  They have to be on the network to receive prior blocks and transmit new ones.  Why would someone go 80% of the way when they could implement a 100% functional pool (or just use any public pool) and draw less attention?  I don't think it's a deliberate attack either because there are much better ways to leverage that much hashpower.

I have no idea what their motives are, and honestly it doesn't matter.  I just want to create the economic incentive where they must fully participate or not get paid, which will definitively, and permanently, solve the problem.

This times 1000.

The 50 BTC block reward is for including transactions. It is money that comes from everyone that owns Bitcoins (in the form of inflation) and you should provide the required service. If we would change the protocol to force you to do so or not get paid this would not be socialistic like some propose because it is not mandated by a central government but in fact by a majority of the people.

In a dumb miner - pool model the fact that the miner doesn't do the workload doesn't mean the workload doesn't exist.

One getwork is ~400 bytes.  Some have speculated mystery has 1.8 million nodes.  To use a traditional full pool server the bandwidth requirements alone would be 5.7Gb and 1.8 million connects every long poll.  Say he staggered those getworks to supply all nodes with work over a 10 second period (less revenue btw).  You are still talking about almost 200K simultaneous connections, 750Mbps, and enough computing power to generate 200K block headers a second.

If you want more insight ask Tycho what kind of load even small botnets do to a pool server.  Now with enough computing power (we are talking a dozen or so high end load balanced servers on a 1 Gbps connection) a 1.8 million node botnet "could" operate in a pool-miner model.  However now compare all that cost vs the incremental reward of .... ready for it ....about 2 cents per block.

That's a matter of opinion.  Bear in mind that including transactions into a block does add some nominial amount of additional processing load for the miner, so it's not free for him.  This null-block miner might simply be a pool that has a higher fee requirement for inclusion than the common amount.  That is his right.  And the only response for the other miners is to increase their own mining power.  This "arms race" benefits the security of the blockchain, even if a not-insignificant minority of blocks are null-set.  I'd be more productive if the miner in question published the minimum fee expected, if that is the case, but it's not necessary.

Becasue we already pay you greedy miners with 33% yearly inflation.

Not enough?

To it is my right to charge what I consider fair but if I do you will exclude the block and I lose everything.  Honestly you think miner's will support this.

It would be like saying:
You can open a restaurant and charge what you want but if 5 people are waiting and you don't feed at least 25% of them you will be shutdown.  Oh an BTW some of them want to eat for free and some want to eat for a penny.  But sure you can charge what you want.

No dice.  Any system that forces me as a miner to use MY HASHPOWER purchased with MY CAPITAL and running on MY ELECTRICITY to include fee and "essentially free" transactions is a non-starter.  The "solution" is worse than the problem.


You "encourage me" but at the same time advocate a system which would orphan my blocks if I do what you are encouraging.  Thanks for your encouragement. 

Even a 1 satoshi minimum would get people to at least start paying fees, and pave the road for the future where free/1-satoshi transactions are slow (just a few miners supporting them) and reasonable fees are required for fast processing.  It's probably helping your goals on the whole.

I am not advocating that.  I am adovcating I set the minimum fee for my hashing power ONLY with no changes to the protocol and you are advocating orphaning my blocks if I do that.

It's not a DOS vulnerablility unless the attacker can mine nearly all blocks empty, in which case he already has 50%+ of the network and can pretty much do anything.  If he has less than a majority of the network, he can't prevent valid transaction processing every 20 minutes or less anyway.  So no, that's not an issue in this context.

And how would it make 51% attacks easier for others?  Again, I think this is an over-reaction.  If you think that this is really a problem, work on a patch that allows miners to simply not forward valid transactions that flag on a user's personal blacklist based upon address or whatever other criteria.  If enough miners agree that this is an issue, they will incorporate your patch and the null-miners will be sanctioned by the community without a community wide action need be taken.

Very true.  +100

The same can be said about the accusation of operating a "botnet", while the facts do not support that theory.

People here speculate about how not processing transactions would benefit a botnet, and then conclude that it must be a botnet.

This lacks both facts and logic.  For the lack of facts, consider this:

The node at 88.6.216.9 did have a complete blockchain and did verify transactions, and also relay them.  It just did not include them into new blocks.  For an example of a relayed transaction, see http://www.blockchain.info/tx-index/3091519/ca264af6271cc40bf8b297e13fc2489bd9564dd18b9e60a1bd4d19046086995b

When it was still operating at 95.120.241.167, it even did include transactions into blocks.  Both free and paid.  For an example, see https://www.blockchain.info/block-height/166667 (the last transaction is a free one).

It stopped including transactions in March.

The decision to not include transactions was made after support for transactions was already in place.  It is a political decision, not a technical necessity.

For the lack of logic, just think about how your own mining client works.  Most of you readers (yes, not all, but most), are mining at a traditional pool.  You consume very little bandwidth, you dont need to store the blockchain and you dont need to know or verify any TXs.  Yet your mining effort works towards blocks with transactions included.

You work successfully under exactly the same conditions, as (some of) you claim would force a botnet to exclude transactions.  (Some of) you say, that downloading the blockchain would make botnet victims suspicious, yet you do not need to download it yourself!  (Some of) you say, the steady influx of transactions would need lots of extra CPU processing and extra network bandwidth, yet you do not need to do that yourself!  (Some of) you even say that a full bitcoind must be in place, yet you do not need it on your own miners!

Your own mining client shows how the technical "problems" can be solved, and source code is available for everyhing (both mining client and pool service).

The assumption of the technical benefit is just plain false.  The only thing that supports the idea of a botnet, is the ever repeating posts here on bitcointalk.  Those who jump to early conclusions and keep reinstating this false idea, are the ones who do damage to bitcoin.

I wonder if you stand up as quick to apologize as you do to condemn.

While it's possible that it's some minority client malfunctioning, 1.8m users and/or 16% of the total network processing power is pretty extreme. The potential advantages for running a stripped down botnet make that the most likely scenario, and whether or not that's the case 16% of the network isn't contributing to the rest while collecting all the benefits.

Also, I think people are still misunderstanding what has been proposed. While Gavin and others have proposed systems which would omit or delay "cheap" blocks, gmaxwell's proposal is simply that in order for a block to be valid they have to prove that they have access to the blockchain. It doesn't mean they have to include any tx if they don't want to, just that they have to prove that they've seen recent valid tx that have been posted to the chain.

This would be trivial for legitimate clients, official or otherwise, but for a botnet running without having the blockchain it would be impossible to fabricate, so they'd be forced to shut down or play fair. The only time "tx lists" would even be used at all is if:

1: Some other miners wanted to "lazy mine" with no blockchain, and are willing to trust full nodes to provide valid tx for them to mine.

AND

2: Miners willingly want to give up their tx to others, or have tx they don't want, and would like a facility for dumping them on lazy miners.

However, while this feature isn't necessary for booting a botnet, it would weaken security since it requires that the "dumb nodes" blindly trust full nodes, so "dumb nodes" could be used to forward an attack. For a normal, fully equipped miner, tx forwarding would be basically useless unless they have a major incentive to dump tx.

I agree that we don't need to centrally control all of the miners, and that we don't have any business telling them which/how many tx to include or not. However, I think there's plenty of good reasons for preventing someone from sitting around doing getwork without actually having the blockchain, but it's easy to prove whether this is the case or not without putting restrictions on including tx. Revalin's suggestion (or similar) would only be relevant under very different circumstances, which are much more unlikely to occur than someone freeloading with an anonymous botnet.


The problem here is that the "Attacker" most likely isn't malicious in the sense of trying to take the network down. They just want free money from stolen computing power, and so far they're getting it. The real problem is that 15% of the BTC that would be going towards miners who actually maintain their rigs are going to someone who more than likely does not.

The other problem is, how the hell do you "block" 1.8 million unconnected stolen computers? If I started manually doing that now, by the time I was dead I'd still be nowhere near caught up. Requiring proof that you've seen the blockchain recently would automatically exclude anyone trying to cheat (at least via that route). Alternatively, if other people start figuring out they can make free coin by running a botnet sans the blockchain, pretty soon "mystery" will make up more like 30% of the network, and even more funds will be diverted from maintaining the legitimate side of the network. At the moment the reward for parasitic mining outweighs any potential losses from not including tx, and that's not likely to change much for at least 10 years.

Checking to see that a miner is using the blockchain is fairly trivial. If they're actually running the blockchain then there's no incentive for them to exclude tx (at the moment), and no need to babysit them to make sure they include some minimum number of tx, either (that's their choice, however arbitrary). The problem is the incentive for running without it, pure and simple.

+100

Of course you should be allowed!  That's well established as a miner's right.  Similarly it is also a relay node's right to exclude blocks they consider insufficient.  Check and slightly imperfect balance.

Relay nodes already enforce some minimum fees; those were actually reduced last year.  Right now it's probably best for free txes to be as low as possible - low friction will help growth in ways much more significant than some 0.01BTC fees.  That said I encourage you to institute this policy.  The completely homogenous one-formula-fits-all isn't good for the network.  I think we would be much better off with a gradient of minimum fees among miners instead of the very sharp instantly accepted / transaction limbo that we have now.

Even a 1 satoshi minimum would get people to at least start paying fees, and pave the road for the future where free/1-satoshi transactions are slow (just a few miners supporting them) and reasonable fees are required for fast processing.  It's probably helping your goals on the whole.

But I don't think we should make a universal mandatory minimum fee a rider on the fix for null blocks; that's really a separate issue.  In my opinion we should just try to solve the latter in a way that doesn't make the former worse or cause other collateral damage.

I elaborated before why forced inclusion of transactions is senseless because it can be evaded without problems.

DeathAndTaxes is constantly approaching this issue from an irrelevant angle (from my point of view). Tx fees do not currently play a significant role in miner rewards, I have repeatedly stated that market based tx fee system is necessary eventually and I have no interest in disrupting that, on the contrary. I like that in many ways.

This issue is different and it has to do with the incentive for a potential botnet owner to simply ignore all transactions. If he has a strong incentive to do this because of a Bitcoin exploit (I consider this issue an exploit) we might have to start paying ridiculous tx fees to make him change his behaviour, which would basically destroy Bitcoin. We are nothing without cheap transactions, that is one of the main benefits of using Bitcoin.

I don't claim transactions should be free either. But they must be cheap. We have no idea what kind of benefits this mystery miner gets from not adding transactions. It could be that he simply can't run that botnet securely without having a system in place that ignores transactions. In that case he would NEVER start adding transactions, not at any tx fees, because his entire mining operation would be at risk.

The only solution I care about is one where a potential botnet miner is forced to at least work with the blockchain. I'm not interested in either keeping Bitcoin transactions free nor am I interested in forcing the miners to do something in general. I am definitely interested in any proposal that forces miners to add at least some transactions, if there are transactions to be added.

And to answer the final point, I believe dummy transactions wouldn't really help if the solution is designed properly. The nodes are aware of all transactions, if x % of them (that have at least the minimum fee) are not included in a block, they simply refuse it. It would definitely be beneficial to Bitcoin users if this could be done. I believe the tools to stop freeleech mining can be built.

Once again making a distinction of paying vs non paying is dubious.

1 satoshi is paying.

If every transaction had a 1 satoshi fee the avg block would contain 0.0000008 in fees.
The entire network annually would produce ~ 0.0292 BTC in fees.

So paying vs non paying is a dubious distinction.

Going forward (once I have modified bitcoind and tested it) I will be excluding ALL transaction with < 0.01 BTC in fees.  I will also share the patch as open source and encourage other miners to adopt similar minimum fee rules (although each miner could choose a different amount).

Do you believe I should not be allowed to exclude fees I consider insufficient?  
More generally:  Do miners have a right to charge that amount they expect in an open and free marketplace for fees?  

People like to use the term boycotting (as in the above post).  If no tx had a fee of 0.01 BTC+ and as a resulted I mined a 1 tx block would that be boycotted.  What if I included a single dummy tx of my own (1 BTC from me to me with 0.01 BTC fee to me)?

I agree with Revalin once again. I don't claim that this is a critical issue, my problem with this is the ethical issue of leeching Bitcoin. That is what the mystery miner is essentially doing because he is only doing half of his job as a miner.

As far as solutions go I like what Revalin is proposing. It would probably be good to make it so that the portion of transactions that must be included is calculated from transactions that have a fee (any that's at least the minimum).

Again this would not remove the market from tx fees, all transactions outside that percentage (whatever percentage we chose) would still be controlled by the miners, they are not forced to add them.

In fact this could actually add incentive for users to start adding higher tx fees to make sure their transaction fits this special slot and thus would be guaranteed in the next block.

A good start would be if something like this was simply built-in to regular nodes so that we could essentially boycott malicious miners by choice. I'd love that. (referring to what MoonShadow proposed)

It's a real, though currently minor and not network-threatening, problem that needs a solution eventually.  It cannot currently be exploited for any real harm other than my indignation that some miners are being paid without doing their proper work.  This discussion is not about emergency damage control; we are just speculating and brainstorming ideas for how to adjust things in the future.

Better?

You said it very well, potential problems, but whoever reads this thread and the other thread at the mining forum will read "real problem in need of a solution" and freak out.

I'm just waiting for some dumb reporter to pick this up and start writing how Bitcoin is controled by botnets who don't even process tx's and other lies...