Topic: Miners that refuse to include transactions are becoming a problem - page 6. (Read 16953 times)

Personally, I favor a drastic rule such as

* Build a list of "likely to be in next block" transactions, from memory pool
* Do not relay blocks unless they contain at least 50% of the transactions in the "likely to be" list

This is "drastic" in my estimation because such a rule has notable downsides,

* Increases possibility of short term fork
* Creates de facto requirement that at least 50% of each block are standard transactions (as defined by isStandard)
* and some other minor fallout

However, it is a strong rule that does address the issue at hand, while permitting valid, no-tx-activity zero-transaction blocks to exist.

reread what you quoted.

To encourage users to use the network.

A network's value increases as its userbase increases (and vice versa).

I think so.


It's not just recent blocks: they'd need to be able to access random unspent transactions in the chain.

The proof of verification could be distributed, but then at least you know that someone is doing the proper verification and miners aren't blindly building onto invalid blocks.

And to that end, this is what I dug up researching this today:


Some food for thought.

Refusing to relay a valid transaction is different than refusing to recognize it as valid.  In theory, a talented coder can set relay rules for his client already.  There is nothing preventing nodes with additional relay rules from rising any more than there is nothing preventing a miner from setting additional block inclusion rules, but those only affect that miner and those who choose to make use of his code & ruleset.  A single miner who refuses to relay valid blocks produced by a competing mining pool amounts to little.  But if a significant number of nodes have a similar relay rule that delays the transmission of a valid block without transactions, then that rule becomes a defacto sanction against the miner who produces null-set blocks. 

Then Haplo's clarification is correct? (Thank you for that, Haplo.) So it means instead of forcing miners to include transactions, they would instead be forced to access recent blocks and extract data from them.

What's still a little confusing about that on a conceptual level is this: mining ALREADY requires access to recent blockchain data, in the form of a hash of the prior block. Even if it required new code, couldn't the method already being used to distribute that prior-block-hash to all the nodes on a botnet be copied/modified to also distribute the newly-required "proof of verification?"

A possibly simpler solution would be a server-to-server version of 'getwork'. This would allow standalone mining clients to connect to any Bitcoin node and ask them which transactions they would include in a block they were mining. Right now, anyone can trivially design a standalone miner that mines blocks with no transactions. This would allow them to trivially design a standalone miner that mines blocks with transactions.

There's a "poisoning the well" problem though. If this is used primarily by botnets, it would be very tempting to include invalid transactions to cause them to mine invalid blocks. (The clients would have no way to validate them.) That would just cause them to go back to mining blocks with no transactions though.

D'oh.


gmaxwell's solution might provide #1. If D&T is correct, then since gmaxwell's solution requires a miner to use the previous block's tx inputs (rather than just the hash) as verification, this would force mystery to either run more complete versions or bow out. The disadvantages are that it requires a change in protocol, and would increase the size of each block by an unknown amount.

For #2 it's been proposed to replace the blockchain with an acyclic directed graph, but I don't know enough about that to compare it in any way. Apparently this would make tx inclusion cheaper, although I have no idea how that would work, or how the filesize compares to the blockchain.

#3 unfortunately is probably not going to be viable for several years, maybe a decade. When the average number of tx increases by a large amount (maybe to 10k+) then even a small tx fee would generate substantial income, but as of right now, and for at least a few years yet, the block reward is much more significant, and has currency appreciation supporting it.

True.  If the botnet operator is paranoid he may wish to minimize the connections to the rest of the bitcoin network (to keep the ip of nodes unknown).

In that case take my theory above and modify it so there is a single bitcoind active at one time.  When it detects a new block it notifies the rest of the swarm.  Using a mesh p2p like protocol the "gateway" nodes notifies 50 nodes who notify 50 each, who notify 50 each.  Each node is aware of the current gateway.  When it finds a block it relays it to the gateway who broadcasts it to the bitcoin network.

Really that would be sub optimal but very isolated.  If one takes the gateway down the operator promotes another node to the gateway.  By keeping the bulk of the swam unknown and with no direct communication with bitcoin network it is opaque and dark.

If the botnet has 1.8 million nodes and only one is active as the gateway at a time shutting it down would be tough.  Even if the ISP did shut it down.  Well there are 1,799,999 more nodes to go.

Likely the next question is "If D&T is right, why doesnt' the gateway simply include txs like any other node would?".  A little thinking I think will reveal a very obvious reason.

Interesting theory, I didnt think of this.

However, it does not address the fact that the blocks all come from the same IP (or small group of IPs).  In your scenario, it would be a random one out of 1.8M IPs.

You're talking about the "discouragement" proposal. I am not.

Discouraging blocks should be a last-resort tool. The incentives are wrong: building onto a discouraged block has no extra cost for miners, but discouraging a block has significant extra costs.

It's all about incentives.

Mining & killing bitcoin? 
Make no sense.

If this is the issue, and there's a good chance that it is, there are three possible solutions:

1) Make it more difficult for listening nodes to get what they need to process a block with no transactions.

2) Make it easier for listening nodes to get what they need to process a block with transactions.

3) Raise the transaction fees so that there's enough incentive for botnet operators to get transactions into their blocks.

But he can't.  If the majority of miners include 10 free tx on the "required list" for the next block then "mystery" (or any miner or pool) is required to include them or have that block invalidated by other miners.  He can't even exclude free tx without taking a risk his block will be orphaned.

Miners already have limited pricing power due to the fact that mining is a commodity business this ensures they have no independent pricing.  If clients know the list used by majority of miners includes free tx there is little reason for them to include a fee.

Regular miners wouldn't use these lists at all. They can do whatever they want. The lists are meant to stop "dumb" miners that can't verify the chain for themselves from just blindly building onto the longest chain. This proposal makes all miners verify the chain or get someone else to do it. If this "mystery miner" is already verifying the chain properly, then there's no problem: he can set fees to whatever he wants.

The problem with delaying blocks is the following.  As a simple user, you may feel personal satisfaction by not relaying the block and it may not affect you in any other way.  But as a miner, you are in a difficult situation when you receive a block that you dont "like".

For as long as you plan to dislike it, you cannot mine on top of it - because your potential next block would be waste without the "evil" block. It is chained to it and eventually you will have to publish the "evil" block along with your own next block.  To make things worse, your chances of success are higher when you relay the "evil" block as soon as possible.  Otherwise you not only depend on your own luck (difficult enough), but also on the luck of the "evil" block (which you have tried to diminish).

Therefore, if you dont like the block, you must not mine on top of it.

This is a very risky strategy.  Other miners may not be as picky as you are, and then your mining efforts may end up waste.  You are intentionally mining on an old block, knowing that a newer block exists.  This can only play out well, when you are absolutely sure that most other miners ignore the block too.

Every time an "evil" block appears, the chances of an orphan produced by (self-proclaimed) "legit" miners raises.  It can be seen as a seed for orphans.  And I mean an orphan of "legit" work, not only the "evil" work itself.

(Remember that I am talking all this under the premise of the poposed intentional relay delay)

Although the percentage of seeds is only 15%, the chance of an orphan arising from it may be much higher.  It depends on the adoption of the protocol change and also on the individual configuration that each miner chooses.  We could very well face a situation where everyones own mining effort has a 50% to be orphaned after each seed.

Effectively every miner would pay a very high price for a just slightly better service.  At 15% seeds and 50% chance to orphan, EVERY "legit" miner would loose 7% income.

A high price to pay.

To pay for what?  Well for the slightly better service quality I suppose.  "Slightly better", because a TX can make it 15% faster into the chain.  However, there is also a negative effect, which is the constant rate of orphans and reorganization of the blockchain.  This could be seen as another kind of "Dos" or attack on bitcoin.  It makes 1 or 2 confirm transactions very unreliable, and can also affect 3 confirm TX or higher.

People who rely on low-confirm transactions will now have to wait CONSIDERABLY longer, not just 15% longer for the first confirm and 0% longer for every subsequent confirms.

We have already seen runs of up to 4 or 5 consecutive blocks with no TX, all from the same source.

The higher price paid by miners (in form of extra electricity with less BTC reward) may actually make the overall service quality worse.

No please listen.  USING A POOL SERVER WITH A BOTNET IS NOT VIABLE DUE TO THE COST OF GETWORKS.  It was your "theory" that the nodes not being able to process tx was bogus because most miners don't.  However conventional pool miners need a pool.  I was just showing the math that current tx fees come nowhere near the cost needed to support a pool.  They don't for a botnet, they don't even for a small conventional pool.

The nodes likely are running a modified stipped down version of bitcoind.  It doesn't keep the blockchain, it doesn't need a db, it doesn't even need logs.  It simply connects to peers and looks for inv messages.  When inv message occurs the nodes internally use the last block hash, current time, no tx, the simplified coinbase, build a merkle tree consisting of 1 tx, build block header and start hashing.  Likely not all nodes are even running this.  To isolate the botnet only a "few" (as a % of total nodes) would need connections to bitcoin network.  They could rely new block notifications in p2p fashion to the rest of the swarm.

Essentially they are listen ony nodes.  They only broadcast to bitcoin network when they find a block.  Which if the avg node is 10 MH/s would only be once every 20 years (per 10 MH/s node).

Or that is how I would do it.  

Nonsense.  The blockheader is tiny.  99%+ of the blockchain is transactions.  The number of blocks is based on time.  Empty or not the block chain will grow by (300 bytes) * 6 * 24 * 365 bytes per year = 15MB annually  (roughly double that if you include the only requires tx - the coinbase).  

That cost is fixed.  The only thing that would change that is a avg block time other than 10 minutes.

Transactions make up the vast majority of the blockchain.  This is why downloading blockchain is fast initially (first 50,000 blocks downloads within minutes but then it slows down).  Those blocks are mostly empty (yup even Satoshi mined empty blocks).


No botnet or even conventional TH/s farm would be running the standard client.  Especially not a farm (conventional or not) which is ignoring tx.

*sigh*, well if that's true then either someone is charging a ridiculous fee, or there's a bug in some version of the miner software. If the difference in load and everything else between including tx and not including tx is basically insignificant, then there should be no reason for anyone to be purposefully excluding them.

On the other hand, if an empty block is really ~1/55th the size of a block with lots of tx, then it might really be someone trying to cheap out on bandwidth while still getting paid.

Either way, it ought to be stopped.