[bitcoind] Transaction fees: One annoying shit i have stumbled upon

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: Pieter Wuille on September 22, 2011, 11:26:22 AM

Quote from: ShadowOfHarbringer on September 22, 2011, 11:20:51 AM

Quote from: Pieter Wuille on September 22, 2011, 11:05:27 AM

The problem was that 0.3.24 reintroduced a bug where that calculation went wrong, and occasionally up to 0.01 BTC would be sent as fee (instead of 0.0005 BTC). It's fixed in 0.4, by the way.

Can you point me to the particular piece of code ?

This commit fixes it: https://github.com/bitcoin/bitcoin/commit/b6d19aefa0cf455dca57ca74eeb75e158a9390d0.

Excellent, thanks.
I will merge it into my fork.

Hopefully, it won't be too difficult for me to merge it into older versions also (< 0.3.24).

Update:
I checked and my forked version already contains this patch, but thanks nevertheless.

Edit2:
I will now try to apply it from trunk into nftf-0.3.24 branch and older versions.

Edit3:
I successfully merged the patch into following branches:

https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/nftf-0.3.21
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/nftf-0.3.22
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/nftf-0.3.23 (was already patched)
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/nftf-0.3.24

I added proper announcement:
https://bitcointalk.org/index.php?topic=22434.40

Pieter Wuille

legendary

Activity: 1072

Merit: 1189

Quote from: ShadowOfHarbringer on September 22, 2011, 11:20:51 AM

Quote from: Pieter Wuille on September 22, 2011, 11:05:27 AM

The problem was that 0.3.24 reintroduced a bug where that calculation went wrong, and occasionally up to 0.01 BTC would be sent as fee (instead of 0.0005 BTC). It's fixed in 0.4, by the way.

Can you point me to the particular piece of code ?

This commit fixes it: https://github.com/bitcoin/bitcoin/commit/b6d19aefa0cf455dca57ca74eeb75e158a9390d0.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: Pieter Wuille on September 22, 2011, 11:05:27 AM

The problem was that 0.3.24 reintroduced a bug where that calculation went wrong, and occasionally up to 0.01 BTC would be sent as fee (instead of 0.0005 BTC). It's fixed in 0.4, by the way.

Can you point me to the particular piece of code ?

2112

legendary

Activity: 2128

Merit: 1073

Quote from: Pieter Wuille on September 22, 2011, 11:05:27 AM

The "change is too small to go to wallet, let's send it to the miner" thing on itself wasn't a bug.

The problem was that 0.3.24 reintroduced a bug where that calculation went wrong, and occasionally up to 0.01 BTC would be sent as fee (instead of 0.0005 BTC). It's fixed in 0.4, by the way.

I apologise for the misrepresentation. I misunderstood somebody's else description. I never looked at the 0.3.24 code. I went straight from 0.3.23 to 0.4rc2. Sorry.

2112

legendary

Activity: 2128

Merit: 1073

Quote from: ShadowOfHarbringer on September 22, 2011, 10:56:11 AM

Can you please point me to the exact source (line of code) ?
I will also remove it from my fork.

I didn't know that fee-skimming scams got so serious these days.

Sorry, no. Please compare 0.3.24 to 0.3.23. The example of the bug:
You have 10.0007 in the wallet and configure it to pay 0.0005 fee. You send 10 plus 0.0005 fee
and expect the wallet to retain the remaining 0.0002. Psych! It went to the miners and your wallet is empty.

The "serious" part mostly pertains to the "conspiracy" part. Traditionally, charges of conspiracy were made when the people involved modified the off-the-shelf accounting software. When "off-the-shelf" really means "off-the-sourceforge" the conspiracy charges are an easy add-on for the plaintiff lawyers.

Pieter Wuille

legendary

Activity: 1072

Merit: 1189

The "change is too small to go to wallet, let's send it to the miner" thing on itself wasn't a bug.

The problem was that 0.3.24 reintroduced a bug where that calculation went wrong, and occasionally up to 0.01 BTC would be sent as fee (instead of 0.0005 BTC). It's fixed in 0.4, by the way.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: 2112 on September 22, 2011, 10:51:25 AM

Also, to the lesser extent, the clients with a bug "change is to small to return it to the wallet, so send it to the miners." It was re-introduced in 0.3.24, which is the current official one.

Can you please point me to the exact source (line of code) ?
I will also remove it from my fork.

I didn't know that fee-skimming scams got so serious these days.

2112

legendary

Activity: 2128

Merit: 1073

Quote from: kjj on September 22, 2011, 10:28:32 AM

What fee-skimming scams? Are we still talking about network transaction fees?

Mining pools mostly. Out of the top 10 is there any that doesn't skim or is upfront about the fact that it skims the "means to assure the security of the network transactions".

Also, to the lesser extent, the clients with a bug "change is to small to return it to the wallet, so send it to the miners." It was re-introduced in 0.3.24, which is the current official one.

It also pertains to you:

Quote from: kjj on September 22, 2011, 08:20:42 AM

The fee is miniscule. Just eat it and call it a cost of doing business.

What if your or ShadowOfHarbinger's software were used to run somebody's else business? You can eat your own fees just fine, but if you make someone else "eat fees" that is somewhere between "negligent accounting" and "conspiracy to commit fraud in accounting". The last one is when someone modifies the software that handles the transactions. In bitcoin ecosystem pretty much everyone runs modified software, so lawyers will have a field day in wringing the settlements from the naive programmers and business-people who hired them.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: kjj on September 22, 2011, 10:28:32 AM

Quote from: 2112 on September 22, 2011, 09:42:28 AM

As to who is bat-shit crazy it will be probably for the courts to decide. This pertains especially to those developers who reside in the USA and other coutries with an adversarial legal system. The various fee-skimming scams are just such an juicy target for a class action lawsuit in the Eastern Texas Circuit that somebody there will just grab this opportunity.

It either that or bitcoin will tank so low that not even a bottom-feeding lawyer will be willing to get out of the bed.

What fee-skimming scams? Are we still talking about network transaction fees?

No, we are talking about not giving user the choice whether he/she should risk his/her money and send the bitcoins without any fee.

Also, check out this topic: https://bitcointalk.org/index.php?topic=22434.0;all

BTW,
Today i did a small test. I re-sent 0.10 BTC having only 2 confirmations to another address, without paying a fee (using my modified client). The resulting transaction was low priority.

Guess what. It was sent without any problems, got 6 confirmations after about 2 hours.

So I wouldn't say that forced transaction fees are so needed.

kjj

legendary

Activity: 1302

Merit: 1026

Quote from: 2112 on September 22, 2011, 09:42:28 AM

As to who is bat-shit crazy it will be probably for the courts to decide. This pertains especially to those developers who reside in the USA and other coutries with an adversarial legal system. The various fee-skimming scams are just such an juicy target for a class action lawsuit in the Eastern Texas Circuit that somebody there will just grab this opportunity.

It either that or bitcoin will tank so low that not even a bottom-feeding lawyer will be willing to get out of the bed.

What fee-skimming scams? Are we still talking about network transaction fees?

2112

legendary

Activity: 2128

Merit: 1073

Quote from: ShadowOfHarbringer on September 22, 2011, 09:49:14 AM

The reasons why people are not rioting against it are probably
1) Larger fees do protect the network from penny flooding attacks
2) This is not annoying enough (yet).

3) Lawyers have sufficient income from other sources. The bitcoin targets are not yet ripe enough. By "ripe" I mean that nobody yet signed off on their audits as "true & correct". Once the word gets out among the lawyers that bitcoin-using businesses routinely sign off the perjurous audits that don't follow the GAAP the writs will start flying.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: 2112 on September 22, 2011, 09:42:28 AM

Quote from: kjj on September 22, 2011, 09:12:50 AM

Actually, no, I don't run a pool. And my mining rigs are almost always on btcguild, which doesn't distribute fees to members, so I have nothing to gain directly by this.
[...]
Oh, and if anyone seriously believes that the devs are obfuscating the mining fees intentionally because they profit from it, you are bat-shit crazy, and I just thought you should know that.

Well, I apologise for being presumptious about your personal involvement in the fee-skimming scam. Its my personal bias I'm getting from the bitcoin promoters that I personally met.

As to who is bat-shit crazy it will be probably for the courts to decide.

I think that involving lawyers is not so necessary.

This is an open-source project, everybody can write their own client. Also, it is a choice of the collective what fees will be enforced. If too many people will be annoyed by fee-skimming, they will move to forked clients, such as mine.

The reasons why people are not rioting against it are probably
1) Larger fees do protect the network from penny flooding attacks
2) This is not annoying enough (yet).

2112

legendary

Activity: 2128

Merit: 1073

Quote from: kjj on September 22, 2011, 09:12:50 AM

Actually, no, I don't run a pool. And my mining rigs are almost always on btcguild, which doesn't distribute fees to members, so I have nothing to gain directly by this.
[...]
Oh, and if anyone seriously believes that the devs are obfuscating the mining fees intentionally because they profit from it, you are bat-shit crazy, and I just thought you should know that.

Well, I apologise for being presumptious about your personal involvement in the fee-skimming scam. Its my personal bias I'm getting from the bitcoin promoters that I personally met.

As to who is bat-shit crazy it will be probably for the courts to decide. This pertains especially to those developers who reside in the USA and other coutries with an adversarial legal system. The various fee-skimming scams are just such an juicy target for a class action lawsuit in the Eastern Texas Circuit that somebody there will just grab this opportunity.

It either that or bitcoin will tank so low that not even a bottom-feeding lawyer will be willing to get out of the bed.

kjj

legendary

Activity: 1302

Merit: 1026

Quote from: 2112 on September 22, 2011, 08:46:56 AM

Quote from: kjj on September 22, 2011, 08:20:42 AM

The fee is miniscule. Just eat it and call it a cost of doing business.

kjj probably runs a mining pool which collects those fees and doesn't distribute them to the actual miners. So your cost will be his gain.

Isn't the bitcoin ecosystem wonderfully egalitarian that way?

Actually, no, I don't run a pool. And my mining rigs are almost always on btcguild, which doesn't distribute fees to members, so I have nothing to gain directly by this.

But finding fees in advance is only possible if you accept some horrible preconditions. If you are willing to lock parts of the wallet in advance and accept the proposed fees before the next block is found, you can do it. Pay attention to the second half of that. If you ignore it, you are absolutely certain to be paying too much in fees over time.

I really like the way flexcoin handles this problem. They charge a predetermined fee when a user wants to send, and the difference (in aggregate) between the fee they charge and the fee they pay to the bitcoin network is distributed as profit to them and as a ~~dividend~~ "refund" to their depositors.

Oh, and if anyone seriously believes that the devs are obfuscating the mining fees intentionally because they profit from it, you are bat-shit crazy, and I just thought you should know that.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: Pieter Wuille on September 22, 2011, 09:08:13 AM

Quote from: ShadowOfHarbringer on September 22, 2011, 08:52:34 AM

Quote from: 2112 on September 22, 2011, 07:51:07 AM

Quote from: ShadowOfHarbringer on September 22, 2011, 07:40:15 AM

Should there be a method in the API like "checksendtransactionfees

" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

That is not a problem really.

The method i proposed could work the following way, by "preparing" the transaction:

1. Make the rand() and prepare the transaction + calculate fees
2. The transaction is stored in a "prepared" or "locked" state - all bitcoins from the transaction are locked, until the transaction is unlocked by user.
3. If the user likes the fees, he confirms the transaction, it becomes unlocked, and it is then relayed to the network.

This shouldn't be extremely hard to implement.

As I said, it already was implemented: https://github.com/TheBlueMatt/bitcoin/commit/5350a907a57b88bd86827c1f3e41b571cf130401. It seems to have been closed awaiting rebase...

Nice !

However I will not be using an untested code in prod evironment.

I hope it will be added to mainline client in the future.

Pieter Wuille

legendary

Activity: 1072

Merit: 1189

Quote from: ShadowOfHarbringer on September 22, 2011, 08:52:34 AM

Quote from: 2112 on September 22, 2011, 07:51:07 AM

Quote from: ShadowOfHarbringer on September 22, 2011, 07:40:15 AM

Should there be a method in the API like "checksendtransactionfees

" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

That is not a problem really.

The method i proposed could work the following way, by "preparing" the transaction:

1. Make the rand() and prepare the transaction + calculate fees
2. The transaction is stored in a "prepared" or "locked" state - all bitcoins from the transaction are locked, until the transaction is unlocked by user.
3. If the user likes the fees, he confirms the transaction, it becomes unlocked, and it is then relayed to the network.

This shouldn't be extremely hard to implement.

As I said, it already was implemented: https://github.com/TheBlueMatt/bitcoin/commit/5350a907a57b88bd86827c1f3e41b571cf130401. It seems to have been closed awaiting rebase...

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: 2112 on September 22, 2011, 08:58:02 AM

Quote from: ShadowOfHarbringer on September 22, 2011, 08:52:34 AM

This shouldn't be extremely hard to implement.

Famous last words. Garbage collecting stale locks on the wallet is a nice gig, you'll spend endless nights debugging it.

Perhaps.

Unfortunately I don't do advanced C++ programming, so I cannot do it myself anyway.
Also there seems to be a lot of unwillingness among current developers for working on any solution to this problem.

That however does not surprise me, as the major devs probably invested heavy money into mining rigs, so why would they damage their business ?

An easy & quick solution could be for example, to return error if transaction fees are set to 0.00, and requested transaction requires a fee.

But obvious mining cartel is obvious.

2112

legendary

Activity: 2128

Merit: 1073

Quote from: ShadowOfHarbringer on September 22, 2011, 08:52:34 AM

This shouldn't be extremely hard to implement.

Famous last words. Garbage collecting stale locks on the wallet is a nice gig, you'll spend endless nights debugging it.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: 2112 on September 22, 2011, 07:51:07 AM

Quote from: ShadowOfHarbringer on September 22, 2011, 07:40:15 AM

Should there be a method in the API like "checksendtransactionfees

" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

That is not a problem really.

The method i proposed could work the following way, by "preparing" the transaction:

1. Make the rand() and prepare the transaction + calculate fees
2. The transaction is stored in a "prepared" or "locked" state - all bitcoins from the transaction are locked, until the transaction is unlocked by user.
3. If the user likes the fees, he confirms the transaction, it becomes unlocked, and it is then relayed to the network.

This shouldn't be extremely hard to implement.

ShadowOfHarbringer

legendary

Activity: 1470

Merit: 1006

Bringing Legendary Har® to you since 1952

Quote from: 2112 on September 22, 2011, 08:46:56 AM

Quote from: kjj on September 22, 2011, 08:20:42 AM

The fee is miniscule. Just eat it and call it a cost of doing business.

kjj probably runs a mining pool which collects those fees and doesn't distribute them to the actual miners. So your cost will be his gain.

Isn't the bitcoin ecosystem wonderfully egalitarian that way?

Actually, i am running my own fork of Bitcoin, which does not enforce fees.
https://bitcointalk.org/index.php?topic=22434.0;all
https://github.com/ShadowOfHarbringer/bitcoin-nftf/

I just wanted to know, if there is a way to do it without using modified clients.

Topic: [bitcoind] Transaction fees: One annoying shit i have stumbled upon (Read 3188 times)