Topic: please restrict "changeback" amount (Read 3648 times)

Right, I haven't seen the wallet fail to notice that it actually sent a transaction before but I can see how that might be annoying if it didn't work. If you see it happen again please send a report to Andreas using the settings screen.

Your node (QT client) will periodically rebroadcast tx it has sent which remain unconfirmed.  It will continue to rebroadcast forever until either the unconfirmed tx is removed from the wallet (using a third party tool like pywallet) or it is eventually included in a block.

Thinking back, that's what I did, eventually, once I realised the transaction had in reality been sent.  And yes... it did figure it out eventually (i.e. when the tx was confirmed).

The problem is that this wa in a point of sale transaction at a pub, and I was standing there waiting while my phone said 'transaction not sent yet'.  Unfortunately, the design of the point of sale system is such that the bar staff have to press a key after I send the transaction - the terminal doesn't get notified asynchrounously.  So, unaware of this protocol failure mode, I just believed what the wallet was telling me.  I now know that what I should have done is ignored what the client was telling me and simply asked the bar staff to check if the payment had arrived.

But I shouldn't have to understand P2P protocol failure modes in order to use the app :-)

EDIT: Oh, it's TCP, so that shouldn't really happen.  Hmm...  (Somehow I was assuming the P2P protocol was UDP - dunno why.)

When you say 'reconnect' does this involve the client actually knowing that the network dropped first?  So if the inv packets from peers were simply dropped by the network, would you not just sit in the 'waiting for transaction to be sent' state?

If there's a network outage then it won't know the broadcast happened successfully, but when you reconnect it'll do a Bloom filtered mempool query and find the transactions at that point, so it's not a big deal. You can just press "back" and it'll figure things out eventually.

It's only happened to me once, and I didn't even know that the client logged by default.  I only mentioned the date to give you an idea of what app version I would have been running.  But if it happens again I'll be sure to grab the logs.

So if the network suffers an outage immediately after sending the tx message, you may miss seeing the transaction advertised by the other peers.  At this point you can try resending the inv and/or tx messages all you like but presumably you'll not get anything more back, right?

roy

EDIT: Oh, I guess you could poll peers with getdata to see if they have recieved the transaction.  That wouldn't even require multiple peers, would it?

No. You have to connect to multiple peers and watch it appear at each of them (which is what the app does). Really the protocol should be way more explicit, but Satoshi didn't make it that way and we never upgraded it to be so.

August 10th is a long time ago now, the logs will be long since disappeared.

I don't really understand the P2P protocol - how does the sender of a tx message know whether it was successfully received?   Does the receiver readvertise the transaction to the original sender in an inv message?

No, I've never used it.

Were you using the trusted peer feature?

Just to say - I had this happen to me on 10 August (can't guarantee the wallet app was bang up to date).  The client got stuck on "this transaction has not been sent yet" even though the recipient had seen the transaction.  AFAICT the client continued to show "this transaction has not been sent yet" until the transaction confirmed.

I think I might indeed have had a slightly flaky Internet connection at the time.

roy

Transactions that don't pay fees, or only pay small fees, are ratelimited so you can't just assume they will be relayed.

Also you have to understand the fine details of how the network protocol works: everything a node knows about is part of its "inventory", and (almost) every time we get a new inventory item we advertise it to our peers by sending them an "inv" message containing the inventory type (block, filtered block or transaction currently) and inventory hash. Our peers then request inventory they don't know about with "getdata" messages; they do this pretty much as fast as they learn about previously unknown inventory items.

Now if your node gets a transaction that either doesn't pay any fee, or pays a very small one, storing it in memory could constitute a DoS attack by filling up your memory, so beyond a certain rate limit free transactions are simply dropped and won't get requested again. On the other hand when the child transaction comes in that does pay the fee arrives we won't even know it paid the fee because we don't have the parent, so it just gets dumped into the orphan tx holding area and aren't relayed to our peers.

In short child-pays-for-parent isn't reliable because you have no guarantee the parent will ever propagate, and thus no guarantee the child will either.

On the other hand if you can tell a peer "If you get transactions a,b and c you'll get xBTC worth of fees for yBytes of space" the child can propagate because peers know it's worth their while to get the parents txs. On the other hand implementing all that is tricky - you want to think about DoS attacks, not to mention the state required.

Your coins should never be "stuck".  The protocol doesn't require that.  This is a bug in the client.  Exactly how/why it believed you had no UXTO to spend I don't know but you should have been able to spend every single coin down to the last satoshi.

Important:  the subsequent tx won't CONFIRM until the first tx (drink) is CONFIRMED.  So since the first tx is small if you didn't include a fee it may be a while and even if you included a fee in subsequent tx they can't be confirmed until the first one is.  So if your buyers complained.  Dude these coins you sent me have been unconfirmed for two hours that would be a different issue however any bitcoin tx (unless it breaks min mandatory fee rules) should be sent immediately and be seen (unconfirmed) by the recipient within seconds at most.  This applies to any client on any OS anywhere in the world.   


This is impossible however it isn't necessary for the real solution.  You simply want to be able to spend all yours coins whenever you want.  That is possible however change is ALWAYS the amount of the input minus the output.

As someone indicated up thread in the real world you open your wallet and inside is a single $50 bill.  You want to buy a beer for $5.  You can't tear off a corner of the bill to pay.  Well you could but the bartender might decide you had one too many.  The only possible way to pay for the beer is to pay $50, the bartender will put $5 in the till and return $45 to you.

Bitcoin works the same way.  If you receive 60 BTC as single tx you have a 60 BTC "bill" you can't tear off a corner of it.  It doesn't matter if you intend to spend 1 satoshi or 20 BTC or 60 BTC the only possible way to "spend" it is in a 60 BTC transaction.

So change will always be the difference.  However this is an xy problem. http://meta.stackoverflow.com/questions/66377/what-is-the-xy-problem

You don't really want to have the change smaller you just want to be able to spend all your coins when and how you want.  Luckily that problem can be solved and would appear (based on your description and assuming you didn't have one too many) to be a client side bug.

They don't need to be in one "packet".  As long as a node can validate the tx and any unconfirmed parents they will be relayed.  Technically it is likely the peer already knows about the unconfirmed parent but sending both just avoids the tx being dropped.

So for unconfirmed tx there is no issue (at the protocol level but based on OP report if accurate there is some issue with the client) in unconfirmed chains.   The main unresolved problem is that the algorithm in bitcoind used by miners to select tx for a block ignores child-pays-parent.

This means usually a tx with a high fee which has a unconfirmed parent w/ low/no fee ends up being delays.   It may take a considerable amount of time before a miner includes the low priority parent into a block and until they do so the high fee child is ignored.

Still all that has to do with first confirmation time.  The issue described by the OP is that down chain tx are "not seen" (meaning not even unconfirmed) by the receiver and reported by the client as unsent.  Unless the OP is mistake or my reading is incorrect that is a client side bug and not a restriction of the protocol.


What should have happened (even assumming empty wallet no other UXTO)

None of the tx would be included in a block until the first one was and since child pays parent is ignored by miners it may be a while but as far as unconfirmed tx there should be no issue.

P2P protocol changes to send multiple tx's in one packet are needed for child-pays-for-parent as well as replace-by-fee secured zero-conf transactions, so it'd be functionality that will get used by a lot of things.

Yes, it needs to send a tx and all its unconfirmed dependencies actually. The wallet can have a method to provide such a collection.

I filed issues against bitcoinj for these two things.

One more thought: The new offline transaction feature would have helped in this case - right? The only caveat is that currently, the feature does not transmit chains of transactions - only the transaction itself.

Although I only saw the result and not his actions, that sounds right. At least I had that happening to my wallet as well. It was never much of a problem, because my wallet is always fragmented enough so I can easily find another output to spend for my beer. But I agree it can be a problem for big transactions.


Bitcoinj would trigger this internally, right?


Please do. The sooner we have this the better. We realized already a year ago how much this would simplify the interaction. An ack message would also be useful, by the way.

Okay you guys, please go about doing technical things to fix this, I'm more of a "user" type.

This is the problematic part:


Building up chains of unconfirmed transactions is not in itself a problem (they can/should all confirm together).

However, if the app says "this transaction has not been sent yet" then this means one of four things:

1) Your internet connection wasn't working.
2) You have set a "trusted peer" (this confuses the ui a bit - known bug, issue in bitcoinj really).
3) There's an unknown bug that was causing the transaction to not be sent.
4) There's an unknown bug that causes the transactions to be sent, but the app doesn't recognise it.

I have a vague feeling I might know what (3) or (4) could be. If you send a transaction and you don't have internet access at the time (or it's flaky etc) then the transaction won't be broadcast. You won't be allowed to extend an unbroadcast transaction chain (you won't be allowed to spend the change). However if you then regain internet access, bitcoinj will announce the transactions to all newly connected peers at once and thus won't get a chance to see them propagate.

Andreas, does that sound about right? If so, it should not be hard to reproduce this and verify the hypothesis. If correct then it should be an easy fix. It means changing how pending transactions are announced to new peers such that rather than announcing them as soon as a peer is connected, all pending transactions in the wallet are broadcast along the standard codepath (using PeerGroup.broadcastTransaction), to allow observation of the propagation.

It'd be even better of course if the P2P protocol informed you if a tx was rejected. Then we could simply this "seen by X peers" tracking code significantly. I might submit a pullreq to do that at some point.

The scenario was (in two cases this hasd happened as of yet) the following:

I get sent a number of BTC, say 100.
I send away a part of those (be it paying for a drink, selling 10 BTC).
So let's say 10 BTC are sent to another person, 90 BTC are "changeback".
I can now send another 10 BTC to yet another person, the remaining 80 BTC being the "changeback".

I have now built a dependency cascade of a number of transactions which all depend on the first one "going through".
In my experience, those do not show immediately on the chain (the app's status shows "this transaction has not been sent yet").

Let's assume that for whatever reason, the next block takes 1h to find (or the app simply doesn't "see" the block).
Andreas was witness to one instance of this scenario (at the first BXB if you remember).

I had received 60 BTC
Paid my drink (0.01987something)
sold 10BTC
sold 10BTC
sold 20BTC
sold 10BTC
and was unable to spend the remaining BTC because the first transaction was still unconfirmed (in this instance, the app had not seen the next block, but there have also been times when a block took 30 minutes).

so I stood there in the rain, having a heap of unconfirmed sales and as number of unspendable BTC (maybe that depends on which version of the app I was using).

I wish there was an "easy" way for any user to circumvent such situations by eg. selecting "spread", which automatically adds 10 keys, makes a backup and then spreads the coins evenly in one transaction.

Wait, everyone calm down for a second.

Ignore the solutions 2weiX is proposing for his problem. Focus on the fact that he/she has a problem in the first place! As Andreas says, it's allowed to send money as many times in a row as you want without waiting for a confirmation. Those transactions will get broadcast as soon as you have a network connection.

So, the real question we need to answer is - why do people not see 2weiX's transactions for "one or two hours"? That's way too long. The recipient should see the transaction immediately, within a few seconds. If the tx doesn't go out for multiple hours, that suggests a bug or failure somewhere.

2weiX - how are you sending these coins? When you press "send" on the screen, you should see the little grey dot grow. Do/did you see that happen?

That's what I can do NOW. And as I pointed out, I can't really control what others send to me.
Why I created the thread is so I can do something different (and more convenient) THEN.

You can't other than make sure that you provide a new address for every incoming tx.

Thing is, I cannot control how others send to me.
I'd really like to control this on MY END.

Problem is (like explained above) the way Bitcoin works.

In simpler terms, when you send BTC50 to an address (or your mobile Wallet) it is like you now have 1 BTC50 Bill.

Just like with an € bill you only can spend this entire Bill. If have want to buy something for 5 € you have to pay with you 50 € bill and get change back. You can't just rip out a pice of the Bill and pay with it.

With Bitcoin it's the same, you have to spend the entire BTC50 and get the change back.

(I know that this is oversimplified)

Simple Solution for your Problem. Instead of sending one BTC50 Transaction to your wallet, use more than one address and for example send a BTC50 Transaction with BTC10 to 5 Addresses in your mobile wallet, so now you carry 5 BTC10 "Bills" instead of 1 BTC50.

I know that, but people freak out when you tell them "sent" and they don't see the trx for another hour (especially in face2face transactions).

Yes - assuming the minimal fee is paid then this is probably the only reasonable solution.

I'd like to add that Bitcoin Wallet does allow you to spend unconfirmed change.

However, the preceding transaction must have been validated at least by hearing it back from the network. You can tell that by watching the grey circle (initially a dot) grow. It can only grow if you're connected to the network after signing/sending the transaction.

The problem is fundamental to the way Bitcoin tx's work - it needs to use UTXOs (unspent transaction outputs - which are Bitcoin tx's that were sent to you) in order for you to send funds to someone else.

Typically the client will make random choices of UTXOs you have to build up enough (or often *more than enough*) BTC for the tx you are sending and will then send the *change* back to yourself.

From an anonymity point of view it is probably best that it actually chooses the UTXOs randomly (although an *exact* match might be a better selection if one exists).

In short though there isn't much you can do about this as fundamentally this is how Bitcoin works (you could of course send *yourself* smaller amounts if you want to create smaller UTXOs to be used in future txs although there is not likely to be any guarantee about which ones it will choose to use).
 

herpderp, I dunno. I'm not technically advanced enough to understand, it seems.
bang rock make fire?

maybe there's another way (a function "auto-spread balance across all addresses in wallet") to circumvent this problem.

If a lower UTXO does not exist there is no choice but to use the one it chose so is the problem that there was a lower UTXO that could have been chosen for this tx (sorry your "changeback" terminology is a little confusing)?

look at this transaction:
http://blockchain.info/de/tx/ee79c0ddc67ac3caeda766cdb1fd1114c87fb4d05a1b50d0a8f523f371c9a694

50BTC in address, I want to send 0.7x BTC to a friend.
Right AFTER that, I need to send 25BTC to somewhere else.... Pustekuchen!

My coins are stuck until the transaction is confirmed.
This REALLY grinds my gears.


Solution: Please restrict the changeback amount to 5% of the wallet balance.