Topic: Message to devs from merchant (Read 5301 times)

You assumed constraints I did not. The scan does not need to use the P2P protocol if it is about the wallet of the node
or if the client have extended ways to communicate with nodes it trusts to some level.

You have it reversed. The client is the one sending the filter (with the filterload message) and the server is scanning the transactions. Of course, what you described could work but it is not the current protocol.

The server might prune but accumulate a bloom filter of addresses it ever saw.

I assume the scan knows the master public key. Addresses can be generated sequentially and checked against the bloom filter instead of the block chain until the bloom filter is negative for longer than the gap. Thereafter transactions can be fetched from pruned set by address.

You seem to be pretty sure of what you say but I really don't see how it would work. The client sends the filter parameters and the server creates a filter per session. The server scans its blocks. If the spent tx are removed from the server, how would it know how to match the filter?
I'm ok being wrong but could you describe the actual process?

false positives only extend the scan. All funds would be found.

BTW, an invertible bloom filter is not a bloom filter.

Yes but it could be fooled by omitting some transactions. I switched to Electrum from Armory. I still think it's the best solution at the moment for someone who doesn't want to spend hours downloading the blockchain but it's not suitable for a merchant.

Back to topic,
In my opinion, there should be a client that lets you:
- use a bootstrap file to catchup to a recent checkpoint
- catchup the remaining part of the blockchain by downloading simultaneously from multiple peers (bitcoin core uses 1 peer)
- performs validation in parallel and progressively as the chain gets reconstructed
- prunes the spent transactions locally. It could also prune the blocks if it doesn't care about relaying the blockchain.

If we are allowed to change the protocol, let there be a way to transfer a verifiable list of spent transactions and real transactions. I think this is what Gavin talks about in his post but I need to see more detail to be sure.

A bloom filter cannot do that. An invertible bloom filter might but it is not decided yet if it is going to used (cf Gavin's message).

This is wrong. Electrum verifies all transactions in its history using SPV.

Nodes might prune transactions but maintain a bloom filter of addresses they saw.

Be careful with assumptions like these.  All (reachable) peers are malicious peers if an attacker controls your network connection. (E.g. compromised your ISP or router, or they are your ISP)

Spv could work, but youd want to be connected to a few nodes at a time to be confident. Just randomly hop around servers in the electrum server list. You can subscribe to callbacks for events from stratum servers if I recall correctly..

Thanks. If a node prunes spent transactions, a hierarchical wallet will not 'see' the empty transactions anymore and it will stop scanning after the gap limit is passed.

A deterministic wallet is simply a way of generating addresses. Bitcoin classically generates new private keys using a true random number generator for each address. When a new address is needed to request a payment, it is generated at random and is not related to any other key. A deterministic wallet instead uses a pseudo-random number formula to create addresses, with only the initial seed state being a generated random number. This prevents the wallet backup from becoming obsolete, because any past or future address you created can be recreated from the seed backup.

Pruning is a technique for reducing blockchain disk space consumption, and is unrelated to your wallet addresses. We assume that the blockchain history has already been thoroughly validated by many miners after it is buried under many blocks and that every transaction in it was properly signed by the owner. Therefore, as long as the hash chain in the headers is correct, we don't need to verify the chain-of-ownership of every single bitcoin amount back to its original creation for our own security. This allows an individual client to remove the spent transactions from the blockchain, potentially freeing hard drive space, and even may enable a 'pruned network' p2p client to exchange a lightweight blockchain between peers.

The "plan out there" for pruning was put forth in the original Bitcoin paper: http://we.lovebitco.in/how-bitcoin-works/bitcoin-paper/#ch7 - it is just that nobody has written a complete client implementation that does this yet.

It's good to know there is a plan out there to have ultra pruned tree verification. The hashes you mentioned are going to incorporated in Merkle trees? I thought that the Bloom filter mechanism was going to be extended to skip over used tx.

In any case, would it be compatible with deterministic wallets? Without the used txs, they will stop scanning for addresses too early.

Its exponential, e^x not x^2

Yes, today. But not at some point in the future. Please read about "UTXO commitments" in https://bitcoinfoundation.org/2014/10/a-scalability-roadmap/

You could use bitcore as far as I understand it - let's you execute all commands on the chain without the need to download the chain.
Open source and fairly easy to use, shall meet your requirement.

You guys say that hard drives are cheap but you still have to scan the block chain at first. It takes a very long time to do but is the only trust less solution, isn't it?

As a merchant, the only way I see things going is to keep the block chain whatever its size becomes, and sync.

It would be good if there was a way to drop the used transactions from the requirement. Last time I checked, there was 27 gb of block data but less than 1 gb worth of active tx.

And then there are the 8TB and 10TB drives from WD (and others).   http://arstechnica.com/information-technology/2014/09/western-digital-readies-first-10tb-hard-drive-ships-new-8tb-drive/

6TB is about $250 and about 270 times the current block chain.  Even growing at 20 GB/year, that is a long time before it is full.  Even IF it grew at 100GB/year (~10 times last year) that is still many, many years to fill it.  And then there are the 8TB and 10TB drives and that is just what is available now.

I still run a full node on a RPi, external hard drives are cheap. I expect they will get cheaper per GB/year in the future. I bought a 3 TB the other day for 130.- USD.