Currently, if I have bitcoind running on 20 machines, or 20 bitcoind running on the same machine ( vps for example ), each of them need to download and store 2 GB blockchain, total 40 GB diskspace used and much bandwidth wasted to download it 20 times.
Code:
ls -alh .bitcoin/blk*
-rw------- 1 neofutur neofutur 1.1G 2012-03-21 00:36 .bitcoin/blk0001.dat
-rw------- 1 neofutur neofutur 401M 2012-03-21 00:36 .bitcoin/blkindex.dat
-rw------- 1 neofutur neofutur 1.1G 2012-03-21 00:36 .bitcoin/blk0001.dat
-rw------- 1 neofutur neofutur 401M 2012-03-21 00:36 .bitcoin/blkindex.dat
The problem is the same for a vps provider like bitvps, hosting bitcoin users, 2 GB blockchain X 20 vps makes much diskspace and bandwidth wasted for no reason.
Also the initial blockchain download is taking hours , often more than one full day . . .
Also its nearly impossible to use bitcoind on a small device like a eeepc ( 4 GB diskspace ), a phone, a raspberrypi . . .
I think the easiest way to make it better would be to have a "read-only" blockchain, one of the bitcoind writes it, and the others only use it read only to check transactions.
Actually bitcoind is not able to use a read only blockchain, that could be written and updated by 1 bitcoind , and used by all the other bitcoinds on the same server ( in the case of VPS ) or mounted readonly as a sshfs or other secure network filesystem.
I thought it could be simple and easy to hack bitcoind to allow this ( I tried ugly debugging on https://github.com/neofutur/bitcoin only to find its really not easy )
So, I start a bounty for whoever can produce a working patch or fork of bitcoind , allowing this "sharing blockchain" feature, allowing a service provider to be his own blockchain provider and save much diskspace and bandwidth.
Also, the "read only blockchain" is just an idea, one way to implement it that seems to be the easy way imho, but you can also provide a better way to do it . . .
another ( better ) way to address the problem, from gmaxwell :
As I've pointed out before on IRC, this isn't the way to accomplish what you want. A "one write many read" blockchain can not be accomplished with a simple patch— it would take either significant and hazardous rewrite of BDB, or a hazardous rewrite of bitcoin to use something custom instead of BDB. Even if someone managed to pull it off I'm doubtful we'd take it upstream.
Instead, it appears to be generally agreed that we want to split the blockchain and the wallet— so you could have one trusted blockchain and N wallets (GUI or CLI/RPC) talking to it across the network.
Instead, it appears to be generally agreed that we want to split the blockchain and the wallet— so you could have one trusted blockchain and N wallets (GUI or CLI/RPC) talking to it across the network.
The patch or fork will have to provide an option like "--readonly" at bitcoind start .
the patch have to be a git pull request on github and be clean enough to have a chance of being accepted upstream.
the implemented solution have to build and work on linux .
I start the bounty with 5 BTC, and I know others will add btcs soon
Feel free to ask for details here or on #bitcoin-hosting channel : http://gw.gd/irc
Read also :
http://bitcoin.stackexchange.com/questions/3199/read-only-blockchain-in-bitcoind-patch-ideas
http://bitcoin.stackexchange.com/questions/2584/is-reclaiming-disk-space-already-implemented-how-effective-will-it-be
Some clues ?
Bluematt have a good beginning :
Code:
< BlueMatt> neofutur: done: take https://github.com/TheBlueMatt/bitcoin/tree/cblockstore and change https://github.com/TheBlueMatt/bitcoin/blob/cblockstore/src/blockstore.h#L127 to return false
useful links from weex :
It appears the devs have discussed this though a bounty like this is probably the only way to get code written. See Mike Hearn's post here: https://bitcointalksearch.org/topic/m.892772
Another reason for this project to happen(linking roughly to where relevant discussion is): https://bitcointalksearch.org/topic/m.892743
Another reason for this project to happen(linking roughly to where relevant discussion is): https://bitcointalksearch.org/topic/m.892743
Rebroad have a good idea to make it more secure for the client :
Code:
(20:32) < rebroad> neofutur, the thing is... each separate bitcoind is going to want to recover an old database if the centralized one is changed in a way they don't like...
(20:33) < rebroad> neofutur, so ideally it would be stored in two places owned by 2 separate parties where users of it can be sure that they can recover to old versions
(20:33) < rebroad> (at least 2)
(20:33) < rebroad> neofutur, so ideally it would be stored in two places owned by 2 separate parties where users of it can be sure that they can recover to old versions
(20:33) < rebroad> (at least 2)
possible problem :
Code:
(21:21) < luke-jr> neofutur: rebroad: the problem you'll run into is that the blk files are NOT identical across bitcoin client instances
(21:21) < rebroad> luke-jr, could they be though, in some theoretical node that's not been written yet?
(21:22) < rebroad> luke-jr, with one file per block
(21:22) < luke-jr> triptyl: have you given any thought to doing it without breaking old clients?
(21:22) < luke-jr> rebroad: probably
(21:22) < gmaxwell> triptyl: it would be better to call it "blinded" rather than anonymous.
(21:23) < rebroad> luke-jr, I think it would be worth adapting bitcoin{d,-qt} to allow the blocks to be stored this way, so that they are more VPS friendly
(21:21) < rebroad> luke-jr, could they be though, in some theoretical node that's not been written yet?
(21:22) < rebroad> luke-jr, with one file per block
(21:22) < luke-jr> triptyl: have you given any thought to doing it without breaking old clients?
(21:22) < luke-jr> rebroad: probably
(21:22) < gmaxwell> triptyl: it would be better to call it "blinded" rather than anonymous.
(21:23) < rebroad> luke-jr, I think it would be worth adapting bitcoin{d,-qt} to allow the blocks to be stored this way, so that they are more VPS friendly
weex is suggesting another way to implement it :
Code:
As someone who has enjoyed using BitcoinSpinner a bit, why not patch bitcoind so it supports direct calls via BCCAPI ( http://code.google.com/p/bccapi/ ) or the protocol it uses?
PS : I have no problem with redefining the bounty goal, my idea of readonly blockchain is possibly a bad idea, and we could need some kind of network protocol and a blockchain server, lets say all comments are welcome, and the goal of the bounty can change, blockchain server is probably a better idea, but I thought it would be a good simple / easy first step to begin with a read only blockchain
Current bounty : 15 btc
+5 btc from raccon under some conditions ( see his comment below : https://bitcointalksearch.org/topic/m.813297 ) : https://bitcointalksearch.org/topic/m.813297
+10 from rapeghost
+5 for weex