Topic: [ANN] BTCplex: an open source block chain explorer written in Go (Read 1240 times)
Bump!
March 06, 2014, 01:30:36 PM
@9inety7even I would love to but sadly I can't afford a 160USD/month server (database is 120+GB), maybe I'll give it a try if I get enough donation.
You could buy it for a few hours just to test the speed; I think the lowest of the high-end plans is like 30 cents an hour.
March 06, 2014, 01:09:00 PM
@btc_enigma Calling bitcoind on itself is fast, but when you add you own server call:
bitcoind <-> your app <-> client
your app <-> client
Even if you can perform multiple RPC calls at once to prevent HTTP round-trip time, I prefer maintaining my own database, when I request a block, I'll want to also fetch every related transactions (sometimes 1000+ txs), that's why BTCplex save the block with every belonging transactions in a single key, so I think it's faster in my use case, but depending on what you want to accomplish, you may achieve better performance calling bitcoind RPC API rather than maintaining your own database.
@9inety7even I would love to but sadly I can't afford a 160USD/month server (database is 120+GB), maybe I'll give it a try if I get enough donation.
bitcoind <-> your app <-> client
your app <-> client
Even if you can perform multiple RPC calls at once to prevent HTTP round-trip time, I prefer maintaining my own database, when I request a block, I'll want to also fetch every related transactions (sometimes 1000+ txs), that's why BTCplex save the block with every belonging transactions in a single key, so I think it's faster in my use case, but depending on what you want to accomplish, you may achieve better performance calling bitcoind RPC API rather than maintaining your own database.
@9inety7even I would love to but sadly I can't afford a 160USD/month server (database is 120+GB), maybe I'll give it a try if I get enough donation.
@9inety7even You're right, the real bottleneck seems to be IO, I think running BTCplex on a server with a SSD should really makes a difference.
I already run everything I can in parallel, but I must parse blocks and transactions chronologically since I need to fetch data about previous TxOuts.
I think 2 days to load the entire block chain is not that bad (when comparing the time it take to start a new bitcoind node), and given the fact that BTCplex keeps track of every known addresses balance.
I already run everything I can in parallel, but I must parse blocks and transactions chronologically since I need to fetch data about previous TxOuts.
I think 2 days to load the entire block chain is not that bad (when comparing the time it take to start a new bitcoind node), and given the fact that BTCplex keeps track of every known addresses balance.
Yeah I would definitely grab a DO vps and see how much of a difference it makes. Chances are you'll see fairly huge speedups.
@btc_enigma thanks! I'm storing block data in SSDB (which is backed by LevelDB internally) for performance, by design the webapp never call directly bitcoind.
Hi,
Can you throw some insights/figures on how this is faster than directly calling bitcoind. This is because I am also writing a similar block explorer and I am thinking about design issues. I was trying to avoid exporting out the whole blocks seeing the huge amount of time its taking. For example , I did
Quote
On testnet3
bitcoind getblock 00000000373403049c5fff2cd653590e8cbe6f7ac639db270e7d1a7503d698df >/tmp/log.txt 2>&1
bitcoind getblock 00000000373403049c5fff2cd653590e8cbe6f7ac639db270e7d1a7503d698df >/tmp/log.txt 2>&1
This completed in 0.013 sec which gives me around 71 requests/second . As far as I know, even bitcoind would just read its leveldb database stored in ~/.bitcoin and give you the output. So it should give similar performance. Actually I see that c++ code is able to read leveldb faster than python.
@9inety7even You're right, the real bottleneck seems to be IO, I think running BTCplex on a server with a SSD should really makes a difference.
I already run everything I can in parallel, but I must parse blocks and transactions chronologically since I need to fetch data about previous TxOuts.
I think 2 days to load the entire block chain is not that bad (when comparing the time it take to start a new bitcoind node), and given the fact that BTCplex keeps track of every known addresses balance.
@SomeoneInNeed Thanks!
I already run everything I can in parallel, but I must parse blocks and transactions chronologically since I need to fetch data about previous TxOuts.
I think 2 days to load the entire block chain is not that bad (when comparing the time it take to start a new bitcoind node), and given the fact that BTCplex keeps track of every known addresses balance.
@SomeoneInNeed Thanks!
It's pretty neat and functional. Me like.
What seems to be the bottleneck? Do you think you could run it in parallel, or is IO speed the problem?
@btc_enigma thanks! I'm storing block data in SSDB (which is backed by LevelDB internally) for performance, by design the webapp never call directly bitcoind.
@9inety7even, I spent a lot of time working on optimization, it takes one week on a small server (1.2Ghz dual core/6GB RAM).
On a more high-end server (i5/16GB RAM, the server btcplex.com is currently hosted), it took 48hours, and I store a lot of data (since disk is cheap, the database is 100+GB), but request are fast, even for querying the balance of an address with thousands of transactions.
@9inety7even, I spent a lot of time working on optimization, it takes one week on a small server (1.2Ghz dual core/6GB RAM).
On a more high-end server (i5/16GB RAM, the server btcplex.com is currently hosted), it took 48hours, and I store a lot of data (since disk is cheap, the database is 100+GB), but request are fast, even for querying the balance of an address with thousands of transactions.
I would say you need to do some optimization, waiting a week for the database to be built seems unreasonable.
Hey,
Nice Work !
Is there are reason you are storing block json again in leveldb ? Shouldn't just maintaining the indexes suffice ? You can always get the raw block from bitcoind
Nice Work !
Is there are reason you are storing block json again in leveldb ? Shouldn't just maintaining the indexes suffice ? You can always get the raw block from bitcoind
I've spent the last 6 months working on a new open source block chain explorer/browser written in Go: https://btcplex.com.
It's an early release (still not production ready), but I would love to hear any feedback!
Some features:
It also provides a full-featured JSON API, a plain-text query API and few server-sent events endpoint to access data pragmatically.
GitHub repository: https://github.com/tsileo/btcplex
Demo website: https://btcplex.com
Hosted documentation: http://docs.btcplex.com
Donation address: 19gzwTuuZDec8JZEddQUZH9kwzqkBfFtDa
Let me know if you have any suggestion/idea/question !
It's an early release (still not production ready), but I would love to hear any feedback!
Some features:
- Browse blocks and transactions.
- Check address balance history.
- Search for blocks, transactions, and addresses.
It also provides a full-featured JSON API, a plain-text query API and few server-sent events endpoint to access data pragmatically.
GitHub repository: https://github.com/tsileo/btcplex
Demo website: https://btcplex.com
Hosted documentation: http://docs.btcplex.com
Donation address: 19gzwTuuZDec8JZEddQUZH9kwzqkBfFtDa
Let me know if you have any suggestion/idea/question !
Jump to: