Topic: Gocoin - totally different bitcoin client with deterministic cold wallet - page 3. (Read 38731 times)

thank you

Brilliant presentation at Future of Bitcoin. Your client looks really good, hopefully it will get you some more recognition!

I have a new update, but first of all I'd like to know if anyone is actually reading this.
Please let me know if you do and if you'd like to read some updates in this topic, or I won't be wasting my energy on this forum thread.

The 1.9.0 release has some reorgs inside the source code and the architecture in general.
It should perform even better.

Since the UTXO db only gets updated on new blocks, I decided to simply the approach and just keep a snapshot of it on the disk.
It's being flushed to disk after each update (new block), but at a vert low priority so it doesn't take too much system resources.

Now the whole UTXO database is caried by the single UTXO.db file, which is about 2.4 GB big.
When you start the new client, it should convert your old "unspent4" DB to the new format automatically.

Also, if you want to bootstrap a new node, you only need this file and the block headers file (about 60MB).
I can distribute the quick bootstrap files, if anyone is interested.

Which brings me to the point that the recent versions of gocoin node can work with purged blocks database.
If your blocks happen to be taking too much space, just close the client and execute:
Or:
This will either purge all blocks or only up ot the given height.
Follow the instructions on the screen as you need some extra copying/deleting, just to be safe and not delete you blocks DB by accident.

Again: make sure to not have the client node running while using the bdb tool.

Sorry for not updating for so long.

I've been travelling for the last couple of months, so no development, just field testing.
It should last for a couple of more months and then I will be back.

Before I left, I made version 1.8.0 - I've just tagged it, as it's been quite stable.
I have it running on my windows laptop and a linux VPS and it has been working fine for the last couple of months.
Once I had to restart it, as it hanged after over 50 days of uptime - I have some ideas why, will try to fix it, but first would like to see it happening again.
Also it well rejected the invalid block mined recently by BU node, although there was also something I will have to look at, as I found one debug of when it happened weird.

Anyway, the current 1.8.0 has segwit support and a few other cool things, and it's stable.
I got rid of the downloader, as the client is now able to sync the chain just as fast (with some additional command line switches).


As for the future development.

When I come back home, I want to make it possible to bootstrap (and further run) the node with only the block headers and a corresponding UTXO snapshot.
Basically it will be possible to run the full node without downloading and keeping the blockchain on the disk.
It will only keep the headers and the UTXO db. At this moment it's about 3GB of data.
Obviously it will be an optional feature and you will still be able to keep (and serve) all the blocks, if you please.

I've been thinking to establish some industry-wide standard for distributing a secured (first by the community signatures, later ideally by the chain protocol itself) UTXO snapshots, but it's not going to happen any soon, as it seems to be involving shit loads of politicking.
So I will just start with my own little solution and see from there, because I find this feature quite necessary already, as new people are very reluctant to try full nodes since they need so much time to bootstrap. Plus the blocks on my travel laptop have already taken so much space that I had to start removing porn, and that's really disturbing

Sorry, I don't have an opinion.

I'm too much involved in my bitcoin stuff and have no time to look into alternative-crypto projects.
I rarely even look in other bitcoin projects - mostly if they have something I might want for myself

Great work you are doing here !
What would you think about decred.org from the developers of other golang bitcoin client ?

Just a quick update on the latest development, to let you know that I am still alive and the project is being maintained

The latest release 1.7.2 comes with some nice features.

I've been happy to learn that the core team finally did something to speed up the blocks' propagation - that came with 0.13.
Therefore with pleasure, I have implemented BIP-152 into Gocoin - I love this new method.

I've also optimised the algorithm of dropping worse (slowest) peer, in order to be connected to the best performing ones.
Now it considers not only the ping, but mostly the number of a new blocks we have received from a peer.

The full list of changes, as always, in the changelog.

Just want to let everyone know that with the new HEAD (to become version 1.7.0) there is kind of a major change in the node.       

The node client does not keep any wallet data anymore - not in the file system, nor it RAM.
Instead the wallets' public addresses are stored in the LocalStorage, on the browser that operates node's WebUI.
The improvement is that a balance of any (P2SH or P2KH) address can be fetched instantly.
Balances of specific wallets are now fetched by the browser while loading a relevant page (Wallet or SendTx) of the WebUI.
The content of the HTTP request (list of addresses) is not stored at the node's side.

At this moment you can only fetch balance of P2SH and P2KH addresses - the support for stealth addresses has been turned off.
Additionally, all the wallet features have been disabled in TextUI.

Because it needs to build the database of all the addresses and their balances, the node launches significantly longer now.
It also requires more RAM - I changed the RAM requirement to 8GB.

Please note that (to save memory and improve performance) by default the balance database does not account for unspent outputs with less than 0.001 BTC.
You can change that by modifying the config value "AllBalances.MinValue" (set it to 1 to have everything listed).


I believe this is a change in a good direction.
It allows multiple users (wallet owners) to share the same gocoin node; improves performance of fetching the balances, as well as the privacy by not storing the content of the requests.

If you like the old functionality (with stealth addresses and wallet files stored at the server), checkout version 1.6.4

A friend of mine modified the original tool for me.
Now it's feeding the test blocks (one-by-one) to an external command.

This allowed me to make a simple wrapper for the gocoin library in order to test the blocks compatibility.
All test pass, except for b61, which I am not going to fix as it would decrease performance of the lib, just for the sake of the test. So I implemented a work-around in the wrapper, for this specific test.
All the rest is passing - up till the block "b1004", which is the last one in the test suite.

If anyone would like to test it by himself, or use the java tool my friend made to test their own software, it's all in here:
https://github.com/piotrnar/btc_block_validator

Start the tool using
.. it will be executing a command next_block.exe in the current folder for each new block.
The block is always in file named dupa.bin - just ignore the first 8 bytes.
Additionally next_block.exe is called with 5 command line arguments. One of them is the hash of the top block expected to be after the block processing is done - that's how you verify whether the block from dupa.bin is properly processed/rejected.

thanks. ipv6 suport will eventually come. it just hasn't' got high enough priority so far.

nice client and the downloader is really fast, cool work!

but a question, i see that you "only" support ipv4 in the client. will you plan to support ipv6 also?
Then I can setup some more nodes

No problem, you're welcome.

OK... After a several changes in Gocoin, currently it does everything up till (including) b89.

Two blocks do not return the expected result:

b48 - block with the time stamp more than 2 hours in the future.
I cannot properly against the blocks data from the disk, but I trust I didn't screw up this check.

b61 - same ID of a coinbase TX.
I'm not going to be adding a check for this, as it can only happen with v1 bocks, which are long gone.

In a future, when I find some more time, I will try to somehow fix the tests tool; make it stable and test all my further gocoin releases against it.
Will also do the remaining "massive reorgs" tests...


Thank you once again, @hhanh00 - I really appreciate your input.
It helped me to find (and fix) another few compatibility issues, on the blocks checking.
Some of the checks in Gocoin were too strict (e.g. b64), others (e.g. b66) were not strict enough.

Yes, I know.
I have been testing it manually, going down the source code.
The code says whether a specific block is expected bo be accepted or rejected (the second parameter to BlockAndValidity)

At this moment all up to block #55 is OK.
Then there are some timestamp tests and things get complicated..
But so far so good.

Cheers

Ok. It's a bit tricky because the block file doesn't have the expected state of the blockchain, just the block data.

In a previous project, I actually did the same thing. You may want to take a look at https://github.com/hhanh00/bitcoin-akka-tutorial
I patched the test generator so that it writes out the chain state and hook up a test driver.

Sorry - haven't noticed.

Thanks, I find the blocks file - will try to feed it to node and see...
Think it's an easier approach, as achieving network compatibility with this tool is not my goal.
But I want to get the block consensus tests out of it.

The pastbins are truncated. Actually the first 100 blocks are not interesting. You could edit that part out.

It creates such a file every time your run it. That's why it takes a while to startup.


Change these lines and you can keep the file around.

Do you know if it is possible to just make this tool output the sequence of blocks and do not talk to my node at the network?

Thank you. That's very useful information. I guess I can somehow imitate it, for a purpose of the test.

I's also sending empty invs - I'm assuming this is also supposed to serve some purpose..

Either way, I think now my biggest mistery is what it doesn't like about the "b4"

test tool: http://pastebin.com/YpkHJMeW

gocoin: http://pastebin.com/QdUCHZrH