.NET Blockchain Parser | Bitcointalksearch.org

Anti-Cen

member

Activity: 210

Merit: 26

High fees = low BTC price

Quote from: lontivero on November 19, 2014, 04:52:27 PM

the github repo: https://github.com/lontivero/BlockchainParser

If you find it interesting and want to collaborate then please do it.

Nice code friend and I did a hatchet job on the code but it does not work on the block.dat
I am using because I think it's been change over these days to use Berkeley-DB format
for the files but useful code none the less and well written

See https://bitcointalk.org/index.php?topic=2748620.new#new

Small, simple and easy to follow and is a good starting point for anyone going down this road

Martijnvdc

sr. member

Activity: 322

Merit: 250

Thank you very much. I like the license!

doof

hero member

Activity: 765

Merit: 503

I wrote this C# code to calculate the hash too https://bitcointalksearch.org/topic/block-hash-code-in-c-888078

Might be worth adding a column in the block table to contain the hash. Seperate thread can hash the block, and then could reference one block to another.

lontivero

full member

Activity: 164

Merit: 128

Amazing times are coming

Quote from: doof on December 03, 2014, 07:20:41 PM

This project has saved me a bunch of time, and we seem to both have the same goal.

Not a lot of btc, but I sent you a tip. Thanks.

I saw it and appreciate it. Thx

doof

hero member

Activity: 765

Merit: 503

This project has saved me a bunch of time, and we seem to both have the same goal.

Not a lot of btc, but I sent you a tip. Thanks.

doof

hero member

Activity: 765

Merit: 503

Difficulty isn't correct

doof

hero member

Activity: 765

Merit: 503

Quote from: lontivero on December 02, 2014, 12:11:51 AM

Quote from: doof on December 01, 2014, 11:21:49 PM

Quote from: lontivero on December 01, 2014, 08:50:16 PM

Quote from: doof on November 26, 2014, 11:47:09 PM

Quote from: lontivero on November 26, 2014, 08:55:06 PM

The couple WPF and Peer-2-Peer sound great but it is no real. If you want a p2p app you have to open a tcp socket.

WCF, not WPF. WCF on IIS for a bitcoind / daemon replacement.

http://msdn.microsoft.com/en-us/library/cc297274.aspx

Yes, I made a mistake when I said WPF instead of WCF however I am right when I say WCF is not a real option. You can share with me a lot of links to msdn and that won't change my opinion. Have you ever seen a p2p app developed with wcf? No real software uses wcf, they use sockets instead.

True, I havent seen any p2p wcf in the wild.

Can you put a sample block in github?

I have an abandoned project that is not the best example https://github.com/lontivero/peer2net (I have to redo it some day)

Ok. Ill be using a project like this and bitcoind in the short term. What happens if the processor is rerun, will duplicates be added? Could the block hash be a primary key to stop duplicates?

lontivero

full member

Activity: 164

Merit: 128

Amazing times are coming

Quote from: doof on December 01, 2014, 11:21:49 PM

Quote from: lontivero on December 01, 2014, 08:50:16 PM

Quote from: doof on November 26, 2014, 11:47:09 PM

Quote from: lontivero on November 26, 2014, 08:55:06 PM

The couple WPF and Peer-2-Peer sound great but it is no real. If you want a p2p app you have to open a tcp socket.

WCF, not WPF. WCF on IIS for a bitcoind / daemon replacement.

http://msdn.microsoft.com/en-us/library/cc297274.aspx

Yes, I made a mistake when I said WPF instead of WCF however I am right when I say WCF is not a real option. You can share with me a lot of links to msdn and that won't change my opinion. Have you ever seen a p2p app developed with wcf? No real software uses wcf, they use sockets instead.

True, I havent seen any p2p wcf in the wild.

Can you put a sample block in github?

I have an abandoned project that is not the best example https://github.com/lontivero/peer2net (I have to redo it some day)

doof

hero member

Activity: 765

Merit: 503

Quote from: lontivero on December 01, 2014, 08:50:16 PM

Quote from: doof on November 26, 2014, 11:47:09 PM

Quote from: lontivero on November 26, 2014, 08:55:06 PM

The couple WPF and Peer-2-Peer sound great but it is no real. If you want a p2p app you have to open a tcp socket.

WCF, not WPF. WCF on IIS for a bitcoind / daemon replacement.

http://msdn.microsoft.com/en-us/library/cc297274.aspx

Yes, I made a mistake when I said WPF instead of WCF however I am right when I say WCF is not a real option. You can share with me a lot of links to msdn and that won't change my opinion. Have you ever seen a p2p app developed with wcf? No real software uses wcf, they use sockets instead.

True, I havent seen any p2p wcf in the wild.

Can you put a sample block in github?

lontivero

full member

Activity: 164

Merit: 128

Amazing times are coming

Quote from: doof on November 26, 2014, 11:47:09 PM

Quote from: lontivero on November 26, 2014, 08:55:06 PM

The couple WPF and Peer-2-Peer sound great but it is no real. If you want a p2p app you have to open a tcp socket.

WCF, not WPF. WCF on IIS for a bitcoind / daemon replacement.

http://msdn.microsoft.com/en-us/library/cc297274.aspx

Yes, I made a mistake when I said WPF instead of WCF however I am right when I say WCF is not a real option. You can share with me a lot of links to msdn and that won't change my opinion. Have you ever seen a p2p app developed with wcf? No real software uses wcf, they use sockets instead.

Nicolas Dorier

hero member

Activity: 714

Merit: 662

I took a look at your code, I think you can win lots of time by switching to raw SQL instead of EF and trying BULK Insert.
http://msdn.microsoft.com/fr-fr/library/ms188365.aspx

I won't be surprised about a x10 perf improvement, maybe even more.

doof

hero member

Activity: 765

Merit: 503

FYI Im getting a null exception on block 215. Ill raise an issue and try fix tonight and make a pull request.

Nicolas Dorier

hero member

Activity: 714

Merit: 662

Quote from: doof on November 26, 2014, 11:49:01 PM

Quote from: Nicolas Dorier on November 26, 2014, 09:12:36 PM

I created an blockchain indexer (NBitcoin.Indexer), I wrote an article about it, and using it in prod. (Colored Coins Compatible recently)
It uses Azure Storage. I will surely provide a JSON and SOAP interface to it in the future. But for now this is at the bottom of my stackoverflow of work. Sad

Looking at it now. I'm doing a similar thing with coloured coins.

Don't hesitate to ask, I have not documented everything yet.
The idea is that the indexer does not deal with fork, the client does. I did lots of unit test you can run directly by setting up your own storage.
The fact that the indexer does not deal with forks make it horizontally scalable.

doof

hero member

Activity: 765

Merit: 503

Quote from: Nicolas Dorier on November 26, 2014, 09:12:36 PM

I created an blockchain indexer (NBitcoin.Indexer), I wrote an article about it, and using it in prod. (Colored Coins Compatible recently)
It uses Azure Storage. I will surely provide a JSON and SOAP interface to it in the future. But for now this is at the bottom of my stackoverflow of work. Sad

Looking at it now. I'm doing a similar thing with coloured coins.

doof

hero member

Activity: 765

Merit: 503

Quote from: lontivero on November 26, 2014, 08:55:06 PM

The couple WPF and Peer-2-Peer sound great but it is no real. If you want a p2p app you have to open a tcp socket.

WCF, not WPF. WCF on IIS for a bitcoind / daemon replacement.

http://msdn.microsoft.com/en-us/library/cc297274.aspx

Nicolas Dorier

hero member

Activity: 714

Merit: 662

Quote from: lontivero on November 26, 2014, 08:55:06 PM

NBitcoin can be useful for you, it doesn't have a way to write to a db (i think so) but it can write to disk.

That's why I am using Sqlite for some basic indexing, it is a good compromise between full db and ease of use. (even if I don't like the package dependency I took on Sqlite)

lontivero

full member

Activity: 164

Merit: 128

Amazing times are coming

The couple WPF and Peer-2-Peer sound great but it is no real. If you want a p2p app you have to open a tcp socket.

Quote from: doof on November 26, 2014, 06:09:29 PM

Then either write to disk or to a db

NBitcoin can be useful for you, it doesn't have a way to write to a db (i think so) but it can write to disk.

doof

hero member

Activity: 765

Merit: 503

Quote from: Nicolas Dorier on November 21, 2014, 08:40:45 PM

For NBitcoin, the interesting part for you is the "Manual scanning" from http://www.codeproject.com/Articles/784519/NBitcoin-How-to-scan-the-Blockchain

Quote

Install-Package NBitcoin

then,

Quote

BlockStore store = new BlockStore("E:\\Bitcoin\\blocks", Network.Main);
foreach(var block in store.Enumerate())
{
...
}

If needed, you can index block headers and blocks location in the directory into an SqlLite database as explained in the article.

Quote

Agree, specially if that project can have decent tests and be modular. http://bitcoinj.github.io/ (http://bitcoinj.github.io/) is a good example how it should be done, it's so good that many developers recommend it over bitcoind reference implementation for use in corporate environments.

Not for throwing flower to myself, but NBitcoin largely match it, and is spreading fast these days. Wink

Flowers are pretty! Nice, I've seen this article before, very good too. Would be good to have a WCF c# project to do the networking / peer to peer. Then either write to disk or to a db. Kinda a project im working on.

Nicolas Dorier

hero member

Activity: 714

Merit: 662

oh lontivero, I did not see you were the guy writing the question ! Cheesy

Sure you know, I remember that you contributed on the NAT part Wink

When you did, I think the BlockStore was already developed, this is an old class.

Anyway, if you need a feature that you think is missing from NBitcoin, you can ask me. If it can be useful to other, I don't care developing it.

lontivero

full member

Activity: 164

Merit: 128

Amazing times are coming

Yes, I know your work Nicolas and it's excellent really. I am a big fan of NBitcoin but I didn't know about the scanning capabilities (I can see it is a recent development).

Topic: .NET Blockchain Parser (Read 4689 times)