Topic: C# Node (Read 4688 times)

Hey, work is definitely still ongoing. There haven't been any source code updates for a few weeks... I've had two other developers join the project, which I'm super excited about!

We're all currently busy getting caught up with each other before making any more updates. Going forward we'll have much more up front design work and documentation; the current source code is a prototype.

I'm planning on posting an update to the site over the weekend. Thanks for checking in!

Paul

Hi!

Do you still work on this?

This weekend I'd like to start getting the blocktester (https://github.com/TheBlueMatt/test-scripts/) setup for when I start implementing things exactly. Does anyone have any experience integrating with this tool?

I've done some work to split my block storage into headers and transactions, I'm able to download and chain the block headers separately from full blocks now. I'm also able to pull individual transactions out of the database, whereas before I was retrieving full blocks just to get one transaction. The initial syncing is working much faster now, I think I should eventually be able to download and sync the full blockchain within an hour, given a proper storage back-end. (not SQLite on a spinning disk )

I'm starting to feel pretty confident in the approach I've taken to my storage system and feeding it into the blockchain calculator. I have a lot of testing and documentation to catch up on now that I've proven that out. I also want to get a very basic lite storage system up and running at some point so I can make sure that I'm designing the blockchain calculator in a flexible enough manner to support it. I'll need to return some additional data so that a lite storage provider can determine what data it can remove, and I need to make sure I don't have any accidental dependencies on full historical storage in the calculator.

The way that I'm currently calculating the UTXO isn't actually correct so I need to get that taken care of. I'll need to properly implement BIP 30, for example. General correctness with regards to bitcoind I've left mostly off the table at this point anyway. That is going to be a huge undertaking that will require significant testing, so I don't want to be approaching that piecemeal. Correctness should ideally be isolated to the blockchain calculator and the scripting projects. The scripting project is essentially unimplemented at this point, I've just done the bare minimum to get me up to about block 158,000.

That's where things stand today, for any of you following along.

Paul

I just found this horrible bug in my == method for 32-byte hashes. I guess it goes to show how unique these hashes really are that I only discovered this now. Note the || operator instead of &&.

        public static bool operator ==(UInt256 left, UInt256 right)
        {
            return left.part1 == right.part1 || left.part2 == right.part2 || left.part3 == right.part3 || left.part4 == right.part4;
        }

I have, thanks. I reworked my SQLite storage provider a bit so that I have an application wide reader-writer lock on a single connection and I've enabled WAL as well. It seems to be performing much better now.

I'm trying to keep my database layer as unintelligent as possible, the main thing I rely on the database for outside of key/value is finding holes in the data. You could write a simplified storage layer that didn't hold onto all historical data and didn't account for holes in the data and then wire that up to the blockchain calculator, if you wanted.

Have you looked at stuff like WAL?

Whatever you use, keep it to standards so that it remains portable.  I would be even tempted to use whatever database you do as a simple key, value store.  Assuming this remains a client tool and not some massive server undertaking you don't need any data level optimizations.  Without the need for data level optimizations you can keep it very portable as far as the back end.

Thanks for the info. Sorry to hear about your troubles with the government, that sucks. Any light at the end of the tunnel?

I took a look at dotCover and I'm able to generate a test coverage report with the trial version of that. They have an open source license but I don't meet the standards yet, it's for more established projects. I'll see about applying for that eventually and I can use the trial version in the meantime.

Well currently our company (Tangible Cryptography) services are halted as we dance with the state but we ran the site fastcash4bitcoins.com The bitcoind interface was used to track customer deposits and update a SQL database.  All high level functions are between the application and the RDBMS.  A service polls bitcoind periodically and updated the database with low level transaction data.

Thanks for the offer! I'll let you know what I find out with my own company, no ulterior motive suspected.

I had started this project as a learning exercise to understand the Bitcoin tech better, but now I think that I've made enough progress to turn it into a real library. Wrapping it up in a proper Windows service and enabling RPC (presumably JSON) are definitely on my TODO list.

What kind of things are you currently using bitcoind for?  Understanding some use cases would really help me flesh things out. All of the current code represents prototyping, not a fully thought-out design.

Yeah very nice set of capabilities.  The fact that it is "built into windows" and requires no installation makes it useful for rapid deployment at least in windows environment.   With a data store abstraction layer a variety of storage systems could be supported.  ESENT could just be the default (providing no install instant availability for windows users).

Performance even on random lookups seems to be exceptional.  The bad news is that it looks like only certain datatypes can be used for keys.  Would be nice to store blocks with blockhash in binary (256 bit BigInteger) as the key.  Not sure if this is a limit of ESENT or just the current implementation of the managed code wrapper).  Still if performance using a hexadecimal string representation of that block hash has good enough performance it many be a non-issue.

One thing that may be worth considering is use ESENT for storing the blockchain and then use something like SQL Server (w/ Express for users without installed RDBMS) to store: wallet data (keypairs, tx history, user metadata), the UXTO, and a smaller table of just block headers.  The UXTO is less than 250MB, block headers since genesis block are about 15MB and most wallets are under a couple MB.  We have one massive (8,000+ keys, 20,000 txs) wallet and it is only 40MB.  All that should fit fine within the constraints of Express.

I am making a project to dump the blockchain into ESENT and benchmark performance (sequential inserts, random reads).

That looks very interesting, particularly the PersistentDictionary, thanks!

If you find nothing else our company has additional visual studio enterprise licenses which are not being used.  I imagine we could work something out (i.e. if necessary hire you as a contractor for 1 bitcent per year for the sole purpose of developing c# open source code to the public domain or under copyleft license, renewable annually).  I need to speak to counsel about it but if interested send me a PM. 

No ulterior motive here.  The public codebase for Bitcoin + .NET has been pretty much been non-existent.  C# is my preferred language so everything I have written has needed to be custom built (use a modified headless version of bitcoind running as a windows service with all interaction through a c# to JSON/RPC wrapper.    It would be useful to get a solid managed code library going (stop reinventing the wheel) if I can support that with a license it works for me.

Haven't used it myself but this appears promising: https://github.com/sawilde/opencover

Unfortunately no. I may see about using my work computer to run unit tests with the full version of Visual Studio, I just want to be careful about keeping this segregated from my work where I want everything released in the public domain.

You aren't by any chance a student are you?

I'd like to integrate code coverage results into the unit tests, but Visual Studio Express doesn't support generating code coverage results. Any recommendations out there on a good, free tool that I could use?

That's definitely the direction I'm coming at things from, although in the future I'd like to support lower trust levels as well.

Tonight I wired up the build to generate documentation, which I've placed on the wiki: http://pmlyon.github.io/BitSharp/apidocs/Index.html. There's no actual documentation yet, but now that it's integrated into the build I can work away at it.