Topic: cbitcoin - Bitcoin implementation in C. Currently in development. - page 10. (Read 20319 times)

Yep; the interpreter is there and most of the ops; except (of course) for the difficult ones that actually check signatures :-)


Ain't that the truth.


Just abstract your string access to a set of routines, then you can make that decision later.  First pass: just use library code.

Thanks. It may be useful to refer to parts that I can't get from bitcoinj such as the script interpreter. 

It seems I've made a little problem. I'm storing bitcoin addresses backwards at the moment since my base-58 conversion code works with little-endian data. Probably makes sense to reverse the result of the base-58 conversion so they are stored the right way around instead of reversing hashes and whatever.

etotheipi was right about the endianness issue! :-)

I have to make some interesting decisions. Should I have a reference counted string structure? It might help with the bitcoin address strings.

Development has kind of stalled lately, but in case it's of any use to you..

https://github.com/andyparkins/additup

It's C++ so won't be copy-and-pastable.  But it talks enough of the protocol to receive and parse blocks and transactions and has base58 (or anything else if you wanted it) support; plus a small test app to manipulate keys and addresses.

The base 58 conversion is working, time to move on. Could do with some more varied testing but for now the tests will do (Contributions to testing are more than welcome).

I'm all ears to suggestions. Now is the best time to pass suggestions to me as it's early on in development.

Remember, anyone would be free to put in cryptography code directly into their own modified versions of the library. Makes sense, at least for now, to use function pointers I think.

All true.

Nevertheless, a wide swath of very fundamental libs provide hooks for similarly fundamental services, most notably memory allocation and m-t locking primitives.

...regardless, all this is really discussing useless details.  The main focus, I hope, should be that cbitcoin is actually doing something unique to bitcoin like parsing blocks and talking network, and not simply wasting a lot of time reimplementing basic software services like crypto or hash or data codec.

For any platform where the above would be useful, the following is sufficient:
1) declare the appropriate entry points as "extern";
2) use the platform's native dynamic linking apparatus.

Using additional layer of indirection is completely pointless and has the following disadvantages:
1) it defeats many optimizations that the tool-chain could apply.
2) it defeats many automatic static-analysis and debugging tools.
3) slows down the resulting code (especially if the CPU doesn't have a branch predictor) and increases the energy consumption during execution (more CPU instructions and more cache misses).

What I meant to convey is the following: the Bitcoin protocol itself contains a multitude of design errors related to byte-order and word-alignment. Those errors are compounded by additional errors in the Satoshi client implementation. If you really plan on writing a portable code you'll need to fix those errors in implementation and work around the errors in protocol to maintain the wire compatibility. This is a nontrivial undertaking. If you don't have access to a native big-endian platform like SPARC or z/Arch, then the cheapest way to debug those problem is by working under OS/X.


What I meant to convery is the following: the Satoshi client implementation is  non-deterministic and non-testable (even while single-threaded) due to to horrible architectural choice of wrapping a stochastic knapsack solver for coin selection and fee calculation inside a control-inverted subroutine that is used in sendtoaddress() API. You'll need to seriously re-architect the API to come up with a deterministic library that will produce deterministic fees with non-trivial wallets.

It is perhaps conceivable that there are some one set of cryptography functions that provide better performance but less security and another set that are slower but provide greater security. A programmer may switch the functions to use the increased security when it is needed or whatever.

But this is not the point in using function pointers, by using function pointers the cryptography functions can be chosen by the programmer using the library and can be configured when the program starts. This is the primary reason. With great flexibility the library can be used as a single dynamic library across various applications; indeed the library will not need to be recompiled for changing requirements.

Even though swapping the crypto layer at runtime is unlikely, function pointers are the more flexible alternative as they allow the developer to choose his crypto layer without having to recompile the bitcoin library.

Just have a look at the OpenSSL library: When you want to swap the memory allocator there, you also do it through function pointers, not preprocessor macros. Still, for obvious reasons nobody will want to switch the allocator at runtime...

1. So you are saying test my code works on both x86-64 and ppc architectures? Well I'm writing it in C with no dependencies or assembly so no reason why not as long as I follow the usual guidelines for programming portable C code.

2. You mean to keep all the components separated? Well this is the plan and I was asking if I should completely separate the cryptography code. With function pointers dependency injection can be done. I'll likely do something like that for threading functionality, without having a dependency on a threading library. I could include the cryptography with the library but it seems it might cause controversy and leaving the cryptography code outside the library satisfies portability and flexibility concerns.


I don't see why not allow people to choose what they want. It cannot be hard to create wrapper functions around these libraries to pass these functions with functionality like void (*sha256)(u_int8_t *, u_int16_t ,u_int8 [32]) or something similar.

Would Nettle fill the bill ? Crypto routines only, no SSL/TLS. I think GnuTLS has a Nettle extension for ECC/ECDSA.

http://www.lysator.liu.se/~nisse/nettle/

No, nobody is planning on swapping the low-level crypto layer at runtime. If you want to abstract it do so using preprocessor macros. I have two other important suggestions for a from-the-scratch implementation:

1) at all times maintain the code as endian-neutral and with proper alignment. If you don't have a Macintosh OSX 10.[456], then build yourself a Hackintosh with Rosetta and test your code using "gcc -arch ppc"
2) resolve the critical inversion of control issue that is present in the Satoshi client. I wrote an post about this a while back. I'll put the link directly to it once I find it. Here they are:

https://bitcointalksearch.org/topic/m.538044
https://bitcointalksearch.org/topic/m.664634

Good luck.

Then link statically.  Or use one of the many embedded tools out there that ensures your lib has only the code your filesystem actually uses.

This is a solved problem, and does not exclude OpenSSL use in any way.


And OpenSSL likely supports outdated OS's and tools...  they maintain support for MS-DOS to this day because that remains a common embedded target.

Would people be more universally happy if my library required developers using the library to pass cryptographic functionality to the library using function pointers? That way people can use whatever cryptography library they like that matches their needs, and my library will focus on everything else unique to bitcoin.

Good idea?

Oh, yes, I should clarify.  VeriFones use a persistent battery-backed RAM-based file system, so disk space essentially equals RAM.  I am referring to the size of the library, not how much memory it allocates.  And some of the other devices I've developed for have very limited flash storage where fitting the entire compiled openssl lib takes up significant space.  I suppose it's laughable to consider a couple megabytes to be "gobs" but in the context of these cheapo devices, it is.  Also for the VeriFones they don't use GCC so sometimes things take refactoring for their dinosaur compiler to be able to handle it (and makefiles in particular).  Of course I understand having outdated tools is sucky, but that's also why you can find these things on eBay for so cheap.  The Omni 3200 model has an even older DOS-based compiler, but someone who bites the bullet and makes a working app for it suddenly breathes new life into a $30-on-eBay all-in-one-computer with a printer to boot.

Even with security bugs (not saying security bugs are OK), but possession is nine tenths of the law with these old school devices.  Suppose somebody finds a way to be able to perform a side channel attack on it and steal keys with RF emissions or something, they also need to gain physical access to the device to do it, making a widespread attack not all that feasible.  Their network support is rudimentary (often the units with Ethernet ports are merely Ethernet-to-Serial converters on an internal tty that is completely silent unless you write an app to access it) so the odds of someone making a key-disclosing network attack against them is pretty much moot.

In order to have a super-lean runtime and dev environment, for example...

Ummm...

OpenSSL is heavily used in the resource-constrained, embedded space, and runs on Windows, Linux, uclinux (embedded linux), VxWorks (an embedded OS), OSX, Solaris, HPUX, OSF1/Tru64, IBM mainframes, ancient MS-DOS systems, netware, and others old and new.  It works very well on almost every known 32- and 64-bit  platform in existence.

It does not require "gobs of memory"; the state structures for AES or SHA algorithms are almost always smaller than 1,024 bytes.  Compiled code size is already small and compact.

Copying OpenSSL code into your own codebase will not save any memory, regardless of linking style.  Libraries are internally split into modules and demand-paged in.  At best it will save disk space for unused code, at a cost of always lagging behind whenever security problems in the code are found.

Under the images I can put "Pictures created by Alan Reiner, developer of Bitcoin Armory (http://bitcoinarmory.com/)"?

I'm hoping following bitcoinj will also help (It's very easy to read), the issue is that bitcoinj doesn't implement everything so I'll have to look into other code that does implement things like the script properly. The C++ client has the code but it harder to read. Documentation no doubt helps me understand what I'm doing rather than do a piece by piece port, so I can make better code.

Go ahead.  Attribution is preferable, but I'm really just happy if other people get use out of them.  Maybe I can save you the 2 days I spent battling endianness and the OP_CHECKSIG wiki page trying to figure out how the hell to implement it!