Topic: subvertx command line utilities (proof of concept using libbitcoin) - page 2. (Read 17823 times)

amazing. just awesome.

This looks very useful.

Nicee!

You don't copy paste though. You write to a file then read it in.

i.e

mktx send < /tmp/serialised-tx

And that file can just as easily be a named pipe (which are faster and not really files).

Or you can run it through a base64 decoder:

echo ... | openssl base64 -d | mktx send

I don't need to derail your thread with my own development activities, so we could PM about this.  In a nutshell, I'm doing Tx construction on the command line, and I usually end up with potential Tx's I want to broadcast, depending whether my SelectCoins did what I wanted.  If so, then I copy the serialized-tx hex into the tool for broadcasting (right now, Nibor's webpage), otherwise I ignore it, tweak my code and try again.  Copying raw binary from ASCII displays always made me uncomfortable... but maybe it works...
(I see that you had direct output from another process in mind for piping into your mktx-send operations, but I didn't)

OK!

But why do you need hex? Just use "openssl base64 ..." to base64 decode your stream and pipe it on.

Also I don't think you can use echo ..., you'd need to use cat to read until EOF. You can create a named pipe then open the mktx send with that pipe in bash.

Good intro to named pipes: http://www.linuxjournal.com/article/2156

Thanks for thinking of me, genjix   How do I put the serialized transaction into the stdin?  raw binary characters?  Would it be:


It's a tad easier if I can pass around hex instead.  On the other hand, I will probably just pick apart your code, adapt to my client, and integrate it into my codebase   Until then, I'll probably use it for experimenting until I get to that point.  Looks really good, thanks for releasing this!

Made updates to subvertx (updated OP but posting below also). Packages have also been refreshed so you may wish to update.

I split the process between creating a transaction and actually broadcasting it into 2 parts rather one command (for etotheipi mainly ). This artifice allows us all the whizz-bang toys like offline transactions, piping a transaction to multiple sends, a flyweight client, piping transactions over network shares to a single bitcoind and whatever guile schemes you can imagine. In shell we trust.


There are 2 commands there. The 'create' actually constructs the transaction and then dumps its binary network format to STDOUT. 'send' reads from STDIN and sends it to the network. By default it sends to localhost:8333, but you can change it using the --host and --port options.

Creating a transaction

A transaction consists of inputs. You can use blockexplorer to look them up. Here are some examples.

1. We want to send from transaction c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c's 0th output (which was sent to an address we own) to 2 addresses: 0.02 BTC to 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s and 0.58 BTC to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z

There is 1 previous output:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1

There are 2 recipients:

12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000
14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000

Note that we use internal BTC amounts (decimal value * 10^8 - see help text).

Now to spend that output, it has to have been sent to an address you own, hopefully generated using the priv tool earlier


If we wish to store that output we can use a redirection > to save it. Maybe store it on your USB and take it to another computer. Or we can pipe it straight to the send command.


The hash at the end is your transaction hash. If you do: tail -f .bitcoin/debug.log, or you look it up on bitcoincharts.com/bitcoin you will see that hash without the spaces (i.e CTRL-F for c53ea3 to see your transaction).

If you noticed, I loaded several private keys there using -k (or --keypair) KEYPAIR-NAME@FILENAME. You then reference which keypair belongs to which input this way.

2. Sending from three, different outputs to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z.

Previous outputs:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

All the outputs (in this case) use a different private key: ./keys/foo, ./keys/bar and ./keys/abc.

We can load the keys and name them (to have a way to refer to them) using:

 -k foo@keys/foo
 -k bar@keys/bar
 -k abc@keys/abc

And then indicate to mktx which key belongs with which inputs from above:

foo@00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
bar@637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
abc@85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

1 recipient:

14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:60000000

Thanks, that looks like a really nice set of tools.

Updated the OP with the ebuild made by Kamil Domański.

No problem for me.  For now, I plan to have the satoshi client open anyway, so I can just connect to localhost and avoid peer discovery entirely.  In the future I might work on something more independent, but for now this is perfect!

I'm not too familiar with the various licenses and terminologies around them.  It looks like your code is also AGPL v3, so there shouldn't be any compatibility problems.  But I would like to do things right, and apply attribution regardless if I really need to -- I like to give credit where it's deserved.  It sounds like it won't be difficult (licensing-wise) to bring in some of your code.  I will battle the code details later.

@grondilu, I changed it according to your suggestions.


I think you want this:
https://gitorious.org/libbitcoin/subvertx/blobs/master/src/mktx.cpp

However it doesn't do node discovery yet, so you have to know who you're connecting to beforehand.

You will need to modify mktx to be a new program that reads in a byte stream for the raw transaction.

See this:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/tests/dialect-test.cpp

If there is a problem reading the stream then it will throw an exception called end_of_stream. That exception is thrown by the internal deserialising object inside that dialect object:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/include/bitcoin/util/serializer.hpp


Of course. It's a library. Don't worry about attribution.

Also, I changed the PPA from zgenjix -> genjix. Please update your sources.list and add the new keyserver
$ wget -q "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x3D1978972EC26E7B" -O- | sudo apt-key add -

+1 To this.  I'm extremely interested in having a minimal networking interface just for getting txs not in the blockchain yet, and transmit new blocks that we just signed.  How much work would that be--it sounds like tests/net is like 90% of the way there.   If I'm working on an AGPL v3 project, will I be able to use your code (modified or not) in my project with proper attribution?

I've been dreading how I'm going to deal with networking in my client, but this sounds like it will be the perfect solution!

This looks awesome.  Is it possible to connect to another host, so that we can make a transaction without having bitcoin actually running nor installed on localhost?

This would be great, as it would be the cheapest (and possibly the safest) to use bitcoin.

I also think this should be related to the Bitcoin Off-The-Grid project.

You're a god damn rock star!

btw in libbitcoin there are a bunch of other tools but I didn't include them as they're more complicated to setup:

examples/poller
download the bitcoin blockchain into a postgresql database

examples/subvertx/balance
find the balance of a bitcoin address using the downloaded database after running the poller above

tests/net
connect, handshake (swap versions, wait for veracks) and then download the first 500 blocks.

These are a simple set of utilities for working with the bitcoin network/protocol. Together you can use them as a 'flyweight' client. I like small sets of inter-working programs because it is the UNIX philosophy about building small bricks which you can piece together to create a set of highly configurable inter-locking mechanism of programs and daemons.

This is a proof of concept built off of the work-in-progress libbitcoin.

Subvertx is a suite of 4 tools:
- poller, downloads the blockchain from a bitcoind into a PostgreSQL database.
- balance, queries the PostgreSQL database downloaded with poller to find the balance of a bitcoin address.
- priv, create a new private key, sign data, verify data or show the bitcoin address of that key.
- mktx, connects to a bitcoind and sends some money from an output to a new address. You can use this in conjunction with a site like blockexplorer.com as a tool to make transactions very simply. Or even make offline transactions or use it like a super lightweight client that can run on embedded devices.
- txrad, like the website http://www.transactionradar.com/. It connects to 100 random nodes and then monitors for transaction messages. Simply run txrad.

Show help:

Make a new private key, sign and verify a transaction hash:

Show your bitcoin address:

Send bitcoins (offline & flyweight transactions possible)

This is done in 2 steps: creating the transaction, then piping it to send the transaction somewhere. Pipes allow you to do really clever things like send to multiple nodes, send it over a network file share, buffer it to send later (like on USB stick with a different machine that doesn't have your private key).


There are 2 commands there. The 'create' actually constructs the transaction and then dumps its binary network format to STDOUT. 'send' reads from STDIN and sends it to the network. By default it sends to localhost:8333, but you can change it using the --host and --port options.

A transaction consists of inputs. You can use blockexplorer to look them up. Here are some examples.

1. We want to send from transaction c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c's 0th output (which was sent to an address we own) to 2 addresses: 0.02 BTC to 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s and 0.58 BTC to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z

There is 1 previous output:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1

There are 2 recipients:

12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000
14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000

Note that we use internal BTC amounts (decimal value * 10^8 - see help text).

Now to spend that output, it has to have been sent to an address you own, hopefully generated using the priv tool earlier


If we wish to store that output we can use a redirection > to save it. Maybe store it on your USB and take it to another computer. Or we can pipe it straight to the send command.


The hash at the end is your transaction hash. If you do: tail -f .bitcoin/debug.log, or you look it up on bitcoincharts.com/bitcoin you will see that hash without the spaces (i.e CTRL-F for c53ea3 to see your transaction).

If you noticed, I loaded several private keys there using -k (or --keypair) KEYPAIR-NAME@FILENAME. You then reference which keypair belongs to which input this way.

2. Sending from three, different outputs to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z.

Previous outputs:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

All the outputs (in this case) use a different private key: ./keys/foo, ./keys/bar and ./keys/abc.

We can load the keys and name them (to have a way to refer to them) using:

 -k foo@keys/foo
 -k bar@keys/bar
 -k abc@keys/abc

And then indicate to mktx which key belongs with which inputs from above:

foo@00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
bar@637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
abc@85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

1 recipient:

14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:60000000


poller

To use the poller you first need to import the schema which can be found in /usr/share/libbitcoin/bitcoin.sql (a pre-downloaded blockchain is here).


Poller is doing full blockchain validation. It will take a long time since it's an SQL database and it's validating all the blocks.

Balance needs the full blockchain otherwise you might not get an up-to-date balance reported back. It's fairly simple to use:

Ubuntu packages

The PPA can be viewed on launchpad.

Add these 2 lines to the end of your /etc/apt/sources.list


This PPA also has a package for libbitcoin-dev although the API is undocumented at the moment.

Gentoo ebuild

Kamil Domański has made a gentoo ebuild.


Source installation instructions

Source code for the non-Ubuntu paupers:
https://gitorious.org/libbitcoin/libbitcoin
https://gitorious.org/libbitcoin/subvertx

You will need:

  - g++ 4.6
  - cppdb
  - boost
  - libpq-dev
  - postgresql

Build cppdb using the instructions on their site.

  $ svn co http://cppcms.svn.sourceforge.net/svnroot/cppcms/cppdb/trunk cppdb-trunk
  $ cd cppdb-trunk
  $ mkdir build
  $ cd build

You can use -DCMAKE_INSTALL_PREFIX if you wish to install to a non-standard place such as a local directory.

  $ cmake -DCMAKE_INSTALL_PREFIX=/tmp/cppdb/ ..  -DPQ_BACKEND_INTERNAL=1
  $ make
  # make install

Clone repo and build.

  $ git clone git://gitorious.org/libbitcoin/libbitcoin.git
  $ cd libbitcoin
  $ autoreconf -i
  $ ./configure
  $ make
  # make install

Don't forget to initialise and create a database (see first post above for more details how).

  $ psql bitcoin < bitcoin.sql

Same for the subvertx suite

  $ git clone git://gitorious.org/libbitcoin/subvertx.git
  $ cd subvertx
  $ autoreconf -i
  $ ./configure
  $ make
  # make install

I can't stress enough for people to use this at their own peril as it's alpha software.

Thanks goes to,

Kamil Domański (kdomanski)
Jan Engelhardt (se7) <[email protected]>
Patrick Strateman (phantomcircuit)
Denis Roio (jaromil)
Amir Taaki (genjix) <[email protected]>