Pages:
Author

Topic: subvertx command line utilities (proof of concept using libbitcoin) - page 2. (Read 17742 times)

sr. member
Activity: 462
Merit: 250
amazing. just awesome.
full member
Activity: 154
Merit: 101
Bitcoin!
This looks very useful.
hero member
Activity: 714
Merit: 500
legendary
Activity: 1232
Merit: 1076
You don't copy paste though. You write to a file then read it in.

i.e

mktx send < /tmp/serialised-tx

And that file can just as easily be a named pipe (which are faster and not really files).

Or you can run it through a base64 decoder:

echo ... | openssl base64 -d | mktx send
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
I don't need to derail your thread with my own development activities, so we could PM about this.  In a nutshell, I'm doing Tx construction on the command line, and I usually end up with potential Tx's I want to broadcast, depending whether my SelectCoins did what I wanted.  If so, then I copy the serialized-tx hex into the tool for broadcasting (right now, Nibor's webpage), otherwise I ignore it, tweak my code and try again.  Copying raw binary from ASCII displays always made me uncomfortable... but maybe it works...
(I see that you had direct output from another process in mind for piping into your mktx-send operations, but I didn't)
legendary
Activity: 1232
Merit: 1076
Thanks for thinking of me, genjix Smiley  How do I put the serialized transaction into the stdin?  raw binary characters?  Would it be:

Code:
echo "" | mktx send

It's a tad easier if I can pass around hex instead.  On the other hand, I will probably just pick apart your code, adapt to my client, and integrate it into my codebase Smiley  Until then, I'll probably use it for experimenting until I get to that point.  Looks really good, thanks for releasing this!



OK!

But why do you need hex? Just use "openssl base64 ..." to base64 decode your stream and pipe it on.

Also I don't think you can use echo ..., you'd need to use cat to read until EOF. You can create a named pipe then open the mktx send with that pipe in bash.

Good intro to named pipes: http://www.linuxjournal.com/article/2156
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
Thanks for thinking of me, genjix Smiley  How do I put the serialized transaction into the stdin?  raw binary characters?  Would it be:

Code:
echo "" | mktx send

It's a tad easier if I can pass around hex instead.  On the other hand, I will probably just pick apart your code, adapt to my client, and integrate it into my codebase Smiley  Until then, I'll probably use it for experimenting until I get to that point.  Looks really good, thanks for releasing this!

legendary
Activity: 1232
Merit: 1076
Made updates to subvertx (updated OP but posting below also). Packages have also been refreshed so you may wish to update.

I split the process between creating a transaction and actually broadcasting it into 2 parts rather one command (for etotheipi mainly Wink). This artifice allows us all the whizz-bang toys like offline transactions, piping a transaction to multiple sends, a flyweight client, piping transactions over network shares to a single bitcoind and whatever guile schemes you can imagine. In shell we trust.

Code:
$ mktx
Usage: mktx COMMAND [ARGS]...

Commands:

  create    Create a new transaction and output the binary data
  send      Send a transaction to the network, reading from STDIN

Options:

 -p, --previous-output  Previous output in the form NAME@OUT:INDEX
 -k, --keypair      Load a keypair with an identifier NAME@FILE
            A single dash - for FILE will load from STDIN
 -r, --recipient    Specify a destination ADDRESS:AMOUNT
            AMOUNT uses internal bitcoin values
              0.1 BTC = 0.1 * 10^8 = 1000000
 -H, --host     Host of bitcoin node
 -P, --port     Port for bitcoin node
 -h, --help     This help text

Please email suggestions and questions to .

There are 2 commands there. The 'create' actually constructs the transaction and then dumps its binary network format to STDOUT. 'send' reads from STDIN and sends it to the network. By default it sends to localhost:8333, but you can change it using the --host and --port options.

Creating a transaction

A transaction consists of inputs. You can use blockexplorer to look them up. Here are some examples.

1. We want to send from transaction c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c's 0th output (which was sent to an address we own) to 2 addresses: 0.02 BTC to 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s and 0.58 BTC to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z

There is 1 previous output:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1

There are 2 recipients:

12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000
14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000

Note that we use internal BTC amounts (decimal value * 10^8 - see help text).

Now to spend that output, it has to have been sent to an address you own, hopefully generated using the priv tool earlier Wink

Code:
$ mktx create -p priv@c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c:0 -k priv@keys/privkey -r 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000 -r 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000
... funny binary output here ...

If we wish to store that output we can use a redirection > to save it. Maybe store it on your USB and take it to another computer. Or we can pipe it straight to the send command.

Code:
$ mktx create -p priv@c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c:0 -k priv@keys/privkey -r 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000 -r 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000 | mktx send
1 peers connected.
s: version (85 bytes)
r: version (85 bytes)
s: verack (0 bytes)
r: verack (0 bytes)
Connected
s: tx (258 bytes)
c5 3e a3 b4 d4 4c cf 67 31 73 17 b2 bd 8d 0a 99 46 d8 2d 67 6c 02 d0 d1 13 2b 11 8f 95 d0 7f 57

The hash at the end is your transaction hash. If you do: tail -f .bitcoin/debug.log, or you look it up on bitcoincharts.com/bitcoin you will see that hash without the spaces (i.e CTRL-F for c53ea3 to see your transaction).

If you noticed, I loaded several private keys there using -k (or --keypair) KEYPAIR-NAME@FILENAME. You then reference which keypair belongs to which input this way.

2. Sending from three, different outputs to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z.

Previous outputs:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

All the outputs (in this case) use a different private key: ./keys/foo, ./keys/bar and ./keys/abc.

We can load the keys and name them (to have a way to refer to them) using:

 -k foo@keys/foo
 -k bar@keys/bar
 -k abc@keys/abc

And then indicate to mktx which key belongs with which inputs from above:

foo@00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
bar@637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
abc@85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

1 recipient:

14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:60000000

Code:
$ mktx create -p foo@00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1 -p bar@637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1 -p abc@85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1 -k foo@keys/foo -k bar@keys/bar -k abc@keys/abc -r 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:60000000 | mktx send
1 peers connected.
s: version (85 bytes)
r: version (85 bytes)
s: verack (0 bytes)
r: verack (0 bytes)
Connected
s: tx (581 bytes)
c5 24 c5 55 aa d1 93 2c 24 c2 6e c2 04 39 a9 ca ef c4 9f 7c 0d f6 d6 bc cc ed 89 0e f9 80 b4 5c

legendary
Activity: 1526
Merit: 1129
Thanks, that looks like a really nice set of tools.
legendary
Activity: 1232
Merit: 1076
Updated the OP with the ebuild made by Kamil Domański.

Code:
layman -a bitcoin
emerge subvertx
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
+1 To this.  I'm extremely interested in having a minimal networking interface just for getting txs not in the blockchain yet, and transmit new blocks that we just signed.  How much work would that be--it sounds like tests/net is like 90% of the way there.

I think you want this:
https://gitorious.org/libbitcoin/subvertx/blobs/master/src/mktx.cpp

However it doesn't do node discovery yet, so you have to know who you're connecting to beforehand.
No problem for me.  For now, I plan to have the satoshi client open anyway, so I can just connect to localhost and avoid peer discovery entirely.  In the future I might work on something more independent, but for now this is perfect!

Quote
If I'm working on an AGPL v3 project, will I be able to use your code (modified or not) in my project with proper attribution?
Of course. It's a library. Don't worry about attribution.
I'm not too familiar with the various licenses and terminologies around them.  It looks like your code is also AGPL v3, so there shouldn't be any compatibility problems.  But I would like to do things right, and apply attribution regardless if I really need to -- I like to give credit where it's deserved.  It sounds like it won't be difficult (licensing-wise) to bring in some of your code.  I will battle the code details later.
legendary
Activity: 1232
Merit: 1076
@grondilu, I changed it according to your suggestions.

+1 To this.  I'm extremely interested in having a minimal networking interface just for getting txs not in the blockchain yet, and transmit new blocks that we just signed.  How much work would that be--it sounds like tests/net is like 90% of the way there.

I think you want this:
https://gitorious.org/libbitcoin/subvertx/blobs/master/src/mktx.cpp

However it doesn't do node discovery yet, so you have to know who you're connecting to beforehand.

You will need to modify mktx to be a new program that reads in a byte stream for the raw transaction.

See this:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/tests/dialect-test.cpp

If there is a problem reading the stream then it will throw an exception called end_of_stream. That exception is thrown by the internal deserialising object inside that dialect object:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/include/bitcoin/util/serializer.hpp

Quote
If I'm working on an AGPL v3 project, will I be able to use your code (modified or not) in my project with proper attribution?

Of course. It's a library. Don't worry about attribution.

Also, I changed the PPA from zgenjix -> genjix. Please update your sources.list and add the new keyserver
$ wget -q "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x3D1978972EC26E7B" -O- | sudo apt-key add -
legendary
Activity: 1428
Merit: 1093
Core Armory Developer
This looks awesome.  Is it possible to connect to another host, so that we can make a transaction without having bitcoin actually running nor installed on localhost?

This would be great, as it would be the cheapest (and possibly the safest) to use bitcoin.

I also think this should be related to the Bitcoin Off-The-Grid project.


+1 To this.  I'm extremely interested in having a minimal networking interface just for getting txs not in the blockchain yet, and transmit new blocks that we just signed.  How much work would that be--it sounds like tests/net is like 90% of the way there.   If I'm working on an AGPL v3 project, will I be able to use your code (modified or not) in my project with proper attribution?

I've been dreading how I'm going to deal with networking in my client, but this sounds like it will be the perfect solution!
legendary
Activity: 1288
Merit: 1076
This looks awesome.  Is it possible to connect to another host, so that we can make a transaction without having bitcoin actually running nor installed on localhost?

This would be great, as it would be the cheapest (and possibly the safest) to use bitcoin.

I also think this should be related to the Bitcoin Off-The-Grid project.
sr. member
Activity: 322
Merit: 251
FirstBits: 168Bc
You're a god damn rock star!
legendary
Activity: 1232
Merit: 1076
btw in libbitcoin there are a bunch of other tools but I didn't include them as they're more complicated to setup:

examples/poller
download the bitcoin blockchain into a postgresql database

examples/subvertx/balance
find the balance of a bitcoin address using the downloaded database after running the poller above

tests/net
connect, handshake (swap versions, wait for veracks) and then download the first 500 blocks.
legendary
Activity: 1232
Merit: 1076
These are a simple set of utilities for working with the bitcoin network/protocol. Together you can use them as a 'flyweight' client. I like small sets of inter-working programs because it is the UNIX philosophy about building small bricks which you can piece together to create a set of highly configurable inter-locking mechanism of programs and daemons.

This is a proof of concept built off of the work-in-progress libbitcoin.

Subvertx is a suite of 4 tools:
- poller, downloads the blockchain from a bitcoind into a PostgreSQL database.
- balance, queries the PostgreSQL database downloaded with poller to find the balance of a bitcoin address.
- priv, create a new private key, sign data, verify data or show the bitcoin address of that key.
- mktx, connects to a bitcoind and sends some money from an output to a new address. You can use this in conjunction with a site like blockexplorer.com as a tool to make transactions very simply. Or even make offline transactions or use it like a super lightweight client that can run on embedded devices.
- txrad, like the website http://www.transactionradar.com/. It connects to 100 random nodes and then monitors for transaction messages. Simply run txrad.

Show help:
Code:
genjix@random:~/subvertx$ priv 
Usage: priv [COMMAND] [ARGS]...

The priv commands are:
  new Generate a new private key and output to STDOUT
  sign Sign the next argument using the private key in STDIN
  verify Verify the next argument using the private key in STDIN
  address show the associated bitcoin address

genjix@random:~/subvertx$ mktx
Usage: mktx COMMAND [ARGS]...

Commands:

  create Create a new transaction and output the binary data
  send Send a transaction to the network, reading from STDIN

Options:

 -p, --previous-output Previous output in the form NAME@OUT:INDEX
 -k, --keypair Load a keypair with an identifier NAME@FILE
A single dash - for FILE will load from STDIN
 -r, --recipient Specify a destination ADDRESS:AMOUNT
AMOUNT uses internal bitcoin values
 0.1 BTC = 0.1 * 10^8 = 1000000
 -H, --host Host of bitcoin node
 -P, --port Port for bitcoin node
 -h, --help This help text

Please email suggestions and questions to .

Make a new private key, sign and verify a transaction hash:
Code:
genjix@random:~/subvertx$ priv new > /tmp/private_key
genjix@random:~/subvertx$ priv sign f5cffc95a4a83c23cb9f666c7cf552f27d9845778bb09a98d5169c461483ba41 < /tmp/private_key > signed_data
genjix@random:~/subvertx$ priv verify f5cffc95a4a83c23cb9f666c7cf552f27d9845778bb09a98d5169c461483ba41 "`cat signed_data`" < /tmp/private_key
1

Show your bitcoin address:
Code:
genjix@random:~/subvertx$ priv address < /tmp/private_key 
14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z

Send bitcoins (offline & flyweight transactions possible)

This is done in 2 steps: creating the transaction, then piping it to send the transaction somewhere. Pipes allow you to do really clever things like send to multiple nodes, send it over a network file share, buffer it to send later (like on USB stick with a different machine that doesn't have your private key).

Code:
$ mktx
Usage: mktx COMMAND [ARGS]...

Commands:

  create    Create a new transaction and output the binary data
  send      Send a transaction to the network, reading from STDIN

Options:

 -p, --previous-output  Previous output in the form NAME@OUT:INDEX
 -k, --keypair      Load a keypair with an identifier NAME@FILE
            A single dash - for FILE will load from STDIN
 -r, --recipient    Specify a destination ADDRESS:AMOUNT
            AMOUNT uses internal bitcoin values
              0.1 BTC = 0.1 * 10^8 = 1000000
 -H, --host     Host of bitcoin node
 -P, --port     Port for bitcoin node
 -h, --help     This help text

Please email suggestions and questions to .

There are 2 commands there. The 'create' actually constructs the transaction and then dumps its binary network format to STDOUT. 'send' reads from STDIN and sends it to the network. By default it sends to localhost:8333, but you can change it using the --host and --port options.

A transaction consists of inputs. You can use blockexplorer to look them up. Here are some examples.

1. We want to send from transaction c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c's 0th output (which was sent to an address we own) to 2 addresses: 0.02 BTC to 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s and 0.58 BTC to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z

There is 1 previous output:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1

There are 2 recipients:

12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000
14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000

Note that we use internal BTC amounts (decimal value * 10^8 - see help text).

Now to spend that output, it has to have been sent to an address you own, hopefully generated using the priv tool earlier Wink

Code:
$ mktx create -p priv@c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c:0 -k priv@keys/privkey -r 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000 -r 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000
... funny binary output here ...

If we wish to store that output we can use a redirection > to save it. Maybe store it on your USB and take it to another computer. Or we can pipe it straight to the send command.

Code:
$ mktx create -p priv@c524c555aad1932c24c26ec20439a9caefc49f7c0df6d6bccced890ef980b45c:0 -k priv@keys/privkey -r 12oabCifvHuxzXtYVGhkxVfWZDvKcU743s:2000000 -r 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:58000000 | mktx send
1 peers connected.
s: version (85 bytes)
r: version (85 bytes)
s: verack (0 bytes)
r: verack (0 bytes)
Connected
s: tx (258 bytes)
c5 3e a3 b4 d4 4c cf 67 31 73 17 b2 bd 8d 0a 99 46 d8 2d 67 6c 02 d0 d1 13 2b 11 8f 95 d0 7f 57

The hash at the end is your transaction hash. If you do: tail -f .bitcoin/debug.log, or you look it up on bitcoincharts.com/bitcoin you will see that hash without the spaces (i.e CTRL-F for c53ea3 to see your transaction).

If you noticed, I loaded several private keys there using -k (or --keypair) KEYPAIR-NAME@FILENAME. You then reference which keypair belongs to which input this way.

2. Sending from three, different outputs to 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z.

Previous outputs:

00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

All the outputs (in this case) use a different private key: ./keys/foo, ./keys/bar and ./keys/abc.

We can load the keys and name them (to have a way to refer to them) using:

 -k foo@keys/foo
 -k bar@keys/bar
 -k abc@keys/abc

And then indicate to mktx which key belongs with which inputs from above:

foo@00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1
bar@637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1
abc@85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1

1 recipient:

14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:60000000

Code:
$ mktx create -p foo@00535291532821f2e4879cf670f61396be32b9579400ae1119497f36f268eb40:1 -p bar@637f001eb4cbe165946c02a56bcb73822610f5886516169f98da6252266b7d8a:1 -p abc@85b423b9c8c5c5277575b87d94dbcd4f87c1be578756eff6a9fde8b7d55749fb:1 -k foo@keys/foo -k bar@keys/bar -k abc@keys/abc -r 14DDgj2r8WQEwfTDEjJFBn3wRnHmXzgB3z:60000000 | mktx send
1 peers connected.
s: version (85 bytes)
r: version (85 bytes)
s: verack (0 bytes)
r: verack (0 bytes)
Connected
s: tx (581 bytes)
c5 24 c5 55 aa d1 93 2c 24 c2 6e c2 04 39 a9 ca ef c4 9f 7c 0d f6 d6 bc cc ed 89 0e f9 80 b4 5c

poller

To use the poller you first need to import the schema which can be found in /usr/share/libbitcoin/bitcoin.sql (a pre-downloaded blockchain is here).

Code:
# do all the postgres stuff to make a new database
$ sudo su postgres
# createuser genjix
...
# psql
> CREATE DATABASE bitcoin;
> quit
# exit

$ psql bitcoin < /usr/share/libbitcoin/bitcoin.sql

# it is a good idea to run this in screen
$ screen -S poller
$ poller bitcoin genjix psqlpassword localhost
... starts downloading the blockchain into postgresql
# now close the terminal. You can re-open the terminal with screen -x poller

Poller is doing full blockchain validation. It will take a long time since it's an SQL database and it's validating all the blocks.

Balance needs the full blockchain otherwise you might not get an up-to-date balance reported back. It's fairly simple to use:
Code:
$ balance postgresql:database:username:password 1jkjsjkdskjb2kkjbkjdsk

Ubuntu packages

The PPA can be viewed on launchpad.

Add these 2 lines to the end of your /etc/apt/sources.list
Code:
deb http://ppa.launchpad.net/genjix/libbitcoin/ubuntu oneiric main
deb-src http://ppa.launchpad.net/genjix/libbitcoin/ubuntu oneiric main

Code:
$ wget -q "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x3D1978972EC26E7B" -O- | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install subvertx

This PPA also has a package for libbitcoin-dev although the API is undocumented at the moment. Grin

Gentoo ebuild

Kamil Domański has made a gentoo ebuild.

Code:
layman -a bitcoin
emerge subvertx

Source installation instructions

Source code for the non-Ubuntu paupers:
https://gitorious.org/libbitcoin/libbitcoin
https://gitorious.org/libbitcoin/subvertx

You will need:

  - g++ 4.6
  - cppdb
  - boost
  - libpq-dev
  - postgresql

Build cppdb using the instructions on their site.

  $ svn co http://cppcms.svn.sourceforge.net/svnroot/cppcms/cppdb/trunk cppdb-trunk
  $ cd cppdb-trunk
  $ mkdir build
  $ cd build

You can use -DCMAKE_INSTALL_PREFIX if you wish to install to a non-standard place such as a local directory.

  $ cmake -DCMAKE_INSTALL_PREFIX=/tmp/cppdb/ ..  -DPQ_BACKEND_INTERNAL=1
  $ make
  # make install

Clone repo and build.

  $ git clone git://gitorious.org/libbitcoin/libbitcoin.git
  $ cd libbitcoin
  $ autoreconf -i
  $ ./configure
  $ make
  # make install

Don't forget to initialise and create a database (see first post above for more details how).

  $ psql bitcoin < bitcoin.sql

Same for the subvertx suite

  $ git clone git://gitorious.org/libbitcoin/subvertx.git
  $ cd subvertx
  $ autoreconf -i
  $ ./configure
  $ make
  # make install

I can't stress enough for people to use this at their own peril as it's alpha software.

Thanks goes to,

Kamil Domański (kdomanski)
Jan Engelhardt (se7) <[email protected]>
Patrick Strateman (phantomcircuit)
Denis Roio (jaromil)
Amir Taaki (genjix) <[email protected]>
Pages:
Jump to: