We just released bitcoinj 0.7 which has a full pruning block chain implementation, with an example using the embedded SQL DB "H2". So you can query the set of unspent outs directly from that. Replacing H2 with a different database backend would be quite easy.
There are also more listeners and other things these days. I doubt you'd need to make many (any?) changes to the core now. So if you're still using the code I'd suggest trying to upgrade to the latest code.
Topic: Is there a remote, trustless bitcoin API that doesn't require local storage? (Read 2645 times)
February 20, 2013, 10:19:01 AM
February 20, 2013, 05:34:15 AM
The changes I needed to make were for listening to pending transaction, this was for an instant payment API.
So I would say this code has been battered around a bit.
However I see it as a proof of concept.
1. It's possible to get BitcoinJ working on Heroku.
2. You can use the JPA as a blockstore and therefore use Postgres as a DB backend.
3. BitcoinJ can be configured just to track unspent outs.
4. BitcoinJ can forward transactions.
With 3 and 4, you can build pretty much any Bitcoin payment API you want. The hard part is tracking unspent outs.
So I would say this code has been battered around a bit.
However I see it as a proof of concept.
1. It's possible to get BitcoinJ working on Heroku.
2. You can use the JPA as a blockstore and therefore use Postgres as a DB backend.
3. BitcoinJ can be configured just to track unspent outs.
4. BitcoinJ can forward transactions.
With 3 and 4, you can build pretty much any Bitcoin payment API you want. The hard part is tracking unspent outs.
February 20, 2013, 05:13:04 AM
I've uploaded to github.
https://github.com/ianpurton/bitcoinj-heroku
The main things to look at are the JPA classes which store the blockchain data.
https://github.com/ianpurton/bitcoinj-heroku/tree/master/src/main/java/com/strongcoin
The startup class is here https://github.com/ianpurton/bitcoinj-heroku/blob/master/src/main/java/com/strongcoin/StrongCoin.java
I did have to make a change to the core code of BitcoinJ as it wasn't easy to add listeners, that may have changed now, this code is over a year old.
The pom.xml in the root directory is what Heroku needs to deploy the project. Heroku will download BitcoinJ etc as part of the deployment, JPA will create the DB tables in Postgres.
I'd be happy to help fund a bounty to get this working again.
https://github.com/ianpurton/bitcoinj-heroku
The main things to look at are the JPA classes which store the blockchain data.
https://github.com/ianpurton/bitcoinj-heroku/tree/master/src/main/java/com/strongcoin
The startup class is here https://github.com/ianpurton/bitcoinj-heroku/blob/master/src/main/java/com/strongcoin/StrongCoin.java
I did have to make a change to the core code of BitcoinJ as it wasn't easy to add listeners, that may have changed now, this code is over a year old.
The pom.xml in the root directory is what Heroku needs to deploy the project. Heroku will download BitcoinJ etc as part of the deployment, JPA will create the DB tables in Postgres.
I'd be happy to help fund a bounty to get this working again.
Thanks, interesting stuff.
Can you document the changes you made to bitcoinj and perhaps submit a pull request to the core project?
It's usually not a good idea to modify a 3rd party library directly, since you lose your update path in the process.
February 20, 2013, 04:50:16 AM
I've uploaded to github.
https://github.com/ianpurton/bitcoinj-heroku
The main things to look at are the JPA classes which store the blockchain data.
https://github.com/ianpurton/bitcoinj-heroku/tree/master/src/main/java/com/strongcoin
The startup class is here https://github.com/ianpurton/bitcoinj-heroku/blob/master/src/main/java/com/strongcoin/StrongCoin.java
I did have to make a change to the core code of BitcoinJ as it wasn't easy to add listeners, that may have changed now, this code is over a year old.
The pom.xml in the root directory is what Heroku needs to deploy the project. Heroku will download BitcoinJ etc as part of the deployment, JPA will create the DB tables in Postgres.
I'd be happy to help fund a bounty to get this working again.
https://github.com/ianpurton/bitcoinj-heroku
The main things to look at are the JPA classes which store the blockchain data.
https://github.com/ianpurton/bitcoinj-heroku/tree/master/src/main/java/com/strongcoin
The startup class is here https://github.com/ianpurton/bitcoinj-heroku/blob/master/src/main/java/com/strongcoin/StrongCoin.java
I did have to make a change to the core code of BitcoinJ as it wasn't easy to add listeners, that may have changed now, this code is over a year old.
The pom.xml in the root directory is what Heroku needs to deploy the project. Heroku will download BitcoinJ etc as part of the deployment, JPA will create the DB tables in Postgres.
I'd be happy to help fund a bounty to get this working again.
February 20, 2013, 04:06:36 AM
Hi,
I had some success deploying BitcoinJ to Heroku, I created a BitcoinJ datastore that would save the whole block to the database not just the headers. For this I used the JPA (Java persistence API).
After that I could do 2 things.
1. Transmit transactions to the network.
2. Query unspent outs (and therefore calculate balances and create transactoins).
I'd be happy to share the code with you.
I had some success deploying BitcoinJ to Heroku, I created a BitcoinJ datastore that would save the whole block to the database not just the headers. For this I used the JPA (Java persistence API).
After that I could do 2 things.
1. Transmit transactions to the network.
2. Query unspent outs (and therefore calculate balances and create transactoins).
I'd be happy to share the code with you.
Post what you have, it's always good to see some examples.
Publicly if you can, or pm if you prefer to keep it more private.
What app framework are you using btw? I'm comfortable with Play Framework which has excellent Heroku support and uses JPA/hibernate out of the box.
February 20, 2013, 03:57:21 AM
Hi,
I had some success deploying BitcoinJ to Heroku, I created a BitcoinJ datastore that would save the whole block to the database not just the headers. For this I used the JPA (Java persistence API).
After that I could do 2 things.
1. Transmit transactions to the network.
2. Query unspent outs (and therefore calculate balances and create transactoins).
I'd be happy to share the code with you.
I had some success deploying BitcoinJ to Heroku, I created a BitcoinJ datastore that would save the whole block to the database not just the headers. For this I used the JPA (Java persistence API).
After that I could do 2 things.
1. Transmit transactions to the network.
2. Query unspent outs (and therefore calculate balances and create transactoins).
I'd be happy to share the code with you.
February 17, 2013, 03:38:01 PM
I don't get the requirement for instant startup. How often do you reinstall your servers? It takes, what, a few minutes to sync for the first time a newly installed server. Are you trying to run your service on a Raspberry Pi or something?
I plan to run my service on heroku. The way heroku works (similar to Google App Engine I guess), is you prepare a server image (git snapshot), and heroku can bring servers up or down as needed. When a version is updated, new servers are started, then the old servers are downed.
In this setup, I do not keep a server that is necessarily long-lived ... I do not manage the lifetime of my servers. Therefore, I wish to make the bootstrapping phase as fast as possible, and spending 0 time on blockchain initialization is desirable.
Relying on a third party site like blockchain isn't just a trust issue. You are externalizing some of your costs onto Ben. Why should he pay for a part of your server costs, for free? He might do so for now, because your site will start small and he's a nice guy, but it's kind of rude to assume that if your website gets really big he'll just pony up for serving all your API traffic.
You could also just use the JSON-RPC API on a regular Satoshi node. It's really not such a big deal.
But the details really depend on what you mean by "various operations".
You could also just use the JSON-RPC API on a regular Satoshi node. It's really not such a big deal.
But the details really depend on what you mean by "various operations".
APIs are meant to be used. Perhaps in a future version of his API, Ben might charge for its usage. Then, I can have the choice of either paying to enjoy the simplicity that I like, or implementing/running my own server layer that handles this API. Even if I were to host everything on my end, I might partition my servers into two kinds - stateless web servers, and a blockchain server (running bitcoinj or bitcoind). I think it just makes the architecture cleaner, and lets me create this service faster without having to build the blockchain server right now, while still letting me create it later if I need it.
February 17, 2013, 06:01:24 AM
This might interest you too:
https://github.com/bitsofproof/supernode/wiki/API
My implementation of the protocol features a language independent (protobuf), dedicated message broker powered (STOMP) merchant centric API.
It is not yet fully tested, but it runs smoothly for months now...
https://github.com/bitsofproof/supernode/wiki/API
My implementation of the protocol features a language independent (protobuf), dedicated message broker powered (STOMP) merchant centric API.
It is not yet fully tested, but it runs smoothly for months now...
February 16, 2013, 11:57:12 AM
I don't get the requirement for instant startup. How often do you reinstall your servers? It takes, what, a few minutes to sync for the first time a newly installed server. Are you trying to run your service on a Raspberry Pi or something?
Relying on a third party site like blockchain isn't just a trust issue. You are externalizing some of your costs onto Ben. Why should he pay for a part of your server costs, for free? He might do so for now, because your site will start small and he's a nice guy, but it's kind of rude to assume that if your website gets really big he'll just pony up for serving all your API traffic.
You could also just use the JSON-RPC API on a regular Satoshi node. It's really not such a big deal.
But the details really depend on what you mean by "various operations".
Relying on a third party site like blockchain isn't just a trust issue. You are externalizing some of your costs onto Ben. Why should he pay for a part of your server costs, for free? He might do so for now, because your site will start small and he's a nice guy, but it's kind of rude to assume that if your website gets really big he'll just pony up for serving all your API traffic.
You could also just use the JSON-RPC API on a regular Satoshi node. It's really not such a big deal.
But the details really depend on what you mean by "various operations".
February 16, 2013, 11:53:33 AM
This class from the android app may be of some help https://github.com/blockchain/My-Wallet-Android/blob/master/src/piuk/MyRemoteWallet.java. It uses the blockchain API and BitcoinJ to sign locally.
Thanks, look promising - I'll take a closer look when I start implementing.
February 16, 2013, 11:36:16 AM
This class from the android app may be of some help https://github.com/blockchain/My-Wallet-Android/blob/master/src/piuk/MyRemoteWallet.java. It uses the blockchain API and BitcoinJ to sign locally.
February 16, 2013, 11:24:08 AM
I will store my private keys in my data tier of course, but I was referring to storing the blockchain/SPV data, that needs constant updating and can't just work instantly when the server is up.
I think bitcoinj is currently the best approach at trustless operation. However, you'd still need to download at least the blockchain headers plus relevant transactions on each start.
You can cut this down dramatically by using checkpoints. However, for this to work you need to update your client with the latest checkpoint data from time to time. Which is of course technically local storage again. Or trust, if you are bootstrapping your client from a remote server each time.
February 16, 2013, 11:11:52 AM
The main architectural difference between using blockchain.info and one based on bitcoinj/ MultiBit is whether you want to rely on a single server backend or the Bitcoin network as a whole.
Using one server backend is simpler, but you have a single point of failure.
blockchain.info is pretty high availability but it does occasionally go down for short periods.
(This is the same if you use BCCAPI - though I think that is multiple physical servers now at the backend - or your own ABE blockchain server).
Using one server backend is simpler, but you have a single point of failure.
blockchain.info is pretty high availability but it does occasionally go down for short periods.
(This is the same if you use BCCAPI - though I think that is multiple physical servers now at the backend - or your own ABE blockchain server).
Well, I'm relying on a single logical endpoint, but that endpoint indeed can have multiple backing servers. I don't believe that blockchain.info runs on a single server.
I would like to separate the concerns - I want my app to focus on the business logic, and not on maintaining a blockchain - the other alternatives mentioned at this thread will probably do a better job at this than I am ... I have no reason to believe I'll have less downtime than them.
Using a more distributed approach is more complicated but as there are working examples that are MIT licenced you don't have to do the dev work to take advantage of the network.
It does not give you your zero bytes starting point though. Only you can decide which of the various design factors are more important for your target application.
One approach you could take is to separate the various areas of:
+ blockchain/ tx reading
+ tx signing
+ tx transmission
So that you can change which service you use in each area (perhaps have multiple services you transmit the tx to for redundancy).
It does not give you your zero bytes starting point though. Only you can decide which of the various design factors are more important for your target application.
One approach you could take is to separate the various areas of:
+ blockchain/ tx reading
+ tx signing
+ tx transmission
So that you can change which service you use in each area (perhaps have multiple services you transmit the tx to for redundancy).
I think of opting to use blockchain.info as my backend as separating my concerns and focusing only on what's important for my app. I can always change the layer that access the blockchain later, and setup my own Abe/bitcoind/bitcoinj server if I need it.
February 16, 2013, 11:05:34 AM
trustless bitcoin API that doesn't require local storage
I don't think its possible. You need to store your private keys somewhere.
Either locally, so you need local storage.
Or remote, so you need trust. At the very minimum, you need to trust that the remote storage does not go away.
I will store my private keys in my data tier of course, but I was referring to storing the blockchain/SPV data, that needs constant updating and can't just work instantly when the server is up.
February 16, 2013, 11:01:17 AM
trustless bitcoin API that doesn't require local storage
I don't think its possible. You need to store your private keys somewhere.
Either locally, so you need local storage.
Or remote, so you need trust. At the very minimum, you need to trust that the remote storage does not go away.
February 16, 2013, 10:16:41 AM
The main architectural difference between using blockchain.info and one based on bitcoinj/ MultiBit is whether you want to rely on a single server backend or the Bitcoin network as a whole.
Using one server backend is simpler, but you have a single point of failure.
blockchain.info is pretty high availability but it does occasionally go down for short periods.
(This is the same if you use BCCAPI - though I think that is multiple physical servers now at the backend - or your own ABE blockchain server).
Using a more distributed approach is more complicated but as there are working examples that are MIT licenced you don't have to do the dev work to take advantage of the network.
It does not give you your zero bytes starting point though. Only you can decide which of the various design factors are more important for your target application.
One approach you could take is to separate the various areas of:
+ blockchain/ tx reading
+ tx signing
+ tx transmission
So that you can change which service you use in each area (perhaps have multiple services you transmit the tx to for redundancy).
Using one server backend is simpler, but you have a single point of failure.
blockchain.info is pretty high availability but it does occasionally go down for short periods.
(This is the same if you use BCCAPI - though I think that is multiple physical servers now at the backend - or your own ABE blockchain server).
Using a more distributed approach is more complicated but as there are working examples that are MIT licenced you don't have to do the dev work to take advantage of the network.
It does not give you your zero bytes starting point though. Only you can decide which of the various design factors are more important for your target application.
One approach you could take is to separate the various areas of:
+ blockchain/ tx reading
+ tx signing
+ tx transmission
So that you can change which service you use in each area (perhaps have multiple services you transmit the tx to for redundancy).
February 16, 2013, 08:26:40 AM
Hmm, we have different definitions of "huge service". Blockchain.info is a site run by one man, if I understand correctly 
You don't have to download the headers every time the server starts, obviously they can be saved to disk.
Try playing with MultiBit. It starts instantly. The need to fetch headers only happens at startup.
You don't have to download the headers every time the server starts, obviously they can be saved to disk.
Try playing with MultiBit. It starts instantly. The need to fetch headers only happens at startup.
Well, huge compared to most other services in the bitcoin world (regarding number of users / transaction volume), and certainly larger than any service that I might start that relies on it.
How will MultiBit help? I think bitcoinj with SPV mode is very close to what I'm looking for ... still, I prefer the instant zero-blockchain-download that is blockchain.info
February 16, 2013, 06:50:42 AM
Hmm, we have different definitions of "huge service". Blockchain.info is a site run by one man, if I understand correctly
You don't have to download the headers every time the server starts, obviously they can be saved to disk.
Try playing with MultiBit. It starts instantly. The need to fetch headers only happens at startup.
You don't have to download the headers every time the server starts, obviously they can be saved to disk.
Try playing with MultiBit. It starts instantly. The need to fetch headers only happens at startup.
February 16, 2013, 06:19:38 AM
And if you don't want to trust blockchain.info, you can always set up your own Abe server.
Abe server? What is that and how do I set one up?
https://en.bitcoin.it/wiki/Abe
But, is it just a browser, or does it also come with a way to push transactions to the network?
Ah, you are right about that.
I use a bitcoind and its api to create, sign and push transactions to the network. I use Abe to fetch unspent transactions.
February 16, 2013, 05:57:01 AM
And if you don't want to trust blockchain.info, you can always set up your own Abe server.
Abe server? What is that and how do I set one up?
https://en.bitcoin.it/wiki/Abe
But, is it just a browser, or does it also come with a way to push transactions to the network?
Jump to: