Topic: How To Correctly Poll New Blocks for Payments? (Read 1052 times)

Thank you. I'm going to look at that.

Just announced: https://bitcointalksearch.org/topic/ann-btc-sniffer-bitcoin-p2p-network-sniffer-events-trigger-102740

The API info states that by default the limit is 300 calls/5 min. or 5760 / 8 hours, and that you can contact them to have the limit increased. I don't see how a call every 3-5 minutes is going to be impolite or a problem given the stated limits. It's very clear from the API docs that using it this way is one of it's intended purposes.

It's not just that I don't want to have bitcoind running on the server and the space it uses but also that AFAIK it doesn't provide a useful interface for monitoring a large collection of randomly generated addresses. There are some calls to check various wallet values but I don't think there is a way to setup callbacks upon payment arrival. So I would end up repeatedly polling by RPC for potentially thousands of addresses that only a few may have a payment arrive on. And then it doesn't seem to provide a way to add addreses without keys. So unless there is a better suited daemon I can run then making use of the JSON API looks like the best way.

After trying some stuff and coding for a while I decided that likely the best way was to poll the simplest "getblockcount" call to see when a new block arrives. This call only returns a number and is very lightweight. When a new block arrives I don't grab it. Instead I grab the one that is N block before it and check those transaction addresses. They already have the needed confirmation level and so can be applied to balances in one step. Much simpler.

One thing that would make such use much lighter on the API server would be to offer a much reduced block-height call. For payment processing it seems you only really need the (trx,address,value) output tuples and that is a small fraction of the overall block data. Blocks that currently are 500K in size would likely be 5-10K if useless data was pared away.

You could always run the bitcoind server on another machine if 3gb of space is a concern on your server.

I don't know if the bitcoind server allows fine grained access control, ie you can check balance but not send etc. But one could write their own api that goes between the bt server and your web server.

Well, they do provide an API that lets you do exactly what OP wants. I don't see how it can be considered impolite to use that API, especially if you only poll it once every few minutes. Your other reason is very valid though.

I really wouldn't recommend that for a few reasons.

One is that it's impolite to everyone else who uses the site, you're draining resources away that people who want interactive exploration of the block chain need, for a very simple functionality. 3GB is really not much storage at all.

Another reason is that if blockchain.info goes down or decides to block you for causing excessive load, your site will break.

There are several ways to do what you want. One is to just use the Satoshi client and send it RPCs.

Another, if you're comfortable with Java, is to write a bitcoinj app. You'll get callbacks when new transactions that are relevant to the wallet arrive. You don't need to have the private keys in the wallet so if the server gets hacked the money can't be stolen.

Here's one concern I have. When I record a pmt transaction from block N and then wait til block N+6 to apply the payment - how do I verify it is still in the main_chain? I should somehow make sure the transaction did not become invalid in the meantime because if I just wait 6 blocks but not check back then I cannot know if it was confirmed after all. Not sure how to best handle that.

I would also like to know this.

I'm going to have a new web service that will require I give out addresses and then monitor if payments are made to me to update an account balance.

I plan to poll blockchain.info every few minutes for each new block. This seems easier than running the bitcoin client and having to keep 3 GB of data on my web server.

Right now I have code and it checks the json data for each block and scans for the block that is marked "main_chain": true and then scans those transactions looking for "out" records in my address database. When one is found I add that transaction to my pmts table (so I can also check for duplicates). After N blocks pass, for confirmation, I can add the amount to a user balance.

This method seems like the most efficient in terms of polling or checking addresses.

I'm just wondering if there is some indicators/flags or gotchas I have to watch out for?