Topic: getrawtransactionsinblock RPC command (Read 1049 times)
For reference, for anyone coming across this post from a search or whatever, have a look at this blog post for a working alternative to the getrawtransactionsinblock rpc command I suggested above..
and.. closed
It turns out there's a second parameter to getblock, (not listed on the wiki) and you can call getblock false to get the raw block data, that can be parsed pretty straightforwardly to obtain the raw transaction data I need..
It turns out there's a second parameter to getblock, (not listed on the wiki) and you can call getblock
Ok, pull request is here: https://github.com/bitcoin/bitcoin/pull/4369
Your best bet is to make the pull request and see. The real discussion will happen on github, in the comments on the pull. If you plan on developing more, either you need practice using git/github anyway, or you are already proficient enough that making the pull won't be very annoying.
Keep in mind that this is not a feature that many people will ever use. And the people that are likely to use it already have other ways to accomplish the same goals, arguably better ways. That said, this is pretty light, and won't have a huge maintenance burden in the future.
Keep in mind that this is not a feature that many people will ever use. And the people that are likely to use it already have other ways to accomplish the same goals, arguably better ways. That said, this is pretty light, and won't have a huge maintenance burden in the future.
Bump!
If I make a pull request for this, is this just going to be ignored?
Is anyone else using bitcoin RPC and wants to be able to quickly process all the transactions in a block?
(Apart from enabling you to drop a requirement for txindex, if you are able to process the raw transactions yourself, doing this with 'getrawtransactionsinblock' is *much* faster than getblock and a bunch of getrawtransaction calls with verbose set to true.)
Is there any reason *not* to add this?
Thomas
If I make a pull request for this, is this just going to be ignored?
Is anyone else using bitcoin RPC and wants to be able to quickly process all the transactions in a block?
(Apart from enabling you to drop a requirement for txindex, if you are able to process the raw transactions yourself, doing this with 'getrawtransactionsinblock' is *much* faster than getblock and a bunch of getrawtransaction calls with verbose set to true.)
Is there any reason *not* to add this?
Thomas
I've added a 'getrawtransactionsinblock' RPC command to bitcoind, in my local source code, and it seems like something that will be useful to other users, particularly if you're trying to split wallet functionality out of bitcoin core.
Is it worth me setting up a pull request for this?
The use case is that I want to scan the transactions in each block, as they come.
I can do this with getblock and a bunch of getrawtransaction calls, *if* txindex is set to 1.
But since I'm not looking up transactions at arbitrary positions in the block chain, I shouldn't need a full transaction index for this.
I know the hash of the block which contains the transactions, and it should then be possible to obtain the transaction information from this block, without a full transaction index.
The code for the new command is very straightforward, and this is essentially just the following, plus a couple of minor changes to make this available as an RPC command:
The name is quite long, but says what it does on the box.
(Could be shortened to something like 'getblocktransactions', I guess, but then it's less clear what it does exactly.
Another possibility could be to add the possibility to pass the containing blockhash in to getrawtransaction, I guess, and not require a full transaction index in this case.
That would be another way to avoid the need for a full transaction index in our use case, but is a lot less efficient because it requires a lot more RPC calls to get all the transactions.
Switching from getblock + getrawtransaction calls to getrawtransactionsinblock gave us quite a significant speedup.
Scanning all transactions as they come seems like it should be quite a common use case for things like wallets that use bitcoind as a backend for network management, and just returning all the transactions then does seem like a good efficient solution for this kind of stuff in general.
For info, we're using this custom method in the SwapBill protocol reference client (announcement thread here:https://bitcointalksearch.org/topic/swapbill-embedded-protocol-preview-and-request-for-feedback-628547, client source code and readme on github here:https://github.com/crispweed/swapbill), but it seems like this is quite a general requirement, and can be useful in a lot of other cases!
Thomas
Is it worth me setting up a pull request for this?
The use case is that I want to scan the transactions in each block, as they come.
I can do this with getblock and a bunch of getrawtransaction calls, *if* txindex is set to 1.
But since I'm not looking up transactions at arbitrary positions in the block chain, I shouldn't need a full transaction index for this.
I know the hash of the block which contains the transactions, and it should then be possible to obtain the transaction information from this block, without a full transaction index.
The code for the new command is very straightforward, and this is essentially just the following, plus a couple of minor changes to make this available as an RPC command:
Code:
Value getrawtransactionsinblock(const Array& params, bool fHelp)
{
if (fHelp || params.size() < 1)
throw runtime_error(
"getrawtransactionsinblock\n"
"Returns txid and raw transaction data for all transactions in the specified block.\n"
"\nArguments:\n"
"1. \"hash\" (string, required) The block hash\n"
"\nResult\n"
"[ (array of json object)\n"
" {\n"
" \"txid\" : \"txid\", (string) the transaction id \n"
" \"hex\" : \"data\", (string) The serialized, hex-encoded data for 'txid'\n"
" }\n"
" ,...\n"
"]\n"
"\nExamples:\n"
+ HelpExampleCli("getrawtransactionsinblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"")
+ HelpExampleRpc("getrawtransactionsinblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"")
);
std::string strHash = params[0].get_str();
uint256 hash(strHash);
if (mapBlockIndex.count(hash) == 0)
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found");
CBlock block;
CBlockIndex* pblockindex = mapBlockIndex[hash];
ReadBlockFromDisk(block, pblockindex);
Array a;
BOOST_FOREACH(const CTransaction &tx, block.vtx) {
Object txEntry;
txEntry.push_back(Pair("txid", tx.GetHash().GetHex()));
CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION);
ssTx << tx;
string strHex = HexStr(ssTx.begin(), ssTx.end());
txEntry.push_back(Pair("hex", strHex));
a.push_back(txEntry);
}
return a;
}
{
if (fHelp || params.size() < 1)
throw runtime_error(
"getrawtransactionsinblock
"Returns txid and raw transaction data for all transactions in the specified block.\n"
"\nArguments:\n"
"1. \"hash\" (string, required) The block hash\n"
"\nResult\n"
"[ (array of json object)\n"
" {\n"
" \"txid\" : \"txid\", (string) the transaction id \n"
" \"hex\" : \"data\", (string) The serialized, hex-encoded data for 'txid'\n"
" }\n"
" ,...\n"
"]\n"
"\nExamples:\n"
+ HelpExampleCli("getrawtransactionsinblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"")
+ HelpExampleRpc("getrawtransactionsinblock", "\"00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09\"")
);
std::string strHash = params[0].get_str();
uint256 hash(strHash);
if (mapBlockIndex.count(hash) == 0)
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found");
CBlock block;
CBlockIndex* pblockindex = mapBlockIndex[hash];
ReadBlockFromDisk(block, pblockindex);
Array a;
BOOST_FOREACH(const CTransaction &tx, block.vtx) {
Object txEntry;
txEntry.push_back(Pair("txid", tx.GetHash().GetHex()));
CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION);
ssTx << tx;
string strHex = HexStr(ssTx.begin(), ssTx.end());
txEntry.push_back(Pair("hex", strHex));
a.push_back(txEntry);
}
return a;
}
The name is quite long, but says what it does on the box.
(Could be shortened to something like 'getblocktransactions', I guess, but then it's less clear what it does exactly.
Another possibility could be to add the possibility to pass the containing blockhash in to getrawtransaction, I guess, and not require a full transaction index in this case.
That would be another way to avoid the need for a full transaction index in our use case, but is a lot less efficient because it requires a lot more RPC calls to get all the transactions.
Switching from getblock + getrawtransaction calls to getrawtransactionsinblock gave us quite a significant speedup.
Scanning all transactions as they come seems like it should be quite a common use case for things like wallets that use bitcoind as a backend for network management, and just returning all the transactions then does seem like a good efficient solution for this kind of stuff in general.
For info, we're using this custom method in the SwapBill protocol reference client (announcement thread here:https://bitcointalksearch.org/topic/swapbill-embedded-protocol-preview-and-request-for-feedback-628547, client source code and readme on github here:https://github.com/crispweed/swapbill), but it seems like this is quite a general requirement, and can be useful in a lot of other cases!
Thomas
Jump to: