Empty blocks - page 9. | Bitcointalksearch.org

jonnybravo0311

legendary

Activity: 1344

Merit: 1024

Mine at Jonny's Pool

Quote from: kano on June 10, 2015, 07:04:07 PM

Quote from: jonnybravo0311 on June 10, 2015, 02:36:46 PM

I got my answer from grue:

Quote from: grue on June 10, 2015, 02:10:10 PM

you need -txindex to view transactions that don't belong in your wallet.

So, I'm currently rebuilding the block index using txindex and once that's done, I'll rerun the code so I'll have all of the data.

My block detection code I run at home, not on the pool, in one of the many checks I do to find our blocks, I simply convert the ascii to hex and grep the coinbase from bitcoin (each new block that appears at home)
Obvious, but easer than having to deal with removing characters below ' ' or above '~' and the problems they can cause in scripts and output

blockchain has this (incomplete) list of coinbase strings they use:
https://github.com/blockchain/Blockchain-Known-Pools

I had one many years ago (last time I updated it was Sep-2012) ... but half the pools in it don't exist any more

I like that idea... no need to bother with the bad actors, just convert the ascii strings you're trying to grep from the coinbase into hex and compare that. I was doing it the other way around and the random characters that don't convert nicely were playing havoc. Thanks for the suggestion!

Now for a stupid question... and I only say it's stupid because I've been mining on p2pool for well over a year and don't know the answer myself Tongue

... is p2pool the only pool to use "/P2SH/" in the coinbase? I swear I saw that in the coinbase of another... maybe I'm wrong and that really is the p2pool identifier. Guess I'll find out when I run my code and do some checks.

kano

legendary

Activity: 4634

Merit: 1851

Linux since 1997 RedHat 4

Quote from: jonnybravo0311 on June 10, 2015, 02:36:46 PM

I got my answer from grue:

Quote from: grue on June 10, 2015, 02:10:10 PM

you need -txindex to view transactions that don't belong in your wallet.

So, I'm currently rebuilding the block index using txindex and once that's done, I'll rerun the code so I'll have all of the data.

My block detection code I run at home, not on the pool, in one of the many checks I do to find our blocks, I simply convert the ascii to hex and grep the coinbase from bitcoin (each new block that appears at home)
Obvious, but easer than having to deal with removing characters below ' ' or above '~' and the problems they can cause in scripts and output

blockchain has this (incomplete) list of coinbase strings they use:
https://github.com/blockchain/Blockchain-Known-Pools

I had one many years ago (last time I updated it was Sep-2012) ... but half the pools in it don't exist any more

jonnybravo0311

legendary

Activity: 1344

Merit: 1024

Mine at Jonny's Pool

I got my answer from grue:

Quote from: grue on June 10, 2015, 02:10:10 PM

you need -txindex to view transactions that don't belong in your wallet.

So, I'm currently rebuilding the block index using txindex and once that's done, I'll rerun the code so I'll have all of the data.

jonnybravo0311

legendary

Activity: 1344

Merit: 1024

Mine at Jonny's Pool

So I got a bit ambitious... and wrote some code to drop the information into a CSV.
Block number, date, decoded script (if available)

As of block 360324 there are 85298 empty blocks.

If anyone wants to look at it, I've uploaded it here: https://www.dropbox.com/s/tisp9mj6a369dds/empty_blocks.csv?dl=0

Strangely, I found a whole lot of coinbase transactions that the core API wouldn't parse (53,612 of them to be precise). I have no idea why the core API wouldn't parse it, but here's an example (block 337104):

Code:

getblockhash 337104
00000000000000000cc790ec61cd20d394f9d0f0c3e34adb1005d2d909b3f691

Code:

getblock 00000000000000000cc790ec61cd20d394f9d0f0c3e34adb1005d2d909b3f691

{
"hash" : "00000000000000000cc790ec61cd20d394f9d0f0c3e34adb1005d2d909b3f691",
"confirmations" : 23233,
"size" : 256,
"height" : 337104,
"version" : 2,
"merkleroot" : "57224de458d9356a471099c5ab9f5e68f395ec5f2822915bdf3c6b296c1428e2",
"tx" : [
"57224de458d9356a471099c5ab9f5e68f395ec5f2822915bdf3c6b296c1428e2"
],
"time" : 1420197655,
"nonce" : 3357110901,
"bits" : "181b0dca",
"difficulty" : 40640955016.57649231,
"chainwork" : "00000000000000000000000000000000000000000003beb652b37a2f4b948166",
"previousblockhash" : "00000000000000001654d3aa9abac3e815391b9b5eca90e15767cc1ee6ce1dc2",
"nextblockhash" : "00000000000000001686943c32233916b9d9485230bf63fc0f9144c94be4c1af"
}

Code:

getrawtransaction 57224de458d9356a471099c5ab9f5e68f395ec5f2822915bdf3c6b296c1428e2 1
No information available about transaction (code -5)

Yet, if I look at that same transaction on blockchain.info I can see the block was mined by Discus Fish:
�$七彩神仙鱼��mmvs��̮��M��,��w, z1FaW��fZ�rMined by andy518038

Anyone have any idea?

jonnybravo0311

legendary

Activity: 1344

Merit: 1024

Mine at Jonny's Pool

Quote from: kano on June 09, 2015, 09:17:10 PM

Quote from: TheRealSteve on June 09, 2015, 07:33:33 PM

Quote from: jonnybravo0311 on June 09, 2015, 07:18:03 PM

Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius. If decoded script contains antpool, then it was found by antpool, etc"?

It's a messy thing. There's actually a write-up about it here: Don't believe everything you read on blockchain.info about block sources

Blocktrail also does the messy thing, as does whomined.com . You can ready whomined's approach here: http://whomined.com/#how-it-works.

I'd imagine there's libraries that'll decode the scriptSig without going through Bitcoin Core.

You don't have to "decode" it.
You just look for the string that the pool always puts there.
e.g. for kano.is it always contains "/Kano"
Of course that only works for pools that do that

Maybe I'm missing something painfully obvious, but when I look at the output of getrawtransaction or decoderawtransaction, this is what I see:

Code:

{
"txid" : "661b8707036fddb8ea2c178a57988b4ed9cc9d9bc800d7aebe5dd58b813a9694",
"version" : 1,
"locktime" : 0,
"vin" : [
{
"coinbase" : "03117f051e4d696e656420627920416e74506f6f6c20626a35180f42fa2055776c0001bc2a0000bf480500",
"sequence" : 4294967295
}
],
"vout" : [
{
"value" : 25.26360849,
"n" : 0,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 9524440a5b54cca9c46ef277c34739e9b521856d OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a9149524440a5b54cca9c46ef277c34739e9b521856d88ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
"addresses" : [
"1Ebb8NfVmKMoGuMJCAEbVMv2dX8GnzgxSa"
]
}
}
]
}

That "coinbase" value is what I need to decode. This transaction was an empty block mined by AntPool (I know this because I looked it up on blockchain.info). This is what it looks like there:

Code:

CoinBase
03117f051e4d696e656420627920416e74506f6f6c20626a35180f42fa2055776c0001bc2a0000bf480500 
(decoded) Mined by AntPool bj5B� Uwl�*�H

So, how do I decode the coinbase hash to get back the string I can parse?

EDIT: I figured it out... it's just a hex to ascii conversion. Duh. Tongue

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: kano on June 09, 2015, 09:14:56 PM

When trying to find empty blocks, of course the transaction count is what matters.

If the data source is no good, the next best bet is the block reward.
If it's only 25BTC then it is very likely to be an empty block (I'd doubt there are few if any recent blocks that have only free transactions)

The block size doesn't help since there are 2 pools with very large coinbase transactions. p2pool is one of them.

Gotcha. I've got a ticket open with Blocktrail, so hopefully I can just re-run that with their API later. If not, I'm sure there's alternatives

kano

legendary

Activity: 4634

Merit: 1851

Linux since 1997 RedHat 4

Quote from: TheRealSteve on June 09, 2015, 07:33:33 PM

Quote from: jonnybravo0311 on June 09, 2015, 07:18:03 PM

Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius. If decoded script contains antpool, then it was found by antpool, etc"?

It's a messy thing. There's actually a write-up about it here: Don't believe everything you read on blockchain.info about block sources

Blocktrail also does the messy thing, as does whomined.com . You can ready whomined's approach here: http://whomined.com/#how-it-works.

I'd imagine there's libraries that'll decode the scriptSig without going through Bitcoin Core.

You don't have to "decode" it.
You just look for the string that the pool always puts there.
e.g. for kano.is it always contains "/Kano"
Of course that only works for pools that do that

kano

legendary

Activity: 4634

Merit: 1851

Linux since 1997 RedHat 4

Just looked up kano.is smallest block:

29 transactions.
https://blockchain.info/block-index/878311/00000000000000000880afbc23f0176372837c3edc9251763a4268edac280299

---

When trying to find empty blocks, of course the transaction count is what matters.

If the data source is no good, the next best bet is the block reward.
If it's only 25BTC then it is very likely to be an empty block (I'd doubt there are few if any recent blocks that have only free transactions)

The block size doesn't help since there are 2 pools with very large coinbase transactions. p2pool is one of them.

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: jonnybravo0311 on June 09, 2015, 07:18:03 PM

Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius. If decoded script contains antpool, then it was found by antpool, etc"?

It's a messy thing. There's actually a write-up about it here: Don't believe everything you read on blockchain.info about block sources

Blocktrail also does the messy thing, as does whomined.com . You can ready whomined's approach here: http://whomined.com/#how-it-works.

I'd imagine there's libraries that'll decode the scriptSig without going through Bitcoin Core.

jonnybravo0311

legendary

Activity: 1344

Merit: 1024

Mine at Jonny's Pool

OK... this has probably been answered before, so forgive me for asking it. Where do I find out which pool mined a block? I can't see anything useful in the output of the core API, and the "relayed_by" value provided by the blockchain.info API doesn't appear to be what I'm looking for, either. I figure it's probably in the output script for the coinbase transaction. Can somebody give me a quick rundown of how to decode that (other than passing it to bitcoin-cli)... like some pseudocode, or a link with some info?

For example, looking at the block TheRealSteve used in his last response, I can clearly see that on blockchain.info it shows it was submitted by Eligius. Looking at the coinbase output script it shows a bunch of garbled crap with Eligius buried in it. Is there a nice way to extract the info from there, or is it just a messy thing like, "if decoded script contains eligius, then the block was found by Eligius. If decoded script contains antpool, then it was found by antpool, etc"?

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: TheRealSteve on June 09, 2015, 06:10:11 PM

Probably - let me check what it's doing.

Stared myself blind on the code, couldn't find any issues. Turns out it's in the API. E.g. "height":360206,"block_time":"2015-06-09T22:17:46+0000",[...],"byte_size":230144,"confirmations":7,"transactions":1
That's not right: https://www.blocktrail.com/BTC/block/00000000000000000f757987312e482d3edd6a019d9eea5eb62e0b4a13e51e05

I'll drop them a line about this later. In the mean time, it looks like the byte_size is correct, so I'll redo the stats image.

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: kano on June 09, 2015, 06:03:08 PM

Your calculations must be wrong.
None of the blocks on my pool kano.is have 0 non-coinbase transactions.
One recently only had about 10 since it was a few seconds after another one on the internet.

Probably - let me check what it's doing.

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: Mikestang on June 09, 2015, 05:57:10 PM

Thank you for the research, the numbers are enlightening. I tend to fall on the side of the discussion that holds that blocks should contain transactions, since that is their ultimate purpose.

Oh I absolutely agree with that. I thought you were referring to disagreement on whether or not the empty-block-first mining strategy was actually advantageous. I seem to recall some reddit chatter about the impact being negligible compared to other factor and minor debate about whether there was in fact an actual advantage

kano

legendary

Activity: 4634

Merit: 1851

Linux since 1997 RedHat 4

Quote from: TheRealSteve on June 09, 2015, 05:04:23 PM

Quote from: jonnybravo0311 on June 09, 2015, 04:37:54 PM

Quote from: os2sam on June 09, 2015, 04:27:33 PM

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?

Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue

.

I ran my parser against a dataset - wasn't really designed for this - but here's what I've got:
http://i.imgur.com/RlhKM97.png

Quote from: jonnybravo0311 on June 09, 2015, 04:37:54 PM

Quote from: TheRealSteve on June 09, 2015, 04:28:11 PM

Quote from: chalkboard17 on June 09, 2015, 04:20:47 PM

Why do some pools do that? Is there an advantage?

Time.

This is where there's plenty of disagreement between pool software writers. On one side of the fence you've got luke-jr and wizkid who support the empty block theory. On the other you've got ck/kano who do not support it at all.

Disagreement is healthy

Your calculations must be wrong.
None of the blocks on my pool kano.is have 0 non-coinbase transactions.
One recently only had about 10 since it was a few seconds after another one on the internet.

Mikestang

legendary

Activity: 1274

Merit: 1001

Quote from: jonnybravo0311 on June 09, 2015, 04:05:35 PM

There's been quite a bit of discussion on this topic throughout the forum. Some feel empty blocks are valuable. Others feel they are of no value.

I thought it might be fun to write a small program to see just how many of these blocks exist in the blockchain. For those who don't know, an empty block is a block that is mined containing only the coinbase transaction that awards the new coins. To be honest, I was quite frankly surprised by the number there were, especially since people seemed to think they were rare occurrences. Sure, I expected there to be quite a few at the beginning of the blockchain when BTC was pretty much an unknown and only a select few were mining it. However, I was not expecting there to be so many empty blocks as there are.

As of block 360189 there are 85295 empty blocks on the chain. For the math challenged, 23.68% of all blocks are empty. 8 of the last 189 blocks are empty. 25 of the last 1000 are empty.

Maybe I'll get ambitious and write the output to a spreadsheet so I can get some good data points out of this to see trends.

Where do you stand on this debate?

Thank you for the research, the numbers are enlightening. I tend to fall on the side of the discussion that holds that blocks should contain transactions, since that is their ultimate purpose.

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: achow101 on June 09, 2015, 05:41:45 PM

Can we see the code that creates this?

I could give you the pseudocode for the poller - all the rest is just LibreOffice Base + Calc bits - but that's not all that interesting either. Basically, take this URL: https://api.blocktrail.com/v1/btc/all-blocks?sort_dir=desc&limit=200&api_key=YOUR_API_KEY_HERE&page=, concatenate with a page number that increments until either you find the page number of interest (or in my case, the block of interest). Download the concatenated URL. Parse result with whatever JSON library you're happy with. I only read out height, is_orphan, block_time and miningpool_slug originally, added transactions and byte_size for this thread.
They offer libraries for Python, PHP and NodeJS, so.. pretty easy to get started with.

You'll need an API key, of course, but that's as easy as signing up, going to the relevant page, and off you go.
( Do keep quota and throttles in mind - they're pretty lax right now, but be a good API citizen and all that. This includes not sharing the actual polled data, as per their T&C. )

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Can we see the code that creates this?

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: os2sam on June 09, 2015, 05:19:09 PM

What is the "dataset"?

Last 10,197 blocks (at the time I ran it), data sourced from blocktrail.com . They have a paginated API (200 results per page, so 51 requests) that at least has info on the block height, timestamp, number of transactions, size of the block, apparent pool that mined it and some other info.

os2sam

legendary

Activity: 3586

Merit: 1099

Think for yourself

Quote from: TheRealSteve on June 09, 2015, 05:04:23 PM

Quote from: jonnybravo0311 on June 09, 2015, 04:37:54 PM

Quote from: os2sam on June 09, 2015, 04:27:33 PM

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?

Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue

.

I ran my parser against a dataset - wasn't really designed for this - but here's what I've got:

What is the "dataset"?

Doesn't look like anyone is purposely mining empty blocks on a regular basis. So, that's a good thing.
Thanks,
Sam

TheRealSteve

hero member

Activity: 686

Merit: 500

FUN > ROI

Quote from: jonnybravo0311 on June 09, 2015, 04:37:54 PM

Quote from: os2sam on June 09, 2015, 04:27:33 PM

Can you tell what pool(s) are currently mining 0 transaction blocks consistently?

Not from the data I pulled since the core API doesn't provide it - I ran this against my local node. I don't know how happy blockchain.info would be if I pummeled them with 86k requests Tongue

.

I ran my parser against a dataset - wasn't really designed for this - but here's what I've got:
Please see update post

Note: This post previously contained an image based on incorrect data. See: https://bitcointalksearch.org/topic/m.11578412

The old image, for reference, can be found at: http://i.imgur.com/RlhKM97.png

Comments below up to kano's post identifying a problem are regarding this old image.

Quote from: jonnybravo0311 on June 09, 2015, 04:37:54 PM

Quote from: TheRealSteve on June 09, 2015, 04:28:11 PM

Quote from: chalkboard17 on June 09, 2015, 04:20:47 PM

Why do some pools do that? Is there an advantage?

Time.

This is where there's plenty of disagreement between pool software writers. On one side of the fence you've got luke-jr and wizkid who support the empty block theory. On the other you've got ck/kano who do not support it at all.

Disagreement is healthy

Topic: Empty blocks - page 9. (Read 23022 times)