how to get block height for given day

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: pooya87 on February 19, 2023, 08:14:11 AM

--snip--

Edit: Yeap. They've changed things. Downloaded Electrum 4.3.4 on Ubuntu and let it sync, even deleted the blockchain_headers file and let it be downloaded again. In both cases the file contains all zeros up until block height 747936. Which is a waste of space if you ask me!!! This also means what I explained above won't work using newer version of Electrum.

That's what i suspect. I just found out Electrum protocol also have API call which can ask single block header[1] and block headers starting from specific height[2]. But those API call i mentioned could be used as alternative to obtain block headers.

[1] https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html#blockchain-block-header
[2] https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html#blockchain-block-headers

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: ABCbits on February 19, 2023, 06:58:49 AM

Do you mind checking beginning of file blockchain_headers with hex editor/viewer? I tried on my device, but mostly it only contain zeroes. I suspect Electrum only request needed block header rather than all block header.

Yes, although I simply read the file as a byte stream not with hex editor, the file starts with the genesis block's header ie. 0x01 followed by 35 zeros (previous hash) then 0x3b (merkle root) and so on.

I've had my Electrum for a very long time though. The file was created in 2019! Maybe they changed some stuff like after the introduction of checkpoint header.
Edit: Yeap. They've changed things. Downloaded Electrum 4.3.4 on Ubuntu and let it sync, even deleted the blockchain_headers file and let it be downloaded again. In both cases the file contains all zeros up until block height 747936. Which is a waste of space if you ask me!!! This also means what I explained above won't work using newer version of Electrum.

LoyceMobile

hero member

Activity: 1659

Merit: 687

LoyceV on the road. Or couch.

Quote from: RoxxR on February 19, 2023, 01:01:48 AM

Thanks everyone for the various approaches!

Let me add my topic on block data: https://bitcointalksearch.org/topic/bitcoin-block-data-available-in-csv-format-5246271
In short, you're looking for https://loyce.club/blockdata/time.txt

ABCbits

legendary

Activity: 2870

Merit: 7490

Crypto Swap Exchange

Quote from: jackg on February 18, 2023, 07:46:31 AM

Can you do by just subtracting 144 from the block height per day you're looking?

This is wrong approach. Current all-time average block time is somewhere about 9.55 minutes.

Quote from: jackg on February 18, 2023, 07:46:31 AM

If not, I'd still recommend doing that and then looking up the api for a block explorer (like blockchain.com) and trying to programmatically go back (or forward) from there based on how many blocks would have come since. I remember blockchain.com's api indexing blocks based on height and hash (not sure how others do it but it's probably an easy enough place to start and get a json of data, and then checking the time the block was received or generated).

Unfortunately blockchain.com API doesn't have filter to choose block based on date.

Quote from: pooya87 on February 18, 2023, 10:23:22 PM

Here is another fun way of doing it which is a lot faster and cheaper to do:
If you run a SPV client like Electrum on your computer you already have all the block headers saved up in a ~60 MB file called "blockchain_headers" which is a stream of bytes. All you have to do is to programatically read this file to get the raw bytes which should be a multiple of 80. Each 80 byte is a block header of a block in chronological order (so you have the block height this way too). Then start from the beginning (byte 0) and extract the time from each 80 byte chunk (ie. one header) knowing that 4th item in it is the timestamp.

Code:

version[4] + prev_hash[32] + merkle_root[32] + time[4] + target[4] + nonce[4]

Here is a pseudocode

Code:

stream = File.Read("blockchain_headers")
while(stream.HasBytesLeft)
stream.Skip(68)
timestamp = stream.Read(4).ConvertToTime_LittleEndian()
resultList.Add(timestamp)
stream.Skip(8)

Now all you have to do is search in the list of timestamps to see when the day in the datetime you converted changes to get the last height of the day. The height is the index of the datetime inside the resultList/array.

Keep in mind that timestamps are in UTC not your local time.

Do you mind checking beginning of file blockchain_headers with hex editor/viewer? I tried on my device, but mostly it only contain zeroes. I suspect Electrum only request needed block header rather than all block header.

RoxxR

full member

Activity: 209

Merit: 148

Thanks everyone for the various approaches!

RoxxR

full member

Activity: 209

Merit: 148

Quote from: pooya87 on February 18, 2023, 10:23:22 PM

Here is another fun way of doing it which is a lot faster and cheaper to do:
If you run a SPV client like Electrum on your computer you already have all the block headers saved up in a ~60 MB file called "blockchain_headers" which is a stream of bytes. All you have to do is to programatically read this file to get the raw bytes which should be a multiple of 80. Each 80 byte is a block header of a block in chronological order (so you have the block height this way too). Then start from the beginning (byte 0) and extract the time from each 80 byte chunk (ie. one header) knowing that 4th item in it is the timestamp.

Code:

version[4] + prev_hash[32] + merkle_root[32] + time[4] + target[4] + nonce[4]

Here is a pseudocode

Code:

stream = File.Read("blockchain_headers")
while(stream.HasBytesLeft)
   stream.Skip(68)
   timestamp = stream.Read(4).ConvertToTime_LittleEndian()
   resultList.Add(timestamp)
   stream.Skip(8)

Now all you have to do is search in the list of timestamps to see when the day in the datetime you converted changes to get the last height of the day. The height is the index of the datetime inside the resultList/array.

Keep in mind that timestamps are in UTC not your local time.

Thanks, this does sound like a fun and effective way. I'll give it a shot!

pooya87

legendary

Activity: 3472

Merit: 10611

Here is another fun way of doing it which is a lot faster and cheaper to do:
If you run a SPV client like Electrum on your computer you already have all the block headers saved up in a ~60 MB file called "blockchain_headers" which is a stream of bytes. All you have to do is to programatically read this file to get the raw bytes which should be a multiple of 80. Each 80 byte is a block header of a block in chronological order (so you have the block height this way too). Then start from the beginning (byte 0) and extract the time from each 80 byte chunk (ie. one header) knowing that 4th item in it is the timestamp.

Code:

version[4] + prev_hash[32] + merkle_root[32] + time[4] + target[4] + nonce[4]

Here is a pseudocode

Code:

stream = File.Read("blockchain_headers")
while(stream.HasBytesLeft)
stream.Skip(68)
timestamp = stream.Read(4).ConvertToTime_LittleEndian()
resultList.Add(timestamp)
stream.Skip(8)

Now all you have to do is search in the list of timestamps to see when the day in the datetime you converted changes to get the last height of the day. The height is the index of the datetime inside the resultList/array.

Keep in mind that timestamps are in UTC not your local time.

o_e_l_e_o

legendary

Activity: 2268

Merit: 18711

You can find this data very quickly using Blockchair. For example, here is a list of all the blocks mined yesterday: https://blockchair.com/bitcoin/blocks?s=id%28desc%29&q=time%282023-02-17%29. They have an API you can use, but there is a cost associated with its use.

They do also have free data dumps of this information which you can download from here: https://gz.blockchair.com/bitcoin/blocks/

paid2

hero member

Activity: 504

Merit: 1065

Crypto Swap Exchange

The easiest way could be to take the list of the blocks mined by a big mining pool.

For example, you can be 100% sure that Antpool does mine at least some blocks per day, so you will have an Block Height associated to a date.

Example : list of blocks mined by Antpool

jackg

copper member

Activity: 2856

Merit: 3071

https://bit.ly/387FXHi lightning theory

Can you do by just subtracting 144 from the block height per day you're looking?

If not, I'd still recommend doing that and then looking up the api for a block explorer (like blockchain.com) and trying to programmatically go back (or forward) from there based on how many blocks would have come since. I remember blockchain.com's api indexing blocks based on height and hash (not sure how others do it but it's probably an easy enough place to start and get a json of data, and then checking the time the block was received or generated).

RoxxR

full member

Activity: 209

Merit: 148

I'm trying to programmatically get the block height at certain given dates - for instance, first day of each month.
I don't have access to a full node.
Is there a block explorer or other platform with an API endpoint (or just a small dump of such data somewhere) that could be helpful?
Thanks!

Topic: how to get block height for given day (Read 163 times)