Topic: your own btc node. (Read 330 times)

Bitcoin Core RPC missing lots of features compared with Blockexplorer API (e.g. blockchair.com).

I don't recall Bitcoin Core have feature such as address index, built-in filter and external information which isn't on blockchain (e.g. BTC price, pool hashrate, mempool history).

You would use a node's RPC function to programmatically get each confirmed transaction, along with what block each transaction was confirmed in, and you can then make queries as to the balance of each address as of whatever date (as of whatever block) you need the information for. You obviously would be unable to get any historical information about the mempool, but you could also periodically programmatically query mempool related RPC commands and store this information into a database.

Yes, your best bet is to have a node and a (managed) relational database that you store information about each block/transaction in.

You would first create a full node, download the entire blockchain, and then use RPC commands to get the inputs and outputs of each transaction contained in each block and put this information into a relational database. You can then query the balance of any address as of the current date, or any date you need the balance on.

You can do this yourself, hire someone to do this for you. If you want to do this yourself, but are not quite sure how to get started, there is a kaggle dataset of the bitcoin blockchain, and on the dataset page, there is a link to github repo that appears to claim to be able to get an ETL pipeline for the bitcoin blockchain setup (I have not looked at the code, and cannot vouch for anything in the repo).

we need a simple balance request ...address/balance but lots of them...no detailed info is needed...in that context (considering that blockchair API charges 500$ a month for 50000 requests a day) wouldn't a separate Node be better?

Poor in what way? You should be able to get literally the same information from RPC commands, and can write a script to get the information by making a series of RPC calls to feed in whatever information you are looking for into a database.

There are a number of cloud providers that will give you enough credit to allow you to spin up a VPS with sufficient resources to obtain (and load into a database) this information for free.

You would do something along the lines of setting the results of RPC commands as variables that are fed into subsequent RPC commands, and from there set more results into more variables that are iterated through, and those results are inserted into your database.

I don't see any good reason to pay for a Block Explorer API if you have the skill set to programmatically obtain this information yourself. 

You should post what information you are looking for and what resources you have. If you have sufficient resources to create a full node, you can do so and use RPC commands to obtain the same information you would be able to get from a block explorer API.

thanks man.

got it. Is there any known, paid alternative to it? with good limits on requests, what do you think?

I'm talking about all known blockexplorers in general, "most block explorer" isn't a name or brand.
Examples of those limits is: capped at x number of queries per hour or day which 'x' can be as high as 1000,
so if you plan on doing that locally, your PC's spec must be higher than the average.

anybody knows what are the limits of the most block explorer API?

Blockchain.com have a form if you want to bypass their "limiters": https://www.blockchain.com/api click "REQUEST API KEY".
That involves registration and providing them some information.

Internet speed will only be an issue during the sync process.
For fetching blockchain data which requires -txindex, you'll be needing fast drive, powerful processor and a quite huge RAM for huge cache size
since you're talking about a scale beyond the limit of most blockexplorer API.

We need our own node for transaction checking purposes, fore block explorer API has their limits on number of request after which their block ip from which the requests are made...And i thought to my self if you set up your own node you will be limited only to a speed of CPU and internet speed of the machine on which node is set up...Dedicated server definitely. May be anybody familiar with the limit situation? Or how to avoid it? Or any paid block explorer API services))? Once again we don't need to post no transactions just for checking purposes. Thanks to everybody. Happy holidays

you could also spin the node on dedicated machine (like raspberry pi) and play with some LN https://github.com/rootzoll/raspiblitz
gonna take couple a few weeks tho

Because of the way that hard drives work, I believe that you should have a minimum of 500Gb, and that it is preferable to have that on a second drive.

You don't need to run your node every day if you aren't making daily transaction, but it pays to run it frequently to avoid long sync periods. The core is pretty robust, and you can interrupt it if it hasn't completely sync'ing - just make sure you wait for it to close down correctly. If you are using an external SSD for the blockchain, then male sure you "eject" it to ensure that the write cache has been emptied.

I just set one up two days ago and this is what it took.

1) laptop with windows 10 8gb of ram and 450gb ssd disk
2) download bitcoin core from https://bitcoin.org/en/download
3) wait to sync, for me it took 2 days over a fast WIFI in nothern Europe

Make sure you have at least 300GB or more as the full node is closing to the 300GB mark and you dont want to move the data again.

Good Luck
/KX

Most likely OP want to help bitcoin network by setting up full bitcoin mode. Or perhaps he/she want to use it for wallet and some development works. Whatever your intention, you need enough patience for that especially during download initial blocks. I were failed multiple time to download (Bitcoin core) due to network issue.

Anyway, I found a thread about your concerns, it would quite helpful for you during setting up your own bitcoin node, here is it, How to run and operate your own bitcoin node. You would found here about basic requirements and useful information. Best of luck for you.

Here are the minimum requirements to run your own full node (as stated on bitcoin.org):

It's a little bit outdated though.

Right now, blockchain size is almost 260Gb (exactly 255.000Mb according to blockchain.com). Therefore, you will need more than 200 gigabytes of free disk space as suggested above.

This depends on the reason you want to operate a full node.

There are a lot of guides available online on how to set up a full node. Following them and setting up a node is not that hard.
But the question is what do you want to achieve with that ?

If you need access to all transactions because you are developing something (e.g. a blockchain explorer), then you need to index all transactions (txindex=1).
If you just want to use it as a wallet without relying on any 3rd party, using the pruned mode is enough (takes less space).


So, if you can tell us more about what you want to do with your node and how much disk space you have available, we can help you way more precisely on how to set it up.

A little effort and spare resources. The easiest way for you would be to use Bitcoin Core. It will download, verify and store all the blocks by default. If you are low on storage then you can use pruning but you won't be sharing old blocks.

what does it take to set it up?