Need Help Setting up Node | Bitcointalksearch.org

beagle1977

newbie

Activity: 35

Merit: 0

Quote from: mocacinno on July 25, 2018, 03:44:11 AM

Quote from: beagle1977 on July 25, 2018, 03:41:58 AM

and lastly, i need the client for running ETH node. can you guys help? sorry to bug you too much

but the project involves running these 3 clients and loading the data into a DB. but by the looks of it , it seems a lost cause, considering the volume !

I've never run an ether node either, but i think you need geth
https://github.com/ethereum/go-ethereum/wiki/geth

But yeah, if you're going to parse the full BTC, BCH and ETH blockchain and insert the data into a relational database, you'll need loads and loads of processing power and fast storage (and a lot of it).

thanks a lot again !

i am planning to use AWS and i guess it will give me the required processing power but it is going to be very costly i guess !

mocacinno

legendary

Activity: 3612

Merit: 5297

https://merel.mobi => buy facemasks with BTC/LTC

Quote from: beagle1977 on July 25, 2018, 03:41:58 AM

and lastly, i need the client for running ETH node. can you guys help? sorry to bug you too much

but the project involves running these 3 clients and loading the data into a DB. but by the looks of it , it seems a lost cause, considering the volume !

I've never run an ether node either, but i think you need geth
https://github.com/ethereum/go-ethereum/wiki/geth

But yeah, if you're going to parse the full BTC, BCH and ETH blockchain and insert the data into a relational database, you'll need loads and loads of processing power and fast storage (and a lot of it).

beagle1977

newbie

Activity: 35

Merit: 0

and lastly, i need the client for running ETH node. can you guys help? sorry to bug you too much

but the project involves running these 3 clients and loading the data into a DB. but by the looks of it , it seems a lost cause, considering the volume !

beagle1977

newbie

Activity: 35

Merit: 0

Quote from: Rath_ on July 25, 2018, 03:38:40 AM

Quote from: mocacinno on July 25, 2018, 03:36:35 AM

I have never run a BCH node on any of my machines, but if i'm not mistaking, bitcoin-abc is the reference client for bitcoin cash.
I think this is the github repo: https://github.com/Bitcoin-ABC/bitcoin-abc

Yes, it is. Here you can find the latest compiled version for Linux, OSX and Windows.

oh...great ! Thanks a lot !

Rath_

legendary

Activity: 1876

Merit: 3139

Quote from: mocacinno on July 25, 2018, 03:36:35 AM

I have never run a BCH node on any of my machines, but if i'm not mistaking, bitcoin-abc is the reference client for bitcoin cash.
I think this is the github repo: https://github.com/Bitcoin-ABC/bitcoin-abc

Yes, it is. Here you can find the latest release for Linux, OSX and Windows.

mocacinno

legendary

Activity: 3612

Merit: 5297

https://merel.mobi => buy facemasks with BTC/LTC

Quote from: beagle1977 on July 25, 2018, 03:31:57 AM

Quote from: mocacinno on July 25, 2018, 03:29:28 AM

Quote from: beagle1977 on July 25, 2018, 03:27:31 AM

Oh ok...its more complicated than i expected i guess.

That's why i proposed to use an existing block parser and edit it to suit your needs Wink

IMHO, usually it's better not to re-invent the wheel...

yeah agreed. And You are a Legend, Mate!

This is a project that i have to complete, so it is going to be very useful. Especially the links that you have shared.

also, i am unable to find the software for running the node for BTC Cash. Can you help?

I have never run a BCH node on any of my machines, but if i'm not mistaking, bitcoin-abc is the reference client for bitcoin cash.
I think this is the github repo: https://github.com/Bitcoin-ABC/bitcoin-abc

Disclaimer: since i've never run a BCH node, i can't be 100% sure this is the correct reference client!

beagle1977

newbie

Activity: 35

Merit: 0

Quote from: mocacinno on July 25, 2018, 03:29:28 AM

Quote from: beagle1977 on July 25, 2018, 03:27:31 AM

Oh ok...its more complicated than i expected i guess.

That's why i proposed to use an existing block parser and edit it to suit your needs Wink

IMHO, usually it's better not to re-invent the wheel...

yeah agreed. And You are a Legend, Mate!

This is a project that i have to complete, so it is going to be very useful. Especially the links that you have shared.

also, i am unable to find the software for running the node for BTC Cash. Can you help?

mocacinno

legendary

Activity: 3612

Merit: 5297

https://merel.mobi => buy facemasks with BTC/LTC

Quote from: beagle1977 on July 25, 2018, 03:27:31 AM

Oh ok...its more complicated than i expected i guess.

That's why i proposed to use an existing block parser and edit it to suit your needs Wink

IMHO, usually it's better not to re-invent the wheel...

But i wasn't kidding when i said this task will be very resource intensive. I tried parsing the blocks and inserting them into a simple mysql db a long, long time ago... I had to quit because the progress was so slow.

beagle1977

newbie

Activity: 35

Merit: 0

Quote from: mocacinno on July 25, 2018, 03:23:38 AM

Quote from: beagle1977 on July 25, 2018, 03:17:48 AM

Quote from: mocacinno on July 25, 2018, 03:15:02 AM

Quote from: beagle1977 on July 25, 2018, 03:05:39 AM

Hi,
I am looking to set up a main BTC node and loading the data into a DB. i downloaded the node software from BTC core and have proceeded with sync..Now i have a set of files but they dont seem to be in a readable format. How do i load them into a DB? Is there a schema? I tried searching the internet but i am unable to find it. Or am i missing something?

There are several options, the easyest one is searching for an existing block parser in a language you understand, and adapting it in order to insert all data into a relational database. Do expect this to be a very time and resource consuming process... You'll need hundreds of gigabytes of diskspace, and if you mess up your relational database, inserting, deleting, updating or selecting records will be painstakingly slow.

In the past, i've found a couple projects you could find interesting:
https://github.com/znort987/blockparser
https://github.com/alecalve/python-bitcoin-blockchain-parser
https://github.com/neocogent/sqlchain

Great! Thanks a lot. But is there a schema or something for the data? I want to load the data using java. so if i have the data structure for the data, i will probably able to write it myself?

No, there's no schema, it's not a relational database.

Here's an old document i found on http://codesuppository.blogspot.com/2014/01/how-to-parse-bitcoin-blockchain.html
http://2.bp.blogspot.com/-DaJcdsyqQSs/UsiTXNHP-0I/AAAAAAAATC0/kiFRowh-J18/s1600/blockchain.png

Here's a link to the wiki: https://en.bitcoin.it/wiki/Block

Good luck

Oh ok...its more complicated than i expected i guess.

mocacinno

legendary

Activity: 3612

Merit: 5297

https://merel.mobi => buy facemasks with BTC/LTC

Quote from: beagle1977 on July 25, 2018, 03:17:48 AM

Quote from: mocacinno on July 25, 2018, 03:15:02 AM

Quote from: beagle1977 on July 25, 2018, 03:05:39 AM

Hi,
I am looking to set up a main BTC node and loading the data into a DB. i downloaded the node software from BTC core and have proceeded with sync..Now i have a set of files but they dont seem to be in a readable format. How do i load them into a DB? Is there a schema? I tried searching the internet but i am unable to find it. Or am i missing something?

There are several options, the easyest one is searching for an existing block parser in a language you understand, and adapting it in order to insert all data into a relational database. Do expect this to be a very time and resource consuming process... You'll need hundreds of gigabytes of diskspace, and if you mess up your relational database, inserting, deleting, updating or selecting records will be painstakingly slow.

In the past, i've found a couple projects you could find interesting:
https://github.com/znort987/blockparser
https://github.com/alecalve/python-bitcoin-blockchain-parser
https://github.com/neocogent/sqlchain

Great! Thanks a lot. But is there a schema or something for the data? I want to load the data using java. so if i have the data structure for the data, i will probably able to write it myself?

No, there's no schema, it's not a relational database.

Here's an old document i found on http://codesuppository.blogspot.com/2014/01/how-to-parse-bitcoin-blockchain.html
http://2.bp.blogspot.com/-DaJcdsyqQSs/UsiTXNHP-0I/AAAAAAAATC0/kiFRowh-J18/s1600/blockchain.png

Here's a link to the wiki: https://en.bitcoin.it/wiki/Block

Good luck

beagle1977

newbie

Activity: 35

Merit: 0

Quote from: mocacinno on July 25, 2018, 03:15:02 AM

Quote from: beagle1977 on July 25, 2018, 03:05:39 AM

Hi,
I am looking to set up a main BTC node and loading the data into a DB. i downloaded the node software from BTC core and have proceeded with sync..Now i have a set of files but they dont seem to be in a readable format. How do i load them into a DB? Is there a schema? I tried searching the internet but i am unable to find it. Or am i missing something?

There are several options, the easyest one is searching for an existing block parser in a language you understand, and adapting it in order to insert all data into a relational database. Do expect this to be a very time and resource consuming process... You'll need hundreds of gigabytes of diskspace, and if you mess up your relational database, inserting, deleting, updating or selecting records will be painstakingly slow.

In the past, i've found a couple projects you could find interesting:
https://github.com/znort987/blockparser
https://github.com/alecalve/python-bitcoin-blockchain-parser
https://github.com/neocogent/sqlchain

Great! Thanks a lot. But is there a schema or something for the data? I want to load the data using java. so if i have the data structure for the data, i will probably able to write it myself?

mocacinno

legendary

Activity: 3612

Merit: 5297

https://merel.mobi => buy facemasks with BTC/LTC

Quote from: beagle1977 on July 25, 2018, 03:05:39 AM

Hi,
I am looking to set up a main BTC node and loading the data into a DB. i downloaded the node software from BTC core and have proceeded with sync..Now i have a set of files but they dont seem to be in a readable format. How do i load them into a DB? Is there a schema? I tried searching the internet but i am unable to find it. Or am i missing something?

There are several options, the easyest one is searching for an existing block parser in a language you understand, and adapting it in order to insert all data into a relational database. Do expect this to be a very time and resource consuming process... You'll need hundreds of gigabytes of diskspace, and if you mess up your relational database, inserting, deleting, updating or selecting records will be painstakingly slow.

In the past, i've found a couple projects you could find interesting:
https://github.com/znort987/blockparser
https://github.com/alecalve/python-bitcoin-blockchain-parser
https://github.com/neocogent/sqlchain

beagle1977

newbie

Activity: 35

Merit: 0

Hi,
I am looking to set up a main BTC node and loading the data into a DB. i downloaded the node software from BTC core and have proceeded with sync..Now i have a set of files but they dont seem to be in a readable format. How do i load them into a DB? Is there a schema? I tried searching the internet but i am unable to find it. Or am i missing something?

Topic: Need Help Setting up Node (Read 160 times)