The size of the blockchain... - page 3.

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Quote from: Peter R on May 01, 2014, 06:21:48 PM

Quote from: skooter on May 01, 2014, 06:17:38 PM

bitcoin QT can sometimes use upwards of 1 GB of RAM.

Well there's your problem skooter up there in bold. We are talking about a custom implementation of bitcoind to create a tiny and low-cost bitcoin node. We don't even need a wallet implementation, and we definitely don't need the QT GUI (there's no display lol).

Well most of that memory usage is the db cache. It probably can be done on a custom ARM based board however it isn't going to be a trivial project and you probably want the board to have a gig of memory. The UXTO is only going to get larger over time.

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: vpitcher07 on May 02, 2014, 07:33:14 AM

I'm not so concerned of the size of the blockchain once it's downloaded. What's more concerning is the bandwidth required to download it. Internet speed (for most) is not keeping pace with the increase in size of harddrives. I personally just got 1MB/s at home and I have cable. That's still a large amount of time to download a 17(?) gig file. Not to mention data caps.

This would be another benefit of low-costs single-purpose plug-and-play bitcoin node hardware. Rather than downloading the blockchain from a single peer node (as is currently the case), the card could come pre-loaded with the blockchain (or you could choose to download it from a faster torrent).

The microSD card could be partitioned to contain a section for storing the growing blockchain, and another section for storing the bitcoin binary that gets bootloaded by the ARM-core processor. You would update your bitcoin program by copying the new binary compiled by a trusted source onto the card.

You could probably do some other useful things too. If these devices became cheap and simple enough, they could come as part of bitcoin miners with added firmware to run P2P pool by default. So you just plug your miner in and immediately it begins mining at P2Pool. The "configuration details" such as where the mined bitcoins get sent can be specified in a simple text file on the SD card. Alternatively, if these devices become cheap enough, they could be an integral part of bitcoin point-of-sales devices in the future.

Massimo80

full member

Activity: 168

Merit: 100

Nice idea, but a device like this would need power and an Internet connection. And possibly a wireless connection, if you are unable to get it near an Ethernet cable. And of course a custom-designed board, which has development and production costs.

On the software side, it would require at the very least a TCP/IP stack and network drivers, and probably much more in order to be able to port bitcoind to it.

What I'm saying it's not this is not feasible, but it could get much more complex and costly than you think. And this calls for the main question: why? What would be the purpose of developing, building, selling, buying and running such a device, which can only give a very small benefit to the Bitcoin network and can't perform any useful function at all for his owner?

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: Lauda on May 02, 2014, 06:58:14 AM

Quote from: DeathAndTaxes

Trade with caution. User interested in generating double spends and willing to share profits with miners to defraud others.

This is why he was negative trust rating. That doesn't tell you if someone is a troll or not.

I know skooter to be a troll from his past comment history, several in response to my own posts. The fact that Gerald (DeathAndTaxes) took the time to report him for pursuing double-spends for the purposes of defrauding others tells me that skooter is also a scammer.

FeedbackLoop

hero member

Activity: 742

Merit: 500

Quote from: Peter R on May 01, 2014, 05:43:10 PM

This got me thinking: does anyone have bitcoind running on an ARM processor? It seems it should be possible to create a "bitcoin node" based on an ARM processor, a microSD card, and a simple ethernet interface IC. The entire node would be only slightly larger than the RJ-45 connector on the mating ethernet cable.

Would be great if someone mounted and sold those for the lazy people who would like to run a full node at lost cost and low work like me Tongue

.

vpitcher07

sr. member

Activity: 342

Merit: 250

I'm not so concerned of the size of the blockchain once it's downloaded. What's more concerning is the bandwidth required to download it. Internet speed (for most) is not keeping pace with the increase in size of harddrives. I personally just got 1MB/s at home and I have cable. That's still a large amount of time to download a 17(?) gig file. Not to mention data caps.

Lauda

legendary

Activity: 2674

Merit: 2965

Terminated.

Quote from: Peter R on May 01, 2014, 09:22:00 PM

We aren't talking about using a SD card for RAM. The SD card would be used to store the blockchain, update the blockchain with new blocks, update the unspent outputs each block, and bootload code segments of the bitcoin binary. Typically, flash memory is rated between 100,000 - 1,000,000 write/erase cycles (2 - 19 years if cycles happen every 10 min) and that's always writing the same segment over and over. With intelligent disk management to avoid "hot spots," this could be greatly increased. So I would expect even low-grade memory cards to last many years.

But skooter you don't really care about what we are talking about anyways, as can be verified by your post history as a troll and the glaring red warning in your profile.

Quote

Trade with caution. User interested in generating double spends and willing to share profits with miners to defraud others.

This is why he was negative trust rating. That doesn't tell you if someone is a troll or not.
Also the blockchain has grown over the past years but it will be growing at an increasing rate.

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: skooter on May 01, 2014, 09:00:30 PM

I think write endurance would be an issue if you're trying to use an SD card as RAM.

We aren't talking about using a SD card for RAM. The SD card would be used to store the blockchain, update the blockchain with new blocks, update the unspent outputs each block, and bootload code segments of the bitcoin binary. Typically, flash memory is rated between 100,000 - 1,000,000 write/erase cycles (2 - 19 years if cycles happen every 10 min) and that's always writing the same segment over and over. With intelligent disk management to avoid "hot spots," this could be greatly increased. So I would expect even low-grade memory cards to last many years.

But skooter you don't really care about what we are talking about anyways, as can be verified by your post history as a troll and the glaring red warning in your profile (EDIT: suggesting you are also a scammer).

skooter

member

Activity: 70

Merit: 10

Quote from: Peter R on May 01, 2014, 07:53:10 PM

Quote from: Soros Shorts on May 01, 2014, 06:47:47 PM

Quote from: telepatheic on May 01, 2014, 06:34:02 PM

Quote

bitcoin QT can sometimes use upwards of 1 GB of RAM.

With some custom settings to limit what gets held in memory, you could get this down to less than 10MB, but it would require a fair bit of work.

Whatever work is done must not slow down the speed at which the node works. You would be doing the network a disservice if your node took a long time to validate and relay messages and blocks because of your low memory footprint and under-powered CPU configuration.

I think people underestimate how powerful low-cost DSPs have become. For example, C. Gouvea was able to verify secp256k1 ECDSA signatures in 700 ms using an ultra-lower power 16-bit microcontroller running at a only 8 MHz (and available as in a tiny 3 x 3 mm package size). Even very cheap ARM-core processors ($2 - $4) will typically run at ~150 MHz (and with 32-bit registers). So let's say signature verification for each received TX takes no more than 20 ms, at which point you can relay the transaction to your other peers. This seems perfectly acceptable to me.

The only other issue I can think of is RAM, but I don't see why you can't store the unspent outputs in the SD card itself (new SD cards permit reads in excess of 30 MB/s so I think the time it takes to cross check the outputs in a TX with the blockchain unspent outputs stored in flash will be short compared to the signature verification times [20 ms]). It seems you just need sufficient RAM to pool the unconfirmed transactions (and some overhead to accept new blocks, etc), so upon first glance the 6x8mm 64 MB IC for $1.50 seems sufficient for this purpose.

I think such a bitcoin node would be useful if it could be made only a few cm in dimensions, cost less than $10-$15 excluding the SD card, and be trivial to set-up.

I think write endurance would be an issue if you're trying to use an SD card as RAM.

cypherdoc

legendary

Activity: 1764

Merit: 1002

Quote from: Peter R on May 01, 2014, 06:58:27 PM

Quote from: cypherdoc on May 01, 2014, 06:43:36 PM

bitcoind requires at least 1GB of RAM and usually more.

Thanks Cypherdoc. I am trying to make sense of this: is this because bitcoind is loading the unspent outputs into RAM for faster checking? My intuition is still telling me that in a custom implementation I just need enough RAM to comfortably pool the unconfirmed transactions.

i'm not sure what's filling up the RAM altho UTXO sounds like a reasonable guess.

according to David Schwartz, bitcoind requires around 650MB. sounds about right. see here:

http://bitcoin.stackexchange.com/questions/8400/to-what-is-bitcoind-memory-usage-bound

and here:

https://bitcointalksearch.org/topic/bitcoind-ram-consumption-313321

here's a top from one of my vps's:

Meuh6879

legendary

Activity: 1512

Merit: 1012

Quote from: telepatheic on May 01, 2014, 06:34:02 PM

Quote

bitcoin QT can sometimes use upwards of 1 GB of RAM.

With some custom settings to limit what gets held in memory, you could get this down to less than 10MB, but it would require a fair bit of work.

yes ... and it's very simple : decrease "cache" setting from 100MB to 10MB for example.
i must do this on my dedicated P2P server (because many others tasks work with bitcoin-QT in mode setting).

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: Soros Shorts on May 01, 2014, 06:47:47 PM

Quote from: telepatheic on May 01, 2014, 06:34:02 PM

Quote

bitcoin QT can sometimes use upwards of 1 GB of RAM.

With some custom settings to limit what gets held in memory, you could get this down to less than 10MB, but it would require a fair bit of work.

Whatever work is done must not slow down the speed at which the node works. You would be doing the network a disservice if your node took a long time to validate and relay messages and blocks because of your low memory footprint and under-powered CPU configuration.

I think people underestimate how powerful low-cost DSPs have become. For example, C. Gouvea was able to verify secp256k1 ECDSA signatures in 700 ms using an ultra-lower power 16-bit microcontroller running at a only 8 MHz (and available in a tiny 3 x 3 mm package size). Even very cheap ARM-core processors ($2 - $4) will typically run at ~150 MHz (and with 32-bit registers). So let's say signature verification for each received TX takes no more than 20 ms, at which point you can relay the transaction to your other peers. This seems perfectly acceptable to me.

The only other issue I can think of is RAM, but I don't see why you can't store the unspent outputs in the SD card itself (new SD cards permit reads in excess of 30 MB/s so I think the time it takes to cross check the outputs in a TX with the blockchain unspent outputs stored in flash will be short compared to the signature verification times [20 ms]). It seems you just need sufficient RAM to pool the unconfirmed transactions (and some overhead to accept new blocks, etc), so upon first glance the 6x8mm 64 MB IC for $1.50 seems sufficient for this purpose.

I think such a bitcoin node would be useful if it could be made only a few cm in dimensions, cost less than $10-$15 excluding the SD card, and be trivial to set-up.

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: cypherdoc on May 01, 2014, 06:43:36 PM

bitcoind requires at least 1GB of RAM and usually more.

Thanks Cypherdoc. I am trying to make sense of this: is this because bitcoind is loading the unspent outputs into RAM for faster checking? My intuition is still telling me that for a custom implementation I just need enough RAM to comfortably pool the unconfirmed transactions.

Soros Shorts

donator

Activity: 1617

Merit: 1012

Quote from: telepatheic on May 01, 2014, 06:34:02 PM

Quote

bitcoin QT can sometimes use upwards of 1 GB of RAM.

With some custom settings to limit what gets held in memory, you could get this down to less than 10MB, but it would require a fair bit of work.

Whatever work is done must not slow down the speed at which the node works. You would be doing the network a disservice if your node took a long time to validate and relay messages and blocks because of your low memory footprint and under-powered CPU configuration.

cypherdoc

legendary

Activity: 1764

Merit: 1002

Quote from: Peter R on May 01, 2014, 06:21:48 PM

Quote from: skooter on May 01, 2014, 06:17:38 PM

bitcoin QT can sometimes use upwards of 1 GB of RAM.

Well there's your problem skooter up there in bold. We are talking about a custom implementation of bitcoind to create a tiny and low-cost bitcoin node. We don't even need a wallet implementation, and we definitely don't need the QT GUI (there's no display lol).

bitcoind requires at least 1GB of RAM and usually more.

telepatheic

jr. member

Activity: 56

Merit: 1

Quote

bitcoin QT can sometimes use upwards of 1 GB of RAM.

With some custom settings to limit what gets held in memory, you could get this down to less than 10MB, but it would require a fair bit of work.

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: skooter on May 01, 2014, 06:17:38 PM

bitcoin QT can sometimes use upwards of 1 GB of RAM.

Well there's your problem skooter up there in bold. We are talking about a custom implementation of bitcoind to create a tiny and low-cost bitcoin node. We don't even need a wallet implementation, and we definitely don't need the QT GUI (there's no display lol).

Mobius7

hero member

Activity: 532

Merit: 500

Quote from: durrrr on May 01, 2014, 05:58:58 PM

I don't understand what u mean here won't the memory stick always be the same size and shouldn't change?

What memory stick? OP was talking about ARM processor and microSD card.

skooter

member

Activity: 70

Merit: 10

Quote from: Peter R on May 01, 2014, 06:12:27 PM

Quote from: telepatheic on May 01, 2014, 05:54:13 PM

Quote

does anyone have bitcoind running on an ARM processor?

People have got it running on raspberry pis.

Thanks. I do know about the raspberry pis, but I'm thinking even smaller: a tiny little custom board with only the bare minimum hardware required to run bitcoind. Perhaps the ARM processor could still run linux, or perhaps in the future we could write a version of bitcoind that implemented its own simple OS (as is commonly done for DSP projects).

Quote

The amount of memory (RAM) you require is the main problem.

I suppose you need enough RAM to pool the unconfirmed transactions, right? A 64 MB chip could hold around 100,000 TXs. Here's a 64 MB SDRAM chip that is only 6mm x 8mm in size and costs about $1.50: http://www.digikey.com/product-detail/en/AS4C4M16S-6BIN/1450-1077-ND/4531924

bitcoin QT can sometimes use upwards of 1 GB of RAM.

Peter R

legendary

Activity: 1162

Merit: 1007

Quote from: telepatheic on May 01, 2014, 05:54:13 PM

Quote

does anyone have bitcoind running on an ARM processor?

People have got it running on raspberry pis.

Thanks. I do know about the raspberry pis, but I'm thinking even smaller: a tiny little custom board with only the bare minimum hardware required to run bitcoind. Perhaps the ARM processor could still run linux, or perhaps in the future we could write a version of bitcoind that implemented its own simple OS (as is commonly done for DSP projects).

Quote

The amount of memory (RAM) you require is the main problem.

I suppose you need enough RAM to pool the unconfirmed transactions, right? A 64 MB chip could hold around 100,000 TXs. Here's a 64 MB SDRAM chip that is only 6mm x 8mm in size and costs about $1.50: http://www.digikey.com/product-detail/en/AS4C4M16S-6BIN/1450-1077-ND/4531924

Topic: The size of the blockchain... - page 3. (Read 4119 times)