Decentralized mining protocol standard: getblocktemplate (ASIC ready!) - page 7.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: Inaba on October 22, 2012, 05:54:16 PM

The blockchain would have to be reduced to under a few megabytes

Getting off-topic here, but this is already possible.

Quote from: Inaba on October 22, 2012, 05:54:16 PM

and RAM usage would have to be a few hundred kilobytes

This is completely unreasonable. Merely being a C++ program requires more than a few hundred KB.

Inaba

legendary

Activity: 1260

Merit: 1000

You forgot to specify what CPU you are using, also how much RAM total you have. Also your HD usage.. 4.5GB. That is larger than the entire usage of my 500 GH/s farm. Combined. That 0 - 2% CPU usage you quote, I'd wager it's a modern Intel or AMD based CPU. You just go ahead and try to run that on an ARM based CPU or a C6 or something, see how far you get (you'll never finish loading the block chain, new blocks will come in faster than you can process old ones).

The blockchain is not suitable for lower power environments. It simply isn't. The blockchain would have to be reduced to under a few megabytes and RAM usage would have to be a few hundred kilobytes before it becomes feasible to run a full node on a mining rig.

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Real world considerations should be part of GBT - not ideal world ...

The GBT data transfer should currently ALWAYS be high at the moment ...

Checking the bitcoind debug.log for the last 3 days:
The smallest poolsz was:
10/20/12 06:58:09 CTxMemPool::accept() : accepted a661e71a68 (poolsz 579)

Yes the minimum outstanding transactions for the last 3 days (which was after block 204102) was 579 waiting txns
(also note that I restarted my bitcoind 4 days ago - so that figure may be low ...)

So unless pools are ignoring lots of transactions, there should never have been a GBT workload sent from any GBT pool with fewer than a few hundred transactions in it for the past 3 days.

... but of course I've very little doubt that isn't the case ... since we know what pool software supports GBT.

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: Inaba on October 21, 2012, 02:50:12 PM

Interesting... I would like to know more about that at some point. I've not run into any problems and my server load is dramatically decreased. Regardless, though, I'm not sure that pools ignoring some transactions in favor of others is necessarily a negative thing. Right now, I limit the amount of transactions I put into any one block and I don't see any major issue with favoring higher transaction fee transactions over lower ones.

If there are no higher fee transactions waiting, then the lower fee ones will get processed and that's as designed as far as I know. I'm not saying it's a bad thing to "fix" GBT, but I've not run into any problems so far and I'd like to know at what point GBT becomes more cumbersome than getwork from a transaction standpoint.

The issue is the payload the longer a block takes to solve as the number of transactions rises. It would not be surprising if the pool was sending 100k sized payloads with all the transactions after 10 minutes on the block. While the load on the pool is definitely less, the data being spewed out to miners will eventually rise, and the more transactions and more popular bitcoin becomes, the larger the payload grows in a linear fashion with transactions. Eventually sending out 100kb payloads say every 30 or 60 seconds from the pool to all your miners will become a burden. Prioritising transactions is your prerogative but with this disincentive to sending transactions we will see a deepbit like situation where pool ops will put a static upper limit on the amount of transactions in total they care about including. This is much worse for bitcoin in general. But with an extension to the gbt protocol that more closely resembles the merkle root base of stratum it can easily be fixed.

Inaba

legendary

Activity: 1260

Merit: 1000

Interesting... I would like to know more about that at some point. I've not run into any problems and my server load is dramatically decreased. Regardless, though, I'm not sure that pools ignoring some transactions in favor of others is necessarily a negative thing. Right now, I limit the amount of transactions I put into any one block and I don't see any major issue with favoring higher transaction fee transactions over lower ones.

If there are no higher fee transactions waiting, then the lower fee ones will get processed and that's as designed as far as I know. I'm not saying it's a bad thing to "fix" GBT, but I've not run into any problems so far and I'd like to know at what point GBT becomes more cumbersome than getwork from a transaction standpoint.

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: Inaba on October 21, 2012, 03:09:20 AM

Well, I've been mining with GBT for about a month now on a lower power netbook with no problems (and no bitcoind), so I'm not sure what you're referring to exactly?

https://bitcointalksearch.org/topic/m.1284623

Inaba

legendary

Activity: 1260

Merit: 1000

Well, I've been mining with GBT for about a month now on a lower power netbook with no problems (and no bitcoind), so I'm not sure what you're referring to exactly?

P_Shep

legendary

Activity: 1795

Merit: 1198

This is not OK.

Quote from: Inaba on October 21, 2012, 12:36:34 AM

Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind. It's far, far, far too resource intensive to put on most serious mining setups. I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.

Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter. A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet. Trying to run a full blockchain on any of those would be an exercise in futility.

I've been trying to get bitcoind on the router... not having much luck so far. Not entirely given up yet.

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: Inaba on October 21, 2012, 12:36:34 AM

Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind. It's far, far, far too resource intensive to put on most serious mining setups. I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.

Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter. A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet. Trying to run a full blockchain on any of those would be an exercise in futility.

Which means you pretty much unintentionally agree with my position: GBT in its current form is not advantageous to miners. Luckily jgarzik is open to modifying it based on what I have recommended. We will be talking about it in the near future.

Inaba

legendary

Activity: 1260

Merit: 1000

Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind. It's far, far, far too resource intensive to put on most serious mining setups. I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.

Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter. A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet. Trying to run a full blockchain on any of those would be an exercise in futility.

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Seriously - what are you looking for?

Decentralisation or control?

Pick one.

Saying that you want decentralisation ... except it has to be how you or Luke-jr or whoever else wants it ... isn't decentralisation.
As soon as you start deciding how things like mining MUST be done, you've lost the plot.

firefop

sr. member

Activity: 420

Merit: 250

Quote from: kano on October 20, 2012, 10:52:58 PM

Sigh - another nut-case.

Hate all you want, but it doesn't invalidate anything I've said. The fact remains that certain posters are more interested in trash talking rather than contributing, while others are overworking themselves to try and get this whole bitcoin thing working to the point where we can make it grow into it's potential.

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Quote from: firefop on October 20, 2012, 09:30:01 PM

...
We need to encourage people like Luke to stick around, add even more sanity to the development of the network and slowly marginalize those who don't think in the right direction.
...

Sigh - another nut-case.

firefop

sr. member

Activity: 420

Merit: 250

Quote from: ?? on ??

Quote from: Luke-Jr on October 20, 2012, 07:41:29 PM

Quote from: ?? on ??

Quote from: Luke-Jr on October 20, 2012, 07:11:22 PM

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.

Unfortunately, simply making it mandatory isn't practical. The closest we could come is giving strong incentives for doing so. For example, I intend to set up Eligius in such a way that those who are solo mining in combination can profit more by claiming the fees from transactions they accept over the base set Eligius provides.

Your efforts are noble, but it might be better for you to start working on alt currency, the one with really strong foundation. Let retards like
those 2 above, who despise you, me and anyone else not fitting their utopian views, finish off Bitcoin. Don't waste your time here, seriously.

I really like your prev-post with the graph substrata - you basically said everything I was thinking but couldn't articulate. Thank you for that!

However, I respectfully disagree. We need to encourage people like Luke to stick around, add even more sanity to the development of the network and slowly marginalize those who don't think in the right direction. If you look at the network as a whole, that's already happening. 2 years ago I wasn't here and you'll see more actual computer scientists become involved in development as bitcoin grows.

@Luke - incentives are nice - but I'd settle for having it be the default option - If you don't have time/inclination to build a custom install script to set that all up (when/if it's supported) I'd be happy to create and submit it to you for approval / publication.

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: jgarzik on October 20, 2012, 07:37:57 PM

Quote from: -ck on October 20, 2012, 05:05:19 PM

Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

Definitely interested in extensions that broaden the userbase as much as possible.

Even for bitcoind (rather than a pool server), extensions like this make sense. The old "getwork" protocol did as much work as possible on the server side, thereby minimizing the miner's work -- notably the miner did not have to care about anything besides the block header itself. No transactions or other parsing to worry about.

getblocktemplate is intended to replace getwork. If there are improvements to be made, I'm quite open to that.

Ping me on IRC, if you want to rapidly iterate some changes to make bitcoind's GBT work better for a wider selection of mining softwares.

I did the final getblocktemplate polish, simplifying and cleaning things up from its original BIP 22/getmemorypool state. If BIP 22 / BIP 23 are currently missing something you want... I want to put it in there.

Thanks. When I get some extended block of time on IRC in the very near future, and you're online, I will most certainly.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: Sitarow on October 20, 2012, 07:20:59 PM

Quote from: Luke-Jr on October 20, 2012, 07:11:22 PM

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That is planned in release BFGMiner's 3.0?

Probably that will have to wait for 3.1... 3.0 is the ASIC release, and that's coming up too soon.

Quote from: ?? on ??

Quote from: Luke-Jr on October 20, 2012, 07:11:22 PM

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.

Unfortunately, simply making it mandatory isn't practical. The closest we could come is giving strong incentives for doing so. For example, I intend to set up Eligius in such a way that those who are solo mining in combination can profit more by claiming the fees from transactions they accept over the base set Eligius provides.

jgarzik

legendary

Activity: 1596

Merit: 1091

Quote from: -ck on October 20, 2012, 05:05:19 PM

Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

Definitely interested in extensions that broaden the userbase as much as possible.

Even for bitcoind (rather than a pool server), extensions like this make sense. The old "getwork" protocol did as much work as possible on the server side, thereby minimizing the miner's work -- notably the miner did not have to care about anything besides the block header itself. No transactions or other parsing to worry about.

getblocktemplate is intended to replace getwork. If there are improvements to be made, I'm quite open to that.

Ping me on IRC, if you want to rapidly iterate some changes to make bitcoind's GBT work better for a wider selection of mining softwares.

I did the final getblocktemplate polish, simplifying and cleaning things up from its original BIP 22/getmemorypool state. If BIP 22 / BIP 23 are currently missing something you want... I want to put it in there.

slush

legendary

Activity: 1386

Merit: 1097

Quote from: ?? on ??

...I think not... ...I think we should not gamble... ...Bitcoin is extreme fragile... ...demand from users... ...I know it can be done... ...it should be mandatory...

Honestly, as I'm reading your posts, I think that you should create separate blockchain with those rules hardcoded. Feel free to name it NaziCoin.

kano

legendary

Activity: 4466

Merit: 1798

Linux since 1997 RedHat 4

Quote from: ?? on ??

Quote from: Luke-Jr on October 20, 2012, 07:11:22 PM

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.

See now you know what you should do.
Go create an alt-coin that tells everyone what they must be doing and see how that fares.
If you are right, then that alt-coin will outlast BTC and you'll be famous ...................

Sitarow

legendary

Activity: 1792

Merit: 1047

Quote from: Luke-Jr on October 20, 2012, 07:11:22 PM

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That is planned in release BFGMiner's 3.0?

Topic: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) - page 7. (Read 32257 times)