Topic: Thoughts and questions on BTC Pools and merged mining (Read 5631 times)

I would really interested in a JK 4diff patch for vinced mm bitcoind version in order to test the poolserverj implementation. Is there any solution out there?

Having played with pushpool and understanding the best simple optimisations to it, I'll have to have a look at poolserverJ now for my home test network.

Java - one of the best oo languages around ... In My Opinion.

Used Java on and off for many years.
Memory limitations are the only issues I've had that are in any way a true issue.
All other issues are simply just bad programmers.
Now if memory is an issue ... try compiling your Java code instead of 'bytecoding' it

My favourite argument in favour of Java is based on most people's lack of understanding of programming
Java code is expected to be faster than most other code.
Why? Simple to understand reason: optimisation.
You write a program today, compile it then release it.
Result: today's optimisation is the fastest it will get.
Java: release is the bytecode that is compiled at runtime, thus as optimisation gets better over time so does you Java bytecode's performance.

Code stability is the programmers fault and no more.
Running out of memory? Well I gotta say, if you have absolutely no idea of how much memory your program requires then you need to understand your code better ...
Debugging is probably the easiest language of all due to tracebacks being available with way more information than most common languages and easy to see/read

So he's "changing over to it" and has not actually changed over to it?  See my next comment in further regards to this:


In my case, and I'm not speaking for other pool authors, I do a lot of work on the getwork server that most other pools do on their backend.  This is mostly because it's exceptionally inefficient to try to do a scored pool on the back end.  BTC Guild is a proportional pool, so it's not really a surprise that it's substantially easier to switch a proportional pushpool over to another bit of software, since there's literally no changes *needed* in pushpool.  I know Eleuthria has done some modifications in Pushpool, but from speaking to him (granted, this was a couple months ago) his changes are not anywhere near as extensive as mine.


My java "hate" comes from countless experiences with Java based products in high capacity enterprise environments (Specifically in the wireless and banking industries).  I have yet to run across a single piece of software that is java based that is a) stable, b) not a resource hog (while still offer sub-par performance) and c) expandable.

I will admit that I have not tried anything with 64 bit Java, since most of the enterprise stuff that's running java based junk is 32 bit, but I'm not talking about 128m of memory limits.. I'm talking about high capacity applications that needs > 8 GB.  This likely doesn't apply to psj, though, so it's probably a moot point. I am just illustrating the limitations of java and why it's not a good choice for applications that are expected to grow or need stable, low latency operation.

Again, I'm not saying psj suffers from any of this, but I've yet to see a java application that doesn't suck and/or have some idiotic limitation because of the exceptionally craptactular JVM.  When a program crashes, I just love having to suss out whether it was the JVM that had a problem or the actual program itself... yay, double my debugging work!

Regardless, I'll take a look at it and see what would be required to convert my setup over to it.  I just have visions of going through that work and then finding myself (once again) up against an nigh impenetrable wall of asinine java design architecture.

Of course I'm not using poolserverj, and I like native code too, but you are a bit wrong in this part :)

It's the merkle root of all the aux chains blockheaders + number of aux chains.

Really?  Is that why the 2nd biggest pool (and the largest that uses pushpool) is changing over to it?  If you have concerns about it's capacity ask Eleuthria how many pushpool instances he replaced with a single poolserverj instance in a production stress test before it choked.


Again suggest you ask Eleuthria how long it took him to make the necessary mods.  You don't even have to rebuild the binary, it's designed with plugin architecture so you can just override the commonly customised parts (like worker auth, db interaction etc) into a separate jar...


Leaving aside the obvious java hate here (I've learned it's not worth having that argument).  poolserverj was designed from the ground up for large pools.  pushpool was not.  That's not an indictment on the pushpool author, it's simply a consequence of the fact that the network was a lot smaller when pushpool was designed than when poolserverj was.   


umm... it's ONE command line switch... -Xmx128m or whatever you want to limit the heap size to.    psj does not use memcached, it keeps all it's caches internal so of course the process is going to use more than pushpool.  It also uses the general principle of preferring performance over limiting memory footprint where there's a choice.  That's a design choice because it's meant to handle high loads.

pooserverj will only have one binary.  wether it uses merged mining or not will be configurable via properties.  There will be no additional overhead if not using merged mining.

Yes a JK 4diff for the mm version of bitcoind is needed however I don't think it's really necessary for namecoind or other aux daemons.  They do not deal with a torrent of rpc requests like the bitcoind does.  Just some aux block updates every few seconds and submission of valid shares which is also rare.

To clarify, poolserverj does not support merged mining natively yet.  It will work behind a merged-mining-proxy instance.  I've been getting my head around it all this last week and plan to make a native implementation tomorrow.  It should be ready by the changeover block unless there's a sudden upsurge in hashrate but it will be alpha.

There was no legitimate complaints, only trolls.

Which is what I said after I realised how the two chains where being mined with the same hash - and I said that coz it is the only way to actually do it - and said that coz I KNOW it is the only way to actually do that.
I didn't think back in my first post in the cgminer thread that they were putting extra data in the bitcoin block chain ... why?
COZ EVERYONE KEEPS TELLING ME THAT THEY DON'T DO THAT.
However, they ARE putting extra data in the bitcoin block chain.
Thus the web page is wrong.
Again this is all a simple understanding of how bitcoin works - nothing to do with reading how namcoin merged mining works.
If namecoin is generating bitcoin blocks, those bitcoin blocks MUST adhere to the bitcoin block chain rules.

Any additional bytes per blocks are an issue since the size of the bitcoin block chain is already an issue.
... and if every block has these extra bytes (as the namecoin people would hope that everyone does merged mining) then it is even more of an issue.
Saying it is 33 bytes doesn't help. I'll look at the code and determine what it really is.

you may be right with that (i just rechecked the thread but did not found anything regard size). do you have a link for that?

anyway: i do not think 33 additional bytes per block is a problem.

and you just confirmed that its possible to set custom data (eg namecoin blockdata) inside the bitcoin block without breaking things for the default bitcoin client.

luke-jr did increase the block size.
You didn't understand that?
That was part of the complaint about what he was doing.
He is current just putting "Eligius" in there (and stopped with the prayers) and thus increasing each block size by 3 bytes
The normal coinbase is 8 bytes.

My blocks (that say Kano) in the other 4 chains have increase each single block that I got by 6 bytes.
Yes I changed the bitcoin code for my solo mining - how on earth else do you think that is possible?
It has nothing to do with the mining software.
(and also has nothing to do with normal pool software - only the server that creates the answers for the getwork requests)

it works with default bitcoin client and the default chain (its not an alternate chain). only reason you need a bitcoin patch is if you want to mine solo (or if you are a pool op)

it's the pool owner's bitcoin who decides what to place in the coinbase.

luke-jr (owner of eligius) did a test and placed prayers in it - without a block size change; without a modified client. is this impossible too?

You CANNOT say "gets 33bytes namecoin data in his coinbase" and "without a bitcoin block size change"

Seriously? You don't see the fallacy of that statement?

Are you telling me that the real bitcoin block chain is going to reject all merged mining bitcoin blocks and only namecoin will accept the namecoin blocks?

If you modify anything in the block (including the coinbase) after calculating the hash, but before putting the block into the real bitcoin block chain, the block will be rejected.

I know this not only by reading but by fact, due to having modified most of the different alternate coin software myself and adding a block into each block chain that says "Kano": I0coin block 29930, SolidCoin V1 block 33533, GeistG block 3676, IXcoin block 21128.
Did it for fun by reading the code and working out how the coinbase works, but more for the sake of understanding how things work.

You see, it has to work with the real bitcoin block chain - doesn't matter if you run a hacked bitcoin, the real bitcoin must accept the bitcoin blocks.

Thus your statement must be wrong (and again, the front page of that site must be wrong)

And Yes I will be reading the code when I have time to go through it (as I said) since no one knows or at least wants to admit what the code actually does (or makes statements that cannot be true)

please do that before posting about merged mining again.

the bitcoin block gets 33bytes namecoin data in his coinbase (without a bitcoin block size change).

there are several people who did run this on testnet (and you can do this right now).

To put this discussion in the right place ...

I didn't link to that document, I referred to where I could gain the source.
That document is wrong.

As I said you completely missed what "create hash" is.
"create hash" is hashing the contents of the block that goes into the block chain and then checking if it is under the difficulty requirement.
If both are true, you then have a block.

Since you are hashing the contents of the block, you cannot even compare that to some other different block.
It's not possible.

What is possible, is to add extra data to the block, that some other block chain can check against and decide it will allow it to be a block.

The hash of the bitcoin block hashes the following information:
version
previous_hash
time
difficulty
nonce
the miner's extranonce
the transactions in the block (this includes the generation block which has the coinbase)

i.e. the whole block

The above information is mandatory for the block chain to be secure.
It ensures that no one can change the block chain somewhere in the middle.

Thus the bitcoin hash is the hash of the complete bitcoin block contents.
It is not the hash of ANYTHING more or less.

For a hash for namecoin to be securely usable in both bitcoin and namecoin, the bitcoin block would have to contain something extra added that namecoin can use.
If this were not true, then namecoin would have to rely completely on the contents of the bitcoin block chain to secure itself i.e. namecoin would become dependent on having a copy of the whole bitcoin block chain to be able work.

P.S. I still haven't read the namecoin code yet

For the big pools, poolserverj is not really an option.  We (not that I'm including myself in the big pools) have our getwork servers, either custom or pushpool highly modified to conform to our database and methods.  Switching over to poolserverj would be a major undertaking and rewrite of the poolserverj software.

Not something that couldn't be done, but it's not something that's going to happen very quickly.  It needs to work with current implementations to gain traction, not foisted off on yet another getwork server.  I don't relish the idea of :

a) Running ANYTHING java based, having to use Java based junk in the enterprise, I can tell you I have never once seen a java based program that was worth anything in high availability/heavy load environments.  Couple that with the fact that without some severe tweaks, the memory limits of the java VM leave a LOT to be desired.  I'll stick with native code, thanks.  That said, it's usually the interface that is complete FAIL with java crap, so maybe Poolserverj doesn't suffer from this, since there's no need for an interface to speak of.  But there are so many other problems with java scalability wise that it's just too unstable and problematic to build a business off of.
b) Rewriting yet MORE code to conform to my standards

Agreed.

Although, I would run the pool on a separate machine until I was 1000% sure it wouldn't interfere with the normal pool.

More important than any of that is DOCUMENTATION. On the protocol, not the patches assuming I want to risk the main pool operation.

As this nears completion, it would be benificial for pool operators if there was:
A bitcoind diff for the merged client (since most of us run custom clients)
A merged-mining poolserverj binary
An easily modifiable namecoind source to apply joel catz fixes (or a premodified one).