Topic: Mining protocol bandwidth comparison: GBT, Stratum, and getwork - page 2. (Read 28241 times)

(Posting delayed by useless IRC conversation)


When did I ever say I was not seeing the network numbers?  Please quote me on that one.


So much hate... and I thought we had some love....

... well, maybe not... but anyway. 


Well, I'll start by pointing out that mining over SSL is a feature that is pretty useless anyway IMO... why waste resources encrypting/decrypting data that is essentially public knowldege already anyway?
that has no bearing on my tests. I stated that the majority of my testing was done against eloipool, which does not support SSL.


So, I obviously never tested that.  And that is also irrelevant to the data, since you said youself bfgminer has the non-SSL cases handled.

I will, however, rerun my longer PCAP tests against your network-bytes code instead of testing it for a short time (since it is newer than the beginning of my tests), to see how it goes with cgminer.

I shall report back soon!

... and then they slink away into the darkness coz they don't want anyone to think that they could be wrong about anything.

So just to clarify in case anyone was wondering where the code is that was committed 4 days ago:

https://github.com/kanoi/cgminer/commit/d3d8772b5ea1d29e6bf8e724fb9a2e662db30c19

... and anyone wondering why my git ... for anyone who bothers to keep track of what is going on with cgminer at the moment ...

https://bitcointalksearch.org/topic/m.1427509

You are not running the latest code if you are not seeing the network numbers ... the old code (as I ALREADY said in here quite a while ago) shows the data byte count, not the network byte count ... and ... as I ALREADY stated in this thread ... I would add the network numbers at a later date.
That change is there and has been for 4 days.
You're probably even looking at the wrong git.

... and of course you've again shown your stupidity.

Will I have to redo the words at 2nd grade level for the moron?
I'll quote them yet again just in case a few extra times sinks in:

The data being ignored is the SSL data - as the example code I linked clearly shows and the cgminer clone code ignores.
No idea why you are unable to spot that damn obvious point in the code there even after being told about it.
My code includes all 6 cases of data, the cgminer clone only includes 4 of them - it ignores the 2 SSL cases that only a halfwit would not be able to see looking at the code supplied by the Curl developers.

I'm not sure how you magically worked out, that when my code has EXTRA cases that the clone is missing, that I am actually getting the wrong answer.
I wrote my code based on the Curl developers code and thus got it correct.
I've no idea where on earth you got the fail code for the cgminer clone.

In my testing I am not using SSL, so it makes no difference to my numbers.
In your testing, if you are using SSL then you are getting the WRONG answer from the cgminer clone.
For anyone who IS using SSL and the clone miner, they will get the wrong answer.
Yeah I know that's not everyone, but when you write code, it's a good idea to get to work in all cases ... and viewing the Curl developers code has a much better chance of getting the right answer than thinking you are always correct when you often aren't

Sigh.

Again, must be psychic. A bad psychic at that. How would I be able to compare PCAP data to your outputs if I wasn't running the code which output the data? Are you really that thick?

And also, do, oh wise one, explain to me how bfgminer's numbers match near perfectly with the PCAP data, yet the as-of-last-night-version of cgminer's api output does not? If there was in fact data being ignored in bfgminer as you claim, then would that not show in a comparison against PCAP data? (I assume kano won't actually try to answer this... wagers?)

i.e. you are not running the latest cgminer version fool.
See, when you don't even take any notice of what is going on with cgminer and only stare into your master's eyes and at your masters crappy clone (as to be expected by the sock puppet you are), don't show your stupidity by assuming you know what you are talking about cgminer, coz you are wrong.
As per my screen, it does indeed.
The code has been there for 4 days.

Though I will repeat what I said above since clearly your ability to read is deficient:

Edit: I'll even add the picture again - since words are too difficult for a sock puppet to understand

I'm pretty sure looking any code is not even necessary considering I'm using PCAP data for my tallies. Learn to read.

Second, it's good to know that you're psychic now and know what version of the software I'm using without even Asking! I'll keep that in mind on my next trip to Vegas.

Edit to clarify: PCAP data matches bfgminers bandwidth efficiency number, while no cgminer api output from any version to date matches. Feel free to run a capture and run the numbers on your own if you don't believe me.

Loser ... using the version of cgminer that doesn't have the change to include the Network byte count ... D'oh
If you are going to make comments about cgminer - then do that when you know what you are talking about.

Anyway, the clone is ignoring data in certain circumstances ... as per that code from the curl developers shows ... that clearly both of you are too dumb to even notice the problem when I stick the code in front of you - lulz.

Meanwhile ... a run log ...



And anyone wanting to generate that with a customsummarypage in miner.php
... and of course you add this to your list of customsummarypages:
'Proto' => array($protopage, $protosum, $protoext)

I've also concluded that, for whatever reason, cgminer does not correctly report bandwidth usage consistent with usage reported by sniffing data directly on the box in question between the pool and box, which is the method I personally used for my tests.  I have several GB of PCAP to back up my "vague sweeping statements".  However, I find it wasteful to post all of that raw data when a summary of the findings is all that is needed.

Also, the new "E" that calculates bandwidth usage in bfgminer as a measure of number of shares per 2KB was in fact accurate to within 1% of my PCAP data. My assumption is that I'm including some packet header data that "E" is not, which would account for the error mostly.

cgminer's counters however almost never matched my PCAP data, being off by huge margins that I could not logically account for, thus I deemed that particular output from the miner useless.

After disabling gzip in bfgminer, both cgminer and bfgminer yield statistically identical results for bandwidth usage for all protocols supported by both softwares.  With gzip compression enabled in bfgminer code, bfgminer, obviously, uses less bandwidth.

All long tests were done against eloipool, and some shorter comparison tests done against various other pools with expected results.

Well I've pointed out the problem and you've ignored it.
To be expected. Nothing new. Source code in front of your face is too difficult for you to understand ...

cgminer doesn't lie about the byte count, it is the correct byte count transferred.
The old version just didn't report the network byte count which can, of course, be different to the data content byte count transferred, if the data is being compressed.
What you didn't bother to point out to the fools who believe you, is that will depend on the pool ...

Of course, as you already know (since I reported it to you), cgminer's counters lie about bandwidth usage. And since you haven't reported any bug to me, you have nobody to blame for that but yourself - IF it's true (which I doubt, since you're not competent and I did in fact read the code you referenced).

As I stated in here before, any test that expects 2 bitcoind's to have the same transaction list is naive and will fail.
If a pool restarts it's bitcoind, the memory pool is empty ...
If you restart your bitcoind, the memory pool is empty ...


I'll reply to your other comment later ... since it's better to supply numbers rather than make vague sweeping statements like you have.
The cgminer API reports those numbers so it's easy to see what they are ... rather than just say something without proof.

Edit: though I will point out that after I added the extra 'network' byte counter to cgminer, I went and looked at the clone code ... and noticed he was ignoring some data ....................
Here's a clue (that he obviously didn't have one of )
https://github.com/bagder/curl/blob/master/src/tool_cb_dbg.c

While this is semi-off-topic, I feel obligated to point out/clarify that Eligius is under new management. 
There are not arbitrary limits on transaction count of block size in place.  The only thing of note in place regarding transactions, that is not part of the reference client, is the filtering of Satoshidice transactions, which means we simply do not include bets directed at Satoshidice in our blocks.  (For the record, this is publicly noted in the original post from when this was implemented here)  The decision to leave that particular piece of code in place was a personal one, as I do believe that SatoshiDice could use a much less wasteful method of doing business.  Passive aggressive vs SatoshiDice.

Anyways, I think that having the transaction data available for miners to verify is worthwhile.  While the software to actually allow any real verification of this data, aside from validating it against what the pool is asking be hashed, is in it's infancy, I think it has potential to help secure the network even more by allowing miners to choose to allow their software to check their transaction list against their own bitcoind's transaction list, note any major issues, transactions that would not be accepted by a standard client, etc.

Obviously not every single miner is interested in doing this, and I'm reasonably certain no one has ever said otherwise.  But, it would not be a bad thing IF every miner were.

----

In any case, this thread is about the bandwidth use of the various mining protocols.

So far, through my own testing, my number's concur with the OP's to within an acceptable margin for error. (No discrepancy sufficient enough to change the order of most to least usage).

I however, do not see any real performance differences between the protocols as far as pool-side accepted shares over time, so, as far as I'm concerned, the bandwidth usage is a moot issue since I would venture to guess that majority of miners are using residential internet connections which are generally unmetered, and even the worst protocol bandwidth-wise (secure stratum) does not use a substantial amount of bandwidth, averaging about 36kbit/sec... and the cheapest broadband connection available in my area is 1500 kbit/sec. So, I think bandwidth usage is a moot issue for mining, especially considering these are non-vardiff tests too.

Have fun, try to stay on topic here...

-wk

You've missed the point that a few of us have asked and been given no answer.

What does verifying the transactions do?

It verifies that the list of transactions provided by the pool ... matches the list of transactions in the block.
Well ... good to know ... but what does that do?
Nothing. There are no 'this is a bad transaction' checks anywhere.

Anyway, once a block is actually found - the list of transaction MUST exist for everyone on the bitcoin network to see or the block will be rejected.

So if people are concerned about mining transactions that some psychotic, religious nutcase thinks are damaging to bitcoin, then oddly enough those dangerous, damaging transactions will be known by everyone as soon as they land in a block.
... and then they can go rant about the pool that did it and lots of people will leave the pool.
If they never get in a block ... well aint that good

Though I will agree that this doesn't always work, there was this pool called Eligius that attacked an alt-coin with the pools hash power ... and also drastically restricted Bitcoin transactions for about 5 months ... and clearly stated that he was doing it in his pool thread ... that no one seemed to care about.

So ... basically ... even adding some 'this is an evil transaction' code into the miner, most likely it will have no effect anyway.
... and we'd also have some centralised 'authority' saying what is an 'evil' transaction ... ... ... ... ... but fortunately only centralised for those who used his holiness' cheap clone miner.

Quoth the wot,


Not an accident.

You seem to have messed up with the URL tag in that quote... I think it's cute so I'm quoting your error as-is.

---

Also, What?

I don't think microsoft or their compiler suite is malware (which somehow magically steals souls)... As a fact-respecting scientist, my usual baseline state don't involve a soul or other philosophical / religious / ethical / GNU baggage, so I'm not worried about this... uh... "philosophically / religiously non-free" distorted price which you speak of, and therefore, somehow different than "free of charge" (zero monetary cost) as most people understand the word...

---

As far as bandwidth goes, I'll stick with stratum, thanks though for helping prove that even your own implementation of stratum makes better sense than GBT...

Most miners / bitcoin users are probably:

1) windows users without "free license" GNU-ism baggage
2) not watching their mining software unless it stops working
3) not watching the contents of the transactions they're (blindly) verifying 24/7
4) not worried about your "philosophically / religiously non-free" dilemma, since most of their software is compiled using commercial compiler toolchains.

The logic would follow that most miners / bitcoin users don't care if your use of MSVC-compatible code will allegedly hurt their soul, or some sort of philosophical / religious / ethical / GNU-ism or other baggage with regard to how they mine or spend bitcoin... Stratum protocol is also acceptable for above-listed reasons 2 and 3...

OK thanks, carry on.

Because they aren't free. If you want to sell your soul to Microsoft for it, go ahead and use their malware to sign your binaries yourself.