Topic: WTF is this? Someone found a trick for fast mining? - page 4. (Read 15839 times)

I don't have any secret. I just observed what I believe is an unusual pattern in block validation that I wanted to discuss. Can I or is this a forbidden topic?

I also have better things to do than to try to teach how scientific inference works.



I see these 4 9's and several other independent facts with small probability, thus it is more like choosing a random numbers between 1 and 100.000.

Please, you, that claims to understand probability, teach us how many 9's you need to see to believe that there is a chance that there is something broken?

Valiron write down your "secret" or stop.  Though I think the secret is that you don't understand the statistics of the situation and you are not willing to admit it.
This is my lay understanding of what you are doing:
1. Someone is picking a "random" number between 1 and 100 a 1000 times.
2. Then you see 4 9s in a row.
3. Based on 4 9s in a row you decide the "random" number generator is broken.

But it's actually worse than that.  People are telling you the nonce "random" number generator is biased for several reasons (miner/asic optimisations) etc. So there is even less reason to suspect issues.

I think gmaxwell has spent enough time on this.  I'd rather see more other/interesting points from him than address your misunderstanding or lack of a secret. As others I have learnt from this thread, but the marginal utility is dropping fast now.

If you really had a "secret" you'd either reveal it to someone responsible or you'd be using it to gain an advantage (but not too much as to destroy bitcoin, if you were sneaky about it).  You certainly won't be posting smoke on the forum.

gmaxwell did nothing of the sort.

You are not even recognising that he's trying to teach you.

I'm not, I am making the most simplistic of statistical arguments against a logical fallacy. To say it technically you've adopted a model so complex that we cannot tell if you've overfit and can not reject the null hypothesis.

I can demonstrate this another way;  lets take your proposed tests:

"having 4 nounces with differences less than 1.8%, 0.12% and 7.2% respectively  of the range in this order."

"the block size, also points to the fact that it is the same miner who mined the blocks. 731 kB blocks are quite common as noted earlier by someone else, but it is not
very likely either to find them consecutively. " you haven't given a numerical threshold, so I'll just take that to be sizes within 10%

"The third piece of evidence is how close in time are these blocks."-- they're not actually unusually close, given hashrate changes, but I'll apply the softer limit of allow below 10 minutes.

"The fourth piece of evidence is the non-chronological timestamps " -- these are also fairly common through Bitcoin's history; but I'll go ahead and apply it too.

"The fifth piece of evidence is that the first block is mined by AntPool and the next 3 by anonymous." -- I can't mechanically apply this one, since it's just bc.i's opinion; but as you'll see leaving out a constraint only aids your argument now.


This is your test for a specific non-disclosed mining optimization. Since you won't disclose the optimization I cannot argue with that point, though it sounds implausibly over-complex.

Now we're going to ask the related question:  "How much of the network is using the optimization identified by this test?";

There are _no_ there matches in the blockchain meeting that criteria other than the single range you've named.   What should we then assume the proportion of hashrate using your optimization is?  Approximately 0.

It's well established that normal people routinely engage in completely defective statistical reasoning. Statistics are unintuitive to people, virtually everyone finds qualitative reasoning more intuitive than quantitative reasoning.

An ideal way to reason about things is to first understand the process; form a hypothesis and from the hypothesis develop a model without looking at the data:  "If vulnerability X is being exploited I will see blocks of with structures and probabilities X, if X is not exploited I will see blocks with structures and probabilities !X". You can then ask what the KL divergence of these two distributions are, and if it is very small then you will gain almost no confidence even with many observations; the question may be undecidable. If there is a separation between the probabilities  then observations one can then apply a statical test to reject one alternative or another with a intentional level of chance of error.

An example of this would be "From the structure of SHA256, and the fact that the 80 byte input requires two runs of the compression function; someone could create slightly more efficient specialized hardware which hardcodes much of the message expansion and initial rounds from the second compression function run-- and scans by rolling the block version.  If this optimization were at play I would expect to find that the block version would have very high entropy, perhaps 32 bits though anything more than a few bits would be suspicious; blocks exploiting this behavior would be expected to have uniform version numbers, rather than a constant in non-exploiting blocks"  We could then apply this test to the blockchain, and because of the big gap between two cases we could decide pretty quickly if the test was indicating. This isn't a definitive test; there could be other reasons for the strange behavior, like miners trying to reduce their network bandwidth usage; we could try to make it more specific by adding a rule like "and we expect blocks with random versions to have nonces with far less entropy".  ---- unfortunately, keeping the reasoning private prevents that kind of thoughtful analysis; all we can go on is how "weird" blocks are in the absence of a reasoned model, and thats not very useful since every block is "weird" by some definition.  Its like if you look for the number 11 you'll find it everywhere.

Sometimes there are free parameters-- we want to ask a question like "does weight over some threshold cause heart attacks? and if so what is that threshold?", without knowing the parameter in advance. This had the danger of fitting the model to the data and telling us nothing at all (like my example of "predicting" the nonces that just happened);  one tool used to address this problem with parametric models is cross validation:  You split your observations up and use one subset to train the model and the other to test.  If the effectiveness goes away during cross-validation the model is likely overfit. The protection isn't perfect because if you tweak the scheme based on the results you miss overfitting the "meta"-parameters..   This is effectively the test I applied above:  You fit a model (a set of gap differences, block timstamps, inter-arrival times) and I excluded a single hit which you appear to have used to set your parameters from the testing set and found your model matched _no_ blocks at all.

What shouldn't be allowed is bad statistical reasoning, which can happen no matter how "natural" your model is, what primarily matters is how many degrees of freedom it has.  What you are looking at there is _not_ dispersion; I tried to give you the benefit of the doubt and assume you meant one of the standard metrics for dispersion (_range_) and you insisted on a complex polytope shape constraint; with probably something on the order of 34 bits of parameter space (three percentages to 0.1 precision without permutation), plus some more model freedom in that its looking at differences instead of absolute values.

If I instead use the standard deviation (another common dispersion metric) we that 0.1875% of 4-block sets (given uniform nonces) would have a standard deviation as low as your selected one; so again, something we'd see every few days.


There is huge amounts of information; everything is open source except for the RTL and mask images of common hardware; though most follow not too far from prior FPGA designs in their overall structure.


It appears unlikely, all four are miners that appear to constantly reuse a static address for their income, but they're each different.  Walletexplorer does an overly aggressive 'taint' analysis and links many addresses to each, but they're disjoint.  The lowest number appears to be antpool, the next appears to be AnxPro.com, the next is likely Polmine.pl (it frequently pays addresses connected with polmine.pl).

There is nothing astonishing that when you pick criteria to fit the data, you find that it fits the data. There is also nothing astonishing that when a model created this way seems unlikely on a uniform basis that it will fail to generalize to anything but the data you fit it on.


Great; so you'll accept concrete proof that these blocks were mined by independent parties as a definitive proof that you were incorrect?

I'm having a hard time figuring out what you mean by retracted and erased;  your posts in this thread continue to claim  "It is clear that someone found a trick for fast mining. I kind of happen to know what might be.", "It is premining at some extend. Won't disclose more for the moment." and so on.  You continue to hold that you have "secret" knowledge which you will not disclose so that it can be discussed on its merits or lack thereof-- this entire thread is a great big advertisement for this claim and seems to have served you little other purpose; which the experienced professionals on this forum find to be non-credible and not supported by the evidence you've presented.


As I explained privately, your behavior so far is indistinguishable from a person who is willfully and fraudulently claiming to know of non-public mining optimizations in the hope of selling them to some greedy sucker who is unable to assess their merit except on the your misleading qualitative claim that the blocks look 'weird'.  Protecting the forum's participants from being deceived themselves or from suffering the traffic from a flood of hopeful scammers demands that we call it out when we see the potential of it-- I say this not as the subfourm moderator or a developer of the Bitcoin system, but just as a community member... There is plenty of room for doubt; I'd say it's even more likely that you're just confused by the statistics of it, but the benefit of doubt can't be given freely, since those looking to exploit people will just sail through that opening.  I offered you a simple mechanism which you can use to distinguish yourself-- share your complete theory with (ideally) the thread or privately with a respected member of the community; in doing so you gain the ability to refine it against the forge of experience, discover what (if any) parts are already known; and, in the possibility that there is something to be concerned about, gain the ability to protect Bitcoin from any potential harm if required.  (I think you greatly underestimate the potential harm of privately held substantial mining optimizations-- beyond some threshold they would result in the complete centralization of the bitcoin system)

The forum trust is a metric of trust; I explained in the rating where the distrust comes from. Your claims here are not credible to any of the subject matter experts and as far as we can tell they're based on erroneous statistical reasoning, you're shielding your theories from criticism by hiding behind secrecy, forcing a debate around statistical minutia and how "weird" the blocks feel qualitatively, rather than the merits of your idea. Only a few weeks ago you were asking questions (using asics to hash files) that showed a profound lack of research and understanding of Bitcoin mining.  I want to be sure someone considering trading with you over these ideas is aware that other community members do not currently consider your claims credible or likely to be well founded. Anyone is free to read the rating and come to their own conclusions.

I'm sorry that you find it heavy-handed.  I would prefer if there were a way to make ratings which were more targeted or conditional. In consideration of this, I'll go and reduce it to neutral to avoid triggering the red flag on you. I am not seeking to cause you distress.

I mentioned it non-specifically in my first response in this thread; (actually some things even more powerful than that). The objection was always the claim of "secret" knowledge supposedly supported by these very non-specifically identified blocks. Sergio actually described the optimizations he was talking about; as have a great many people. Not said they were keeping them secret and then "oh yea that" when other people play guessing games with other already known optimizations (which there have been easily a dozen discussed over the years);  especially not using alarming words like "premining" (which is a term used for a oft-perceived dishonest move in pump-and-dump altcoins).

I was unable to reconstruct suspected vulnerability from proposed defense. The discussion in the linked thread was mostly about compatibility, not about cryptography. Also, new Sergio's idea about approximation of double SHA256 is not discussed here, as well as his guess about Satoshi using Gray code optimization for early mining. Strange.

Is it the case?

Thank you for the links. The second link hints at something but it is not implementable as described (this is well known to any cryptographer and I am not hinting at any secret knowledge. If workable as described it could be used to find collisions of the hashing algorithm).

If I understand properly Sergio Demian states the weakness of the current mining algorithm and proposes a partial solution modifying the block headers. So, since this information is in a public blog, this weakness, or people claiming it, is also public knowledge, thus there is no reason any one gets mad when someone mentions it. Right?

Agreed, there were a few new things I learned from gmaxwell's posts. He is a gentleman and scholar. What is amazing about Bitcoin is that there is so much information to absorb and so many nuances and developments that you really should approach it with a bit of humility. I learn something new almost everyday.

Clearly you did it but I don't believe it was in vain, the discussion have very useful information and many of us are learning from this. Furthermore, your explanations were absolutely necessary. 

As an aside-- I suspect your time is somewhat off, since there is basically a constant offset of a couple minutes between your and my numbers. (My clock is timed off a local atomic clock which is wsynced to UTC with a GPS timing receiver; it agrees with a remote stratum 1 NTP server by better than 1ms).
Indeed.

A node of mine received these blocks at the following times

2015-05-02 12:29:45 height=354640

2015-05-02 13:06:01 height=354641
2015-05-02 13:09:46 height=354642
2015-05-02 13:12:59 height=354643
2015-05-02 13:17:45 height=354644

meh.

I explained specifically how and why miners select nonces non-uniformly-- I would ask if you missed the post, but you responded to it; https://bitcointalksearch.org/topic/m.11266349.

You gave no indication that you were "waiting" on anything-- you even responded "This makes sense"--, if I didn't see this post would you just go ahead and declare default as you did earlier in this thread to jl2012?  I've already wasted an incredible amount of time responding to you, but I won't make claims and fail to back them up if someone finds them implausible.

However, you're not going to get a compact "algorithm" because there isn't a compact description for it, as the behavior is a product of the physical geometry of the part, random decisions decisions, and even the failed engines on particular chips; ranges are also truncated for load distribution across chips, and to control latency.  The behavior is well known, documented in various data sheets (see, for example register 0x7f), and even sometimes used for device autodetection by some pools and mining software (Even for just mode detection, E.g. the icarus driver sends a dummy work task to the miner which was chosen to have 4 different solutions, and figures out what mode the chip is in based on which solution it returns). Historically, before extranonce rolling existed, nonce restrictions were used to reduce work on pools. Of course, it's not the _most_ commonly discussed thing; because its fundamentally uninteresting-- all nonces are equally good so if some device only uses some subset of them the response of an engineer working on software for them is just "okay, thanks for telling me.", they write it into their implementation, and they move on with life; they don't go and author a press release.

And yes, reducing the range it results in some additional extranonce work, but extranonce work is trivial already-- a single cheap micocontroller can do the extranonce work for many TH/s of mining.


And what are your other _exact_ criteria? Please lay out the exact test you suggest which shows those blocks to be improbable in a single post in a list so there is no confusion... I've not seeing _anything_ exceptional about these blocks, excepting the fallacious post-hoc selection of criteria to exactly fit them around nonces.

Just pot luck with them blocks being mined had it myself when mining other tpyes of crypto mining shead loads blocks then a drout of mining and no blocks and then a burst of blocks. often not is just pure luck to get something like that happen. Having a lot of hash power does help too

The algorithm to do the SHA256d could be in software but it does make sense to put the entire algorithm on the chip to have dedicated bitcoin mining hardware.

The ASICs don't have algorithms per se to hash SHA256d. From what I understand, the ASIC chips are just a bunch of physical circuits that perform a SHA256 hash. These are mostly not public domain.
I also thought that the nonces were determined by the software from the computer that the miner is connected to e.g. cgminer or bfgminer. These software are open source.

Huh, you found nothing special in blockchain (yes, I could be wrong here) but the reaction to your post is very, very interesting.
For reference:
Redesign of bitcoin block header
Potentially faster method for mining on the CPU

EDIT:
Did Sergio_Demian_Lerner published details about found attacks?

I also don't understand so much aggressivity. It is disturbing to say the least.

If a charlatan comes to me and tells me that he has a 2 page proof of the Riemann Hypothesis, this doesn't really disturbs me much... I guess I will thank him and let him know that I will read his extraordinary proof...

For the record:

Until further notice I believe it has been shown that the pattern detected can only appear every several years. For me this is unusual. Maybe not for you, maybe not for gmaxwell that claims from his knowledge that the odd  distribution of nounces can be due to mining hardware. I am willing to believe that but I am waiting for the links to algorithms in the mining hardware that produce this type of bias in the nounces. If these links do not exist it means that these facts are not in the public knowledge. And the nounce is just one of the facts that make this sequence of validations unusual.  

A related question is: Are the algorithms in different ASICs in the public domain or are they proprietary? And then if they are not, I ask you guys asking for transparency if you didn't ask to the ASIC manufacturers for transparency. Did you? Where you as aggressive? Did the moderators label them as scammers for not revealing the inside proprietary algorithms? Or on the contrary were they allowed to advertise their products in the forum?