Pages:
Author

Topic: WTF is this? Someone found a trick for fast mining? - page 4. (Read 15839 times)

sr. member
Activity: 311
Merit: 250
Valiron write down your "secret" or stop.  

I don't have any secret. I just observed what I believe is an unusual pattern in block validation that I wanted to discuss. Can I or is this a forbidden topic?

I also have better things to do than to try to teach how scientific inference works.


Though I think the secret is that you don't understand the statistics of the situation and you are not willing to admit it.
This is my lay understanding of what you are doing:
1. Someone is picking a "random" number between 1 and 100 a 1000 times.
2. Then you see 4 9s in a row.
3. Based on 4 9s in a row you decide the "random" number generator is broken.

I see these 4 9's and several other independent facts with small probability, thus it is more like choosing a random numbers between 1 and 100.000.

Please, you, that claims to understand probability, teach us how many 9's you need to see to believe that there is a chance that there is something broken?

sr. member
Activity: 362
Merit: 262
Valiron write down your "secret" or stop.  Though I think the secret is that you don't understand the statistics of the situation and you are not willing to admit it.
This is my lay understanding of what you are doing:
1. Someone is picking a "random" number between 1 and 100 a 1000 times.
2. Then you see 4 9s in a row.
3. Based on 4 9s in a row you decide the "random" number generator is broken.

But it's actually worse than that.  People are telling you the nonce "random" number generator is biased for several reasons (miner/asic optimisations) etc. So there is even less reason to suspect issues.

I think gmaxwell has spent enough time on this.  I'd rather see more other/interesting points from him than address your misunderstanding or lack of a secret. As others I have learnt from this thread, but the marginal utility is dropping fast now.

If you really had a "secret" you'd either reveal it to someone responsible or you'd be using it to gain an advantage (but not too much as to destroy bitcoin, if you were sneaky about it).  You certainly won't be posting smoke on the forum.

You attribute yourself the privilege to talk in the name of the whole community. On which grounds? Your position as moderator and developer gives a valuable opinion (more than mine in the eyes of the community, no question about that),
but you cannot talk for other people, nor predetermine that they have your same opinion. We all have seen posts of people disagreeing with you and willing to understand what is going on. There are people around here that know more math
than you do. are open minded, and understand perfectly well the arguments I gave. Speak for yourself as I do and let others speak for themselves.
gmaxwell did nothing of the sort.

With respect to "matter experts" what we have discovered is that you don't have any serious background on statistics or probabilities. And in your case, you don't know how research inference works.  As for me I do recognize that
I know little about mining hardware, certainly you know more, but I probably can teach you some lessons about cryptography and hashing.
You are not even recognising that he's trying to teach you.
sr. member
Activity: 311
Merit: 250
staff
Activity: 4242
Merit: 8672
You are manipulating and you know it.
I'm not, I am making the most simplistic of statistical arguments against a logical fallacy. To say it technically you've adopted a model so complex that we cannot tell if you've overfit and can not reject the null hypothesis.

I can demonstrate this another way;  lets take your proposed tests:

"having 4 nounces with differences less than 1.8%, 0.12% and 7.2% respectively  of the range in this order."

"the block size, also points to the fact that it is the same miner who mined the blocks. 731 kB blocks are quite common as noted earlier by someone else, but it is not
very likely either to find them consecutively. " you haven't given a numerical threshold, so I'll just take that to be sizes within 10%

"The third piece of evidence is how close in time are these blocks."-- they're not actually unusually close, given hashrate changes, but I'll apply the softer limit of allow below 10 minutes.

"The fourth piece of evidence is the non-chronological timestamps " -- these are also fairly common through Bitcoin's history; but I'll go ahead and apply it too.

"The fifth piece of evidence is that the first block is mined by AntPool and the next 3 by anonymous." -- I can't mechanically apply this one, since it's just bc.i's opinion; but as you'll see leaving out a constraint only aids your argument now.


This is your test for a specific non-disclosed mining optimization. Since you won't disclose the optimization I cannot argue with that point, though it sounds implausibly over-complex.

Now we're going to ask the related question:  "How much of the network is using the optimization identified by this test?";

There are _no_ there matches in the blockchain meeting that criteria other than the single range you've named.   What should we then assume the proportion of hashrate using your optimization is?  Approximately 0.

Quote
If it is (as for 99.99% of normal people), how would you go about proving that is out of normal? Won't you compute the probability of seeing this occurence?
But then it will be fallacious according to your comment since it is "after the fact"!!!
Nonsense!! Otherwise empirical knowledge won't exist!!
It's well established that normal people routinely engage in completely defective statistical reasoning. Statistics are unintuitive to people, virtually everyone finds qualitative reasoning more intuitive than quantitative reasoning.

An ideal way to reason about things is to first understand the process; form a hypothesis and from the hypothesis develop a model without looking at the data:  "If vulnerability X is being exploited I will see blocks of with structures and probabilities X, if X is not exploited I will see blocks with structures and probabilities !X". You can then ask what the KL divergence of these two distributions are, and if it is very small then you will gain almost no confidence even with many observations; the question may be undecidable. If there is a separation between the probabilities  then observations one can then apply a statical test to reject one alternative or another with a intentional level of chance of error.

An example of this would be "From the structure of SHA256, and the fact that the 80 byte input requires two runs of the compression function; someone could create slightly more efficient specialized hardware which hardcodes much of the message expansion and initial rounds from the second compression function run-- and scans by rolling the block version.  If this optimization were at play I would expect to find that the block version would have very high entropy, perhaps 32 bits though anything more than a few bits would be suspicious; blocks exploiting this behavior would be expected to have uniform version numbers, rather than a constant in non-exploiting blocks"  We could then apply this test to the blockchain, and because of the big gap between two cases we could decide pretty quickly if the test was indicating. This isn't a definitive test; there could be other reasons for the strange behavior, like miners trying to reduce their network bandwidth usage; we could try to make it more specific by adding a rule like "and we expect blocks with random versions to have nonces with far less entropy".  ---- unfortunately, keeping the reasoning private prevents that kind of thoughtful analysis; all we can go on is how "weird" blocks are in the absence of a reasoned model, and thats not very useful since every block is "weird" by some definition.  Its like if you look for the number 11 you'll find it everywhere.

Sometimes there are free parameters-- we want to ask a question like "does weight over some threshold cause heart attacks? and if so what is that threshold?", without knowing the parameter in advance. This had the danger of fitting the model to the data and telling us nothing at all (like my example of "predicting" the nonces that just happened);  one tool used to address this problem with parametric models is cross validation:  You split your observations up and use one subset to train the model and the other to test.  If the effectiveness goes away during cross-validation the model is likely overfit. The protection isn't perfect because if you tweak the scheme based on the results you miss overfitting the "meta"-parameters..   This is effectively the test I applied above:  You fit a model (a set of gap differences, block timstamps, inter-arrival times) and I excluded a single hit which you appear to have used to set your parameters from the testing set and found your model matched _no_ blocks at all.

Quote
Come on....what is not allowed is to pick an artificial or stupid complicate criteria. To look at nounce dispersion is natural.
What shouldn't be allowed is bad statistical reasoning, which can happen no matter how "natural" your model is, what primarily matters is how many degrees of freedom it has.  What you are looking at there is _not_ dispersion; I tried to give you the benefit of the doubt and assume you meant one of the standard metrics for dispersion (_range_) and you insisted on a complex polytope shape constraint; with probably something on the order of 34 bits of parameter space (three percentages to 0.1 precision without permutation), plus some more model freedom in that its looking at differences instead of absolute values.

If I instead use the standard deviation (another common dispersion metric) we that 0.1875% of 4-block sets (given uniform nonces) would have a standard deviation as low as your selected one; so again, something we'd see every few days.


Quote
or more precise description how miners work.
There is huge amounts of information; everything is open source except for the RTL and mask images of common hardware; though most follow not too far from prior FPGA designs in their overall structure.


Quote
Let me ask you something simple: Do you think these four blocks were mined by the same miner?
It appears unlikely, all four are miners that appear to constantly reuse a static address for their income, but they're each different.  Walletexplorer does an overly aggressive 'taint' analysis and links many addresses to each, but they're disjoint.  The lowest number appears to be antpool, the next appears to be AnxPro.com, the next is likely Polmine.pl (it frequently pays addresses connected with polmine.pl).

Quote
We can compute/estimate the probability of all the other "pieces of evidence" that are independent events with small probabilities. What is astonishing is the coincidence of all these facts.
There is nothing astonishing that when you pick criteria to fit the data, you find that it fits the data. There is also nothing astonishing that when a model created this way seems unlikely on a uniform basis that it will fail to generalize to anything but the data you fit it on.

Quote
I believe they prove that all 4 blocks were mined by the same miner (then these facts would not be independent and it makes more sense that they occur simultaneously).

Great; so you'll accept concrete proof that these blocks were mined by independent parties as a definitive proof that you were incorrect?

Quote
I have retracted and erased all claims. I wait for your gentleman's word of removing your negative rating.
(also, I understood that the forum "Trust system" is for trading (since the first thing you see is "Trade with extreme caution!"). I doesn't seem to be designed to be used for academic discrepancy.
I'm having a hard time figuring out what you mean by retracted and erased;  your posts in this thread continue to claim  "It is clear that someone found a trick for fast mining. I kind of happen to know what might be.", "It is premining at some extend. Won't disclose more for the moment." and so on.  You continue to hold that you have "secret" knowledge which you will not disclose so that it can be discussed on its merits or lack thereof-- this entire thread is a great big advertisement for this claim and seems to have served you little other purpose; which the experienced professionals on this forum find to be non-credible and not supported by the evidence you've presented.

Quote
I haven't traded anything with you, and everybody with whom I have traded are 100% satisfied. I would appreciate that you don't use your trustworthy position in order to discredit me for disagreeing academically with you.
Moreover your comments "I suspect Valiron is either trying to scam someone out of paying for his "knowledge" or that he is attempting to manipulate the market price of Bitcoin; " are diffamatory and without ground. Also it is
ridiculous to pretend that I can manipulate the market by discussing anomalies in the block validations! I believe that the bitcoin market is a bit more robust...).

As I explained privately, your behavior so far is indistinguishable from a person who is willfully and fraudulently claiming to know of non-public mining optimizations in the hope of selling them to some greedy sucker who is unable to assess their merit except on the your misleading qualitative claim that the blocks look 'weird'.  Protecting the forum's participants from being deceived themselves or from suffering the traffic from a flood of hopeful scammers demands that we call it out when we see the potential of it-- I say this not as the subfourm moderator or a developer of the Bitcoin system, but just as a community member... There is plenty of room for doubt; I'd say it's even more likely that you're just confused by the statistics of it, but the benefit of doubt can't be given freely, since those looking to exploit people will just sail through that opening.  I offered you a simple mechanism which you can use to distinguish yourself-- share your complete theory with (ideally) the thread or privately with a respected member of the community; in doing so you gain the ability to refine it against the forge of experience, discover what (if any) parts are already known; and, in the possibility that there is something to be concerned about, gain the ability to protect Bitcoin from any potential harm if required.  (I think you greatly underestimate the potential harm of privately held substantial mining optimizations-- beyond some threshold they would result in the complete centralization of the bitcoin system)

The forum trust is a metric of trust; I explained in the rating where the distrust comes from. Your claims here are not credible to any of the subject matter experts and as far as we can tell they're based on erroneous statistical reasoning, you're shielding your theories from criticism by hiding behind secrecy, forcing a debate around statistical minutia and how "weird" the blocks feel qualitatively, rather than the merits of your idea. Only a few weeks ago you were asking questions (using asics to hash files) that showed a profound lack of research and understanding of Bitcoin mining.  I want to be sure someone considering trading with you over these ideas is aware that other community members do not currently consider your claims credible or likely to be well founded. Anyone is free to read the rating and come to their own conclusions.

I'm sorry that you find it heavy-handed.  I would prefer if there were a way to make ratings which were more targeted or conditional. In consideration of this, I'll go and reduce it to neutral to avoid triggering the red flag on you. I am not seeking to cause you distress.
staff
Activity: 4242
Merit: 8672
If I understand properly Sergio Demian states the weakness of the current mining algorithm and proposes a partial solution modifying the block headers. So, since this information is in a public blog, this weakness, or people claiming it, is also public knowledge, thus there is no reason any one gets mad when someone mentions it. Right?
I mentioned it non-specifically in my first response in this thread; (actually some things even more powerful than that). The objection was always the claim of "secret" knowledge supposedly supported by these very non-specifically identified blocks. Sergio actually described the optimizations he was talking about; as have a great many people. Not said they were keeping them secret and then "oh yea that" when other people play guessing games with other already known optimizations (which there have been easily a dozen discussed over the years);  especially not using alarming words like "premining" (which is a term used for a oft-perceived dishonest move in pump-and-dump altcoins).
hero member
Activity: 524
Merit: 500
If I understand properly Sergio Demian states the weakness of the current mining algorithm and proposes a partial solution modifying the block headers. So, since this information is in a public blog, this weakness, or people claiming it, is also public knowledge, thus there is no reason any one gets mad when someone mentions it. Right?
I was unable to reconstruct suspected vulnerability from proposed defense. The discussion in the linked thread was mostly about compatibility, not about cryptography. Also, new Sergio's idea about approximation of double SHA256 is not discussed here, as well as his guess about Satoshi using Gray code optimization for early mining. Strange.
sr. member
Activity: 311
Merit: 250
sr. member
Activity: 311
Merit: 250
A related question is: Are the algorithms in different ASICs in the public domain or are they proprietary? And then if they are not, I ask you guys asking for transparency if you didn't ask to the ASIC manufacturers for transparency. Did you? Where you as aggressive? Did the moderators label them as scammers for not revealing the inside proprietary algorithms? Or on the contrary were they allowed to advertise their products in the forum?
The ASICs don't have algorithms per se to hash SHA256d. From what I understand, the ASIC chips are just a bunch of physical circuits that perform a SHA256 hash. These are mostly not public domain.
I also thought that the nonces were determined by the software from the computer that the miner is connected to e.g. cgminer or bfgminer. These software are open source.

The algorithm to do the SHA256d could be in software but it does make sense to put the entire algorithm on the chip to have dedicated bitcoin mining hardware.

Is it the case?
sr. member
Activity: 311
Merit: 250
I also don't understand so much aggressivity. It is disturbing to say the least.
Huh, you found nothing special in blockchain (yes, I could be wrong here) but the reaction to your post is very, very interesting.
For reference:
Redesign of bitcoin block header
Potentially faster method for mining on the CPU

EDIT:
Do we try to hide something disturbing?
Only valiron does. He claims there's something wrong with the mining process but won't explain what. Nobody else is trying to hide it, whatever "it" is.
Did Sergio_Demian_Lerner published details about found attacks?
Quote
Today is the third time I find an attack to the way Bitcoin uses SHA-256 to perform mining. Two of the attacks belong to a new family of attacks that involve terribly technical details about the inner workings of SHA-256. These are attacks that may impact on Bitcoin probably not before 5 years, and they could even never have a real impact on Bitcoin. I will talk openly about them when I can really tell if they could affect Bitcoin.


Thank you for the links. The second link hints at something but it is not implementable as described (this is well known to any cryptographer and I am not hinting at any secret knowledge. If workable as described it could be used to find collisions of the hashing algorithm).

If I understand properly Sergio Demian states the weakness of the current mining algorithm and proposes a partial solution modifying the block headers. So, since this information is in a public blog, this weakness, or people claiming it, is also public knowledge, thus there is no reason any one gets mad when someone mentions it. Right?
hero member
Activity: 658
Merit: 501
I've already wasted an incredible amount of time responding to you

Clearly you did it but I don't believe it was in vain, the discussion have very useful information and many of us are learning from this. Furthermore, your explanations were absolutely necessary.  

Agreed, there were a few new things I learned from gmaxwell's posts. He is a gentleman and scholar. What is amazing about Bitcoin is that there is so much information to absorb and so many nuances and developments that you really should approach it with a bit of humility. I learn something new almost everyday.
full member
Activity: 164
Merit: 126
Amazing times are coming
I've already wasted an incredible amount of time responding to you

Clearly you did it but I don't believe it was in vain, the discussion have very useful information and many of us are learning from this. Furthermore, your explanations were absolutely necessary. 
staff
Activity: 4242
Merit: 8672
A node of mine received these blocks at the following times
As an aside-- I suspect your time is somewhat off, since there is basically a constant offset of a couple minutes between your and my numbers. (My clock is timed off a local atomic clock which is wsynced to UTC with a GPS timing receiver; it agrees with a remote stratum 1 NTP server by better than 1ms).
Quote
meh.
Indeed.
newbie
Activity: 54
Merit: 0
A node of mine received these blocks at the following times

2015-05-02 12:29:45 height=354640

2015-05-02 13:06:01 height=354641
2015-05-02 13:09:46 height=354642
2015-05-02 13:12:59 height=354643
2015-05-02 13:17:45 height=354644

meh.
staff
Activity: 4242
Merit: 8672
For me this is unusual. Maybe not for you, maybe not for gmaxwell that claims from his knowledge that the odd  distribution of nounces can be due to mining hardware. I am willing to believe that but I am waiting for the links to algorithms in the mining hardware that produce this type of bias in the nounces. If these links do not exist it means that these facts are not in the public knowledge.

I explained specifically how and why miners select nonces non-uniformly-- I would ask if you missed the post, but you responded to it; https://bitcointalksearch.org/topic/m.11266349.

You gave no indication that you were "waiting" on anything-- you even responded "This makes sense"--, if I didn't see this post would you just go ahead and declare default as you did earlier in this thread to jl2012?  I've already wasted an incredible amount of time responding to you, but I won't make claims and fail to back them up if someone finds them implausible.

However, you're not going to get a compact "algorithm" because there isn't a compact description for it, as the behavior is a product of the physical geometry of the part, random decisions decisions, and even the failed engines on particular chips; ranges are also truncated for load distribution across chips, and to control latency.  The behavior is well known, documented in various data sheets (see, for example register 0x7f), and even sometimes used for device autodetection by some pools and mining software (Even for just mode detection, E.g. the icarus driver sends a dummy work task to the miner which was chosen to have 4 different solutions, and figures out what mode the chip is in based on which solution it returns). Historically, before extranonce rolling existed, nonce restrictions were used to reduce work on pools. Of course, it's not the _most_ commonly discussed thing; because its fundamentally uninteresting-- all nonces are equally good so if some device only uses some subset of them the response of an engineer working on software for them is just "okay, thanks for telling me.", they write it into their implementation, and they move on with life; they don't go and author a press release.

And yes, reducing the range it results in some additional extranonce work, but extranonce work is trivial already-- a single cheap micocontroller can do the extranonce work for many TH/s of mining.


Quote
And the nounce is just one of the facts that make this sequence of validations unusual.
And what are your other _exact_ criteria? Please lay out the exact test you suggest which shows those blocks to be improbable in a single post in a list so there is no confusion... I've not seeing _anything_ exceptional about these blocks, excepting the fallacious post-hoc selection of criteria to exactly fit them around nonces.
legendary
Activity: 1820
Merit: 1001
Just pot luck with them blocks being mined had it myself when mining other tpyes of crypto mining shead loads blocks then a drout of mining and no blocks and then a burst of blocks. often not is just pure luck to get something like that happen. Having a lot of hash power does help too
hero member
Activity: 672
Merit: 508
LOTEO
A related question is: Are the algorithms in different ASICs in the public domain or are they proprietary? And then if they are not, I ask you guys asking for transparency if you didn't ask to the ASIC manufacturers for transparency. Did you? Where you as aggressive? Did the moderators label them as scammers for not revealing the inside proprietary algorithms? Or on the contrary were they allowed to advertise their products in the forum?
The ASICs don't have algorithms per se to hash SHA256d. From what I understand, the ASIC chips are just a bunch of physical circuits that perform a SHA256 hash. These are mostly not public domain.
I also thought that the nonces were determined by the software from the computer that the miner is connected to e.g. cgminer or bfgminer. These software are open source.

The algorithm to do the SHA256d could be in software but it does make sense to put the entire algorithm on the chip to have dedicated bitcoin mining hardware.
sr. member
Activity: 268
Merit: 258
A related question is: Are the algorithms in different ASICs in the public domain or are they proprietary? And then if they are not, I ask you guys asking for transparency if you didn't ask to the ASIC manufacturers for transparency. Did you? Where you as aggressive? Did the moderators label them as scammers for not revealing the inside proprietary algorithms? Or on the contrary were they allowed to advertise their products in the forum?
The ASICs don't have algorithms per se to hash SHA256d. From what I understand, the ASIC chips are just a bunch of physical circuits that perform a SHA256 hash. These are mostly not public domain.
I also thought that the nonces were determined by the software from the computer that the miner is connected to e.g. cgminer or bfgminer. These software are open source.
hero member
Activity: 524
Merit: 500
I also don't understand so much aggressivity. It is disturbing to say the least.
Huh, you found nothing special in blockchain (yes, I could be wrong here) but the reaction to your post is very, very interesting.
For reference:
Redesign of bitcoin block header
Potentially faster method for mining on the CPU

EDIT:
Do we try to hide something disturbing?
Only valiron does. He claims there's something wrong with the mining process but won't explain what. Nobody else is trying to hide it, whatever "it" is.
Did Sergio_Demian_Lerner published details about found attacks?
Quote
Today is the third time I find an attack to the way Bitcoin uses SHA-256 to perform mining. Two of the attacks belong to a new family of attacks that involve terribly technical details about the inner workings of SHA-256. These are attacks that may impact on Bitcoin probably not before 5 years, and they could even never have a real impact on Bitcoin. I will talk openly about them when I can really tell if they could affect Bitcoin.
sr. member
Activity: 311
Merit: 250
Why are people so ferociously attacked every time they say there are some unusual patterns in mining? Do we try to hide something disturbing? In my view valiron has a valid point. This deserves getting a closer look.

I also don't understand so much aggressivity. It is disturbing to say the least.

If a charlatan comes to me and tells me that he has a 2 page proof of the Riemann Hypothesis, this doesn't really disturbs me much... I guess I will thank him and let him know that I will read his extraordinary proof...

For the record:

My conclusion is that the nounces produced by this miner are likely not independent and the mining procedure is not the usual one and it uses previous block computations or doesn't uses much the nounce variable.

But this is just one piece of evidence.

The second one, about the block size, also points to the fact that it is the same miner who mined the blocks. 731 kB blocks are quite common as noted earlier by someone else, but it is not
very likely either to find them consecutively. Moreover I bet that they cluster more often than expected and this can be checked running statistics on the blockchain.

The third piece of evidence is how close in time are these blocks. THe probability is not alarmingly small and can be computed by the Poisson distribution that follow times between blocks.

The fourth piece of evidence is the non-chronological timestamps that suggest that the timestap maleability is also used as nounce (this fact was already noted for blocks with only one transaction).

The fifth piece of evidence is that the first block is mined by AntPool and the next 3 by anonymous. It is not so common to have consecutive anonymous blocks,
This indicates that the miner is trying to hide that he is the same one mining.
 
sr. member
Activity: 311
Merit: 250
Why are people so ferociously attacked every time they say there are some unusual patterns in mining?
They aren't. They're only ferociously attacked when they continue to say so after it is repeatedly explained to them why such patterns are, in fact, not unusual in the slightest.

Until further notice I believe it has been shown that the pattern detected can only appear every several years. For me this is unusual. Maybe not for you, maybe not for gmaxwell that claims from his knowledge that the odd  distribution of nounces can be due to mining hardware. I am willing to believe that but I am waiting for the links to algorithms in the mining hardware that produce this type of bias in the nounces. If these links do not exist it means that these facts are not in the public knowledge. And the nounce is just one of the facts that make this sequence of validations unusual.  

A related question is: Are the algorithms in different ASICs in the public domain or are they proprietary? And then if they are not, I ask you guys asking for transparency if you didn't ask to the ASIC manufacturers for transparency. Did you? Where you as aggressive? Did the moderators label them as scammers for not revealing the inside proprietary algorithms? Or on the contrary were they allowed to advertise their products in the forum?

Pages:
Jump to: