Topic: 1GH/s, 20w, $700 (was $500) — ButterflyLabs 3rd party testing for Dummies - page 2. (Read 3714 times)

This is the key here for all the sheeps out there.

BFL even has a nerve to tell us "mind your own business"

So you think that people under a pseudonym calling each other names without any actual evidence are being courageous?
We must have a very different notion of courage.

To answer what I think is your question, I am not at all sure what to think about Inaba.
As I said earlier, in one hand, if BFL turns out to be scam, Inaba would be the ideal candidate for the role of the con artist, but he could also be a naive victim however unlikely that would be.
In the other hand, it is also very possible that BFL is legit in spite of their questionnable marketing policy, in which case Inaba would really be the helpful guy he appears to be.
Short of evidence, I am not going to jump to any hastly conclusion, nor to call Inaba and BFL names, neither am I going to believe BFL and Inaba without questionning.

That being said, there is definitely something fishy with this 30 Nov testing so I'd like to understand what happened.
In any case, until evidence is clearly made in favor or agains BFL's legitimacy, I'd like to advise people to apply healthy skepticism in everything they read on this forum, including testimony of seemingly trustable but isolated people.

You have to expect that tracking what has been said over 87+ pages and two threads is going to take some space.
This is the price to pay to reproduce the entire context, and have a discussion backed with facts and not blurry gut feelings.
Now, if you do not have any doubt about the way the testing was done and reported, no one forces you to read it all..

I am not being critical. I am being inquisitive.
I decided to lay down all the evidence and start from there after Makomk asked exactly what I had on my mind on 9 Dec, and Inaba somehow dodged the question that IMO was legitimate (cf quotation below)


If only BFL could stop taking orders for products that they are not ready to ship yet, and resume ordering once they have enough stock to ship within the charge back delay of paypal.
That would eliminate once and for all the growing doubt in the community that they are just trying to buy some time.

The title is a copy / paste of previous thread title.
The question of the original price being totally irrelevant to the topic, that shouldn't be a big issue.

There are countless occurences of you and BFL referring to the test of 25 and then 30 Nov as "proper testing", "live test" "prerelease demo" and whatnot.
You clearly mentionned after the fact that 25 Nov wasn't a live test.
I have found no evidence of you telling anywhere that 30 Nov wasn't a live demo (appart from your quotation of yesteday, of course).
Anyway, whether the test of 30 Nov was to be called a "live test", a "prerelease demo" or "proper testing" or a "small demo" isn't the real problem.

The problem is that the test of 30 Nov was arguably meant to prove that the product of BFL was legit, and that no countrary statement has been made to moderate the original announcement.


BFL calls the test of 30 Nov the "pre-release demo" so let's keep this term to stop playing with the words.
The "pre-release demo" had for ambition to establish once and for all the their product is legit.
Following the announcement, you didn't comment to moderate the announcement of BFL or clarify the location or circumstances.


After the test, BFL confirm with triumph that the test was a success in establishing that their product is legit.
Again, you didn't comment to moderate the announcement of BFL.
Later posts show that you actually vouch for their conclusion.

Here you are pretty affirmative about the fact the test demonstrates that their product is legit.

Again, this is pretty affirmative. You even go to the length of doing comparative marketing on behalf of BFL.

To summarize :

You knew that our requirement to establish that BFL's product is legit was to follow a certain number of constraints, including limitations on the location, hardware, network access and software build.
You acknowledged, and made a rigourous protocol that was covering all these requirements and commited to apply it for all the tests.
So you knew without any ambiguity that there was no way we could ever reach the conclusion "BFL's product is legit" without abiding to the rules that we had agreed on.

On 30th Nov, you and BFL announced a test that had all the appearance of a formal test, and aimed at no less than prove that BFL's product is legit.
But the test that you performed was not valid as it did not follow in any way the protocol that was agreed on.
At that point you knew that there was a flaw in the way the test had been executed, and that no evidence had been made.
Nonetheless on 1st Dec, you and BFL announced that the test was successful at establishing that BFL product is legit.

The least that can be said while remaining objective is that the way you reported about the testing so far has been extremely misleading.
I won't make any further assumptions here simply because I think there is no ground for making additional assumptions.

  I did.......

https://bitcointalksearch.org/topic/m.639056

Tell the other OP to fix his title too if you really care. You're being a nitpicking?

As far as the laptop, i always assume it was BFL's, as its obvious to me from the pics (i doubt Inaba would have left the username and computer name as default. BFL, on the other hand, would do a fresh install of ubuntu to avoid exposing any personal identity)

I'm surprised someone thought it was Inaba's laptop from the pics.

Now, isnt the test scheduled for this weekend?

First I need to criticize you a bit, Freequant. Assuming you are being sincere in your questioning and not just lacking the balls to say "I think you are full of shit". Then is the title including, "testing for Dummies" really condusive to getting ones questions answered? And why the need for so much text? It just seems way overdone for just being thorough.

Second though, I am also curious about why the changes in 'plans'. But not from the perspective of any kind of assumption that it implies anything.

Was it BFL or Inaba that changed the venue for meeting one?
Meeting two?
Why?
I too naturally expected that it was Inaba's laptop that I was looking at in the pictures of test 1. And from some of the verbage that mentioned the unit being plugged into the laptop. I am not certain, but don't recall the indentifiers of 'mine' or 'theirs' being present in the text.

  There were more questions in my mind but those and the ones I pressent here seemed meaningless to me at the time. And, I would have assumed anyone else following along would not be so concerned since the final result was that there was still not a unit that was in a ready state for a truely proper test. I do agree testing and reporting should be as complete as is possible. But I also appreciate that as far as we know Inaba is doing this on his own time. For no agreed compensation. And has said nothing to make any intelligent person want to rush out and pay for something they would not do otherwise. I mean, if the intention of being so critical is to help 'protect' the community that's great. But at the same time it also basicly is saying to everyone that they are stupid and not capable of making an informed decision on their own.

  Well, cheers to you both for taking action on behalf of the community whatever the outcome may be.

 
P.S. Fix the freaking "1GH/s, 20w, $700 (was $500)". It comes off as being very purposely biased to everyone who knows you are smarter than that...
The unit was never priced at $500. It was $599. There were pre-orders with a set date at $499. Anyone who ordered by that date has not been asked to pay more money.

Both the test of 25th Nov and 30th Nov were meant to take place in your datacenter according to yourself or BFL.
Let me put things back in context.


Without question, the context here is the first meeting with BFL planned on 25th at 18:00 CST.
You are planning to do the live test if enough time.


In the next post, you mention that the test will take place in your data center.
There is no ambiguity of context : this is the next post, right after previous one where you were speeking about the meeting with BFL.
You even invited people to join, so assuming some people had joined, they would have been waiting in front of your datacenter.

At this point :
- you mentionned you would try to do the live test
- you mentionned it will take place in the data center.

So everyone who followed so far would naturally expect :
- that you will bring your laptop
- that the test will indeed take place in the data center

Luckily, the next day someon mentions about the data center.

And you clarify : the test did not take place in the data center after all (based on the photo, it looks like a restaurant, or someone's home).
So on 25 Nov, you wanted to do the live test in the data center with your laptop, but none of that happened.
You mentionned that the live test was postponed, but you didn't mention about the computer and the location until people ask more explicitely.
What happened? Who changed the plan and why? Why not be more clear about the change of place and the circumstances?

So anyway, that was the first test, and the first confusion.

Now let's look the announcement of the second "pre-release" test.


Here, it is the BFL official annoucement for the test of 30th Nov.
By BFL's own words, it is unambiguously stated that the test will take place in the data center.
By BFL's own words, it is a pre-release (so almost final) test with a live unit, so that should clearly qualify as the previously mentionned "live test".
By BLF's own words, it is going to establish the proof that the their product is real, so we are talking about a test with a formal experimental protocol here.

Following BFL's announcement, you did not make any comment to retarget before or after the test.

So everyone who followed so far would naturally expect :
- that you will bring your laptop
- that you will apply the experimental protocol we agreed one.
- that this was the so-called "live test" you were talking about all along.
- that the test will indeed take place in the data center

Now, you just clarified : this was not in the data center (in spice of BFL's announcement)
You clarified a few days ago that you in fact did not use your laptop, that the network was in fact not really segregated, and that this was in fact not the live test after all.

So on 30 Nov, as on 25 Nov before that, you wanted to do the live test in the data center with your laptop but, again, none of that happened.
Same pattern as last time, you mentionned vaguely that you would do more testing (but not that this wasn't the live test), and omitted to mention about the computer and the location until people ask more explicitely.
What happened again? Who changed again the plan and why? Again why not be more clear about the change of place and the circumstances?

I will answer to your points, one per post, to keep things clearly separated.

It was indeed extremely ambiguous because, we (as the community) could not care less if you respected your agreement with BFL, and could not care more if you did not respect your agreement with us.
What the "the ground rules that were set forth PRIOR to the test" refer to is a question of context.
If you were answering to BFL, it would make sense that the above would refer to your agreement with them.
And if you were answering to the community (which was the case here), it was only logical that the ground rules would refer to our agreement on the experimental conditions.

There's a few things that are inaccurate here.  First is the "Ground rules" statement referred to my inability to give hard numbers, not to test protocol.  This is an important point and at the time I was aware of no one who was confused by that statement or I would have clarified it.

Secondly, the building we have been in is my office building, not the data center, thus why there is wifi available.  Also why I know what the wifi available in that building is and what it congestion rate is.  At no time have we (BFL and I) been to the data center and I don't believe I have stated we have done any testing inside the data center at this point.

The "small demo" was NOT a live demo, and I stated as much.  It was a demo of me taking (by hand) a dataset from my pool and pasting it into the mining testbed to return the nonces I reported and offered to independent verification.

I thought I had mentioned it, but perhaps not, I have yet to receive software source that will compile from BFL.  The one set of software I received and reviewed would not compile and I am suppose to receive a new codeset this weekend.  However, given the nature of the codeset and it's author, I'm inclined to believe that is more likely legit than not, but that does not mean I will not be reviewing it regardless.

I noticed that many people here are totally lost about what has be told and done, told but not done, done but not told and neither told nor done regarding the third party testing of ButterflyLab BitForce Single product.
With all the stuff scattered over two lengthy threads and weeks of endless debate, it is geting so confusing that even Inaba seems to be lost about what he has said and done.

This is why I have written for you the little pocket guide of "ButterflyLabs 3rd party testing for Dummies".
All the quotations are taken from the "is it a scam" and "is it for real" threads (courtesy of BFL and Inaba).
The emphasis in bold and comments are from me.

I have tried to remain objective, but I am of course totally biaised  , so do not base your opinion solely on my comments.
Please base your opinion on the facts, quotations, and linked messages.

This is a recommanded reading for :
- People waiting for their pre-order to be delivered
- Inaba, to help him sort out what he has said already
- Everyone else who wonders what the heck is going on with that ButterflyLabs testing nonsense.

----------------------


Friday 25 November, no test have been done yet.
Inaba starts defining the experimental protocol and commits to use his own hardware and compile himself the binaries.

Commitments about the experimentation protocol:
- miner run on own computer
- self built software


Inaba announces the tentative date for a first test : Saturday 26 Nov or Sunday 27 Nov.
The basic demo is done, but was more like simulation testing. Let's call that preliminary test "the basic demo".
The "proper testing" will be on Saturday 26 Nov or Sunday 27 Nov.

Status of the experiment:
- "basic demo" test done
- "proper testing" test pending => planned for 26 or 27 Nov


The test (as called earlier "proper testing") will be on Sunday 27 Nov.

Status of the experiment:
- "basic demo" test done
- "proper testing" test pending => planned for 27 Nov


Commitment to do the necessary to segregate the computer used for the test from the Internet.
Full awareness that any possibility of external communication would make the test invalid.

Commitments about the experimentation protocol:
- miner run on own computer
- self built software
- pool run on own server
- segregated network environment


Technical problem, actual (proper testing) test delayed.
Test rescheduled for in a couple days.
This (proper testing) test is called "live demo" here. This is an important term. Let's remember it.
So the "live demo" will take place in a couple days.

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing") test pending => posponed to "in a couple days or so"


The "live demo" will take place on 30 Nov afternoon.


This is a BFL anouncement.
The test in question (so the "live demo") will take place 30 Nov afternoon CST.
This test involved running a "live unit" (in case it wasn't clear enough this is a live demo) in Inaba's data center.
According to BFL's own words, the test aims to no less than proving that the product is real.

The test in question (the "live demo") will take place 30 Nov afternoon at 16:00 CST.
No correction to what BFL said, so it is implicitely assumed that we are talking about the "live demo" that will prove that the product is real.


The test in question (the "live demo") took place on 30 Nov afternoon CST.
Positive feed back : the hardware does what it was designed to do.
At this point, no mention if the experimental protocol that was agreed on as been followed.

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing") test done


Inaba confirms after the fact that the purpose of the earlier "live demo", also called "pre-release demo", was to confirm if the product was real of fake.
Strong suggestion that the test was successfull in showing that the product is in fact real.
At this point, no mention if the experimental protocol that was agreed on as been followed, and therefore whether the test was reliable.

Commitments about the experimentation protocol:
- miner run on own computerassumed
- self built software assumed
- pool run on own server assumed
- segregated network environment assumed

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done


Inaba confirms after the fact that the test (the "live demo") followed the experimental protocol that was agreed.
He also mentions that he reported exactly, and without bias.
This effectively clears any doubt about the reliability of the test, the applicablity of the agreed commitment, and therefore the correctness of the conclusions given earlier.

Commitments about the experimentation protocol:
- miner run on own computerapplicable
- self built software applicable
- pool run on own serverapplicable
- segregated network environmentapplicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done


One week later, the original "scam thread" has been closed and a new "is it for real" thread has been created.
On 9 Dec, Bulanula uncovers that the laptop used for the "basic demo" (which photo have been published) is the same as the one used by BFL for their screen shots.

Inaba confirms that the laptop that was used for the "basic demo" and the "live demo" is BFL's own laptop.
His own laptop was actually never seen or accessed to by BFL.

Earlier commitments about the experimentation protocol:
- miner run on own computerapplicable not applicable
- self built software applicable
- pool run on own serverapplicable
- segregated network environmentapplicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done

[NR]
It is doubtful that the drivers / miner built by Inaba for his own computer / distribution would work straight away on BFL's own laptop under Ubuntu.
It is also rather unlikely that Inaba thought of transferring the drivers he built to some portable storage, but didn't think of bringing his own laptop as agreed.
Even assuming that Inaba really gave the binaries he built himself to BFL, the laptop being BFL's, the path could have been set so as to run their own version of the binary.
Under these conditions, the condition "self built software" cannot be ascertained.


Earlier commitments about the experimentation protocol:
- miner run on own computerapplicable not applicable
- self built softwareapplicable doubtful
- pool run on own serverapplicable
- segregated network environmentapplicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done


Makomk question regarding the objectivity and reliability of the live test.
Inaba clarifies the situation as he understands / remember it : the live test did not take place.
[NR] Note for people who are not familiar with block chain reorganizations : the live test that took place on 30 Nov at 16:00 CST, did in fact not take place (i.e. the live test has been orphaned).

Earlier commitments about the experimentation protocol:
- miner run on own computerapplicable not applicable
- self built softwareapplicable doubtful
- pool run on own serverapplicable
- segregated network environmentapplicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done not done


Later in the same post, after referring to quotations of the "basic demo" test and highlighting that this was not a live test, Inaba states that there hasn't been a second live test (which also means that there was a first live test).
[NR] Note for people who, like Inaba, would be confused at this point : the live test that took place on 30 Nov at 16:00 CST, and that subsequently did not take place, actually took place but it was the first (this is important).

Earlier commitments about the experimentation protocol:
- miner run on own computerapplicable not applicable
- self built softwareapplicable doubtful
- pool run on own serverapplicable
- segregated network environment applicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done not done kind of done
- "second live demo" test pending


Inaba recognizes that after all, it is possible that BFL's laptop was able to communicate outside (albeit with a high latency) using 3G or WiFi routers that he knows of in the data center.
Under these conditions, the "segregated network environment" doesn't apply.
The fact he knows of the WiFi router in the building suggests that he could scan the WiFi, which suggests in turn that the data center building isn't isolated from WiFi signal.

Earlier commitments about the experimentation protocol:
- miner run on own computerapplicable not applicable
- self built softwareapplicable doubtful
- pool run on own serverapplicable
- segregated network environmentapplicable not applicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done not done kind of done
- "second live demo" test pending


Later in the same post, in fact the datacenter is a giant faraday cage that would have prevented WiFi and 3G to get through.
[NR] Note : the faraday cage there prevented BFL to connect from inside the building to the routers that Inaba detected in spite of the faraday cage by scanning from inside the building.

Earlier commitments about the experimentation protocol:
- miner run on own computerapplicable not applicable
- self built softwareapplicable doubtful
- pool run on own serverapplicable
- segregated network environmentapplicable not applicable kind of applicable

Status of the experiment:
- "basic demo" test done
- "live demo" (aka "proper testing" aka "pre-release demo") test done not done kind of done
- "second live demo" test pending

----------------------

Conclusion :

Meeeehh