Pages:
Author

Topic: Modular Python Bitcoin Miner - Official Thread - page 10. (Read 74196 times)

hero member
Activity: 784
Merit: 500
Somehow i'm unable to get it running.... Ztex told me how to revert my "serial number problem" and every board has back its unique one. When i start mpbm (the windows packaged) on my VM the hot plug worker recognizes the 5 Boards. But they are in some kind of constant reconnection loop. The don't start mining since they are kicked out of the work immediately when the connect to it.

even one board doesn't function with it (and it did before)?

Some Screens (I'm on an iPad so I'm only able to make screenshots):



It could be som VM related problems .... so its not neccecary that this works:)

I'm only curious whats the cause of this is?


Ill hook them to my gnu miner if i have some time tomorrow.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I've had my Icarus + Ztex combo up and running with MPBM for 1D 12h now.

I'd like to understand some of the numbers.

For instance, why are cancelled jobs on the Ztex boards 11.50%, 14.58% and 13.61% vs. only between 1.11% and 1.16% on my Icarus boards?

A Ztex board takes about twice as long to process a job than an Icarus. This means that the probability that a job is hit by a long poll is twice as high.
The actual probability for a job to be canceled on an Icarus should be around 1.3% on average, twice that on Ztex. So your Icarus numbers look perfectly fine to me.
The reason that your Ztex boards show higher numbers is a bit more complicated: MPBM fetches new work as soon as the number of jobs in the queue that expire in more than 10 seconds goes below some threshold. However, if jobs are received in bursts due to X-Roll-NTime, it's rather likely that all jobs in the queue have the same expiry time, and that this is somewhere between 10 (fetcher threshold) and 15 (Ztex board job processing time) seconds in the future. Because of this, there are no jobs with a remaining life of >15 seconds in the queue, and the Ztex board is forced to work on a job that it can't possibly finish in time. As soon as the remaining jobs cross the 10 second boundary, MPBM will fetch new jobs, and as soon as the remaining life of the accepted job hits zero, the board will behave as if a long poll came in, cancel its current job and get a new one (with longer life). To account for network latencies, a safety margin of about 5 seconds is subtracted from the expiry time reported by the pool before inserting the jobs into the queue.
TL;DR: This is caused by MPBMs inner workings, and nothing to worry about.

The 'GHashes total' on the Icarus seems lower per FPGA (ie. Ztex 28290 vs. Icarus 25045) than on the Ztex, but then the jobs per hour seems higher (Ztex 240-250 Icarus 308-318).

This is partially caused by the effect I described above, and partially by the fact that MPBM pushes new work to the boards about 1 second before the board actually exhausts the noncerange of the previous job, because none of the boards that we have today support real queueing of jobs (which means we need to do this to prevent the board from idling briefly between jobs). This absolute time value doesn't depend on the hashrate, and thus offsets the number of jobs against the number of gigahashes.

Do any other numbers seem off in the screenshot? I realize that Slush has a lot of upload errors. The timeout is 5 now (default), would increasing that possibly solve that problem? 5 seems already very long to me.

5 Seconds should be plenty. If a pool fails to respond within this timeframe, I'd avoid that pool. I wouldn't increase that timeout, because that would decrease MPBM's aggressiveness to deliver a share, thereby possibly causing more stales, especially if the timeout was not caused by the pool software being overloaded, but rather by network packet loss (retrying is likely to be faster than waiting for the protocol layer to detect the packet loss, trigger retransmissions, and finally get a response). However it seems like stales are still within a reasonable margin on this instance, so I wouldn't care too much about those retries.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
So far the only thing that shows up in my log file (log level 300) with 'long poll' or 'longpoll' is those error messages. How can I actually confirm that long polling is working? Canceled Jobs = long polling works?

Canceled jobs do not necessarily mean that long polling actually works, because MPBM tries to detect new blocks by other means as well.

Code:
2012-03-26 23:23:20.042000 [200]: ABCPool long poll failed: Traceback (most recent call last):
2012-03-26 23:23:20.042000 [200]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\bcjsonrpc\bcjsonrpcworksource.py", line 181, in _longpollingworker
2012-03-26 23:23:20.042000 [200]:     response = conn.getresponse()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 1046, in getresponse
2012-03-26 23:23:20.042000 [200]:     response.begin()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 346, in begin
2012-03-26 23:23:20.042000 [200]:     version, status, reason = self._read_status()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 316, in _read_status
2012-03-26 23:23:20.042000 [200]:     raise BadStatusLine(line)
2012-03-26 23:23:20.042000 [200]: http.client.BadStatusLine: ''

I have two other pools (slush + eclipse) but as said nothing that confirms any long polling in my log file.

And long polling with ABCPool works with Ztex BTCMiner.

This message does not mean that long polling does not work at all. It just means that a long poll connection died without getting a response. MPBM will automatically reconnect it, so if you only get that message occasionally, this is probably just a sign of a slightly unstable pool, connection issues (bad router? have seen this before...) or just someone restarting something on the pool side.

I'd like to see a message in the logs when a long poll is received. Is that possible?

If you increase the loglevel to 500, you should occasionally see these messages:
Code:
: Got jobs from long poll response
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
There are two libusb implementations, both called the same thing but incompatible with each other.

This actually shouldn't matter for MPBM. PyUSB (which MPBM relies on) supports both libusb versions, and something called openusb, which I think is apple's own libusb equivalent. So you might possibly get away without any libusb at all.

Gotta run now... Antirack, I'll explain the stats to you later today.
brand new
Activity: 0
Merit: 250
Just a tip from the guy who managed to build the entire Ztex SDK from scratch on Mac OS X.

There are two libusb implementations, both called the same thing but incompatible with each other. With the Mac, which is Unix underneath but left mostly to the Mac hackers, no package management system is installed by default (for FOSS and CLI stuff, at least - the Apple Installer system is pretty sophisticated).

Most Mac hackers who bought OS X because it promised 'Linux power and Mac usability and Microsoft apps' (like me) use MacPorts, which is a BSD-type package management system for the CLI.

Of course, sudo port install libusb will get you the latest libusb. And the Ztex Java bindings are built on the previous legacy libusb.

Not sure what OS X itself comes packaged with because I'd started compiling my own before checking (OS X 'unix' libs often either lag version-wise or have dependencies that aren't necessary on non-Apple systems (Kerberos turns up a lot, for example)) - but if separate modules require both incompatible libusb implementations concurrently, then I'd consider statically linking the 'modules' to avoid confusion.

Mac OS X also has a habit of caching what it thinks it 'knows' each USB device is, and seems to periodically enumerate the bus on various signals. I wouldn't be surprised if the main issue behind this is iTunes and a daemon called usbmuxd, which handles the proprietary interface of iPods and iPhones over USB (and ramps up the current available to iPads, which can't charge well off 500mA).

However, get the right libraries and OS X is like any other Unix. I'm doing insane amounts of idiotic things with my Mac Pro right now, and the USB buses have hubs and high-bandwidth devices galore (I don't need 4 high-res webcams, microphones and sets of speakers on ONE workstation, but when they're integrated into your displays...) - yet my single Ztex FPGA, running Stefan's code with my mods to his Java USB bindings (which use the *legacy* libusb), is happily using the 64-bit jnilib and hashing away merrily. It hasn't broken - ever - I've had to bork the USB bus by sticking completely fecked mass storage devices in (cue 'uninterruptible wait' - which the Penguinboys get *just* as bad - knackered hardware is knackered hardware) to duff up the BTCMiner.

If you're running a farm though - give each board a different 'serial number' - it doesn't change the USB vendor / device ID and Stefan's code requires it. I've only got one so far, so can't thoroughly test the ramifications of ignoring this. It shows up nicely in the Mac System Profiler though Smiley


Make no mistake - when my 25 other boards arrive, I will certainly give this software a try, and if everyone on OS X is still having trouble then I'll work out a way round it. Hope this info is helpful and not already redundant... when I found out that there were two incompatible libusb implementations, I wanted to force the developers to program in x86 assembler for the rest of the year  Angry
hero member
Activity: 489
Merit: 500
Immersionist
So far the only thing that shows up in my log file (log level 300) with 'long poll' or 'longpoll' is those error messages. How can I actually confirm that long polling is working? Canceled Jobs = long polling works?

Code:
2012-03-26 23:23:20.042000 [200]: ABCPool long poll failed: Traceback (most recent call last):
2012-03-26 23:23:20.042000 [200]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\bcjsonrpc\bcjsonrpcworksource.py", line 181, in _longpollingworker
2012-03-26 23:23:20.042000 [200]:     response = conn.getresponse()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 1046, in getresponse
2012-03-26 23:23:20.042000 [200]:     response.begin()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 346, in begin
2012-03-26 23:23:20.042000 [200]:     version, status, reason = self._read_status()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 316, in _read_status
2012-03-26 23:23:20.042000 [200]:     raise BadStatusLine(line)
2012-03-26 23:23:20.042000 [200]: http.client.BadStatusLine: ''

I have two other pools (slush + eclipse) but as said nothing that confirms any long polling in my log file.

And long polling with ABCPool works with Ztex BTCMiner.

I'd like to see a message in the logs when a long poll is received. Is that possible?
hero member
Activity: 489
Merit: 500
Immersionist
I've had my Icarus + Ztex combo up and running with MPBM for 1D 12h now.

I'd like to understand some of the numbers.

For instance, why are cancelled jobs on the Ztex boards 11.50%, 14.58% and 13.61% vs. only between 1.11% and 1.16% on my Icarus boards?

The 'GHashes total' on the Icarus seems lower per FPGA (ie. Ztex 28290 vs. Icarus 25045) than on the Ztex, but then the jobs per hour seems higher (Ztex 240-250 Icarus 308-318).

Do any other numbers seem off in the screenshot? I realize that Slush has a lot of upload errors. The timeout is 5 now (default), would increasing that possibly solve that problem? 5 seems already very long to me.

hero member
Activity: 489
Merit: 500
Immersionist
Is this a bad board?

If your other two boards work, and switching around the cables, changing the socket on the USB hub, switching around power supply etc doesn't work, then that would be my guess.

I'd mail to [email protected] and see if they have any other ideas what the problem might be.

I haven't personally used the X6500 yet, but it may just be something stupid. I forgot to switch my Icarus boards ON since I was used to Ztex boards which don't have a on/off switch. You plug them in and they start mining (if the software is already running). This reminds me, I should test if Ztex hotplug really works with MPBM (edit: working very well).




sr. member
Activity: 308
Merit: 250
Is this a bad board?
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I've made some packages for windows users who haven't got Python installed, see the first post.
Can somebody who doesn't have the neccessary stuff installed on his system yet please try these out?

If I test them locally, I never know if they're really self-contained or if something just picked up a library that was present on my system anyway.
Just extract both zip files to the same directory and run mpbm.exe.



Interesting... I need to check why this is happening tomorrow.
Temporary workaround: Move that dll file into c:\windows\system32 (on 32bit windows) or c:\windows\syswow64 (on 64bit windows).
sr. member
Activity: 308
Merit: 250
I switched the cable with one of the good ones and its the same thing.  The other 2 work great.  Once it gets going then I get this.
X6500 A4014OEL: Discovering FPGA 0...
2012-03-27 22:33:25.213000: X6500 A4014OEL: Error while booting board: Traceback (most recent call last):
2012-03-27 22:33:25.213000:   File "worker\fpgamining\x6500.pyc", line 194, in main
2012-03-27 22:33:25.213000:   File "worker\fpgamining\util\fpga.pyc", line 83, in detect
2012-03-27 22:33:25.213000:   File "worker\fpgamining\util\jtag.pyc", line 88, in detect
2012-03-27 22:33:25.213000: NoDevicesDetected

X6500 A4014OEL: Reading FPGA temperatures failed: list index out of range
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
sr. member
Activity: 308
Merit: 250
I just recieved 3 x6500 boards and 2 of them work fine but one of them I keep getting

X6500 A4014OEH: Error while booting board: Traceback (most recent call last):
2012-03-26 16:29:52.619000:   File "worker\fpgamining\x6500.pyc", line 194, in main
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\fpga.pyc", line 83, in detect
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\jtag.pyc", line 90, in detect
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\jtag.pyc", line 331, in _readIdcodes
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\jtag.pyc", line 166, in read_dr
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\jtag.pyc", line 163, in shift_dr
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\jtag.pyc", line 170, in read_tdo
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\ft232r.pyc", line 203, in read_data
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\ft232r.pyc", line 163, in write
2012-03-26 16:29:52.619000:   File "worker\fpgamining\util\ft232r.pyc", line 410, in write
2012-03-26 16:29:52.619000:   File "usb\legacy.pyc", line 148, in bulkWrite
2012-03-26 16:29:52.619000:   File "usb\core.pyc", line 619, in write
2012-03-26 16:29:52.619000:   File "usb\backend\libusb01.pyc", line 474, in bulk_write
2012-03-26 16:29:52.619000:   File "usb\backend\libusb01.pyc", line 556, in __write
2012-03-26 16:29:52.619000:   File "usb\backend\libusb01.pyc", line 384, in _check
2012-03-26 16:29:52.619000: USBError: [Errno None] libusb0-dll:err [_usb_reap_async] timeout error
2012-03-26 16:29:52.619000:
2012-03-26 16:29:52.619000:

My other two work fine.
I am using Windows 7 64 bit.  Downloaded the software off of fpgamining.com.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
MPBM's support of fallback pools is the best. Any idea to make it support phatk2 video card mining?
Well, in theory you should be able to rip the relevant parts of code out of poclbm or some other python gpu miner, and implant it into MPBM as a worker module. I'd figure that this should probably be an out of process worker module like ztex/x6500, to prevent GIL congestion and have some isolation to prevent a buggy GPU driver from taking down the whole MPBM instance.
sr. member
Activity: 324
Merit: 260
MPBM's support of fallback pools is the best. Any idea to make it support phatk2 video card mining?
sr. member
Activity: 410
Merit: 252
Watercooling the world of mining
Hello everybody,

As promised i release some images and data of my FPGA family at its daily work.

The setup (please excuse the messy workbench)                                                                                                                                                                

     

[From left to right: ARM board "Gumstix" (the first i developed on), Icarus(rev0),120mmfan ,x6500(rev2),120mmfan,Pandaboard ES(revB2),Bitforce single(1.0)]

And the mining data thanks to TheSeven


hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I'm assuming a bitstream file is missing. (which is correct, i'm managing bitstreams myself)
Your assumption is correct. Just put the right bitstream there, or disable bitstream upload at all (which means you have to boot the boards manually).

Any clues as to disabling bitstream upload? (or how to configure anything at all?)

I see a "config" directory with a default.cfg which seems to be a binary format (what the?)

the old version had a config.py or similar which was nicely commented. I see no such file with this release, unless i'm missing something obvious.

There is no such thing any more. The new configuration store is a Python pickle, so that it is easily machine read/write/editable. Configuration is now done through user interface modules, currently only WebUI supports that: http://localhost:8832 (user: admin, password: mpbm)

I'm fairly sure i've read the whole thread, and of course first post still says documentation "coming soon...". perhaps a wiki is in order?

Thanks
-bitcowok

Yes, a wiki sounds like a plan. Maybe put it into http://wiki.btcfpga.com?
newbie
Activity: 48
Merit: 0
I'm assuming a bitstream file is missing. (which is correct, i'm managing bitstreams myself)
Your assumption is correct. Just put the right bitstream there, or disable bitstream upload at all (which means you have to boot the boards manually).

Any clues as to disabling bitstream upload? (or how to configure anything at all?)

I see a "config" directory with a default.cfg which seems to be a binary format (what the?)

the old version had a config.py or similar which was nicely commented. I see no such file with this release, unless i'm missing something obvious. I'm fairly sure i've read the whole thread, and of course first post still says documentation "coming soon...". perhaps a wiki is in order?

Thanks
-bitcowok
hero member
Activity: 784
Merit: 500
I'm on an iPhone right now. Pics are easier to make Cheesy 

hero member
Activity: 489
Merit: 500
Immersionist
Quote

Programming one device:
java -cp ZtexBTCMiner-120208.jar BTCMiner -m p -f ztex_ufm1_15d2.ihx -s 01-02-01

Reprogramming that specific device:
java -cp ZtexBTCMiner-120208.jar BTCMiner -m p -ps 01-02-01 -f ztex_ufm1_15d2.ihx

Reprogramming all devices that had been programmed with the firmware .....:
java -cp ZtexBTCMiner-120208.jar BTCMiner -m p -pt ztex_ufm1_15d2 -f ztex_ufm1_15d2.ihx


Somewhere between these steps it must have happened...... -s is the serial number of a device (it could be anything even a name) I took one of the boards serial numbers when I tried to run this for the first time:) <--- had no clue what I was doing then.....

When I run BTCminer with the option -I i get this printout:
Pic is comming soon;)

You could always just copy and paste the output, no pic needed.

Can't you just unplug all your devices but one, then program with a new serial number? Rinse and repeat for all your boards. Problem fixed. I am not very experienced with the programming mode, so I can't think of any other solution for now.
Pages:
Jump to: