Pages:
Author

Topic: Modular Python Bitcoin Miner - Official Thread - page 13. (Read 74196 times)

sr. member
Activity: 410
Merit: 252
Watercooling the world of mining
My BFL box has been working for some days now hooked up to cgminer.

If theres anyone interested , maybe you TheSeven or anybody else i will create a ssh access for him to
develop a module.
I would very much appreciate this.
newbie
Activity: 48
Merit: 0
Anyone with a BFL board around who wants to help adding the last missing FPGA board interface module?

As soon as mine arrives, i'll set you up with ssh access, if you still need it. It was ordered in jan, so i'm hoping in the next month or so.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
New feature on testing branch: ZTEX board support

Even though ztex himself still does not seem to want to cooperate, a user of MPBM was kind enough to let me test things on a couple ZTEX boards.

For now, the dynamic overclocking behavior is 100% cloned from BTCMiner to avoid warranty issues. Feel free to check things out! Smiley


Anyone with a BFL board around who wants to help adding the last missing FPGA board interface module?
legendary
Activity: 1022
Merit: 1000
BitMinter
Hey Raize here a little write-up for Windows.

Download and install the USB to Serial driver:

http://www.prolific.com.tw/support/files/%5CIO%20Cable%5CPL-2303%5CDrivers%20-%20Generic%5CWindows%5Callinone%5CPL2303_Prolific_DriverInstaller_v1.5.0.zip

Install Python:

http://www.python.org/getit/

Get the python libs from Zhang (need them to run the board) and put them into the python27/Lib folder:

https://github.com/ngzhang/Icarus/tree/0945ce06be6ed85fc2623ef36aa15c70ae82e56f/Downloads/python_Libs

For .exe curses go to http://www.lfd.uci.edu/~gohlke/pythonlibs/#curses, scroll down to curses and install the one you need for your Windows version !

Download MPBM, unzip in a location of your choice:

https://github.com/TheSeven/Modular-Python-Bitcoin-Miner/zipball/master

Here is my default_config.py file:

Code:
# Modular Python Bitcoin Miner demonstration configuration file
# Copy this file to config.py and modify it to fit your needs.


################################
# List of modules to be loaded #
################################

import frontend.theseven.cursesui
import frontend.theseven.simplelogger
import pool.theseven.bcjsonrpc
#import worker.theseven.simplers232
import worker.theseven.icarus
#import worker.fpgamining.x6500
#import worker.fpgamining.x6500hotplug


###################
# Global settings #
###################

#bufferseconds = 50 # Work buffer size in seconds (default: 50). This should generally be no
# # more than 60 seconds, otherwise you will get increased stale rates!
#getworktimeout = 2 # Work source response timeout in seconds (default: 2)
#sendsharetimeout = 10 # Share upload timeout in seconds (default: 10)
#longpolltimeout = 900 # Long poll connection inactivity timeout in seconds (default: 900)
#longpollgrouptime = 30 # Long poll aggregation timeout in seconds (default: 30)
#longpollgrouptime = 30 # Long poll aggregation timeout in seconds (default: 30)

# DON'T PLAY WITH THESE UNLESS YOU KNOW WHAT YOU'RE DOING!
#getworkbias = -1 # Bias (in MHashes) that is credited to the work source for every work
# # request (default: -1). This punishes work sources which cancel their
# # work very often, but the default value effectively disables this
# # behavior. This needs to be negative (non-zero) though, in order
# # to ensure that work requests are distributed evenly between work
# # sources during startup.
#longpollkillbias = 0 # Bias (in MHashes) that is credited to the work source for every piece
# # of work that was invalidated by a long poll (default: 0). This is
# # used to compensate for anomalies caused by getworkbias.
#getworkfailbias = -3000 # Bias (in MHashes) that is credited to the work source for every
# # failed work request (default: -3000). This punishes work source
# # downtime in general.
#jobstartbias = 0 # Bias (in MHashes) that is credited to the work source everytime
# # a job of that work source starts being processed on a worker (default: 0).
#jobfinishbias = 0 # Bias (in MHashes) that is credited to the work source everytime
# # a job of that work source ends being processed on a worker (default: 0).
#sharebias = 4000 # Bias (in MHashes) that is multiplied with the difficulty and credited
# # to the work source for each found share (default: 4000). This rewards
# # work sources with high efficiency. Keep it near the default value to
# # ensure that work sources which produce junk work (that never yields
# # shares) can not consume much hashing power.
#uploadfailbias = -100 # Bias (in MHashes) that is and credited to the work source for
# # each share upload retry (default: -100). Because a huge bias
# # doesn't keep a work source from retrying to upload the share,
# # you should keep this relatively low to ensure that the work
# # source will be used again when it pops back to life. Work source
# # downtime should be punished using getjobfailbias instead.
#stalebias = -15000 # Bias (in MHashes) that is multiplied by the difficulty and credited
# # to the work source for each stale share (default: -15000). With the
# # default settings this will half the work source's hashing power at
# # a stale rate of about 2%.
#biasdecay = 0.9995 # Decay factor that is multiplied onto all work sources' bias on every
# # getwork on any work source (default: 0.9995). Helps ensuring that
# # work sources will be favored after they recover from temporary
# # failures until they have caught up with the configured priority.


###########################
# List of user interfaces #
###########################

interfaces = [ \

  # Curses UI
  { \
    # User interface module
    "type": frontend.theseven.cursesui.CursesUI, \
    # Update stats every second (default)
    "updateinterval": 1, \
  }, \

  # Simple logger
  { \
    # User interface module
    "type": frontend.theseven.simplelogger.SimpleLogger, \
    # Log file location (default)
    "logfile": "miner.log", \
  }, \

]


###################
# List of workers #
###################

workers = [ \

# # SimpleRS232 worker
# { \
# # Worker module
# "type": worker.theseven.simplers232.SimpleRS232Worker, \
# # Worker module parameters, in this case serial port name (default: /dev/ttyS0)
# "port": "/dev/ttyUSB0", \
# }, \

# # Single X6500 worker
# { \
# # Worker module
# "type": worker.fpgamining.x6500.X6500Worker, \
# # Worker module parameters, in this case board serial number (default: take first available)
# #"deviceid": "ABCDEFGH", \
# }, \

# # X6500 hotplug worker
# { \
# # Worker module
# "type": worker.fpgamining.x6500hotplug.X6500HotplugWorker, \
# }, \

 # Icarus worker
 { \
 # Worker module
 "type": worker.theseven.icarus.IcarusWorker, \
 # Worker module parameters, in this case serial port name (default: /dev/ttyUSB0)
 "port": "COM4", \
 }, \

]


########################
# List of work sources #
########################

# MPBM supports multiple blockchains and aggregates long poll responses
# from all pools within a blockchain to further reduce stales.
# Work from all pools within a blockchain is flushed as soon as any of
# the pools responds to a long poll (subsequent long poll responses from
# other pools within the next seconds will be suppressed).
# This means that e.g. P2Pool will need to be put into a separate blockchain
# definition because its long poll responses don't correlate with other pools.

# The available hashing power (from all workers) can be distributed across
# different work sources by adjusting the priority values. MPBM will attempt
# to distribute hash rate proportionally to these priorities, unless a work
# source fails. If that happens, the work source's priority will be reduced
# and slowly be increased again when the work source comes back to life.
# You can see this in action with the P2Pool entry in the default config
# if you don't have a P2Pool instance running on localhost.

# If you want to support further development of MPBM, you can do me a favor by
# leaving the demo pool entries active, and just adding your own one below them.
# If you give your own pool entries a total priority of 1000, you will donate
# 1% (on average) of your total hash rate to the author of this software.
# This won't make a big difference for you, but it does for me.
# Templates for that can be found below the demo pool entries.

blockchains = [ \

  # Regular bitcoin blockchain
  { \
    # Pools to be used for this blockchain
    "pools": [ \

# # BitMinter
# { \
# # Pool interface module
# "type": pool.theseven.bcjsonrpc.JSONRPCPool, \
# # Display name of the pool (default: host name)
# "name": "BitMinter", \
# # Priority (default: 2)
# "priority": 1, \
# # Host name of the pool
# "host": "mint.bitminter.com", \
# # HTTP port of the pool (default: 8332)
# "port": 8332, \
# # HTTP authentication user name (default: no authentication)
# "username": "XXX", \
# # HTTP authentication password (default: empty)
# "password": "XXX", \
# }, \

 # GPUMAX
 { \
 # Pool interface module
 "type": pool.theseven.bcjsonrpc.JSONRPCPool, \
 # Display name of the pool (default: host name)
 "name": "GPUMAX", \
 # Priority (default: 1)
 "priority": 1, \
 # Host name of the pool
 "host": "gpumax.com", \
 # HTTP port of the pool (default: 8332)
 "port": 8332, \
 # HTTP authentication user name (default: no authentication)
 "username": "XXX", \
 # HTTP authentication password (default: empty)
 "password": "XXX", \
 }, \

    ], \
  }, \

# # P2Pool sharechain
# { \
# # Pools to be used for this blockchain
# "pools": [ \
#
# # P2Pool instance on localhost
# { \
# # Pool interface module
# "type": pool.theseven.bcjsonrpc.JSONRPCPool, \
# # Display name of the pool (default: host name)
# "name": "P2Pool", \
# # Priority (default: 1)
# "priority": 1000, \
# # Host name of the pool
# "host": "127.0.0.1", \
# # HTTP port of the pool (default: 8332)
# "port": 9332, \
# }, \
#
# ], \
# }, \

]

Once edited for your needs, start the miner with the command prompt, something like below if you installed the software to user:

C:\Users\Samxxx Xxxxxx>miner.py

If the board is well, it "should" fire up. If you have questions, feel free to ask.

 
legendary
Activity: 1022
Merit: 1000
BitMinter
Never used Python in my life. Anyone willing to give me a step-by-step on exactly how to do this? I've got Python 2.7 installed and I downloaded default_config.py and miner.py. I have literally no idea what is safe to edit. I use P2Pool. I've made the necessary edits and nothing seem to be working. CGminer also doesn't work, and nor does RG7Miner under Java. I'm starting to think I've gotten a bad Icarus device.

Bounty of 2 Bitcoin to whoever provides a successful step-by-step.

What OS ? Windows ? I spent some hours to get my board running. You need some important files to make it work. It's very unlike that the board is bad. Can help you this evening (european time) if your on IRC.
donator
Activity: 1419
Merit: 1015
Never used Python in my life. Anyone willing to give me a step-by-step on exactly how to do this? I've got Python 2.7 installed and I downloaded default_config.py and miner.py. I have literally no idea what is safe to edit. I use P2Pool. I've made the necessary edits and nothing seem to be working. CGminer also doesn't work, and nor does RG7Miner under Java. I'm starting to think I've gotten a bad Icarus device.

Bounty of 2 Bitcoin to whoever provides a successful step-by-step.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I'm running it both with 2.7 and 3.2, everything >=2.6 should work.
donator
Activity: 1654
Merit: 1354
Creator of Litecoin. Cryptocurrency enthusiast.
USBError: [Errno None] Unknown error

Now that's the kind of libusb errors that I like most.
What OS is that? Mac? Does this affect only the testing branch or both? Does it work with the D2XX driver?

OS is Mac. Only affects testing branch. Haven't tried D2XX drivers. Will try that later.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
USBError: [Errno None] Unknown error

Now that's the kind of libusb errors that I like most.
What OS is that? Mac? Does this affect only the testing branch or both? Does it work with the D2XX driver?
donator
Activity: 1654
Merit: 1354
Creator of Litecoin. Cryptocurrency enthusiast.
I can't seem to get my X6500 workers to work.

2012-03-18 17:17:44.525   [100]   X6500 worker - AH00WI18: Proxy: Exception caught: Traceback (most recent call last):
  File "/Users/charlie/Modular-Python-Bitcoin-Miner-testing/modules/fpgamining/x6500/boardproxy.py", line 77, in run
    else: self.device = FT232R(FT232R_PyUSB(self.serial, self.takeover))
  File "/Users/charlie/Modular-Python-Bitcoin-Miner-testing/modules/fpgamining/x6500/util/ft232r.py", line 314, in __init__
    for bus in usb.busses():
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/legacy.py", line 333, in busses
    return (Bus(),)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/legacy.py", line 329, in __init__
    self.devices = [Device(d) for d in core.find(find_all=True)]
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/core.py", line 851, in find
    return [d for d in device_iter(k, v)]
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/core.py", line 821, in device_iter
    for dev in backend.enumerate_devices():
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/backend/libusb10.py", line 457, in enumerate_devices
    return _DevIterator()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/backend/libusb10.py", line 445, in __init__
    byref(self.dev_list))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/backend/libusb10.py", line 403, in _check
    raise USBError(_str_error[ret], ret, _libusb_errno[ret])
USBError: [Errno None] Unknown error
2012-03-18 17:17:44.528   [100]   X6500 worker - AH00WI18: Traceback (most recent call last):
  File "/Users/charlie/Modular-Python-Bitcoin-Miner-testing/modules/fpgamining/x6500/x6500worker.py", line 211, in main
    elif data[0] == "dying": raise Exception("Proxy died!")
Exception: Proxy died!
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I'd like to invite everyone who's willing to take a small risk of deadlocks and other annoyances, to check out the testing branch of MPBM.
It isn't quite ready for production use that's unattended over months yet, but it has at least survived a week of testing on my own low-hashrate board.
It would be especially interesting for me to collect testing results from bigger FPGA rigs (>2GH/s, or even >10GH/s), but please only do that if you can react quickly in case of failure, to prevent wasting valuable hashing time.
Please note that latencies seem to be too high to work reasonably well with P2Pool. This seems to be the price you have to pay for flexibility. We'll see if that can be improved.

One of the most important new features is the web based user interface, which allows to configure the miner on the fly, and removes the need for a configuration file.

If you want to check it out, do this:
  • Check out the testing branch (git clone git://github.com/TheSeven/Modular-Python-Bitcoin-Miner.git -b testing)
  • Run "python run-mpbm.py"
  • Open http://localhost:8832
  • Log in with user name "admin", password "mpbm"
  • Configure the WebUI frontend to set your own password
  • Add your own work sources to the "user work sources" group
  • Right click on the created work sources and assign them to the "Bitcoin" blockchain (or whichever blockchain they belong to), to enable long poll aggregation
  • When you're done, hit "save configuration" (will be done during a clean shutdown anyway, but it can't hurt to do it manually after lots of changes to reduce the risk of data loss)

Btw. the correct way to shut down MPBM is just hitting ctrl+c on the console.

Known issues:
  • Hashrate/priority settings do reach their targets after days of runtime, but shortly after starting the miner the hashrate distribution depends more on X-Roll-NTime support of the work source than anything else.
  • The WebUI doesn't play nice with Firefox. Anyone got a fix for that?

If you hit some issues, or just want to provide some feedback, feel free to post here or contact me at #mpbm on irc.freenode.net.
In case you should hit a deadlock, please obtain a thread state dump from the debug menu on WebUI and pastebin that.
legendary
Activity: 1022
Merit: 1000
BitMinter
Runs stable at 0.1% rejects. No more sleep mode. Only thing that's not working after some time is the average MH/s rate and the efficiency. It says 600+ MH for one board. Restart helps.
legendary
Activity: 1022
Merit: 1000
BitMinter
Finally working Cheesy no curses, no miner... donation sent !
Are you sure that you prefer CursesUI over WebUI+StderrLogger with ANSI coding enabled?

 Shocked i'm a carpenter, no idea what you talk about Cheesy
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
Finally working Cheesy no curses, no miner... donation sent !
Are you sure that you prefer CursesUI over WebUI+StderrLogger with ANSI coding enabled?
legendary
Activity: 1022
Merit: 1000
BitMinter
Finally working Cheesy no curses, no miner... donation sent !
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I've uploaded an MPBM v0.1.x prototype to the testing branch of the git repository.
It's working sufficiently to fetch work from the demo work sources somehow, so people can start porting worker interfaces. The SimpleRS232Worker module has already been ported, and might serve as kind of a porting guide.

Please not that this is not ready for production use yet. At the moment it supports neither long polling nor any other getwork extensions, and it's likely that there's still quite a lot of bugs. There also is no UI for the statistics yet (but the statistics should be collected in the background).

Tasks left to be done before the beta release:
  • Clean up BCJSONRPC module
  • Implement getwork extensions
  • Possibly implement UBBP module
  • Implement statistics UI (still needs some cosmetic improvements)
  • Add work queue debug UI
  • Possibly port CursesUI (likely not to be done)
  • Port Icarus worker module (mostly working, minor statistics issues)
  • Implement out of process worker support (might make sense for the X6500/ztex interfaces)
  • Port X6500 worker modules (untested)
  • Possibly implement support for uploading firmware to multiple X6500 boards at once, with only a single shared preparation process (will not be implemented)
  • Possibly implement BFL worker module
  • Decide what to do about the ztex worker module
  • Possibly implement GPU mining (cherrypick from poclbm?) (not anytime soon)
  • Fix bugs
  • Fix even more bugs
  • Improve documentation
  • Clean up the code
  • Do some benchmarks and identify/fix bottlenecks
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
And again:
what are your experience values  with MPBM and p2pool ( stales,rates, ..)
And wich values do you adjust to improve performance.
Should be between 10% and 20% stales if p2pool performs well (which my installation clearly doesn't, due to a sluggish bitcoind).
And you can't really adjust any values to improve that, maybe increase the polling frequency if it's a x6500/ztex board.
The only thing you could do about it would be latency optimizing the code, which will break the modular approach (and multi-worker/multi-pool capabilities).
sr. member
Activity: 410
Merit: 252
Watercooling the world of mining
I am using a modified Ubuntu 11.04 natty.
I recompiled both bitcoind and bitcoin-qt for the plattform.
Have you done a writeup on this anywhere?

I will create thread for Bitcoin on ARM devices as soon as i find the time.
Maybe this weekend.
I will post descriptions  downloadable system images, build instructions....
For the "Gumstix" ARM , the panda board , and maybe for a rasberry pi
if i get one.

So you will have to be a little patient.
I would appreciate anybody else who has created systems on ARM devices
to post his system there.

And again:
what are your experience values  with MPBM and p2pool ( stales,rates, ..)
And wich values do you adjust to improve performance.
hero member
Activity: 489
Merit: 500
Immersionist
I just got this running with Icarus on Windows 7.. nice!
Bitcoin donated.
Thank you!

Is there a proper way to shut down MPBM? I've just been killing the command window as things like ctrl-c ctrl-z don't seem to have any effect.
Hm, usually Ctrl+C should kill it. If it doesn't, there's probably something that locks up while attempting to shut things down. The Icarus worker module is currently kind of a 5 minute hack to make it work somehow, so it might suffer from that kind of thing. This will be improved in the upcoming major release.
Just killing the miner should be fine, it doesn't do any real cleanup anyway (yet).

Ctrl-C doesn't work on my Windows 7/Icarus setup neither. Ctrl-Break works.

I now also have 3 ZTEX boards, but I will be traveling for the next 10 days so I can't try it using the new code in MPBM.
hero member
Activity: 504
Merit: 500
FPGA Mining LLC
I just got this running with Icarus on Windows 7.. nice!
Bitcoin donated.
Thank you!

Is there a proper way to shut down MPBM? I've just been killing the command window as things like ctrl-c ctrl-z don't seem to have any effect.
Hm, usually Ctrl+C should kill it. If it doesn't, there's probably something that locks up while attempting to shut things down. The Icarus worker module is currently kind of a 5 minute hack to make it work somehow, so it might suffer from that kind of thing. This will be improved in the upcoming major release.
Just killing the miner should be fine, it doesn't do any real cleanup anyway (yet).
Pages:
Jump to: