(OLD) BFGMiner: modular FPGA/GPU, GBT, Stratum, RPC, Avalon/Lnx/OpnWrt/PPA/W64 - page 34.

Hpman

full member

Activity: 176

Merit: 100

Quote from: Hpman on May 02, 2012, 09:57:15 AM

Hi, I'm using ubuntu 12.04. For preparation i'm using ./configure --enable-ztex but it stops with

Code:

checking for libusb_init in -lusb-1.0... no
configure: error: Could not find usb library - please install libusb

I thought libusb is already installed per default on ubuntu ? Do i need to copy any header file in the bfgminer directory ?
I installed libsusb-dev too but it doesn't help.

Thanks Hpman
PS: BTCMiner works, i guess without libusb it would not.

libusb-dev wasn't enough, I've added "libusb-1.0-0-dev" too and it works. Now it stops at

Code:

checking for LIBCURL... no
checking for LIBCURL... no
configure: error: Missing required libcurl dev >= 7.10.1

It seems that there is no libcurl.pc existing (for pkg-config)
curl-config --version shows libcurl 7.22.0

Hpman

Luke-Jr

legendary

Activity: 2576

Merit: 1186

NEW VERSION: 2.4.0 - May 3, 2012

Note this is an "upstream changes from CGMiner only" release.
However, since BFGMiner has had reasonably accurate Icarus hash measurement from the start, my fix in CGMiner/BFGMiner 2.4.0 makes BFGMiner as ideally accurate as reasonably possible.

This version has a fairly significant upgrade to the way networking is done, so there is a minor version number update instead of a micro version, but it has already been heavily tested.

Human readable changelog:

A whole networking scheduler of sorts was written for this version, designed to scale to any sized workload with the fastest networking possible, while minimising the number of connections in use, reusing them as much as possible.
The restart feature was added to the API to restart BFGMiner remotely.
If you're connected to a pool that starts rejecting every single share, BFGMiner will now automatically disable it unless you add the --no-pool-disable option.
Once a pool stops responding, BFGMiner won't keep trying to open a flood of extra connections.
Failing BFL won't cause cgminer to stop; it'll just disable the device, which an attempt may be made to re-enable it.
Hashrates on FPGAs are now reasonably accurate.
Longpoll messages won't keep going indefinitely while a pool is down.

Full changelog:

Only show longpoll warning once when it has failed.
Convert hashes to an unsigned long long as well.
Detect pools that have issues represented by endless rejected shares and disable them, with a parameter to optionally disable this feature.
Bugfix: Use a 64-bit type for hashes_done (miner_thread) since it can overflow 32-bit on some FPGAs
Implement an older header fix for a label existing before the pthread_cleanup macro.
Limit the number of curls we recruit on communication failures and with delaynet enabled to 5 by maintaining a per-pool curl count, and using a pthread conditional that wakes up when one is returned to the ring buffer.
Generalise add_pool() functions since they're repeated in add_pool_details.
Bugfix: Return failure, rather than quit, if BFwrite fails
Disable failing devices such that the user can attempt to re-enable them
Bugfix: thread_shutdown shouldn't try to free the device, since it's needed afterward
API bool's and 1TBS fixes
Icarus - minimise code delays and name timer variables
api.c V1.9 add 'restart' + redesign 'quit' so thread exits cleanly
api.c bug - remove extra ']'s in notify command
Increase pool watch interval to 30 seconds.
Reap curls that are unused for over a minute. This allows connections to be closed, thereby allowing the number of curl handles to always be the minimum necessary to not delay networking.
Use the ringbuffer of curls from the same pool for submit as well as getwork threads. Since the curl handles were already connected to the same pool and are immediately available, share submission will not be delayed by getworks.
Implement a scaleable networking framework designed to cope with any sized network requirements, yet minimise the number of connections being reopened. Do this by create a ring buffer linked list of curl handles to be used by getwork, recruiting extra handles when none is immediately available.
There is no need for the submit and getwork curls to be tied to the pool struct.
Do not recruit extra connection threads if there have been connection errors to the pool in question.
We should not retry submitting shares indefinitely or we may end up with a huge backlog during network outages, so discard stale shares if we failed to submit them and they've become stale in the interim.

Tinua

hero member

Activity: 871

Merit: 1000

Hi Luke-Jr

Like promise in the cgminer-tread, i make the same test with 15 BFL's.

Thats the result:

http://www.fileswap.com/dl/QELC51peid/bfgminer2.jpg.html

With CGminer runs everything now!

Regards
Tinua

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: rjk on May 02, 2012, 11:20:54 AM

Quote from: Luke-Jr on May 02, 2012, 11:19:22 AM

Quote from: Inaba on May 02, 2012, 11:17:04 AM

How is Ufasoft handling the hot-plugging? I'd *really* like to see that functionality in cgminer/bfgminer.

It's polling every serial port every few seconds. Too ugly to put in any sane software, IMO. The real problem for CGMiner/BFGMiner is that the initialization has to happen in a certain order right now, so adding devices later isn't possible. Changing that will require some refactoring.

I'd argue that this is very acceptable for a dedicated miner, even if it isn't for a non-dedicated miner, therefore I would like the same thing to be implemented in cgminer/bfgminer. It could be an option, such as -S aggressive or similar.

I was considering a "-S all"; but in any case, hotplug support still requires a refactor of CGMiner's initialization.

af_newbie

legendary

Activity: 2702

Merit: 1468

Quote from: Epoch on May 02, 2012, 10:05:45 AM

Quote from: preparation on May 02, 2012, 09:34:46 AM

Quote from: Inaba on May 02, 2012, 08:54:44 AM

Yes, please... if one of my units fails or otherwise goes off line, it basically brings the entire cluster down... very annoying.

this is exactly what I wanted to say

Guys, as a temporary workaround, you can wrap your cgminer call in a .bat file (Windows), or the Linux equivalent. Something like this (again, Windows code):

Code:

:loop
cgminer 
waitfor /t 10 signame
goto loop

If cgminer quits for any reason (such as a Single going offline), the batch file will pause for 10 seconds (adjust this to whatever you want) and then runs cgminer again. If you ever want to stop the infinite loop and get out of the batch file, hit and answer 'y' to the prompt that appears (again, Windows).

Epoch, cgminer crashes. If process is still running, your script will not work.
It will only work when cgminer quits gracefully or calls exit()/abort().

rjk

sr. member

Activity: 448

Merit: 250

1ngldh

Quote from: Luke-Jr on May 02, 2012, 11:19:22 AM

Quote from: Inaba on May 02, 2012, 11:17:04 AM

How is Ufasoft handling the hot-plugging? I'd *really* like to see that functionality in cgminer/bfgminer.

It's polling every serial port every few seconds. Too ugly to put in any sane software, IMO. The real problem for CGMiner/BFGMiner is that the initialization has to happen in a certain order right now, so adding devices later isn't possible. Changing that will require some refactoring.

I'd argue that this is very acceptable for a dedicated miner, even if it isn't for a non-dedicated miner, therefore I would like the same thing to be implemented in cgminer/bfgminer. It could be an option, such as -S aggressive or similar.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: Inaba on May 02, 2012, 11:17:04 AM

How is Ufasoft handling the hot-plugging? I'd *really* like to see that functionality in cgminer/bfgminer.

It's polling every serial port every few seconds. Too ugly to put in any sane software, IMO. The real problem for CGMiner/BFGMiner is that the initialization has to happen in a certain order right now, so adding devices later isn't possible. Changing that will require some refactoring.

Inaba

legendary

Activity: 1260

Merit: 1000

Same with my experience ... cgminer just stops, it doesn't hang... so the batch file idea would work until it's fixed.

How is Ufasoft handling the hot-plugging? I'd *really* like to see that functionality in cgminer/bfgminer.

Epoch

legendary

Activity: 922

Merit: 1003

Quote from: af_newbie on May 02, 2012, 10:20:31 AM

Quote from: Epoch on May 02, 2012, 10:05:45 AM

Quote from: preparation on May 02, 2012, 09:34:46 AM

Quote from: Inaba on May 02, 2012, 08:54:44 AM

Yes, please... if one of my units fails or otherwise goes off line, it basically brings the entire cluster down... very annoying.

this is exactly what I wanted to say

Guys, as a temporary workaround, you can wrap your cgminer call in a .bat file (Windows), or the Linux equivalent. Something like this (again, Windows code):

Code:

:loop
cgminer 
waitfor /t 10 signame
goto loop

If cgminer quits for any reason (such as a Single going offline), the batch file will pause for 10 seconds (adjust this to whatever you want) and then runs cgminer again. If you ever want to stop the infinite loop and get out of the batch file, hit and answer 'y' to the prompt that appears (again, Windows).

Epoch, cgminer crashes. If process is still running, your script will not work.
It will only work when cgminer quits gracefully or calls exit()/abort().

That is not what I am seeing. I have tried this exact scenario on 4 separate Windows 7/64 machines. When I unplug one of the Singles, cgminer simply quits. Perhaps not gracefully, but it does not hang or lock up. It merely quits ... essentially back to the command prompt. My script then gets control and loops back to re-run it; it has worked fine on 4 different machines.

Some of the recent earlier versions of cgminer used to crash/lock-up. But the recent releases (certainly 2.3.6) do not and merely quit to the command prompt.

I realize that if cgminer hangs, this workaround will not work. The real solution is a fix in cgminer. I offer this looping .bat file as something that may work for some of us (Windows 7 at least, perhaps not Linux ... that's something I cannot comment on).

edit: sorry, Luke, I don't mean to go off-topic here with cgminer comments ... I'll stay back on track with bfgminer! Lips sealed

I haven't 'unplugged a Single' running with bfgminer yet to see how it behaves ... I'll try that this evening.

Epoch

legendary

Activity: 922

Merit: 1003

Quote from: preparation on May 02, 2012, 09:34:46 AM

Quote from: Inaba on May 02, 2012, 08:54:44 AM

Yes, please... if one of my units fails or otherwise goes off line, it basically brings the entire cluster down... very annoying.

this is exactly what I wanted to say

Guys, as a temporary workaround, you can wrap your cgminer call in a .bat file (Windows), or the Linux equivalent. Something like this (again, Windows code):

Code:

:loop
cgminer 
waitfor /t 10 signame
goto loop

If cgminer quits for any reason (such as a Single going offline), the batch file will pause for 10 seconds (adjust this to whatever you want) and then runs cgminer again. If you ever want to stop the infinite loop and get out of the batch file, hit and answer 'y' to the prompt that appears (again, Windows).

Hpman

full member

Activity: 176

Merit: 100

Hi, I'm using ubuntu 12.04. For preparation i'm using ./configure --enable-ztex but it stops with

Code:

checking for libusb_init in -lusb-1.0... no
configure: error: Could not find usb library - please install libusb

I thought libusb is already installed per default on ubuntu ? Do i need to copy any header file in the bfgminer directory ?
I installed libsusb-dev too but it doesn't help.

Thanks Hpman
PS: BTCMiner works, i guess without libusb it would not.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

OK, I wrote the code to do it and sent it to Con for CGMiner. Whether he merges it or not, it'll be in the next BFGMiner.

preparation

member

Activity: 63

Merit: 10

Quote from: Inaba on May 02, 2012, 08:54:44 AM

Yes, please... if one of my units fails or otherwise goes off line, it basically brings the entire cluster down... very annoying.

Hotplug would be the best, but disabling the device is a second best, since the rest can keep mining at the point.

this is exactly what I wanted to say

Inaba

legendary

Activity: 1260

Merit: 1000

Yes, please... if one of my units fails or otherwise goes off line, it basically brings the entire cluster down... very annoying.

Hotplug would be the best, but disabling the device is a second best, since the rest can keep mining at the point.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: preparation on May 02, 2012, 01:58:59 AM

I'd like to request a protection of BFGMiner in case a BFL unit will be disconnected. If you unplug the usb cable the miner crashes as 'BFwrite failed'.

What would you expect it to do? BFGMiner isn't hotplug-capable (yet). Maybe it could disable the device...

preparation

member

Activity: 63

Merit: 10

I'd like to request a protection of BFGMiner in case a BFL unit will be disconnected. If you unplug the usb cable the miner crashes as 'BFwrite failed'.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

NEW VERSION 2.3.6, APRIL 29 2012

Note this is a "upstream changes from CGMiner only" release.

Human readable changelog

Important: --submit-stale option no longer exists. Con has replaced it with --no-submit-stale and made it submit stale shares by default now. The reason for this change is that with the new longpoll code in cgminer, it can detect block changes if you have backup pools almost always faster than the primary pool can detect the block change (unless the primary pool found the block). This means the primary pool is still accepting shares for the old block, so it won't consider them stale yet.
Con has further revised the networking code to be able to cope with multigigahash machines that can overwhelm the one upstream and one downstream connection that was introduced into 2.3.5. Now it tries to get the best of all worlds. It does this by keeping one connection open for up and one for down and reuses that at all times, but, when it detects a backlog of share submission or getworks will occur, it starts recruiting extra connections. Thus it will be as nice as it can be to networks, but get nasty if it needs to.
Two crashes were fixed: One was that hot-adding of pools was broken in 2.3.5. The second crash would very rarely occur across a longpoll when many pools were set up.
The double longpoll message from the same pool should be fixed.
Slight changes to make the screen output even neater.

Full changelog:

Shorten stale share messages slightly.
Protect the freeing of current_hash under mutex_lock to prevent racing on it when set_curblock is hit concurrently.
Change default behaviour to submitting stale, removing the --submit-stale option and adding a --no-submit-stale option.
Make sure to start the getwork and submit threads when a pool is added on the fly. This fixes a crash when a pool is added to running cgminer and then switched to.
Faster hardware can easily outstrip the speed we can get work and submit shares when using only one connection per pool.
Test the queued list to see if any get/submits are already queued and if they are, start recruiting extra connections by generating new threads.
This allows us to reuse network connections at low loads but recuit new open connections as they're needed, so that cgminer can scale to hardware of any size.

BR0KK

hero member

Activity: 784

Merit: 500

Unable to test it atm .... working on some of my Singles that have some weird problems (one thinks to be a quad 1.15y and the other one seems to be dead for no reason?)

bombo999

member

Activity: 107

Merit: 10

lol ... kano is such a cry baby

kano

legendary

Activity: 4634

Merit: 1851

Linux since 1997 RedHat 4

Aw, come on, do a proper release post.
If it's just 'cgminer' go read the release over there, then it's pretty much ... and go use cgminer also Tongue

Topic: (OLD) BFGMiner: modular FPGA/GPU, GBT, Stratum, RPC, Avalon/Lnx/OpnWrt/PPA/W64 - page 34. (Read 260061 times)