Pages:
Author

Topic: (OLD) BFGMiner: modular FPGA/GPU, GBT, Stratum, RPC, Avalon/Lnx/OpnWrt/PPA/W64 - page 15. (Read 260061 times)

legendary
Activity: 2576
Merit: 1186
Abstract: getblocktemplate more wasteful than getwork

As a solo miner, I switched one of my FPGA hosts to bfgminer 2.9.3 in order to mine with getblocktemplate against bitcoind 0.7.1. This particular host has 37 FPGA devices (mixture of bfl singles, cm1's, icarus's) and I notice that bfgminer is sub-optimal: every 2-3 minutes, it does about 20-40 getblocktemplate calls in a row (1 for each FPGA device?). These 20-40 getblocktemplate calls amount to about 3-4MB total. So on average, getblocktemplate generates about 1-2 MB of network traffic per minute.

On the other hand, when mining with getwork, these 37 FPGA devices amount to about 15 Ghash/sec, so it generates about 4 getwork calls per second, or about 600 kB/minute as measured by a packet sniffer.

Bottom line, getblocktemplate as it is implemented in bfgminer causes my host to generate more network traffic than getwork (but fewer RPC calls). I haven't taken the time to read the code yet, but, Luke, isn't there something trivial to optimize to reduce the number of getblocktemplate calls?
bitcoind is not intended for use of slow networks, and doesn't even attempt to minimize traffic. It also does not currently implement long-polling (though next-test has a patch to support it), so the template needs to be refreshed regularly in case of the old one being stale.

As you note, above matters with bitcoind aside, bfgminer does not implement GBT optimally at this time, since it is using code originally built around getwork; cgminer has bypassed the getwork paths for its GBT support, but also intentionally designed the GBT path to use more bandwidth (conman wants to make GBT look bad). Rewriting the pool networking code in bfgminer has been on my todo list for a while (mainly due to other problematic bugs in it), and hopefully I'll get to that before 3.0. But even without it, each template is still able to scale per device, so the problem of ASICs hashing much faster is still dealt with in practice.
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
mrb, seriously ... read:

https://bitcointalksearch.org/topic/m.1349123

... and of course it is also way less traffic.

I dunno what bugs Luke-Jr was talking about he has in bfgminer, that he was mentioning in the cgminer thread, but try it all with stratum and cgminer and see for yourself.

Luke's already been told what the problems are with GBT and how to fix it but his response was along the lines of ... that's just an implementation issue ...
mrb
legendary
Activity: 1512
Merit: 1028
Abstract: getblocktemplate more wasteful than getwork

As a solo miner, I switched one of my FPGA hosts to bfgminer 2.9.3 in order to mine with getblocktemplate against bitcoind 0.7.1. This particular host has 37 FPGA devices (mixture of bfl singles, cm1's, icarus's) and I notice that bfgminer is sub-optimal: every 2-3 minutes, it does about 20-40 getblocktemplate calls in a row (1 for each FPGA device?). These 20-40 getblocktemplate calls amount to about 3-4MB total. So on average, getblocktemplate generates about 1-2 MB of network traffic per minute.

On the other hand, when mining with getwork, these 37 FPGA devices amount to about 15 Ghash/sec, so it generates about 4 getwork calls per second, or about 600 kB/minute as measured by a packet sniffer.

Bottom line, getblocktemplate as it is implemented in bfgminer causes my host to generate more network traffic than getwork (but fewer RPC calls). I haven't taken the time to read the code yet, but, Luke, isn't there something trivial to optimize to reduce the number of getblocktemplate calls?
legendary
Activity: 2912
Merit: 1060
start "BFG Miner BFL" bfgminer.exe -c cgminer.conf -S bitforce:\\.\COM3 -S bitforce:\\.\COM4
legendary
Activity: 1764
Merit: 1002
can one run bfgminer in windows 8 using a *.bat file?

nvm, i guess you can.
legendary
Activity: 1764
Merit: 1002
can one run bfgminer in windows 8 using a *.bat file?
full member
Activity: 154
Merit: 100
Can you elaborate on the 3(?) different warnings more? Maybe also figure out which was the last/first version to raise the warnings?

Luke, here is the full screenshot with the error on 2.8.3:



I'll try to find some time tomorrow and check the last good / first bad version.
legendary
Activity: 2576
Merit: 1186
I do not see any virus warning with previous versions. Last warning that I remember was long time ago and it was about the pdcurses.dll.
Can you check again? If 2.8.5 really doesn't trigger this and 2.8.6 does, I imagine it should be easy to workaround.

Sorry I think I was just lucky and as you said ... I do not use much the windows version but today I was trying to check something (the stats issue with --scrypt) and I jumped from 2.6.6 to 2.9.3 / 2.8.6 and I notice the warnings.

As you suggested I retested all versions from 2.8.0 to 2.8.6 and 2.9.0 to 2.9.3. See table below for the results ...

VersionTest result
2.6.6No Warnings
2.8.0Same warning as 2.9.3
2.8.1Same warning as 2.9.3
2.8.2Same warning as 2.9.3
2.8.3New warning: "This is a known Trojan/Backdoor. It is recommended that you quarantine this threat."
2.8.4Same warning as 2.9.3
2.8.5Same warning as 2.9.3
2.8.6Warning with full description
2.9.0Same warning as 2.9.3
2.9.1Same warning as 2.9.3
2.9.2Same warning as 2.9.3
2.9.3Same warning as 2.9.3
Can you elaborate on the 3(?) different warnings more? Maybe also figure out which was the last/first version to raise the warnings?
full member
Activity: 154
Merit: 100
I do not see any virus warning with previous versions. Last warning that I remember was long time ago and it was about the pdcurses.dll.
Can you check again? If 2.8.5 really doesn't trigger this and 2.8.6 does, I imagine it should be easy to workaround.

Sorry I think I was just lucky and as you said ... I do not use much the windows version but today I was trying to check something (the stats issue with --scrypt) and I jumped from 2.6.6 to 2.9.3 / 2.8.6 and I notice the warnings.

As you suggested I retested all versions from 2.8.0 to 2.8.6 and 2.9.0 to 2.9.3. See table below for the results ...

VersionTest result
2.6.6No Warnings
2.8.0Same warning as 2.9.3
2.8.1Same warning as 2.9.3
2.8.2Same warning as 2.9.3
2.8.3New warning: "This is a known Trojan/Backdoor. It is recommended that you quarantine this threat."
2.8.4Same warning as 2.9.3
2.8.5Same warning as 2.9.3
2.8.6Warning with full description
2.9.0Same warning as 2.9.3
2.9.1Same warning as 2.9.3
2.9.2Same warning as 2.9.3
2.9.3Same warning as 2.9.3

AV software looks for miners specifically.

I think this fully explain the warnings...

Sorry for the false alarm!

Jake
legendary
Activity: 2576
Merit: 1186
It's not saying BFGMiner is a virus, merely that users should be aware it's installed. Since some viruses tend to bundle miner programs, this seems quite reasonable.
What exactly you and ckolivas are using which causes AV software to trigger alarms? Miners can't work without those components, or it's just .exe and .dll packing method?
AV software looks for miners specifically.
legendary
Activity: 2576
Merit: 1186
Using latest win32 builds for bfgminer I start seeing virus warnings:

Here is the warning for 2.8.6 build:
Please read the warning; I agree with it: "This is a potentially unwanted application. These are programs that computer users wish to be made aware of."
It's not saying BFGMiner is a virus, merely that users should be aware it's installed. Since some viruses tend to bundle miner programs, this seems quite reasonable.

I do not see any virus warning with previous versions. Last warning that I remember was long time ago and it was about the pdcurses.dll.
Can you check again? If 2.8.5 really doesn't trigger this and 2.8.6 does, I imagine it should be easy to workaround.
full member
Activity: 154
Merit: 100
Using latest win32 builds for bfgminer I start seeing virus warnings:

Here is the warning for 2.8.6 build:
...
and here is the one for 2.9.3:
...

I do not see any virus warning with previous versions. Last warning that I remember was long time ago and it was about the pdcurses.dll.

Thanks,
Jake
pls just stfu... we dont need AV spam here seriously!

I see this as a possible major issue if the AV warning prove to be valid ... Seriously!

If you do not see it this way then please just ignore it.
legendary
Activity: 1792
Merit: 1008
/dev/null
Using latest win32 builds for bfgminer I start seeing virus warnings:

Here is the warning for 2.8.6 build:


and here is the one for 2.9.3:


I do not see any virus warning with previous versions. Last warning that I remember was long time ago and it was about the pdcurses.dll.

Thanks,
Jake
pls just stfu... we dont need AV spam here seriously!
full member
Activity: 154
Merit: 100
Using latest win32 builds for bfgminer I start seeing virus warnings:

Here is the warning for 2.8.6 build:


and here is the one for 2.9.3:


I do not see any virus warning with previous versions. Last warning that I remember was long time ago and it was about the pdcurses.dll.

Thanks,
Jake
sr. member
Activity: 850
Merit: 331
vitruvio, if you have a problem, you need to open an issue in github so the devs can troubleshoot it, as that is most likely where they look for the active issues when they are coding, rather than scouring forum threads.

I will.

Regards
hero member
Activity: 504
Merit: 500
vitruvio, if you have a problem, you need to open an issue in github so the devs can troubleshoot it, as that is most likely where they look for the active issues when they are coding, rather than scouring forum threads.
sr. member
Activity: 850
Merit: 331
Stats with --scrypt param still wrong for me in  2.9.3.
Is there an issue for this open on GitHub?

I download Windows version already comipiled, so nothing to try or check until next w32 binary release.

Regards
legendary
Activity: 2576
Merit: 1186
I'm thinking that the dynamic clocking of 2.9.2 and 2.9.3 (using the older bitstream doesn't help, still getting SICK and DEAD later on) is probably more aggressive compared to 2.9.0 and 2.9.1.
I don't think SICK/DEAD can be caused by the bitstream or dynclock code. This is something around the FT232R chip; unfortunately, the only change in this that I recall between 2.9.1 and 2.9.2 was a memory corruption bug. There were no changes to the dynamic clocking besides the default frequency.
full member
Activity: 173
Merit: 100
After 3 days of testing, I get better stability with the overclocker bitstream and 2.9.1 on my x6500s.
Any chance we could bisect the SICK problem on IRC sometime? Smiley

Ran 2.9.0 and overclocker bitstream for 12 hrs, it doesn't seem to affect the performance while using Chrome and other tasks unlike using the default bitstream included with 2.9.0 and 2.9.1 that seem to lower the performance of the FPGAs (not a problem on a dedicated mining machine, of course).

     

I'm thinking that the dynamic clocking of 2.9.2 and 2.9.3 (using the older bitstream doesn't help, still getting SICK and DEAD later on) is probably more aggressive compared to 2.9.0 and 2.9.1.
legendary
Activity: 2576
Merit: 1186
Stats with --scrypt param still wrong for me in  2.9.3.
Is there an issue for this open on GitHub?
Pages:
Jump to: