Author

Topic: OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.0 - page 476. (Read 5806057 times)

-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
can't compile with opencl support under windows!
Result after
Code:
CFLAGS="-O2 -msse2" ./configure
Code:
Configuration Options Summary:

  curses.TUI...........: FOUND: pdcurses
  OpenCL...............: NOT FOUND. GPU mining support DISABLED
configure: error: No mining configured in
folder ADL_SDK contains:
adl_defines.h
adl_sdk.h
adl_structures.h
Last time this happened it was a packaging error on my part. I'll reupload shortly. I've reuploaded it.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
if you could, it'd be extremely helpful to add a flag to the program to simply ignore the memory size checking and create the pad on the GPU anyway (for some reason, cgminer is only reporting that my video cards have 512mb of memory when they have 3gb.  I don't know why this is)

It will set it to whatever you choose regardless if it can instead of what it detects as the maximum.

The reported size is just the reported max size allocatable by opencl, it is NOT the gpu ramsize. I already said that it will try to set it to what you set it to. It fails, and we are back to my original response - I have no idea why it fails, but that is the response to the command asking for that much memory.
member
Activity: 125
Merit: 10
that is for ADL support, now go back and re-read the windows-build.txt file
But it all worked while compiling cgminer 2.7.5
Have APPSDK 2.5
Do I need the latest APPSDK 2.7 ?
legendary
Activity: 1484
Merit: 1005
high HW values are normal? also utility is very low!
i haven't found yet correct ltc settings for 79xx cards. changing to btc  Wink
yeah, i realized it after posting. still broken.
hero member
Activity: 607
Merit: 500
high HW values are normal? also utility is very low!
i haven't found yet correct ltc settings for 79xx cards. changing to btc  Wink
legendary
Activity: 1484
Merit: 1005
You should be able to easily replicate this just by setting --thread-concurrency 12288 (which works fine on reaper).

I'm pretty sure the problem has to do with these lines,
Code:
clState->padbufsize = bufsize;
clState->padbuffer8 = clCreateBuffer(clState->context, CL_MEM_READ_WRITE, bufsize, NULL, &status);

For whatever reason your program is calculating 0 for the bufsize.  You should be able to step through this with a debugger and figure it out pretty easily I would presume.
I'm unable to reproduce this anywhere. Can you give me your whole command line minus any account details?

--scrypt -I 20 -g 1 -v 1 -w 256 --shaders 1792 --thread-concurrency 12288 or 24000

Off the top of my head

It's been a noted bug in the windows version since the tittiez beta builds
Now that is just bizarre. I tried it even on a windows machine and it didn't give me zero...

EDIT: Nm, can now reproduce.
Okay I've done quite a bit of investigation around this "0" displayed issue. Ironically, that is a display bug in windows. The buffer size is actually being worked out to something like 1.5 billion, and if it's put on a separate line you can see that bufsize is  not zero (I'll do it in the next version). However this does not fix your original complaint that you can't set very high thread concurrency counts like you could on raper [sic]. But you've reminded me of what happened when I investigated this originally.

There are a number of problems with the way raper uses the padbuffer there. Firstly it is reused between threads which means that if you set multiple threads per device they fight over and can trash the data in the buffer. That's not a huge problem with raper because its threading is pretty primitive, unlike cgminer which is heavily multithreaded. However, the main problem is that there is NO error checking on setting values to run the opencl commands. If it  returns invalid values, raper just does it again, and assumes the hashes have been done. So it intermittently works, and intermittently just counts up a number of hashes that never happened. So what happens is you get a displayed hash rate that is really high that does not translate into a proportional rise in number of shares generated.

Summary: I implore you to compare the best share generation rate of raper to cgminer rather than the displayed hashrate.

I only go by pool hashrate.  Reaper pulls 550kh/s per card versus 460kh/s per card with cgminer.  That's a 20% improvement.  Reaper gets ~3% stales which is more or less the same as compared with cgminer.

edit: tried these settings
Code:
--scrypt --thread-concurrency 8000 --shaders 1792 -I 20 -g 1 -v 1 -w 64


Okay.  The problem is the same as the one with reaper, in that when thread_concurrency is too low 7xxx cards will only yield hardware errors at intensity = 20.  to fix this, you need to use more memory.  if you could, it'd be extremely helpful to add a flag to the program to simply ignore the memory size checking and create the pad on the GPU anyway (for some reason, cgminer is only reporting that my video cards have 512mb of memory when they have 3gb.  I don't know why this is)



basically, i'm pretty sure the problem is that cgminer is calculating the memory size available on the card incorrectly.
legendary
Activity: 2772
Merit: 1277
I see a problem with stratum support, too. I use the following pool config:
Code:
"pools" : [
        {
                "url" : "stratum+tcp://api-stratum.bitcoin.cz:3333",
                "user" : "",
                "pass" : ""
        },
        {
                "url" : "stratum+tcp://stratum.btcguild.com:9332",
                "user" : "",
                "pass" : ""
        },
        {
                "url" : "http://mining.eligius.st:8337",
                "user" : "",
                "pass" : ""
        }
]

In cgminer the stratum pools that are not immediately selected are dead. The example config above leads to the following picture in cgminer:

Code:
0: Enabled Alive Priority 0: api-stratum.bitcoin.cz  User:
1: Enabled Dead Priority 1: stratum.btcguild.com  User:
2: Enabled Alive Priority 2: http://mining.eligius.st:8337  User:
legendary
Activity: 1386
Merit: 1097
I can confirm the bug, cgminer doesn't detect connection failure.
legendary
Activity: 1750
Merit: 1007
Looks like there's a bug in the stratum support Sad.  If the pool is turned off, cgminer doesn't register that the pool is dead.  This means it never attempts to reconnect when the pool comes back up.  Unable to test if the same thing happens on a local network failure since I'm not in a situation to turn off my router or the PC my FPGAs are plugged into.
legendary
Activity: 1750
Merit: 1007
Any stats on how much kb/s would be transferred to a pool for a generic hashrate of like 60Ghash/s ?

On BTC Guild, Stratum also includes variable difficulty to control the rate of share submissions, with a target rate of approximately 1 share per 5 seconds.  

Each share submission is roughly 110 bytes.  So your upstream bandwidth will be approximately 30 bytes per second upstream (a bit higher when you first connect due to not being adjusted to variable difficulty).  A share response is roughly 50 bytes, meaning 10 bytes per second downstream (for share submissions).

New work jobs are delivered every 30 seconds, and clock in at about 700-1100 bytes.  Using 1kb averages, that is 2 kb/min, or roughly 35 bytes per second downstream.

In the end, you're looking at less than 0.5 KB/s for both downstream and upstream communications.  Since backup pools will be receiving jobs as well, add maybe 0.1 KB/s downstream per connected backup pool.
sr. member
Activity: 467
Merit: 250
New version - 2.8.0, 7th October 2012

This version is a massive upgrade compared to the stable 2.7 releases and involves over 1000 lines of changed code and a lot of new work implementing the stratum protocol for pooled mining, which is currently only implemented on slush's and btcguild pools. With such a massive change, breakage is likely but there have been numerous people who have graciously tested along the way paving the way for a much more bug tested release to the public.




In the case of X-Stratum headers, what's the behavior on failure of the "promoted" stratum pool? Does it fail back to "demoted" pool, or mark the entire "pool" dead and fall down to the next one?

sr. member
Activity: 383
Merit: 250
can't compile with opencl support under windows!
Result after
Code:
CFLAGS="-O2 -msse2" ./configure
Code:
Configuration Options Summary:

  curses.TUI...........: FOUND: pdcurses
  OpenCL...............: NOT FOUND. GPU mining support DISABLED
configure: error: No mining configured in
folder ADL_SDK contains:
adl_defines.h
adl_sdk.h
adl_structures.h


that is for ADL support, now go back and re-read the windows-build.txt file

**************************************************************************************
* Install AMD APP SDK, latest version (only if you want GPU mining)                  *
**************************************************************************************
Note: You do not need to install the AMD APP SDK if you are only using Nvidia GPU's
Go to this url for the latest AMD APP SDK:
 http://developer.amd.com/sdks/AMDAPPSDK/downloads/Pages/default.aspx
Go to this url for legacy AMD APP SDK's:
 http://developer.amd.com/sdks/AMDAPPSDK/downloads/pages/AMDAPPSDKDownloadArchive.aspx
Download and install whichever version you like best.
Copy the folders in \Program Files (x86)\AMD APP\include to \MinGW\include
Copy \Program Files (x86)\AMD APP\lib\x86\libOpenCL.a to \MinGW\lib
Note: If you are on a 32 bit version of windows "Program Files (x86)" will be
"Program Files".
Note2: If you update your APP SDK later you might want to recopy the above files
member
Activity: 125
Merit: 10
can't compile with opencl support under windows!
Result after
Code:
CFLAGS="-O2 -msse2" ./configure
Code:
Configuration Options Summary:

  curses.TUI...........: FOUND: pdcurses
  OpenCL...............: NOT FOUND. GPU mining support DISABLED
configure: error: No mining configured in
folder ADL_SDK contains:
adl_defines.h
adl_sdk.h
adl_structures.h
hero member
Activity: 896
Merit: 1000
Hi,

I just got a Sapphire 5970 4GB with a very odd behavior. I already own another of the very same model (verified the SKUs) which works perfectly.
This Sapphire refuses to report its fan speed with aticonfig
Code:
--pplib-cmd "get fanspeed 0"
.

I usually don't use aticonfig, but for an odd reason, cgminer's TUI freezed with this card and a "-D" run reported a problem getting the fan speed.

I tried to disable everything linked to the fanspeed, leaving voltage/engine/memory controls in my cgminer config. These controls work (aticonfig at least reports engine and memory speed being adjusted). But the freeze still happens: the log is displayed for a while below the status lines at the top of the curses interface, but it eventually stops and the status lines remain empty (which from my experience should only be a temporary state for a couple of seconds when I start cgminer).

The only way to make cgminer work properly with this card is to disable ADL entirely at configure time and recompiling it. Removing all command-line parameters linked to ADL isn't enough.

I suspect that with ADL compiled in, cgminer attempts to read the fan speed for the status lines and is somehow blocked here, interfering with the other threads: some shares are submitted but probably at between 10 and 30% of the expected rate (didn't let it mine in these conditions long enough to have a good estimation) and often with large delays (I mine on p2pool and it rejects most of them as stale).

I'm tempted to dig in the cgminer sources to find out if there's a way to disable fan speed reading for a subset of GPUs, but if anyone has suggestions, I'll happily take them!
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
cut/paste ...

2.8.0
An Xubuntu 11.04 x86_64 executable is in my github downloads called cgminer-2.8.0a
https://github.com/kanoi/cgminer/downloads
(it also works on Fedora 16 and 17)

For anyone who didn't realise, it's just the executable file to put in place of 'cgminer'
Nothing else needs changing
First get and extract the full binary release from ckolivas and then copy my file in place of 'cgminer'

No problems so far on my BFL or my '2xGPU+2xIcarus' (for half an hour) on normal pools

The same configure options as cvolivas' binary version
In case anyone was wondering:
CFLAGS="-O2 -W -Wall" ./autogen.sh --enable-icarus --enable-bitforce --enable-ztex --enable-modminer --enable-scrypt
make clean
make
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
Any stats on how much kb/s would be transferred to a pool for a generic hashrate of like 60Ghash/s ?
Not offhand because it depends on how often the pool sends out a notify update and whether they support variable diff targets. The difference is there is only one lot of new information sent from the pool every 30-60 seconds regardless of the hashrate of the miner which is starkly different to the standard "getwork" protocol. If you tie that into variable diff shares, then you can theoretically make the network transfer volume STATIC regardless of the miner's hashrate with stratum.
420
hero member
Activity: 756
Merit: 500
Any stats on how much kb/s would be transferred to a pool for a generic hashrate of like 60Ghash/s ?
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
New version - 2.8.0, 7th October 2012

This version is a massive upgrade compared to the stable 2.7 releases and involves over 1000 lines of changed code and a lot of new work implementing the stratum protocol for pooled mining, which is currently only implemented on slush's and btcguild pools. With such a massive change, breakage is likely but there have been numerous people who have graciously tested along the way paving the way for a much more bug tested release to the public. Basically you need to do nothing to make use of this new protocol, as cgminer will try to switch to it automatically if it detects it, you can force it by giving it the direct url of stratum+tcp://pool:port if you have the details from the pool. There were intentionally 2 releases on the say day, the 2.7.7 and 2.8.0 as 2.7.7 builds on the stability of the 2.7 series for those who just want a stable release and 2.8.0 has such massive changes that it should be considered relatively experimental.

More information about the stratum protocol can be found here:
https://bitcointalksearch.org/topic/ann-stratum-mining-protocol-asic-ready-108533


Human readable changelog:
Stratum support.


From the readme:

Q: What is stratum and how do I use it?
A: Stratum is a protocol designed for pooled mining in such a way as to
minimise the amount of network communications, yet scale to hardware of any
speed. With versions of cgminer 2.8.0+, if a pool has stratum support, cgminer
will automatically detect it and switch to the support as advertised if it can.
Stratum uses direct TCP connections to the pool and thus it will NOT currently
work through a http proxy but will work via a socks proxy if you need to use
one. If you input the stratum port directly into your configuration, or use the
special prefix "stratum+tcp://" instead of "http://", cgminer will ONLY try to
use stratum protocol mining. The advantages of stratum to the miner are no
delays in getting more work for the miner, less rejects across block changes,
and far less network communications for the same amount of mining hashrate.


Full changelog

- Major upgrade - support for the stratum mining protocol.
- Fix various modminer warnings on mingw.
- Fix sign warning on windows build for bitforce.
- Cast socketfail to integer since SOCKET is an unsigned int on windows.
- Use strtod not strtol for bitforce temp backup.
- Cope with broken drivers returning nonsense values for bitforce temperatures.
- Minor warning fixes.
- Use the stratum thread to detect when a stratum pool has died based on no
message for 2 minutes.
- Only set the stratum auth flag once and once the stratum thread is started,
use that to set/unset the stratum active flag.
- Only hand off to stratum from getwork if we succeed in initiating the
protocol.
- Target should only be 32 bytes copied.
- Use a static array for work submission data instead of stack memory.
- Clear the buffer data before sprinting to it.
- Clear work stratum strings before setting them and add them to debug output.
- Drop stratum connect failed message to verbose level only since it's a regular
probing message.
- TCP Keepalive in curl is only in very recent versions and not required with
regular messages on stratum anyway.
- Move stratum sockets to curl infrastructure with locking around send+recv to
begin support for proxies and ssl.
- Make detect stratum fail if a proxy has been set up.
- Stratum does not currently have any proxy support so do not try to switch to
stratum if a proxy has been specified.
- Windows doesn't work with MSG_PEEK on recv so move to a continuously updating
buffer for incoming messages.
- Alloca is unreliable on windows so use static arrays in util.c stratum code.
- Begin support for mingw stratum build.
- Add space to reject reason.
- Parse the reject reason where possible from stratum share submission.
- Pass json error value to share result function to be able to parse reject
reason in stratum.
- Don't try to parse unneeded parameters in response to mining.subscribe.
- Remove the sshare hash entry if we failed to send it.
- Change notify message to info level to avoid spamming repeatedly when a pool
is down.
- Check the stratum pool difference has not changed compared to the work diff
when testing whether a share meets the target or not and retarget if necessary.
- Bit error in target calculation for stratum.
- Set work_block in gen_stratum_work for when work is reused to avoid thinking
it's all stale.
- Offset the current block detection to the prev block hash.
- We should be testing for id_val, not id in parse stratum response.
- Make target on stratum scale to any size by clearing sequential bits according
to diff.
- Correct target calculation in gen_stratum_work.
- If a share result has an error code but still has an id, it is likely a
reject, not an error.
- Initiate stratum the first time in pool_active only, allowing us to switch to
it on getting a failed getwork and detecting the presence of stratum on the url
at that time.
- Use 5 second timeout on sock full for now as a temporary workaround.
- If no stratum url is set by the end of the detect stratum routine, copy the
sockaddr url.
- Make all buffers slightly larger to prevent overflow.
- Make the stratum recv buffer larger than the recvsize.
- Userpass needs to be copied to user and pass earlier to allow stratum
authorisation to work with it.
- Store a sockaddr url of the stripped url used in determining sockaddr to not
confuse it with the stratum url and fix build warnings.
- Decrease the queued count with stratum work once it's staged as well.
- Allow the stratum retry to initiate and auth stratum in pool_alive to make
sure the stratum thread is started.
- Avoid duplicating pool->rpc_url and setting pool->stratum_url twice to itself.
- Detect if a getwork based pool has the X-Stratum header on startup, and if so,
switch to the stratum based pool.
- Comment update.
- Minor message change.
- Create a work item from a "clean" request from stratum allowing the new block
to be detected and the appropriate block change message to be given.
- Use statically allocated stratum strings in struct work to cope with the
inability to safely deallocate dynamically allocated ram.
- Use the current pool when deciding whether to reuse work from a stratum source
rather than the work's previous pool.
- Copy the stratum url to the rpc url to avoid none being set.
- Provide locking around stratum send operations to avoid races.
- Submit shares from stratum through the abstracted submit share function
detecting what message they belong to and showing the data from the associated
work, and then deleting it from the hash.
- Use a more robust mechanism to obtain a \n terminated string over a socket.
- Abstract out share submit as a function to be useable by stratum.
- Rename parse_stratum to parse_method as it is only for stratum messages that
contain methods.
- Display stratum as mechanism in status line when current pool is running it.
- Count each stratum notify as a getwork equivalent.
- Correct nonce submitted with share.
- Extranonce2 should be added before coinbase2.
- We should be hashing the binary coinbase, not the hex one.
- Fix endianness of nonce submitted for stratum.
- Check that stratum is already active in initiate_stratum to avoid
de-authorising ourselves by subscribing again.
- Begin implementing a hash database of submissions and attempt sending results.
- Copy parameters from stratum work required for share submission.
- Set lagging flag on first adding a pool to prevent pool slow warning at
startup.
- Fix work->target being a 32 byte binary in gen_stratum_work.
- Store and display stripped url in its own variable.
- Create machinery to divert work requests to stratum.
- Generate the work target in gen_stratum_work, setting default diff to 1 in
case it is not yet set.
- Generate work data, midstate and hash1 in gen_stratum_work.
- Generate header created from stratum structures in gen_stratum_work.
- Generate merkle root hash in gen_stratum_work.
- Generate the coinbase for generation of stratum based work.
- The number of transactions is variable so make merkle a variable length
dynamically allocated array and track how many there are for stratum.
- Rename nonce2 to n2size reflecting that it's a size variable and not the
actual nonce.
- Provide rudimentary support for stratum clean work command in the stratum
thread.
- Cope with pools being removed in the stratum thread.
- Use the pool sock value directly in the stratum thread in case it changes
after reconnecting.
- Create a stratum thread per pool that has stratum that monitors the socket and
serves received data.
- Check return value of stratum_parse.
- Complete authorisation in stratum.
- Implement stratum parsing of notify parameters and storing them in the pool
stratum work structure.
- Create helper functions for duplicating json strings to avoid keeping json
references in use.
- Append \n in the sock_send function instead of adding it when constructing
json in stratum.
- Don't keep any json references around with stratum structures.
- Create parse_stratum function that hands off stratum parameters to other
functions to manage pool stratum work struct variables. Implement mining
difficulty setting.
- Create helper functions for checking when a socket is ready to read on and
receive a single line at a time. Begin stratum authorisation process.
- Provide a helper function for reading a single \n terminated string from a
socket.
- Create a stratum work structure to store current work variables.
- Test specifically for stratum being active in pool_active.
- Detect stratum in common place when adding urls, and use a bool to tell us
when it's active.
- Fix warnings.
- Extract and store various parameters on stratum init confirming successful
mining notify.
- Use existing socket macros and close the socket on failure in init stratum.
- Initiate stratum and grab first json result.
- Get detailed addressinfo from the parsed URL for future raw socket usage when
possible. IPV4 only for now.
- Prepare for getaddrinfo call.
- Add data structures to pool struct for socket communications.
- Put all socket definitions in util.h to allow reusing by added socket
functions to be used in util.c.
hero member
Activity: 518
Merit: 500
Manateeeeeeees
Running 2.7.7 after upgrading from 2.7.5, things for some reason seem to be a little hotter on my 7970s.  It might just be a fluke...
legendary
Activity: 4634
Merit: 1851
Linux since 1997 RedHat 4
cut/paste ...

2.7.7
An Xubuntu 11.04 x86_64 executable is in my github downloads called cgminer-2.7.7a
https://github.com/kanoi/cgminer/downloads
(it also works on Fedora 16 and 17)

For anyone who didn't realise, it's just the executable file to put in place of 'cgminer'
Nothing else needs changing
First get and extract the full binary release from ckolivas and then copy my file in place of 'cgminer'

No problems so far on my BFL or my '2xGPU+2xIcarus'

The same configure options as cvolivas' binary version
In case anyone was wondering:
CFLAGS="-O2 -W -Wall" ./autogen.sh --enable-icarus --enable-bitforce --enable-ztex --enable-modminer --enable-scrypt
make clean
make
Jump to: