Topic: OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.0 - page 110. (Read 5805527 times)

You're welcome

Yes Con designed all of the cglock code and wrote almost all of it.
I've had a few deadlocks I've coded in the past, so I wrote the lock tracking to be able to find them easily when I cause them.

Heh no problem Appreciate extra eyes on the code, always.

BTW if it wasn't obvious: The cg write lock variant would have grabbed the mutex successfully but be unable to grab the write lock while the other thread holds the read lock.

Oki You are the boss..
Thank you for your comments.

Best

That can't happen.

So what will happen when  if (strcmp(work->job_id, pool->swork.job_id)) and free(pool->swork.job_id); are executed simultaneously?

Actually the unfortunate thing is that cglocks are unique upgradeable read write locks, and it is normal for a write lock to be able to grab the mutex component of the cglock while a read variant is holding the read lock - but it will not be able to grab the write lock. The same rules don't quite apply as per regular locks.

Yes they are an implementation of my own, originally developed for the linux kernel scheduler code I maintain. See:
http://ck-hack.blogspot.com/2012/06/upgradeable-rwlocks-and-bfs.html

Hello Con,

I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!

Here are my findings:

First Issue
From the following code



static inline void _cg_wlock(cglock_t *lock, const char *file, const char *func, const int line)
{
   _mutex_lock(&lock->mutex, file, func, line);
   _wr_lock(&lock->rwlock, file, func, line);
}

static inline void _cg_rlock(cglock_t *lock, const char *file, const char *func, const int line)
{
   _mutex_lock(&lock->mutex, file, func, line);
   _rd_lock(&lock->rwlock, file, func, line);
   _mutex_unlock_noyield(&lock->mutex, file, func, line);
}

It seems that when a thread acquires a read lock (_cg_rlock) another thread can acquire a write lock (_cg_wlock)

Please take a look at the flowing peace of code

stale_work(struct work *work, bool share)   
.....
   cg_rlock(&pool->data_lock);
      if (strcmp(work->job_id, pool->swork.job_id))
         same_job = false;
      cg_runlock(&pool->data_lock)

It may turn that while we are performing strcmp
static bool parse_notify(struct pool *pool, json_t *val)
.......

   cg_wlock(&pool->data_lock);
   free(pool->swork.job_id);
   pool->swork.job_id = job_id;

frees pool->swork.job_id

Resulting in flowing line in BLOCKED gets (9) id=248038834 by cgminer.c stale_work()....
I have changed stale_work
      cg_rlock(&pool->data_lock);
      if (strcmp(work->job_id, pool->swork.job_id))
         same_job = false;
      cg_runlock(&pool->data_lock);
to
    cg_wlock(&pool->data_lock);
   
      if (strcmp(work->job_id, pool->swork.job_id))
         same_job = false;
      cg_wunlock(&pool->data_lock);
I have moved

gen_stratum_work.....

work->job_id = strdup(pool->swork.job_id); under
cg_wlock(&pool->data_lock);
Second:

   cg_wlock(&control_lock);
   local_work++;
   work->id = total_work++;
   cg_wunlock(&control_lock);

I do think that everywhere total_work++; and local_work++ shod be changed under &control_lock. This is not happening currently. I am digging to find all places to do it

New version: 4.2.1, 24th March 2014

Bugfixes.


Human readable changelog:

- Bitcoind did not like lots of persistent connections at once meaning it would fall over if more than one cgminer instance was trying to mine at the same time from the one bitcoind instance. Cgminer now opens and closes the connection every time it talks to bitcoind allowing any number of cgminer instances to mine solo from the one instance. Confirmed working with 200TH of miners aimed at the one bitcoind...
- Big endian hosts (like the antminer S1, avalon) did not work with solo mining.
- Solo mining setups will not mine unless a btc address is specified now, and the address is displayed on startup if it exists.
- Solo mining disconnections to bitcoind are handled better, not spawning more polling threads every failure.
- Low level optimisations for solo mining
- AntminerS1 fixes to decrease CPU usage and actually honour overheat conditions.
- Network diff when submitting a block is shown correctly on screen when >2billion.
- Build fixes for avalon2
- miner.php improvements


Full changelog:

- Fix various ava2 build issues generically
- Minimise the amount of heap memory allocations/frees when submitting gbt
shares.
- Make varint in gbt submission a stack object.
- Fix big endian problems with gbt submissions.
- Fix 32bit overflow on relative diff shown.
- ants1 - stop results read hard looping
- ants1 - slow down mining if overheat occurs
- miner.php allow gen before (bgen) and after (gen) grouping
- Change default solo mining to failing when no btc address is specified.
- Use upgrade cglock variants in get_gbt_curl
- Provide a cg_uilock to unlock the intermediate variant of cglocks.
- Use the one curl instance for all gbt solo operations, protecting its use with
a bool set under gbt lock.
- Only start block detection with gbt solo if setup succeeded
- One less block detection message
- Toss out the curl handle after each solo poll
- Don't reuse any curl handles for solo mining and break out of the lp thread if
the pool is removed.
- Make sure to only start the lognpoll thread once on gbt solo.
- Don't keep RPC connections open for solo mining since bitcoind doesn't like
having many persistent connections.
- GBT solo pools should be considered localgen pools.
- miner.php - speed up formatting and allow calc on gen fields
- Always show the address we're solo mining to to avoid confusion for when no
address is set.

Glad to help, was an easy tweak to grep out "lastsharetime" recursively down the lua/luci dir chain and work back from there. Like I said, I am new to all this (mining) and have 3 Antminer S1's running kano's great stuff (thanks mucho kano, i will be sending some coin your way), running real stable at low temps 200GH/s with nice low temps, but I am also a hacker at heart and these little rigs are proving fun for the interface side and cgminer side. This little test seems to be earning it's keep so-far (initial hardware costs and recurring electricity cost vs the payout I'm seeing)... but we'll see at the next difficulty increase... which is all too quickly approaching ... and the erratic bit-coin pricing. Anyway, long term investment, we'll see how it works out.

My "day job" these past 12 years has been writing enterprise software (shop floor controls, low level machine control for component placement, health system logs to pull/burn onto ROMS/B2B,ERP, web reporting & eCommerce) all day long for a PCBA fabrication facility, systems integration factory, engineering and program management site, our eCommerce group, and an a couple of RMA facilities (1 man "manager of 1", waaaaayyyyyy many hats)..... with a wife who designs the boards in the engineering wing of the same company (won't mention any names, but we make just about some percentage everyone's retail stuff they sell like servers, PC's, phones, etc... , they just have us design it and then re-brand or we build on their specs as a CM).... if anyone ever wants to design and build a "better ant", maybe we can figure something out and come up with a team... pretty sure I can do NPI's and get PCB's (since we own a PCB fab as well) without any hassles Weekdays suck for comms, but nights and weekends are usually free to mess around in my hardware lab all day.

Thank you kano for the S1 binary, running successfully on two S1s and using your luci pool password hack to pass temp params to cgminer.  Thank you artpego for the last share time fix, also working on both ants.  Thank you ckolivas for cgminer 4.2.0 working on win7 with 12 BE's and the Avalon 4.2.0 image running on a Mini.  The Mini is running at 59 gh/s clocked at 400, turns out miner 15 has failed, but 1-14 and 16 are working.  It might be a loose connection, but I'm not sure it's worth attempting to disassemble the Mini to check.   

Edit: there's a later 4.3.2a update here: https://bitcointalksearch.org/topic/m.6533537

Here's another new AntS1 binary.

An endian issue was found in the solo code for the ant (no issue for desktop computers), so this update includes the fix for that.
I've removed both old AntS1 binaries

This uses the same patch as before for the luci display.
My source for this version is here:
https://github.com/kanoi/cgminer/tree/ants1-4.2.0-00567a4

The binary and README are in my cgminer-binaries git here:
https://github.com/kanoi/cgminer-binaries/tree/master/AntS1  <-- Follow this link to get the new binary

Read the README on the screen there for how to replace the /usr/bin/cgminer binary in your AntS1 with the new one.

This binary includes all cgminer changes up to 4.2.0 and forward after that up to:
https://github.com/ckolivas/cgminer/commit/cec48cee5f1759e5996de5339537fab4aeff416c

The previous report of having trouble with running the binary turned out to be caused by incorrectly specifying cgminer options.

I do have two different machines I'm mining with and 4.2 seems to be working pretty good on one.  But the other machine if when I try to run any 4.x version a bunch of AMU's get turned off and mining is very erratic.  It's a low end P3 with 512MB RAM but it works fine with CGMiner versions 3.11 and earlier.  Currently running 3.9 with 31 BE's.

Would this include the connections to the Bitcoin network?  My Bitcoin client currently has 70 connections.  Should I be restricting the max connections in the client itself?

Sam

If there is going to be an update soon could I ask for

1.  Bitcoin Address displayed to verify which address the block reward would go to?  I saw that you said you would add this.

2.  Ability to sign the block?

I'm pretty happy with my testing so far.  But the memory utilization is still climbing, 408MB so far.
Thanks,
Sam

CKOlvias its wierd that at least for the moment under both conditions they were all detected on this turn. Whatever the deal they are almost imediatly in both situations being piped to the end of the list

I got 1 anu listed device showing its only had 2 sharse to work on and zero hardware errors and one with 146 with 2 errors that just zombied with the overclock its showed about 7 minutes in the zombie. They're both mostly doing the normal blinking but no continous solid green light. When I see a light go on I cant corespond it to any errors on the screen.

In verbatim I think when one of them goes solid for a moment its getting discarded work. The zombie one is still in a work state(not solid green) but doing nothing.

All I know for certain is I got nothing from them when I place them in my aitech 3.0 hub with 5 anus and a drill bit thumb and I've got about the same on my roswill 2.0 device. Each have 10 ports and 4 amps to spare.

Well he was mining with almost 200TH (aimed at just one bitcoind!) confirming nicely the scaleability of cgminer's solo mining. And here's another just for good measure:
https://blockchain.info/tx/1f59b91615ea0f9e6c633f7c426daa367cde119b03a36ad235def2236b9c0f7d

There are a number of small improvements/fixes going into the solo mining code as a consequence of this testing which should be wrapped up into a minor bugfix release soon. There is one showstopper that makes solo mining not work properly on big endian machines (like antminers) which warrants this release. The other limitation is that bitcoind does not like a lot of persistent connections at the same time so I have reworked the code to drop connections as much as possible allowing multiple cgminer instances to connect to the one bitcoind.

They're all seen there so it's not a usb recognising issue. Amuse me and run them at default clock without specifying 250 since you are then overclocking them compared to the default.

I think at least the u1's were 2 watts each and that the eroupters I replaced the u2's with are 2 watts so I think I should be fine. I have all set at 250.

Here the devices. When I run it there should be 8 anu's and 3 amu's among other devices and an unsupported tecnobit device(*yes theres an unsupported patch but compiling has been a pain so far).

The new binary works great!

I got it back to a normal appearance rather than just the UNIX timestamp.

vi /usr/lib/lua/luci/controller/cgminer.lua

Find the part where it says: "--lst_date = os.date("%c", lst)" and get rid of comment (the --).

Had to do a reboot to get the change to show, buy you guys here probably know a better solution to reload luci/lua/whatever it is. I'm very new to all of this so hope it was somewhat helpful.

From:
         if lst == "0" then
            lst_date = "Never"
         else
            --lst_date = os.date("%c", lst)
           lst_date = lst
         end

To:

         if lst == "0" then
            lst_date = "Never"
         else
            lst_date = os.date("%c", lst)
            --lst_date = lst
         end

Heh, well that just tells us the first node that relayed the transaction, not where it originated from. I'm pretty sure it was actually from North America.