Topic: Flexible mining proxy - page 15. (Read 88812 times)

One more that I meant to send:

mysql> select * from work_data where data = '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa';
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
| worker_id | pool_id | data                                                                                                                                                     | time_requested      |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
|         1 |       1 | 000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa | 2011-05-07 06:21:42 |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
1 row in set (0.00 sec)

mysql>

That script returns "No rows found."  Preemptively I'm going to send you the contents of tables "worker", "worker_pool" and "pool" with passwords removed, since it looks like that's the information you're looking for with the aforementioned php script:

mysql> select * from worker;
+----+-------+-----------+
| id | name  | password  |
+----+-------+-----------+
|  1 | w0113 | xxxxx |
+----+-------+-----------+
1 row in set (0.00 sec)

mysql> select * from worker_pool
    -> ;
+---------+-----------+------------------------+---------------+----------+---------+
| pool_id | worker_id | pool_username          | pool_password | priority | enabled |
+---------+-----------+------------------------+---------------+----------+---------+
|       1 |         1 | [email protected]_10 | xxxxxxxxxxx     |        1 |       1 |
+---------+-----------+------------------------+---------------+----------+---------+
1 row in set (0.00 sec)

mysql> select * from pool;
+----+---------+-------------------------+---------+
| id | name    | url                     | enabled |
+----+---------+-------------------------+---------+
|  1 | deepbit | http://deepbit.net:8332 |       1 |
+----+---------+-------------------------+---------+
1 row in set (0.00 sec)

Based on my lay knowledge of php, I can't imagine why it doesn't return anything from the script you sent.  Is it possible that my installation (default php 5.1 from CentOS 5.x with php-* installed, mysql-* installed, and the add-in json libraries) is different, misconfigured, or missing a dependency?  For reference, it already works fine with other popular MySQL/PHP off the shelf applications.

Also sent you 5 BTC to the address in your signature for all the work you're putting in; this is in addition to my previous commitment to send you 10 when it appears that this works and is viable.  Thanks again for your help.

Hmm, weird.  Can you put the contents of this script in a PHP file alongside index.php (in other words, directly in the web root) and paste me its output?  Of course, sanitize the username and password if they get displayed.

Yes, it's in there.

mysql> SELECT * FROM `work_data` WHERE `data` = '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa';
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
| worker_id | pool_id | data                                                                                                                                                     | time_requested      |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
|         1 |       1 | 000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa | 2011-05-07 06:21:42 |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
1 row in set (0.00 sec)

mysql>

Hmm.  Can you run the query I gave and see what it returns?  Also, you shouldn't need to clone anything again -- just git pull from inside the git root.

My work_data table had 192 entries.  I cloned the tree again and replaced the contents of htdocs, copying over the same config file, and the error is still the same on the client side.  Let me know if there is anything else I can do to help troubleshoot.

Perfect, that's exactly the data I needed.

The error you are getting back from the work submission should indeed not be happening; the submitted work does match one of the getwork requests.  Can you execute this query against your database and see if it gets any results?  If it doesn't, can you verify that your work_data table has any records in it at all?


Edit: I just checked the code and if the INSERT into the work_data table fails, it will do so silently, still returning the work to the client.  I need to patch this, since the client shouldn't be given work that can't be registered, or the proxy won't know which pool to route it too (I think this is the problem you are seeing).  We still need to figure out why the INSERT is failing though.

Edit 2: I think I found the problem with work submissions.  Please update to the latest code from Git.

Here is a complete exchange including both sides from tcpdump.  The IPs were masked with "xxx" on the first 3 octets as you'll see:

Hmm.  Can you post the replies to the previous getworks too?  Somehow the work either isn't getting saved in the database correctly, or the worker isn't sending the work back correctly, or something.  I'd have to see the getwork responses and the work submission requests to diagnose this further.

Thanks for the reply, I've sanitized things further in the last post, even though the worker password is of no consequence for me.  Here is the work being sent and the corresponding error:

POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": ["0000000129d09b9af4d37ae0641aeaf3618fb62692477035e4f4a17300009697000000004a84ffa 3bb5f69b95dd4656b1267f99f0675fca59a2899b31a07f3eb31992d8e4dc4a7561b0098fa7958ce a500000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000"], "id": 1}

Date: Sat, 07 May 2011 01:59:14 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=6r9lc3ejja26ok9fv5ve84dc44; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 66
Connection: close
Content-Type: application/json-rpc

{"error":"Work not found in proxy database.","result":null,"id":1}

Yes, better validation and logging are on the cards.  I was more concerned with getting the code ready for a proper release than adding features, so this should be coming pretty soon.

Can you post some more logs, but include the replies from the proxy?  It's hard to diagnose problems when I can only see one half of the conversation.

If I were you I would sanitize these headers too; your username and password are exposed to anyone who understands HTTP basic authentication.

Yes, this is a long polling request.

It's possible that your client is pre-fetching the next chunk of work so that when it's done with the current chunk it can move on to the next right away.

This is where response logs from the proxy would be most helpful, since I can't really tell why it's failing without knowing what the proxy is saying in response.

Thanks for the replies.  I should start by saying that I think those errors I was seeing in the errorlog were related to my incorrect entering of the parameters for deepbit (username/password, and the port number).  Although I did have it entered in the correct format as you mentioned.  In a future version you might want to do some more extensive error handling to let the user know in the admin panel that the "last result" from the pool is that it can't connect because of x or y, and the actual error sent back from the pool was z.

I sniffed out the communication going from Phoenix to the proxy only, and this is what it looks like (including timestamps above):

1) Initial connection:

19:14:20.136363
POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}

2) One more (not sure why this was necessary):

19:14:20.693905
POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}

3) Another for long polling?:

19:14:20.717374
GET /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1 HTTP/1.1
Connection: close
Host: bitproxy.xxx.com
Authorization: Basic xxx
User-Agent: phoenix/v1.2

4) And one more.. not sure why this was necessary, the client didn't say it had seen a new block:

19:14:21.213894
POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}

5) Ahh now we're getting somewhere....14 seconds later when we have some work done:

19:14:35.187166
POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": ["00000001856410222014682e1d4361839ddae0adf1f7415584a4e6d8000006d20000000086ed4af 00f553972b297b1cdbd24d6b99e4720df26ff41229dbd2384c1d0cab44dc480c81b0098fa35cafe bd00000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000"], "id": 1}

At this point, the client disconnects from the proxy and throws the following in the client:

[06/05/2011 19:15:02] Disconnected from server
[06/05/2011 19:15:02] Result: a8b4a195 rejected

Then immediately reconnects to the server:

[06/05/2011 19:15:03] Connected to server

And sends the same message as in 1) above.  I should also note that the apache logs are consistent with the above and I do not see any signs of an error related to these http transactions in the error log.  Hope that helps.

Yeah, there's definitely a blank line being output prior to the XML header.  I don't see the same behavior on my install, in fact the dashboard validates as XHTML strict (as should every other page, but I just checked the dashboard right now and it validates).  I can't find anywhere in the code where the blank line would be getting written.

This notice is expected.  PHP likes to whine if you access array keys that aren't defined, but it's impractical to always check if they're set first. 

This is a bit troubling, as $http_response_header is a reserved variable.  Which version of PHP are you using?  Did you make sure to enter "http://" at the beginning of each pool URL?  (Something like "deepbit.net:8332" will not work; you need "http://deepbit.net:8332".)  You may also get this error if the HTTP host can't be reached at all, for example due to a DNS lookup failure.

This is related to the previous issue; fixing it should make this one go away.

Yes, this is PHP being picky again.  You can ignore these. 

MySQL does not have a real boolean type; BOOLEAN is an alias for TINYINT(1).

Hmm, this is almost certainly a bug.  Can you edit other pools, or is it just this one?  Does anything show up in the Apache logs when you hit this page?

This is definitely weird.  I use poclbm without any issues, so you should be able to use it too.  I'm mostly worried about the rejected shares in the first example.  Were all shares rejected or just those two?

Using a sniffer might be a good idea, since it's hard to tell just from these logs exactly what's going wrong.

So insomnia set in, and I decided to dig deeper.  I found a couple more things:

1) In your database schema, in the worker_pool table, the type for the 'enabled' field is a tinyint and should be boolean it looks like, unless you have some special state info to store that's not 0 or 1?
2) I discovered #1 while attempting to go back and edit the workers and pool info that I'd setup the first time, in an effort to debug what was happening with the behavior I reported above.  While using the various "edit" dialogs, I'm hitting blank pages being returned, i.e. in the "edit pool" dialog at http://xxx/admin/pool.php?id=1&action=edit - it returns nothing in View Source as well. 

Having said that, and after editing the databases by hand (because it looks like I had the port number to the pool AND my login credentials entered incorrectly), I got further with the miner now (I'm guessing before it would not connect because the credentials at the pool were incorrect.  Here's what happens now: 

[06/05/2011 03:18:17] Connected to server
[06/05/2011 03:18:23] Disconnected from server
[06/05/2011 03:18:23] Result: 5b433c62 rejected
[06/05/2011 03:18:24] Connected to server
[06/05/2011 03:18:26] Disconnected from server
[06/05/2011 03:18:26] Result: 9e17d0db rejected
[06/05/2011 03:18:27] Connected to server
[06/05/2011 03:18:31] Disconnected from server

My first instinct was to turn on keepalives in the webserver, I did so and set them to allow an unlimited number with a 30 second timeout, but it only prolonged the generation of the same errors by 10 or 20 seconds.  In the end it still disconnected my client from the server.  I tried m0mchil's miner with similar results:

06/05/2011 03:25:22, Unexpected error:
Traceback (most recent call last):
  File "BitcoinMiner.pyo", line 141, in mine
  File "BitcoinMiner.pyo", line 168, in sendResult
  File "BitcoinMiner.pyo", line 177, in getwork
  File "BitcoinMiner.pyo", line 204, in request
TypeError: string indices must be integers
248779 khash/s

This after running 10 to 20 seconds also.  Not seeing anything in the logs that would give me any indication of the problem.  Without cranking up a sniffer, I'm out of ideas; any suggestions?

Sweet!  Maybe you can advise on the following.  I broke down and decided to stay up past my bedtime to pull your code and put it online.  I took your advice and put this on a separate vhost in apache, because my mining client (phoenix) would not hit http://domain.com/some/big/url as you pointed out.  I threw this on a CentOS box that already has working php/mysql sites on it, so any dependencies other than those two are hopefully met already.  When I connect Phoenix, it tells me 'failed to connect, retrying'.  When I examine my apache error log, I see this:

[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Notice:  Undefined index:  lpurl in /home/rrb/bitproxy/index.php on line 70
[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Notice:  Undefined variable: http_response_header in /home/rrb/bitproxy/common.inc.php on line 117
[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Warning:  Invalid argument supplied for foreach() in /home/rrb/bitproxy/index.php on line 215

Additionally, you might find this of interest.  When I hit the admin url, the admin interface seems to work fine but merely hitting /admin/ throws the following into the apache error log:

[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  format in /home/rrb/bitproxy/mvc.inc.php on line 38
[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  tempdata in /home/rrb/bitproxy/common.inc.php on line 177
[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  tempdata in /home/rrb/bitproxy/common.inc.php on line 177

(IPs redacted).  I'm pretty sure I followed your instructions to the letter, but have not yet taken a look at the code to try and debug this (maybe tomorrow).  Any ideas?

i only get an xml-parsing error when trying to access /admin/

guess it's just a blank line hiding somewhere, but i can't find it.

He is serious about this, worked with me on and off for a good hour helping me figure stuff out step by step.  Already made a small donation, well worth it.

Cool, thanks!  Hope the software works for you.

If you run into any trouble, let me know.  I intend to support the software I publish.  (That goes for everyone.)

Thanks for the release!  I'll dedicate some time to pulling and installing it tomorrow, and pass that money along once it's up.  After all, a promise is a promise   Thanks again.