SOLVED [Bitcoins.lc] Issue with invalid shares (10 BTC bounty) - page 2.

maxcorrads

newbie

Activity: 42

Merit: 0

Quote from: Jine on June 10, 2011, 06:19:18 PM

I'm planning to keep that to me self, due to i think the other pools based on pushpoold will eventually face the same problem.
This issue costed the pool/me personally 10 BTC + a whole day of hard work with debugging, tcpdump and ~10 people that tried in all ways to help us.

If I'm about to release this, the pool need something in return.
We'll see if this is just an isolated issue with our pool, or if anyone else will face the same problem.

I agree with you

Dusty

hero member

Activity: 731

Merit: 503

Libertas a calumnia

The server is up but the pool is down now...

d3wo

member

Activity: 69

Merit: 10

Kupo!

another server attack ?

Chick

member

Activity: 70

Merit: 10

Quote from: WiseOldOwl on June 11, 2011, 03:45:23 AM

Server is down no website, connection problems mining.

D:

WiseOldOwl

full member

Activity: 238

Merit: 100

Server is down no website, connection problems mining.

rb1205

hero member

Activity: 797

Merit: 1017

Quote from: ?? on ??

If there's a bug with pushpool (which is what it sounded like from IRC) then you should let it be known. The developer of that was good enough to let you use it freely... share the love.

edit: Also it'll save me a weekend of serious debugging. If I track it down myself I won't be keeping secrets!

+1

If it's about pushpool, then you should tell them.

Jine

sr. member

Activity: 403

Merit: 250

I'm planning to keep that to me self, due to i think the other pools based on pushpoold will eventually face the same problem.
This issue costed the pool/me personally 10 BTC + a whole day of hard work with debugging, tcpdump and ~10 people that tried in all ways to help us.

If I'm about to release this, the pool need something in return.
We'll see if this is just an isolated issue with our pool, or if anyone else will face the same problem.

Jobbernowl

member

Activity: 222

Merit: 12

What was the problem?

Jine

sr. member

Activity: 403

Merit: 250

We finally solved our issue with rejected shares!
Credits to ius over at #bitcoins.lc for helping us out.

His account was just credited with 10 BTC ($270)
Thanks all for trying to help us! We really appreciate it!

If any other pool operator want to know what it is, please contact me and we can work something out...

lizthegrey

newbie

Activity: 56

Merit: 0

I'm chasing a hunch. If anyone currently using bitcoins.lc would like to compare results from disabling Long Polling vs. leaving LP on, I'd find it instructive.

Jine

sr. member

Activity: 403

Merit: 250

#bitcoins.lc at irc.quakenet.org or use the webchat:
http://www.bitcoins.lc/chat

/ Jim

kjj

legendary

Activity: 1302

Merit: 1026

Where the hell is your IRC channel? I've been lurking in what I thought was the channel all day and haven't seen a peep.

Jine

sr. member

Activity: 403

Merit: 250

The bounty have not been claimed yet.
Still looking for a solution(!)

A few people over at our IRC-channel is working HARD to figure out the problem.

Quote from: ?? on ??

I think a packet sniffer would also tell us the answer to this, and wouldn't require a restart of anything?

We tcpdumped all bitcoind->pushpoold comminucation and founded 4 duplicate getwork's.
What we should do about it, or why it's that way - we don't know.

kjj

legendary

Activity: 1302

Merit: 1026

I think you are going to need to restart pushpoold with full logging on, and hope that you get enough information from the logs.

Right now, I can't tell if the problem is in get_work handing out the same work to many people, or if the problem is in check_hash (or hist_lookup) finding improper duplicates. (both are in msg.c)

The answer is in the history elist, but I don't think there is any way to interrogate it while running. Looks like the logs might show enough info.

Jine

sr. member

Activity: 403

Merit: 250

Quote from: SomeoneWeird on June 10, 2011, 09:12:39 AM

What are the specs of the computer it's running on (including bandwidth), too little ram could potentially cause shares to not be accepted.

Enough for both DB and MySQL.

Quote

jine@bitcoins:~$ free -m
total used free shared buffers cached
Mem: 2022 1372 649 0 105 611
-/+ buffers/cache: 655 1366
Swap: 234 4 230

Bandwidth is redundant 100mbit (200+200mbit full duplex)
CPU for main VM is 8 total cores of two Xeon E5504

Slasklitta

newbie

Activity: 53

Merit: 0

Quote from: Jine on June 10, 2011, 07:21:59 AM

Worker's doing duplicated work.

See screenshot below.

It's from all workers, all over the pool.
Duplicated work, and we really can't figure out why or what to do about it.

I can see my IP Cheesy

kjj

legendary

Activity: 1302

Merit: 1026

Has nothing to do with the midstate. The midstate is calculated on the first 512 bits, which is identical in the two examples given in IRC. The difference is in the timestamps, which is in the second half and will not change the midstate at all.

Also, the extraNonce is in the generation transaction. The mining client never sees it and can't change it.

SomeoneWeird

hero member

Activity: 700

Merit: 500

Quote from: Jine on June 10, 2011, 09:05:53 AM

bitcoind isn't threaded AFAIK (yet)

Some more data:

14:56:31 <@jine> 28 133 total - SELECT * FROM `shares` WHERE `reason` = 'unknown-work'
14:56:54 <@jine> 193 114 total - SELECT * FROM `shares` WHERE `reason` = 'stale'
14:57:40 <@jine> 239 898 total - SELECT * FROM `shares` WHERE `reason` = 'duplicate'
14:58:18 <@jine> 10 424 570 total - SELECT * FROM `shares` WHERE `reason` IS NULL

SELECT * FROM shares WHERE solution ="*one of the duplicated solutions*"
Returns: http://jine.be/2

Example shares - query: SELECT * FROM `shares` WHERE `reason` IS NOT NULL LIMIT 465990 , 30
Returns: http://bitcoins.lc/_files/shares.csv

Jobbernowl:
No, but it only caches authentication AFAIK - not shares.

SomeoneWeird:
We were running stable of both versions before, when the problem started occouring.
Upgraded and restarted both pushpoold and bitcoind without any change.

What are the specs of the computer it's running on (including bandwidth), too little ram could potentially cause shares to not be accepted.

Jine

sr. member

Activity: 403

Merit: 250

bitcoind isn't threaded AFAIK (yet)

Some more data:

14:56:31 <@jine> 28 133 total - SELECT * FROM `shares` WHERE `reason` = 'unknown-work'
14:56:54 <@jine> 193 114 total - SELECT * FROM `shares` WHERE `reason` = 'stale'
14:57:40 <@jine> 239 898 total - SELECT * FROM `shares` WHERE `reason` = 'duplicate'
14:58:18 <@jine> 10 424 570 total - SELECT * FROM `shares` WHERE `reason` IS NULL

SELECT * FROM shares WHERE solution ="*one of the duplicated solutions*"
Returns: http://jine.be/2

Example shares - query: SELECT * FROM `shares` WHERE `reason` IS NOT NULL LIMIT 465990 , 30
Returns: http://bitcoins.lc/_files/shares.csv

Jobbernowl:
No, but it only caches authentication AFAIK - not shares.

SomeoneWeird:
We were running stable of both versions before, when the problem started occouring.
Upgraded and restarted both pushpoold and bitcoind without any change.

Jobbernowl

member

Activity: 222

Merit: 12

I might be completely wrong here, but it like there could be some threading issues with bitcoind when using long polling.

edit: Never mind, answered my own question, before the next one is sent to it... duh.

Topic: SOLVED [Bitcoins.lc] Issue with invalid shares (10 BTC bounty) - page 2. (Read 16718 times)