Code:
File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 187, in loop
rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'
rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'
Cheers,
Topic: New pure-python CPU miner, for fun and testing - page 2. (Read 25535 times)
Pyminer takes as its single argument the name of the config file. Please see message #10 in this thread.
Cheers,
Are you guys getting near triple efficiency with this vs over poclbm?
Definitely not, I get about 900 KHash per core on poclbm and only 182 KHash per core on pure-python.
I'm trying to run this and getting an exception:
It seems that the settings dictionary is empty, because when we're using separate processes the children don't have access to the parent's variables. Probably the settings should be passed in as constructor arguments. I might put this up on GitHub and hack on it a bit if you don't mind
Code:
Process Process-1:
Traceback (most recent call last):
File "C:\Python26\lib\multiprocessing\process.py", line 232, in _bootstrap
self.run()
File "C:\Python26\lib\multiprocessing\process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 197, in miner_thread
miner.loop()
File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 187, in loop
rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'
1 mining threads started
Sun Feb 27 15:51:18 2011 Miner Starts - 127.0.0.1:8332
Sun Feb 27 15:51:18 2011 Miner Stops - 127.0.0.1:8332
Traceback (most recent call last):
File "C:\Python26\lib\multiprocessing\process.py", line 232, in _bootstrap
self.run()
File "C:\Python26\lib\multiprocessing\process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 197, in miner_thread
miner.loop()
File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 187, in loop
rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'
1 mining threads started
Sun Feb 27 15:51:18 2011 Miner Starts - 127.0.0.1:8332
Sun Feb 27 15:51:18 2011 Miner Stops - 127.0.0.1:8332
It seems that the settings dictionary is empty, because when we're using separate processes the children don't have access to the parent's variables. Probably the settings should be passed in as constructor arguments. I might put this up on GitHub and hack on it a bit if you don't mind
Are you guys getting near triple efficiency with this vs over poclbm?
Psyco should speed things up.
Psyco only runs on “obsolete” versions of Python (and it doesn't work on any 64-bit systems at all).Cheers,
Psyco should speed things up.
No, just the URL.
Okay, no pull-request for you, then. 
Please consider the following patch for inclusion:
Code:
--- pyminer.py 2011-02-19 19:05:29.794843964 +0100
+++ pyminer-opt.py 2011-02-19 19:20:06.136093826 +0100
@@ -113,28 +113,22 @@
targetbin_str = targetbin.encode('hex')
target = long(targetbin_str, 16)
- hashes_done = 1
- for nonce in range(MAX_NONCE):
+ for nonce in xrange(MAX_NONCE):
# encode 32-bit nonce value
nonce_bin = struct.pack("
# hash final 4b, the nonce value
- hash1_o = hashlib.sha256()
- hash1_o.update(blk_hdr)
+ hash1_o = hashlib.sha256(blk_hdr)
hash1_o.update(nonce_bin)
hash1 = hash1_o.digest()
# sha256 hash of sha256 hash
- hash_o = hashlib.sha256()
- hash_o.update(hash1)
+ hash_o = hashlib.sha256(hash1)
hash = hash_o.digest()
- hashes_done += 1
-
# quick test for winning solution: high 32 bits zero?
- H = struct.unpack('
+ if hash[-4:] != '\0\0\0\0':
continue
# convert binary hash to 256-bit Python long
@@ -147,12 +141,12 @@
# proof-of-work test: hash < target
if l < target:
print time.asctime(), "PROOF-OF-WORK found: %064x" % (l,)
- return (hashes_done,
+ return (nonce+1,
static_data[:76] + nonce_bin)
else:
print time.asctime(), "PROOF-OF-WORK false positive %064x" % (l,)
- return (hashes_done, None)
+ return (nonce+1, None)
def iterate(self, rpc):
work = rpc.getwork()
+++ pyminer-opt.py 2011-02-19 19:20:06.136093826 +0100
@@ -113,28 +113,22 @@
targetbin_str = targetbin.encode('hex')
target = long(targetbin_str, 16)
- hashes_done = 1
- for nonce in range(MAX_NONCE):
+ for nonce in xrange(MAX_NONCE):
# encode 32-bit nonce value
nonce_bin = struct.pack("
# hash final 4b, the nonce value
- hash1_o = hashlib.sha256()
- hash1_o.update(blk_hdr)
+ hash1_o = hashlib.sha256(blk_hdr)
hash1_o.update(nonce_bin)
hash1 = hash1_o.digest()
# sha256 hash of sha256 hash
- hash_o = hashlib.sha256()
- hash_o.update(hash1)
+ hash_o = hashlib.sha256(hash1)
hash = hash_o.digest()
- hashes_done += 1
-
# quick test for winning solution: high 32 bits zero?
- H = struct.unpack('
+ if hash[-4:] != '\0\0\0\0':
continue
# convert binary hash to 256-bit Python long
@@ -147,12 +141,12 @@
# proof-of-work test: hash < target
if l < target:
print time.asctime(), "PROOF-OF-WORK found: %064x" % (l,)
- return (hashes_done,
+ return (nonce+1,
static_data[:76] + nonce_bin)
else:
print time.asctime(), "PROOF-OF-WORK false positive %064x" % (l,)
- return (hashes_done, None)
+ return (nonce+1, None)
def iterate(self, rpc):
work = rpc.getwork()
The patch eliminates 2 calls to hashlib.sha256(), 1 call to struct.unpack() and 1 one variable increment per work loop.
Cheers,
Updated.
Do you happen to have a git repository of the code?No, just the URL.
Updated.
Do you happen to have a git repository of the code?Cheers,
Do I interpret your meaning right when I say that the code — for each value of nonce — intends to calculate the SHA256 digest of the precalculated value (intended to stay constant throughout the loop) updated with the nonce? If so, that is not what the code does. Please see this simplified example:
Well, that is disappointing. If true, yes, that is a bug. Updated.
I wonder if this is a bug:
Do I interpret your meaning right when I say that the code — for each value of nonce — intends to calculate the SHA256 digest of the precalculated value (intended to stay constant throughout the loop) updated with the nonce? If so, that is not what the code does. Please see this simplified example:
So pre and post (like hash1_precalc_o and hash1_o) both points to the same object, and so I guess we should really write:
or even
Or did I misunderstand the whole concept?
Cheers,
Code:
# hash final 4b, the nonce value
hash1_o = hash1_precalc_o
hash1_o.update(nonce_bin)
hash1 = hash1_o.digest()
hash1_o = hash1_precalc_o
hash1_o.update(nonce_bin)
hash1 = hash1_o.digest()
Do I interpret your meaning right when I say that the code — for each value of nonce — intends to calculate the SHA256 digest of the precalculated value (intended to stay constant throughout the loop) updated with the nonce? If so, that is not what the code does. Please see this simplified example:
Code:
>>> from hashlib import sha256
>>> pre = sha256('abc')
>>> post = pre
>>> post.update('xyz')
>>> pre.digest() == post.digest()
True
>>>
>>> pre = sha256('abc')
>>> post = pre
>>> post.update('xyz')
>>> pre.digest() == post.digest()
True
>>>
So pre and post (like hash1_precalc_o and hash1_o) both points to the same object, and so I guess we should really write:
Code:
# hash final 4b, the nonce value
hash1_o = hashlib.sha256(blk_hdr + nonce_bin)
hash1 = hash1_o.digest()
hash1_o = hashlib.sha256(blk_hdr + nonce_bin)
hash1 = hash1_o.digest()
or even
Code:
# hash final 4b, the nonce value
hash1 = hashlib.sha256(blk_hdr + nonce_bin).digest()
hash1 = hashlib.sha256(blk_hdr + nonce_bin).digest()
Or did I misunderstand the whole concept?
Cheers,
Oh, and one more thing:
In the work loop, the line “hashes_done += 1” is effectively useless since nonce is already running from 0 to MAX_NONCE. So at the end of the loop — either because a winning solution has been found, or because MAX_NONCE has been reached — hashes_done equals nonce+1. Dropping the “hashes_done += 1” line gives a few extra khash/sec since this line was executed so many times.
Cheers,
In the work loop, the line “hashes_done += 1” is effectively useless since nonce is already running from 0 to MAX_NONCE. So at the end of the loop — either because a winning solution has been found, or because MAX_NONCE has been reached — hashes_done equals nonce+1. Dropping the “hashes_done += 1” line gives a few extra khash/sec since this line was executed so many times.
Cheers,
Ok, implemented the H==0 test shortcut (hey, everyone uses, might as well document it) and pyminer.py now gets 250 Khash/sec for one thread, on my box.
That's cool! 
If you drop the unpack stuff and compare the last 4 bytes directly you can gain another 10 khash/sec or so:
Code:
if hash[-4:] != '\0\0\0\0':
continue
continue
Cheers,
Ok, implemented the H==0 test shortcut (hey, everyone uses, might as well document it) and pyminer.py now gets 250 Khash/sec for one thread, on my box.
I'm not sure you caught my point.
You're right, I didn't catch your point. I have to chew on that, though. What we need is a binary buffer with the right endianness, not a python long integer, right?No, we really do need a 256-integer, because that is the fundamental proof-of-work test in the bitcoin system, comparing two 256-bit integers:
hash < target
Almost every practical miner simplifies this test to simply verify that the final, most-significant 32 bits are zero.
I'm not sure you caught my point.
You're right, I didn't catch your point. I have to chew on that, though. What we need is a binary buffer with the right endianness, not a python long integer, right?Cheers,
And remember, that bufreverse/wordreverse/encode/long sequence exists solely to build a 256-bit integer.
If you accept the patch that basically makes “hash = hash[::-1]” instead of a call to bufreverse() followed by a call to wordreverse(), there is no longer any use for wordreverse(), plus bufreverse() is ever only used in one place.The “hash[::-1]” (i.e., “reverse the entire buffer”) could be explained in a comment above the invocation.
I'm not sure you caught my point. If one (a) iterates over each 4-byte sub-string, (b) uses struct.unpack to perform endian conversion, and (c) uses Python integer math to build a 256-bit long, there should be no need for hash=hash[::-1] or bufreverse/wordreverse.
And remember, that bufreverse/wordreverse/encode/long sequence exists solely to build a 256-bit integer.
If you accept the patch that basically makes “hash = hash[::-1]” instead of a call to bufreverse() followed by a call to wordreverse(), there is no longer any use for wordreverse(), plus bufreverse() is ever only used in one place.The “hash[::-1]” (i.e., “reverse the entire buffer”) could be explained in a comment above the invocation.
Cheers,
I don't mind optimizing bufreverse/wordreverse, but I would like to avoid optimizing Miner.work() so heavily that it cannot be read. That's the function that must be most-readable to other humans.
And remember, that bufreverse/wordreverse/encode/long sequence exists solely to build a 256-bit integer. It would probably be more optimal to simply build a 256-bit integer using a per-word loop and shifts, such as
According to the docs, we can specify the byte order to struct.unpack()
And remember, that bufreverse/wordreverse/encode/long sequence exists solely to build a 256-bit integer. It would probably be more optimal to simply build a 256-bit integer using a per-word loop and shifts, such as
Code:
s = 'binary string...'
r = 0L
for i in range(8):
w32 = struct.unpack('>I', s[i:i+4])
r = r | (w32 << (i * 32))
r = 0L
for i in range(8):
w32 = struct.unpack('>I', s[i:i+4])
r = r | (w32 << (i * 32))
According to the docs, we can specify the byte order to struct.unpack()
More optimized functions:
Code:
def bufreverse(ch):
return "".join([ch[i:i+4][::-1] for i in range(0,len(ch),4)])
def wordreverse(ch):
return "".join([ch[i:i+4] for i in range(0,len(ch),4)][::-1])
return "".join([ch[i:i+4][::-1] for i in range(0,len(ch),4)])
def wordreverse(ch):
return "".join([ch[i:i+4] for i in range(0,len(ch),4)][::-1])
Jump to: