Love the program, I too wish to see a stratum version soon, as well.
One annoyance at the moment I would like to know how to solve, if possible, is getting the connection error to reset to normal after the miner disconnects. Right now it keeps saying:
"[Errno 10054] An existing connection was forcibly closed by the remote host"
After the miner is disconnected but I know it is running just fine after I reconnect. The current solution is to simply just close and reopen Rollproxy but I will lose my collected stats if I do so. So it's either lose my stats or not have a log in the mean time.
Thank you.
Topic: RollProxy - a bandwidth-saving mining proxy (Read 14396 times)
Is this proxy works with quark algorithm?
Its working but it auto crashes at random times (10 mins or 30 mins) by itself for me
I am running in a centos dedicated server with python 2.6
I am running in a centos dedicated server with python 2.6
Hello. i setup pushpool + bitcoind. is there appropriate mechanism to proxy to bigger mining pull to share profit with them? we are young and it will be difficult at first time to earn something.
it is possible to create something like that:
miners-> pushpool -> proxy -> another_pool?
will i be able to gather statistics from Rollproxy via SimpleBTC? now simplebtc connects to bitcoind via rpc and get all stat.
i looked at rollproxy and it seems it can help me.
Thanks in advance.
it is possible to create something like that:
miners-> pushpool -> proxy -> another_pool?
will i be able to gather statistics from Rollproxy via SimpleBTC? now simplebtc connects to bitcoind via rpc and get all stat.
i looked at rollproxy and it seems it can help me.
Thanks in advance.
I used rollproxy to force reuse of work units when solo-mining ppcoin for a few days. It worked very well, fixing my problem with running out of work when pointing miners directly to ppcoind. I found 2 blocks and those were the only shares submitted to rollproxy. Why would you want any other difficulty "shares" on a setup like that, or are you thinking of something different?
When you have multiple machines accessing the same pool server/proxy, it's nice to know that the miners are online and functioning properly. If they submit lower difficulty shares I will be able to tell this (even if they aren't difficulty 1.. 1 share every 5 or 10 minutes would be fine.)
You could turn this into a simple pool software that could mine on litecoind if you did the 2 following things:
#1) Allow the rollproxy to rewrite target difficulty
#2) Add the option to only submit shares that meet difficulty upstream.
Would you be willing to implement this for solo miners?
I used rollproxy to force reuse of work units when solo-mining ppcoin for a few days. It worked very well, fixing my problem with running out of work when pointing miners directly to ppcoind. I found 2 blocks and those were the only shares submitted to rollproxy. Why would you want any other difficulty "shares" on a setup like that, or are you thinking of something different?
You could turn this into a simple pool software that could mine on litecoind if you did the 2 following things:
#1) Allow the rollproxy to rewrite target difficulty
#2) Add the option to only submit shares that meet difficulty upstream.
Would you be willing to implement this for solo miners?
Hey. I need a small code addon. Is it possible?
Basically, I would like for the proxy to restart if there are >5% stale shares.
I think it's just a few lines of code. I'd do it myself, but I don't know Python.
Thanks a bunch!
Basically, I would like for the proxy to restart if there are >5% stale shares.
I think it's just a few lines of code. I'd do it myself, but I don't know Python.
Thanks a bunch!
any way to log mining data like shares submitted per worker etc ? like in a text file or something?
Done. There's now a --sharelog option that allows you to specify a file for logging submissions.
Sweet thanks !
any way to log mining data like shares submitted per worker etc ? like in a text file or something?
Done. There's now a --sharelog option that allows you to specify a file for logging submissions.
any way to log mining data like shares submitted per worker etc ? like in a text file or something?
I'm getting noticeably higher rejects when mining through rollproxy. Long poll is enabled on both cgminer and rollproxy, but my rejects are still 1-2% compared to the 0.5% max when directly connecting. There's no additional network delay since rollproxy is running on localhost. Has anyone else noticed higher reject rates? What might be the cause?
Looks like you've downloaded the github website but not the python code...
Try this:
Try this:
Code:
wget https://raw.github.com/pooler/rollproxy/master/rollproxy.py -O rollproxy.py
2to3 -w rollproxy.py
python3 rollproxy.py -v "your pools"
2to3 -w rollproxy.py
python3 rollproxy.py -v "your pools"
help me :
Quote
robin@ubuntu:~$ 2to3 -w rollproxy.py
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
RefactoringTool: Can't parse rollproxy.py: ParseError: bad input: type=20, value=u'<', context=(u'', (1, 0))
RefactoringTool: No files need to be modified.
RefactoringTool: There was 1 error:
RefactoringTool: Can't parse rollproxy.py: ParseError: bad input: type=20, value=u'<', context=(u'', (1, 0))
robin@ubuntu:~$
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
RefactoringTool: Can't parse rollproxy.py: ParseError: bad input: type=20, value=u'<', context=(u'', (1, 0))
RefactoringTool: No files need to be modified.
RefactoringTool: There was 1 error:
RefactoringTool: Can't parse rollproxy.py: ParseError: bad input: type=20, value=u'<', context=(u'', (1, 0))
robin@ubuntu:~$
Quote
robin@ubuntu:~$ python rollproxy.py -v "pool"
File "rollproxy.py", line 1
^
SyntaxError: invalid syntax
robin@ubuntu:~$
File "rollproxy.py", line 1
^
SyntaxError: invalid syntax
robin@ubuntu:~$
AFAIK that's not possible but RollProxy provides a webinterace where you can check your miners status...
Hi,
is it possible to connect with different workernames to a pool ? i lose controll which worker is down.
I could start a bunch of rollproxies on different ports, but that looks for me like a waste of recources.
Miner_1 =>> Rollproxy ==> worker1@abcpool
Miner_2 =>> same Rollproxy ==> worker2@abcpool
Thats what i mean.
Thanks
is it possible to connect with different workernames to a pool ? i lose controll which worker is down.
I could start a bunch of rollproxies on different ports, but that looks for me like a waste of recources.
Miner_1 =>> Rollproxy ==> worker1@abcpool
Miner_2 =>> same Rollproxy ==> worker2@abcpool
Thats what i mean.
Thanks
Sure, no problem. Just switched back to 0.5.2 and started logging to a file. If anything shows up I'll send it to you.
Ok, so after six days of using v0.5.2 (again) everything is fine. Whatever my problem was I'm unable to reproduce it...
Unfortunately that doesn't help. What is needed is a way for RollProxy to connect to the Internet through a SOCKS4/5 proxy, not a way for the miners to connect to RollProxy through an additional local proxy.
Use socat.
for mining through tor,
socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050
Sure, no problem. Just switched back to 0.5.2 and started logging to a file. If anything shows up I'll send it to you.
Rolled back to v0.5.1. On pool errors v0.5.2 started rotating over all configured pools in a endless loop...
I just had a look at the 0.5.2 commit. All I did is adding a 1-second pause between retries; I don't think I've changed anything else in the failover mechanism.
So I'd say if the problem you describe exists in 0.5.2 it should also exist in 0.5.1.
Can you please send me some logs if the issue shows up again?
Rolled back to v0.5.1. On pool errors v0.5.2 started rotating over all configured pools in a endless loop...
Thanks again!
Whenever rollproxy has a connection issue between itself and a pool, CPU usage goes crazy (it's stuck in a loop, I'd imagine). With enough rigs running against it, rollproxy can no longer keep up with providing work while this is happening.
I have added a 1-second pause between retries as a quick-and-dirty fix for this issue.What's this "cannot decode byte"-message? Happens everytime I try to access rollproxy with a browser... the webinterface is not displayed.
After exchanging a few private messages with ShadesOfMarble, this turned out to be a OS-and-locale-dependent issue; it should be fixed now.Version 0.5.2 should also support miners that (like Phoenix) issue long polling requests using the GET method.
What's this "cannot decode byte"-message? Happens everytime I try to access rollproxy with a browser... the webinterface is not displayed.
Getting this endlessly:
however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?
however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?
I have the same issue (although the error message in rollproxy varies a bit). Whenever rollproxy has a connection issue between itself and a pool, CPU usage goes crazy (it's stuck in a loop, I'd imagine). With enough rigs running against it, rollproxy can no longer keep up with providing work while this is happening.
Getting this endlessly:
however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?
however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?
I have the same issue (although the error message in rollproxy varies a bit). Whenever rollproxy has a connection issue between itself and a pool, CPU usage goes crazy (it's stuck in a loop, I'd imagine). With enough rigs running against it, rollproxy can no longer keep up with providing work while this is happening.
This is pretty slick. Thanks!!
For those windows users out there like me:
I couldn't get ez_setup to install urllib. It complained about a config file be missing.
So instead of I copied the urllib3 folder into my python\lib folder, and voila, it works.
For those that aren't used to the somewhat odd URL format this uses (I've seen it elsewhere, only reason I could figure it out), it's:
http://username:password@server:port
M
For those windows users out there like me:
I couldn't get ez_setup to install urllib. It complained about a config file be missing.
So instead of I copied the urllib3 folder into my python\lib folder, and voila, it works.
For those that aren't used to the somewhat odd URL format this uses (I've seen it elsewhere, only reason I could figure it out), it's:
http://username:password@server:port
M
Do you plan to add support for ntime rolling to the miners ? I'm asking because my config is a little odd and I have to cascade 2 rollproxy and I notice that second one does not see any support for ntime rolling from the first proxy.
If miners were allowed to do ntime rolling themselves, they could end up working on the exact same work unit, so it's not a good idea. Maybe it is possible to allow for little variations using a very low "expire" parameter, assuming of course that miners interpret that correctly, but honestly I can't see why you need the miners to roll the work themselves when they can just ask the proxy to do it.Quote
Do you plan to support the new getblocktemplate protocol in the proxy ?
Support for additional protocols is not planned at this time. I have very limited time to dedicate to this project, so right now I can only fix bugs and maybe add a few minor features.
Do you plan to add support for ntime rolling to the miners ? I'm asking because my config is a little odd and I have to cascade 2 rollproxy and I notice that second one does not see any support for ntime rolling from the first proxy.
Do you plan to support the new getblocktemplate protocol in the proxy ?
Do you plan to support the new getblocktemplate protocol in the proxy ?
Getting this endlessly:
however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?
however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?
Just for the fun - cascading the proxies (miner -> rollproxy -> stratum-proxy -> pool) works.
Yes, it's quite expected, because stratum proxy provides full getwork API, including long polling and ntime rolling.
pooler, looks like you're concerned about mining efficiency. Maybe you would be interested in supporting Stratum in your proxy as well? 
Just for the fun - cascading the proxies (miner -> rollproxy -> stratum-proxy -> pool) works.
pooler, looks like you're concerned about mining efficiency. Maybe you would be interested in supporting Stratum in your proxy as well?
I'm really busy with other stuff in this period, so no, or at least not any time soon.
pooler, looks like you're concerned about mining efficiency. Maybe you would be interested in supporting Stratum in your proxy as well?
As I understand, this helps for situation where miner don't support ntime rolling and pool does, right? [...]
The way I understand it, a miner can only do rolling within itself. If you have several miners on different machines, this proxy will help a lot. But please correct me if this is nonsense!Yes, macavity is right, that's the true raison d'être of this program, and why it was developed.
RollProxy allows multiple miners to reuse the same work unit. And this can bring huge bandwidth savings.
Use socat.
for mining through tor,
socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050
But that would still leak DNS requests, wouldn't it? That has always been the issue with tools like socat or tsocks.for mining through tor,
socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050
Your miner connects to 127.0.0.1 - no dns there
if you use the socks4a option on socat, it does the dns lookup for the pool through the proxy.
As I understand, this helps for situation where miner don't support ntime rolling and pool does, right? What miners don't support ntime rolling yet?
I lived in the opinion that ntime rolling is widely supported everywhere. Then I found that some big pools still don't support such simple and important extension, like Deepbit or even 50BTC. But as I understand, this proxy don't help in such situation - pool have to support ntime rolling itself.
Maybe the best thing is just to switch to miner and pool which supports ntime rolling, or even give a try to Stratum which implements mining protocol in quite innovative way (yes, this is kind of advertisement for my pool and BtcGuild.
The way I understand it, a miner can only do rolling within itself. If you have several miners on different machines, this proxy will help a lot. But please correct me if this is nonsense!I lived in the opinion that ntime rolling is widely supported everywhere. Then I found that some big pools still don't support such simple and important extension, like Deepbit or even 50BTC. But as I understand, this proxy don't help in such situation - pool have to support ntime rolling itself.
Maybe the best thing is just to switch to miner and pool which supports ntime rolling, or even give a try to Stratum which implements mining protocol in quite innovative way (yes, this is kind of advertisement for my pool and BtcGuild
.Besides, even without this feature, mining proxies are great tools to have if you use many machines. It makes monitoring, maintenance, setup of new machines or changing the pool and propagating that change to all clients so much easier.
I especially like Rollproxy because it's so small and efficient. I've used another proxy before, but it required apache, php, mysql, regular database cleanups, and while it worked well, it was a bit cumbersome to use.
As I understand, this helps for situation where miner don't support ntime rolling and pool does, right? What miners don't support ntime rolling yet?
I lived in the opinion that ntime rolling is widely supported everywhere. Then I found that some big pools still don't support such simple and important extension, like Deepbit or even 50BTC. But as I understand, this proxy don't help in such situation - pool have to support ntime rolling itself.
Maybe the best thing is just to switch to miner and pool which supports ntime rolling, or even give a try to Stratum which implements mining protocol in quite innovative way (yes, this is kind of advertisement for my pool and BtcGuild.
I lived in the opinion that ntime rolling is widely supported everywhere. Then I found that some big pools still don't support such simple and important extension, like Deepbit or even 50BTC. But as I understand, this proxy don't help in such situation - pool have to support ntime rolling itself.
Maybe the best thing is just to switch to miner and pool which supports ntime rolling, or even give a try to Stratum which implements mining protocol in quite innovative way (yes, this is kind of advertisement for my pool and BtcGuild
.Unfortunately that doesn't help. What is needed is a way for RollProxy to connect to the Internet through a SOCKS4/5 proxy, not a way for the miners to connect to RollProxy through an additional local proxy.
Use socat.
for mining through tor,
socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050
Your miner connects to 127.0.0.1 - no dns there
if you use the socks4a option on socat, it does the dns lookup for the pool through the proxy.
Unfortunately that doesn't help. What is needed is a way for RollProxy to connect to the Internet through a SOCKS4/5 proxy, not a way for the miners to connect to RollProxy through an additional local proxy.
Use socat.
for mining through tor,
socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050
say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support
cgminer does:
--socks-proxy
Unfortunately that doesn't help. What is needed is a way for RollProxy to connect to the Internet through a SOCKS4/5 proxy, not a way for the miners to connect to RollProxy through an additional local proxy.
Use socat.
for mining through tor,
socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050
say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support
cgminer does:
--socks-proxy
Unfortunately that doesn't help. What is needed is a way for RollProxy to connect to the Internet through a SOCKS4/5 proxy, not a way for the miners to connect to RollProxy through an additional local proxy.
say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support
cgminer does:
--socks-proxy
In any case, it seems my debian squeeze server is stuck on python 2.6.6 anyway, which neither offers urllib3 packages, nor satisfies rollproxy's version requirement. Oh well.
Rollproxy on squeeze works fine for me:
Code:
apt-get install python-pip
pip install argparse
pip install urllib3
python /usr/local/bin/rollproxy.py -v
[2012-08-29 08:58:05] Serving HTTP on port 8345
thank you very much, it is working now.
and thank you, too, pooler. rollproxy is great!
In any case, it seems my debian squeeze server is stuck on python 2.6.6 anyway, which neither offers urllib3 packages, nor satisfies rollproxy's version requirement. Oh well.
Rollproxy on squeeze works fine for me:
Code:
apt-get install python-pip
pip install argparse
pip install urllib3
python /usr/local/bin/rollproxy.py -v
[2012-08-29 08:58:05] Serving HTTP on port 8345
say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support
I realize that SOCKS support would be a very nice feature to have. The problem is that the master branch of urllib3 doesn't support SOCKS proxies yet, although developers have been working on it for some time now (see this pull request).
One option would be to rewrite RollProxy to use PycURL instead of urllib3, but the problem is that urllib3 seems to be the only decent HTTP library that supports connection pooling.
Another option would be to use SocksiPy, which however is buggy, old, and unmaintained.
For the above reasons, it's probably better to wait till the urllib3 guys finish implementing SOCKS support.
In any case, it seems my debian squeeze server is stuck on python 2.6.6 anyway, which neither offers urllib3 packages, nor satisfies rollproxy's version requirement. Oh well.
say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support
I realize that SOCKS support would be a very nice feature to have. The problem is that the master branch of urllib3 doesn't support SOCKS proxies yet, although developers have been working on it for some time now (see this pull request).
One option would be to rewrite RollProxy to use PycURL instead of urllib3, but the problem is that urllib3 seems to be the only decent HTTP library that supports connection pooling.
Another option would be to use SocksiPy, which however is buggy, old, and unmaintained.
For the above reasons, it's probably better to wait till the urllib3 guys finish implementing SOCKS support.
What about just using p2pool?
say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support
Great idea. I will try it out
So i connect them to localhost:port x ?
You connect miners to the computer running the proxy. If a miner is on the same computer as the proxy, you can just connect it to localhost:8345. If the proxy is running on, say, 192.168.1.123, connect miners to 192.168.1.123:8345.
So i connect them to localhost:port x ?
And for the windows dummies out there (like me) ?
Last time I used Windows was about 10 years ago, so it's best if I let someone else answer this question...?how do i connect a miner to that proxy?
Normaly i give my mining program the credentials to my pool. For what i understand is that i give them to your proxy and connect my miner to it instead?
Yes. You start the proxy specifying the address and credentials for one or more pools; then you connect your miners to the proxy, which by default runs on port 8345. Miners need not authenticate, but if they provide a username you will see per-worker stats in the web interface of RollProxy (which is also served on port 8345 by default). Normaly i give my mining program the credentials to my pool. For what i understand is that i give them to your proxy and connect my miner to it instead?
Note that cgminer does this already intrinsically and adding this proxy with it serves no real purpose.
That's if you only have one computer mining. If you have more than one, no miner can do what this proxy does.
Note that cgminer does this already intrinsically and adding this proxy with it serves no real purpose. I can't speak for all other mining software though.
And for the windows dummies out there (like me) ?
how do i connect a miner to that proxy?
Normaly i give my mining program the credentials to my pool. For what i understand is that i give them to your proxy and connect my miner to it instead?
?how do i connect a miner to that proxy?
Normaly i give my mining program the credentials to my pool. For what i understand is that i give them to your proxy and connect my miner to it instead?
Great work, pooler! Thanks a lot for sharing this.
Nice work.
Nice, will be using this
Announcing RollProxy, a mining proxy... with a twist!
RollProxy takes advantage of the X-Roll-NTime extension to minimize the number of getwork requests you send to pools. By keeping more requests local, it minimizes latency, while drastically reducing the load on upstream servers.
The proxy will prove especially useful to those who run many miners on a single local network.
Features
Please note that your miners need not support the X-Roll-NTime extension; it's the pools you connect to that need to. If you connect to a pool that doesn't support the extension, the proxy will still work, but you will see no increase in getwork efficiency. In that case, RollProxy will warn you with a "Work cannot be reused" message.
Major pools that are known to support the extension include:
RollProxy is written in Python and is distributed under the terms of the AGPLv3. Please refer to the readme for details about setting it up.
https://github.com/pooler/rollproxy
This first public release (version 0.5.0) should already be pretty stable; I wish to thank Pontius, who helped me by testing preliminary versions of the program.
RollProxy takes advantage of the X-Roll-NTime extension to minimize the number of getwork requests you send to pools. By keeping more requests local, it minimizes latency, while drastically reducing the load on upstream servers.
The proxy will prove especially useful to those who run many miners on a single local network.
Features
- Works with Bitcoin as well as with Litecoin and other cryptocurrencies that use the getwork protocol
- Makes persistent HTTP connections
- Supports long polling
- Per-worker statistics available via web interface
- Round-robin failover mechanism (more advanced strategies may be implemented in the future)
- Ability to connect through an additional proxy server
Please note that your miners need not support the X-Roll-NTime extension; it's the pools you connect to that need to. If you connect to a pool that doesn't support the extension, the proxy will still work, but you will see no increase in getwork efficiency. In that case, RollProxy will warn you with a "Work cannot be reused" message.
Major pools that are known to support the extension include:
- For Bitcoin: Eclipse, Eligius, MaxBTC, Mt. Red, OzCoin, P2Pool, Slush
- For Litecoin: burnside's, LitecoinCash, notroll.in, OzCoin, P2Pool, Pool-X, Xurious's
RollProxy is written in Python and is distributed under the terms of the AGPLv3. Please refer to the readme for details about setting it up.
https://github.com/pooler/rollproxy
This first public release (version 0.5.0) should already be pretty stable; I wish to thank Pontius, who helped me by testing preliminary versions of the program.
Jump to: