Topic: [Stratum] Overlay network protocol over Bitcoin - page 4. (Read 37882 times)

So does this mean that electrum servers will eventually be part of the stratum network? That's what I was hoping, I just want to make sure.

@slush:
I have rewritten the Electrum network interface, so that it uses jsonrpc over http when the http port is used.
I used the rpc calls that you provided in your document where it was possible, but some calls are missing for session management.
please have a look and update the protocol.

I shut it down cuz something broke in the database.  I don't really have the desire to fix it right now. I'll post my errors in the electrum thread.

Not needing to run bitcoind would be neat. I can see the arguments for having multiple clients on the bitcoin network, but I feel like it is also a dangerous road to go down.  It can potentially slow down adoption of features and definitely has the potential to introduce bugs. Sticking with satoshi's bitcoind is definitely a good idea for now.

I think libbitcoin will be a good choice once it and stratum are a bit more stable.

I agree Abe is lowest risk.  I was just thinking that looking at other people's code may give us some inspiration.

I also found https://github.com/samrushing/caesure which hasn't been active in a while, but it is all in python.  

Please don't shut down Electrum now, Stratum is still not ready as a replacement for Electrum backend. But I'm hoping it will be ready soon.

I'm working on two fronts. I like libbitcoin project a lot and I think it's the future of the Stratum project, because it following all bitcoin rules, does all validations, it's programmed in really good way and don't need to be running with satoshi's Monolith. However at this point libbitcoin is not mature enough. As an example, I have no idea how to really use it, because the lack of documentation. And I'm not hiding that my C++ skills are almost non-existing.

Thanks to this I see Abe as a better candidate for the startup. Although it is bounded to original satoshi's client, it's pure python and I can understand it better ATM. And I'm trying to implement Abe in the way that it can be easily replaced by another blockchain backend.

I didn't read the armory code yet, but it's also early alpha and memory usage is outragenous (but I understand it!). I still see the lowest risk in using Abe.

Edit: About running Stratum over Tor. I'm also thinking about this, it should be as easy as point hidden service to local transport ports. But I'm thinking about changing node.get_peers() method and IRC peer lookup to be more flexible and enable also exposing alternative channels like onion address. Then it should be easy to ask any Stratum node or IRC channel for all known Tor-enabled Stratum servers (it's the feature which satoshi's client is missing).

Maybe pip grabs everything while easy_install doesn't. I'm not sure though.

I've shut down my electrum server since I am liking this project more.  Are you still planning on using abe for indexing the block chain?  Maybe looking into some of the armory code would give us some ideas.

Interesting. This is needed for IRC support, but I see you in #stratum-nodes already. I'll need to check pip, if it also install some recommended dependencies or what. But if it didn't raise any exception during startup, then everything is fine.

Definitely checkout out http://www.doughellmann.com/projects/virtualenvwrapper/ if you are new to virtualenv. It adds some really nice commands.

I don't know if it makes much of a difference.  I've just been using pip for other things and am used to it.

I didn't.  It hasn't thrown any errors though, so it looks like I don't need to. Running "python -c "from twisted import words"" works without problem. Hopefully nothing is broken lol

Yeah it is weird. apt is "python-openssl" but pip/easy_install use "pyopenssl"

Red Emerald, great stuff. Although I'm using those packages in other applications as well and I don't need virtualenv, I think it's good candidate to use in official installation howto. I wanted to learn how to use virtualenv, so thanks for the command, I'll try it on my server and update installation howto.

a) easy_install and pip are using the same repository. You're right that pip is newer and should be used instead of easy_install. It's my fault.

b) how did you managed to install "twisted words" package from pip? I failed to use easy_install because it has space in the name (wtf?). It's why I used apt-get.

c) I fixed installation of openssl package in README. Didn't realized that it's called "pyopenssl" instead of "openssl". Stupid ambiguous name conventions in pypi...

I've got a server up on stratum.stitthappens.com.  Right now just 3333.  I haven't setup nginx to do any proxying yet, although that won't take long.

I don't like using apt to manage my python packages and I also prefer pip over easy_install.  I used Doug Hellmann's virtualenvwrapper, but virtualenv is also easy to use. 


I'll probably also get the server running on stratum.stitthappens.bit which will be able to securily use self signed certificates and also potentially resolve to a tor hidden service (v7g2e7rdrjpgsmlu.onion)

Great project, slush. Keep up the good work!

Nice! Yesterday, during the installation of chicago instance I realized that root privileges are necessary for running software on ports < 1024 (well, I known it years ago, but I simply forgot). So I'll need to implement mechanism (+init script) which will downgrade privileges after startup from root to some normal user.

For now the cleanest solution is to install (for example) Nginx on 80/443 and use reverse proxy module to route traffic to Stratum instance running on >1024 ports. I think that most servers are using 80/443 ports already, so this proxied setup will be the most common use case anyway. For this purpose I defined "application/stratum" HTTP content type, so stratum traffic can be easily splitted by simple rules in http servers like Apache or Nginx.


It isn't documented anywhere, but I'll write something to README file soon. So far protocol + TCP/HTTP trasport implementations are complete, some example services are already working. What isn't finished and what's important: blockchain service (which use indexed blockchain to ask for address history/balance) and library for broadcasting messages subscribed by client. Blockchain service is under development by my friend and I'll implement broadcasting messages very soon, so stay tuned :-).

I just moved around a couple of my VMs and reworked a firewall.  I should have stratum.stitthappens.com (California, US) up soonish.  I'll play around with ports 80 and 443, but that might take a little longer to get online.

Is there a KNOWN_ISSUES file or something somewhere? What works so far? What doesn't?

Thanks to support from BtcVPS.net guys, second Stratum node is running!

Currently there are two nodes: london.stratum.bitcoin.cz (Linode, UK), chicago.stratum.bitcoin.cz (BtcVPS, US).

One guy asked me how to install the server, so I prepared short installation howto.

I created repository for Stratum client implemented in PHP. Although it is already doing RPC requests, response handling and handling of broadcast messages isn't finished yet. I didn't code PHP for few years, so please excuse if some constructs looks old-fashioned (I'm receiving merge requests :-) ).

The simplest possible usage (it is plain HTTP Poll) is following:


Method add_request() collects calls into internal buffer, but only communicate() performs real network transfer. This looks pretty unintuitive, but it's minimizing network overhead. With this mechanism, you can submit tens or hundreds transactions with single HTTP call. communicate() will also call proper callbacks when client accepts some broadcasted messages (not implemented yet).

I'm still not sure what is the best solution for exception handling. communicate() shouldn't raise an exception when some call failed, so I'm thinking that callback won't be the value itself, but just a wrapper object containing real value OR exception object. When result is firstly accessed by the application (var_dump in example above), the exception is raised. But I don't know if such construction is even possible in PHP... Any suggestions?

Edit: Implemented basic idea  of exposing services on the client. See example.php in the repo.

Added. I hope that I'll configure wiki soon. Google docs did the job from the beginning, but I'd like to add some examples and Docs aren't the best choice anymore.

slush - you should add a link to the Google Doc from the Stratum Homepage.

I wrote this example of HTTP Push protocol for https://bitcointalksearch.org/topic/m.677355, but see it interesting enough to repost here, too:

Following HTTP call will subscribe URL http://yourdomain.com/callback.php for receiving notifications about 1YourAddress Bitcoin address. It may be useful for receiving realtime notifications for web application. Please note that method blockchain.address.subscribe is not fully implemented yet, it's just a protocol example. This subscription will expire in X-Session-Timeout seconds (presented in HTTP response), but can be prolonged by any other HTTP call with the given session ID (also returned in response):

(Note: there must be newline ("\n") on the end of every JSON command).

Response:

I finally found some time to learn basics of IRC protocol. Thanks to it, I implemented IRC peer lookup mechanism into Stratum server. See node.get_peers() method...